An agentic team of data scientists that takes any dataset from raw data to model inference — using AI.

Avaloka is a LangGraph-powered team of specialized agents you direct in plain language. Point it at a dataset and it will sample and profile the data, plan an analysis, write and validate the Python to carry it out, execute that code locally or on a distributed Ray cluster, train a model, serve it for inference, and persist every artifact along the way. One chat session can go from a raw CSV to a deployed model endpoint — with a human in the loop at each decision.

New here? Three links, in the order most people want them:

1. 📦 Install it — one command, or fully offline.
2. 🚶 Walk through it — sign in, upload, ask, train.
3. 🗺️ Understand it — the agent team and the request lifecycle.

Five minutes, on your own machine, no cloud account.

git clone https://github.com/guruvaidev/avaloka.git && cd avaloka
./scripts/install.sh                 # creates .venv, installs deps, prints an edition report

Give it one model key — any provider — and start the API:

export GROQ_API_KEY="<your key>"     # or OPENAI_API_KEY / OPENROUTER_API_KEY / a local Ollama
uvicorn app.api.server:app --port 9000
curl localhost:9000/health

Then upload something and ask a question:

curl -F "file=@sales.csv" localhost:9000/api/upload

Port 9000, not 8000. The container image serves on 9000 and the Helm service maps 8000 → 9000. Uvicorn's own default is 8000, which is the single most common reason a fresh install "cannot reach the API" — pass --port 9000 and every path lines up.

Prefer a cluster, a UI, or no internet at all? That is all in the Install Guide — including running Avaloka completely offline with a --network none check so you can prove it rather than trust it.

Something not working? The install guide's First run: what to expect, and what goes wrong section lists every failure we hit bringing this stack up on a clean cluster. Each one is silent — the symptom never names the cause — so it is worth a read before you start debugging.

The open-source edition is not a limited trial of the hosted ones. It is the same engine on your own hardware, and it stays useful with no account, no key and no network.

Under the hood, a Planner agent acts as the team lead: it converses with you and delegates to Sampling, Profiling, Coder, Validator, Execution, Data-Transfer, Model-Training, and Visualization specialists, all coordinating through one shared state object. Generated code is authored pseudocode-first and passes a three-layer validation gate, so it stays grounded in the source schema even when the primary LLM is unavailable.

Avaloka is an orchestration layer, not a reimplementation. The heavy lifting — distributed execution, storage, vector search, experiment tracking, the database — is done by mature, widely deployed projects that were load-bearing in production long before Avaloka existed. What Avaloka adds is the agent team, the evidence layer and the conversation; what it deliberately does not add is a homegrown substitute for any of the following.

Every one of these is independently maintained, independently released and independently audited by communities far larger than ours. When Avaloka reports a number, the computation underneath it was performed by software that has been vetted by many more eyes than a single vendor could bring — and you can verify any of it yourself, because none of it is hidden behind our abstraction.

The full per-project attribution, with licences, is in Acknowledgements and NOTICE.

Everything below is in this repository. Start wherever your question is.

Avaloka is open source under the Apache License 2.0.

Jim Gray (1944–2007), Turing Award laureate, and The Fourth Paradigm: Data-Intensive Scientific Discovery (Microsoft Research, 2009).

In a January 2007 lecture — his last before he was lost at sea — the database pioneer Jim Gray described a fourth paradigm of science. The first three were empirical observation, theoretical models, and computational simulation. The fourth is data-intensive discovery: science conducted by exploring and analyzing vast, messy data. Gray's dream was not more data for its own sake — it was to give scientists effective tools so they could spend their time on questions and discovery instead of on the plumbing of capturing, cleaning, moving, and analyzing data. That dream is collected in the book that bears his idea's name, published in his honor.

Two decades on, most of the plumbing is still there — cloud accounts, clusters, data pipelines, schema wrangling, visualization boilerplate, statistical setup — and it still stands between a curious person and an answer.

Avaloka is built to remove that cruft. Drawing on decades of data-engineering and data-science experience, it is an AI-native data OS for ETL, ML, and experimentation — deliberately not code-first, and not another chatbot code generator. It is analysis-first and model- and inference-first: you lead with the question, and the AI builds the scaffolding beneath it — the cloud and Kubernetes, the data pipelines, the visualization, and the statistical analysis — so a raw dataset becomes clean data, actionable intelligence (not just charts), a trained model, and a served prediction.

That is Gray's dream, made practical by a team of AI agents that work like a data science team — so the scientist, the analyst, and the curious can stay with the science.

Upload a spreadsheet to Claude or ChatGPT and you get a genuinely useful answer about that spreadsheet. In an enterprise the interesting facts are not in one spreadsheet. They are spread across an operational database, a warehouse, an object store and a document store, and the question worth asking is almost always a cross-correlation between them — the churn number next to the support history next to the billing record. A file you can upload is already the answer to a much smaller question than the one you have.

An agentic application is necessary but still not sufficient. An agent that writes and runs code against your systems removes the copy-paste step, which is real progress. It does not, by itself, tell you whether the answer is right. An agent that silently samples the first thousand rows, or leaks the target into its own features, or agrees with you when you push back on a number, is more dangerous than a spreadsheet — it is wrong at machine speed, fluently, with supporting prose.

Three things have to hold at once, and it is the combination that is hard:

Connected, not uploaded. The agent reaches the systems where the facts already live — PostgreSQL, MongoDB, DuckDB, SQLite, S3/GCS, warehouses — and joins across them. Data flows, not data files, are what business processes actually look like.

Conversational, because analysis is a conversation. The first question is never the real one. "Why are people leaving?" becomes a question about tenure, then about support tickets, then about one plan tier. A system that answers once and stops has not done the job; a system that asks which of two readings you meant, and says when the evidence contradicts you, has.

Verified, because fluent and correct are different properties. Avaloka runs its own checks against its own work: integrity checks for leakage and train/test contamination, a mandatory baseline every model must beat before it is reported as good, and a claim verifier that checks the prose against the computed evidence. When a result does not hold, the system says so. That is the part a model cannot supply on its own, because a model has no privileged access to whether its own answer is true.

Transparency is the thread through all three: every number is traceable to the query that produced it, every sample says it is a sample, and every claim is checkable.

The industry position is that the best model is not sufficient for enterprise work, and that what actually decides the outcome is the harness, the learning loop and the knowledge graph around it. We agree, and the reason is measurable rather than rhetorical: swap the model underneath a weak harness and the answers change; swap the model underneath a strong one and the errors change but the guarantees do not.

The harness is what stands between a fluent answer and a true one. In Avaloka it is not a wrapper around a prompt — it is an execution graph with checks that can stop it: integrity checks that refuse to train on leaked or contaminated data, a baseline every model must beat before it is reported as good, optimizer-aware validation of generated queries before they run, and a claim verifier that checks the prose against the computed evidence rather than against the model's own narration. A model cannot supply any of this, because a model has no privileged access to whether its own answer is true.

The knowledge graph is what makes an answer traceable and a question answerable across systems. Avaloka records nodes and edges for datasets, columns, transformations and derived artifacts, so a claim resolves back through its lineage to the rows that produced it — and so a question like "where did this PII end up" has an answer that is computed rather than remembered.

The learning loop is what makes the second analysis cheaper than the first: episodic memory over past queries, retrieved by similarity, plus stated preferences that persist across a session and style hints that shape generated code. This is the youngest of the three, and the section below says plainly where it stands.

None of these three is a model feature. All three are the product.

We would rather publish the gaps than claim three pillars and evidence one.

The honest summary is that Avaloka today is a strong harness, a real graph, and a working learning loop whose value is still ahead of it. Anyone evaluating the three-pillar claim should weight it that way, and the test report contains the per-case evidence for each row above, including which cases are blocked and on what.

On the loop specifically: it is deployed, it persists across restarts, and it retrieves what earlier turns established. That is the point of it today — to hold context so that memory-aware behaviour has somewhere to live. We are not claiming it makes analyses measurably better, and we have not measured that. It is foundation, not a result, and we would rather say so than dress a scaffold up as an outcome.

Avaloka is evaluated against external and internal benchmarks rather than described in adjectives:

• DataAgentBench (DAB) — the UC Berkeley agentic data-analysis benchmark: 54 queries over 17 datasets spanning PostgreSQL, MongoDB, SQLite and DuckDB, graded by the official per-query validators. Run against the unmodified upstream scaffold.
• Conversational integrity — sycophancy resistance, evidence fidelity under social pressure, honest delivery of bad news, and ambiguity handling, each bracketed by scripted faithful and sycophantic reference systems so the numbers have a floor and a ceiling rather than floating free.
• Adaptive sampling, swarm ablation, reliability, and a model matrix — what each agent contributes, what sampling costs in accuracy, how the system behaves under injected faults, and how much conversational integrity the cheap model tier gives up.

Current results, their bounds, and the places the system is weakest are in docs/ and in the research paper. Where a number is not yet measured it is marked unmeasured rather than estimated.

This repository is the open-source edition. Everything here is Apache 2.0 and yours to run.

./scripts/install.sh          # open source
./scripts/install.sh --check  # what does my install actually allow?

Two things people reasonably expect to be different:

Self-hosted open source has the scheduler and distributed Ray execution. It is your cluster and your bill, so we do not cap it. The Free hosted tier does not, because we pay for those cycles. Same code — the difference is who is paying.

Generated code is yours in every edition, Free included. It is your work product; review it and run it at your own discretion, as you would any code.

Commercial capabilities are not flags you can flip — they live in a separate package that an open-source install never pulls. Full detail, including how offline licence verification works for air-gapped deployments, is in docs/EDITIONS.md.

Two independent questions decide what you get, and collapsing them is the usual source of confusion:

This is why "does the free version have a scheduler?" has no single answer. Self-hosted open source: yes — it is your cluster and your bill. Free on Avaloka's cloud: no — we pay for every scheduled cycle. Same code, opposite answer; the difference is who is paying.

A capability is unavailable for exactly one of three reasons, and every gate declares which (GATE_REASON in app/core/editions.py):

A capability disabled in a self-hosted OSS install for a COST reason is a bug. The boundary is enforced by import probe, not configuration — no environment variable can claim a capability whose module is absent — which is what makes the split honest rather than advisory. See docs/EDITIONS.md.

./scripts/install.sh                 # open source
./scripts/install.sh --check         # what does my install resolve to?
./scripts/install.sh --edition professional --license-key "$KEY"

A laptop, or a small Kubernetes cluster you run yourself. That is the target, it is what gets tested, and it is what the capability matrix grants. Everything below follows from it.

If you need scale — provisioned and autoscaled clusters, larger distributed jobs, scheduled cloud workloads, SLAs — that is what the commercial editions are for. See avaloka.ai, or email support@avaloka.ai.

Included, and fully functional

• The whole analysis path — profile, transform, analyse, visualise
• Distributed Ray execution on a cluster you already run, and the local scheduler (Celery + Redis via docker-compose.scheduler.yml). These are not capped: it is your cluster and your bill, so capping them would protect nothing. They are sized for a small cluster, not limited to one.
• Model training and inference locally
• The generated code, always. Whatever Avaloka writes, you can export and run in your own environment. There is no black box.
• File connectors: CSV, TSV, JSON, XML, Excel, Parquet, Avro, Delta, Iceberg
• Any model provider, including local models

Not included

• Cloud infrastructure provisioning. Avaloka will not create a GKE, EKS or AKS cluster for you. get_provider("gcp"|"aws"|"azure") refuses in an open-source build and tells you where to go. This is a genuine limitation, not a switch: standing up and paying for cloud infrastructure on your behalf is what the commercial editions do.
• Scheduling work onto a cloud cluster. The scheduler runs locally and schedules local and own-cluster work. What it cannot do is dispatch a job to a managed cluster it did not provision.
• Scheduling a training run onto a cloud cluster. MTA itself is yours — train and serve models locally or on your own Ray cluster, with MLflow tracking and the model registry. What is commercial is dispatching that work to managed cloud infrastructure Avaloka provisioned for you.
• Scale operations — autoscaling policy, node-pool management, cost controls, multi-tenant isolation.
• Team features — shared analyses, comments, notifications, SSO and audit.

What "connect" still does. Avaloka can attach to any cluster your kubeconfig already reaches, in