Build call trees from call detail records (CDRs): chains of calls linked in time that either converge on a number — who fed into it — or diverge from one — how something spread outward.

• Two deployables, one repository. backend/ is the service and the engine, frontend/ is the UI, and they meet over HTTP — one Deployment each, with the OpenShift manifests in deploy/openshift/.
• Connect your own providers by config. Point it at your CDR providers in a JSON file — each with its own field names and request shape — no code changes.
• Built for rate-limited providers. Batched, cached, adaptively paced — and a closed circle search prefetches every possible number in one batch.
• Offline by design. The HTML output inlines everything; nothing fetches a CDN.
• Measurable. Every build reports its runtime and provider query counts, and backend/tools/bench.py runs whole suites of cases and diffs two versions.

--- tree 1: 8 call(s), 2 root call(s), 4 tier(s), width 3
+15550100003 → +15550100001  [11:59:00-12:00:30, 90s]  #c002
└── +15550100006 → +15550100003  [11:52:00-11:55:00, 180s]  #c005
+15550100002 → +15550100001  [12:00:00-12:02:00, 120s]  #c001
├── +15550100004 → +15550100002  [11:50:00-11:56:00, 360s]  #c003
│   └── +15550100007 → +15550100004  [11:40:00-11:47:00, 420s]  #c006
│       └── +15550100009 → +15550100007  [11:30:00-11:38:00, 480s]  #c008
└── +15550100005 → +15550100002  [11:58:00-11:59:30, 90s]  #c004
    └── +15550100008 → +15550100005  [11:51:00-11:54:00, 180s]  #c007

Two calls reached the sink 30 seconds apart, so that is one tree with two root calls rather than two trees telling you about the same event twice.

The two halves install separately, because they deploy separately.

# the service
cd backend
python3 -m venv ../.venv && source ../.venv/bin/activate
pip install -r requirements.txt -r requirements-dev.txt
cp config/providers.example.json config/providers.json   # point it at your providers

# the UI
cd ../frontend && npm install

cd backend  && python src/main.py    # API on :8000, API docs at /docs
cd frontend && npm run dev           # UI on :5173, proxying /api to :8000

Everything configurable on the backend is an environment variable (all optional): CALLTREES_HOST, CALLTREES_PORT, CALLTREES_REGISTRY, CALLTREES_LOG_LEVEL (debug logs every outgoing provider request body), CALLTREES_LOG_FORMAT (json for Filebeat → Elastic) — full list in docs/service.md. The UI needs one thing, and only when the API is not on the same origin: CALLTREES_API_BASE, read at container start-up rather than baked into the bundle, so one image works against any backend.

Everything in one go, docker:

docker compose up --build      # UI on :8080, API proxied under /api

On OpenShift, as two Deployments:

docker build -t <registry>/calltrees-backend:1.0.0  ./backend
docker build -t <registry>/calltrees-frontend:1.0.0 ./frontend
./deploy/openshift/deploy.sh <registry>/calltrees 1.0.0

New to OpenShift? deploy/openshift/README.md opens with a click-by-click walkthrough in the web UI that assumes no Kubernetes background — no command line at all — and the oc version above is the second half of the same page.

The tests:

cd backend && PYTHONPATH=src pytest tests -q     # engine + API in one suite

Mocks, to try it before wiring anything real: backend/mocks/ holds a stand-in for each outside service — sample-records.json for a CDR provider, scores.json for the scoring service's answer, identities.json for the identity directory's and transcripts.json for the speech service's (see backend/mocks/README.md). Each is a mock of a contract, and all three answer files are wired by pointing the feature's url at an absolute file: URL — urllib reads it and ignores the body it would have POSTed, so the client, the fold and the views run exactly as they do over HTTP.

The CDR sample is 10 numbers (each with an identification) across scenario bands, one per calendar day, so different cases never bleed into each other. Point a registry entry of type file at it (the local-sample entry in backend/config/providers.example.json shows the shape) and build from the UI with seed +15550000001, range 2026-07-20T11:00:00Z–13:00:00Z, window 600, min call length 30. Other days hold a fan-out hub (07-21), a 6-hop chain (07-22), cycles (07-23), a down-tree (07-24), overlapping calls (07-25), an exact boundary gap (07-26), several separate trees on one seed (07-27), parse rejects (07-28) and repeated calls in a row (07-29).

A larger randomized set is also generatable:

python -c "import json, sys; sys.path.insert(0, 'src'); \
  from calltrees.engine.sampledata import build_sample; \
  json.dump(build_sample(), open('records.json', 'w'))"

This is the point of the tool: run it on an internal network and query your own CDR providers, each with its own field names and request shape.

Providers are configuration, not code. Each entry in backend/config/providers.json pairs how to reach a provider with how that provider names its fields, so several services returning the same kind of rows under different key names all work unchanged — and per-provider chunk_size / start_rate / max_rate tune the batching and pacing to each provider's rate limit:

"billing-cdr": {
  "source": {
    "type": "http",
    "url": "https://billing.internal/v2/cdr/search",
    "headers": { "X-Api-Key": "${BILLING_TOKEN}" },
    "chunk_size": 200, "start_rate": 4.0, "max_rate": 8.0
  },
  "fields": { "call_id": "cdr_ref", "caller": "calling_party",
              "receiver": "called_party", "start": "connect_time",
              "end": "disconnect_time" }
}

${ENV_VAR} is substituted from the environment at startup, so tokens never live in the file. Adding or re-pointing a provider is a config edit plus a restart.

Any field may name one key or a list of keys tried in order — "caller": ["calling_party", "a_number"] — which covers a provider whose rows are not uniform without a second registry entry per shape. A key present but null counts as absent, so a feed that returns every column and nulls the empty ones still falls through to the next name.

fields may also map the optional caller_name / receiver_name, which carry each party's identification: free text your provider may put anything in — a person, a role, an employer, an area, or several of those in one string. (The keys keep the older _name spelling because they are the wire contract; the views call it identification, because "name" promises one thing and the field holds many.) It shows up in every output — text, CSV, DOT, the HTML views and tooltips — and every view leads with it, falling back to the bare number when there is none. Linking always uses the numbers.

If a provider expects a nested request body rather than a flat object, give it a body_template — the exact JSON it wants, with {{numbers}}, {{since}}, {{until}}, {{role}}, {{exclude}}, {{circle}}, {{min_call_length}} placeholders dropped wherever they belong, plus time_format (epoch / epoch_ms / iso) and records_path for where the records sit in the response. The filter placeholders are also enforced locally, so a provider that ignores them still yields correct trees — honouring them just returns fewer wasted rows. Full details and examples in docs/service.md.

If one database is reached through several endpoints partitioned by phone prefix (endpoint A for +1…, endpoint B for the rest), give the provider routes instead of a source — each route names its prefixes and source, sharing one fields mapping. A single build then routes each number to its endpoint as the tree crosses prefixes. A number matching no route is reported in the build's progress and left unexpanded rather than failing the build — a foreign number two hops out is a normal thing for a cascade to do. See "One provider, partitioned by prefix" in docs/service.md.

When a provider rejects a request, the job fails with the provider's own explanation (its response body, not just the status line). Logs are structured for production: set CALLTREES_LOG_FORMAT=json (the docker default) for one JSON object per line that Filebeat ships straight to Elasticsearch, with a job_id on every record so a failed build is one filter to investigate. See "Logs" in docs/service.md.

Optional, off by default: one short sentence per hop of a tree — "John called 3 taxi drivers and his brother" — written by Claude from the free-text identification on each party, which is where the roles, employers, areas and family relationships live. A hop is one expanding party and everyone it reached in one step, so each sentence is about one caller rather than a whole tier.

cd backend
cp config/llm.example.json config/llm.json    # then set your key / endpoint
pip install -r requirements-llm.txt           # the one optional dependency

Then tick Summarise each layer in the UI, or send "summarize": true with the build. Everything about the model — which one, the endpoint, effort, batch sizes, the prompt — lives in config/llm.json with the same ${ENV_VAR} interpolation as providers.json, so retuning it is an edit and a restart. Set base_url to an Anthropic-compatible gateway when the deployment cannot reach api.anthropic.com, which on these networks is the normal case.

A broken or unreadable config/llm.json turns the feature off and says so in the log; it never stops the service. Delete it (or set "enabled": false) and the feature disappears completely — no summaries, no dependency needed, and the checkbox is hidden rather than shown greyed out. A summary that fails never fails the build: the forest is the product and the sentences are a commentary on it, so an unreachable endpoint costs you a paragraph, not a run. Read them as a lead, not as evidence — the underlying calls are always right there beside them.

Optional, off by default: a directory service is asked who each number belongs to and answers with a card — a picture, a display name, a line of context, tags, and however many labelled fields it wants to send. The card's name is then what every view calls that party, and the whole card is laid out in the details panel beside the forest.

cp backend/config/identity.example.json backend/config/identity.json   # then point it at your directory

Tick Identify numbers, or send "identify": true. Two things are worth knowing before you wire a service up:

• Every key you send shows up. Only the keys named in the config get a place of their own; every other key in the row becomes a labelled field, in the order it arrived. Adding a field to what your directory returns needs no change here — send it and it appears.
• Pictures are carried, not linked. The HTML page makes no external requests, so a photo URL is fetched once at build time and written into the page as a data: URI (inline_photos, capped by max_photo_bytes). A photo that is too big, unreachable or not an image is left off — it is a picture, and no picture is worth failing a build for.

Optional, off by default: a speech-to-text service returns per-call segments, and they appear in the call's own details — click any call and the transcript is there, laid out as the conversation it was: bubbles, one side per speaker, the offset into the call tucked under each turn. It sits with the call rather than in a view of its own, because a transcript is only ever about one call and a separate tab made you pick the call twice.

cp backend/config/transcripts.example.json backend/config/transcripts.json

Tick Fetch transcripts, or send "transcribe": true. Segment times are offsets from the start of the call in seconds (time_scale converts an engine that answers in milliseconds), and speaker labels are the engine's — each run of turns is tagged with the label the engine gave it, and the sides are only a way of separating them on screen. Nothing claims which of the two numbers was speaking, because a mapping the engine did not make is a guess.

A build's sweep is capped, because transcribing a 400-call forest up front is a long wait for text nobody has opened. Any call it did not cover has a Fetch transcript button in its details, which pulls just that one from GET /api/builds/{id}/transcript/{call_id}. The filter box searches transcript text too, so a call can be found by something said on it.

Builds run as async jobs: POST /api/builds validates immediately and returns a job id; poll GET /api/builds/{id} (or subscribe to /stream for live progress), then fetch /result?format=html|json|csv|dot|tree|unicode. Interactive API docs at /docs. Details: docs/service.md and frontend/README.md.

You do not have to wait for a build to finish to look at it. The forest is published after every expansion step: the first layer appears as soon as it exists and the rest arrive on top of it, so a slow provider is a picture that grows rather than a progress log that might be stuck. The page carries a standing "still building" banner while that is true — an unfinished forest must never be read as a finished one — and the page is loaded once, with later layers pushed in as data, so a landing layer never costs you the tab you were on, the numbers you dragged, the call you selected or where you had scrolled to. Scripts can ask for the same thing with /result?format=html|json&live=true.

And you do not have to fetch the same calls twice. Once a build has run, Re-run on these calls runs the whole pipeline again over the rows it already has: a different window, different filters, the other direction, or a commentary you did not ask for the first time — with no provider traffic and no waiting. It cannot reach calls that were never fetched, so widening the range finds less, not more; a genuinely new question is still a new build. Over the API it is {"from_build": "<job id>", "params": {...}}.

The UI and the HTML page are available in English and Hebrew; the switch is in the header, and choosing Hebrew turns the whole interface right-to-left — including the graph, which mirrors so a tree grows outwards from its seed in the reading direction. /result?format=html&lang=he renders the same page for a script or a download. The data formats are unaffected.

For an up-tree, child Q extends parent P when:

Q.receiver == P.caller       Q reaches the number that placed P
Q.start <= P.end             Q is not strictly after P
P.start - Q.end <= window    the gap is within the window

The gap goes negative when the calls overlap, so concurrent calls link naturally — B relaying to C while still connected to A is exactly what a cascade looks like.

Anchoring on start times instead is always stricter (since Q.start <= Q.end), so "link if either anchor fits" reduces exactly to the test above. There is no third case. tests/test_linking.py pins this.

Down-trees are the mirror image.

Within a seed, a call appears in exactly one tree, exactly once. Reading the same event twice is worse than missing it: two findings that are really one finding are two people-hours of someone reconciling them. Three rules keep that true, and none of them throws anything away.

Roots within the window are one episode. A wide range finds every root call in it, but roots no further apart than time_difference_between_calls are one burst of activity, not one tree each. The seed working through five numbers in two minutes is one tree with five root calls; the same seed ringing someone an hour later is a second tree. The same rule folds a pair that spoke repeatedly into one tree rather than one tree per call.

A pair speaking more than once says so. Every call carries where it falls in the run between those two numbers in that tree — ×2/3 in the HTML views, [repeat 2/3] in the text one, repeat_index / repeat_count in JSON and CSV. Counted per tree, so the same pair in a later episode starts again at 1.

A call several parents reached is linked twice, never copied. Expansion materialises each call once; a parent that reaches an existing call records the link on itself (also_links_to) instead of growing a second copy of that call and everything beneath it — and the two branches then belong to one tree. The graph view draws calls as edges between numbers, so it shows this as it is; the indented views place the call once and name the extra link, clickable through to where it sits.

Expansion never walks up to find a call's other parents — only calls already reached from a seed are linked. Hunting parents would have no natural stop.

Seeds stay independent: two seeds are separate analyses, so a call may appear under each. Each seed's forest is meant to read on its own.

The naive approach — one lookup per node, depth-first — is pathological: request count scales with tree size, requests are sequential, and a number on several branches is fetched once per branch. That produces both 429s and long runtimes.

calltrees expands breadth-first, one layer at a time:

The cache is exact rather than range-merged because search_bounds() fixes identical time bounds for every request in a build. A number is either fetched or it is not.

With no documented rate limit, AdaptiveLimiter uses AIMD — creep up while succeeding, halve on every 429 — the same control law TCP uses. It converges just under the real ceiling without being told what it is. And because the limiter is shared per provider across concurrent builds, several jobs hitting the same provider at once cannot collectively exceed its ceiling.

You configure all of this per provider in the registry (chunk_size, start_rate, max_rate, max_workers) — see Connecting your providers. The knobs map onto the RemoteCallSource the service builds for you; if a provider's request body is more than payload-key renaming can express, give it a body_template, and if it differs by even more, write a fetch_batch (contract documented on RemoteCallSource in backend/src/calltrees/engine/sources/remote.py: take a list of numbers, return records for all of them, raise RateLimited on a 429).

When the investigation is about a known group, set phone_numbers_circle and trees are restricted to those numbers. That is also the strongest optimization available: since every number a tree can ever contain is known before the first request, the builder prefetches the whole circle in one batched fetch and every layer is then answered from the exact per-number cache. Round trips stop scaling with tree depth entirely:

The circle also reaches HTTP providers as the {{circle}} body-template placeholder, so a provider that supports filtering by the other party returns only rows that can possibly link.

Every finished build reports how much it cost: the job view (and the UI's performance block, under the result tiles) carries the wall-clock runtime and the source's fetch counters — requests sent to the provider, numbers answered from cache, batched lookups (≈ one per layer), distinct numbers queried, and rows received. Local file sources report the same shape with requests/cache at zero, so a case is comparable across sources. Routed providers report the sum over their endpoints.

For systematic testing — many use cases, against the real providers on your internal network — there is a benchmark harness:

# Run a suite of cases against a deployment, save the measurements (from backend/)
python tools/bench.py run config/bench-cases.example.json -o baseline.json \
    --base-url http://calltrees.internal:8000 --api-key "$KEY" --label "v1.0"

# The same suite against another deployment (or another version)
python tools/bench.py run config/bench-cases.example.json -o current.json \
    --base-url http://localhost:8000 --label "working tree"

# Side by side: runtime and request deltas, and whether the output matches
python tools/bench.py compare baseline.json current.json

tools/bench.py is standard-library-only — copy the one file anywhere Python 3.11+ runs, no installs. A cases file is a list of named POST /api/builds bodies (so anything the API accepts can be a case) plus a default base_url and repeat; each case runs several reps and records median/min/max runtime, the query counters and the result summary. Results files are stamped with the service's own /api/health version, your --label and a timestamp, so they stay self-identifying.

compare prints per-case runtime and request deltas — and checks output equality: a faster run that changes the resulting trees is flagged DIFFERS and the command exits non-zero, because that is a regression, not a win. Older service versions that predate some counters show — and fall back to client-measured time, so comparing against a previous release works.

To benchmark against a previous version of this repo locally:

git worktree add ../calltrees-old <old-rev>
# start it on another port, run the same cases file against both, compare

config/bench-cases.example.json ships ready to run against local-sample and covers the shapes worth measuring: deep chains, hub fan-out (with and without fan_out_limit), down-trees, wide ranges, multi-seed, and a circle/no-circle A/B pair.

Fetch a finished build as ?format= on the result endpoint — tree (default), unicode, json, csv, dot, or html.

• html — interactive, six views over the same forest. One file, no external requests. This is what the web UI shows, and the only view it shows: the other formats are for scripts and pipelines, and were a worse way to read a forest than the page that already has all six views in it.
• csv — one row per call with a parent_call_id, plus repeat_index / repeat_count and also_links_to, for pivoting in a spreadsheet.
• dot — Graphviz, if you want to lay the forest out properly. One cluster per tree; links outside the spanning tree are dashed.