A local-first, autonomous agent operating system for AI experimentation and personal automation.

Cognition + Thor — Intelligence with Power

cognithor.ai · Agent Packs · Docs · Blog

19 LLM Providers · 17 Channels · 6-Tier Memory · 4-Channel Search · Knowledge Vault · Security · Apache 2.0

Pre-v1.0 Beta — Cognithor is under active development. APIs may change between releases. Not recommended for production customer-facing deployments. Bug reports and feedback welcome via Issues.

While the test suite is extensive (17,000+ test functions, 89% coverage gate), the project has not been battle-tested in production environments. Expect rough edges, breaking changes between versions, and some German-language strings in system prompts and error messages. See Status & Maturity for details. For non-technical users, wait until version 1.0.0 for stable long-term support.

Weekly Recap: Rank #1 | $1,644 spent vibe-engineering

Vibe-Engineered, not vibe-coded. Cognithor is not a weekend hack held together by AI-generated spaghetti. Every module follows a deliberate architecture (PGE-Trinity, 6-phase gateway init, 3-layer security, TRUST-1..10 operational-trust stack), backed by 17,000+ test functions, structured plans, spec compliance reviews, and code quality gates. The AI writes the code — but a human engineers the system. There's a difference.

Most AI assistants send your data to the cloud. Cognithor runs entirely on your machine — with Ollama or LM Studio, no API keys required. Cloud providers are optional, not mandatory.

It replaces a patchwork of tools with one integrated system: 17 channels, 141 MCP tools across 30 modules, 6-tier memory with 4-channel hybrid search, knowledge vault, voice, browser automation, Computer Use, cross-platform social listening, signed audit receipts (TRUST-1..10), resumable batch workflows (CRWE), TUF-Light-signed pack registry, and more — all wired together from day one. The test suite is extensive (17,000+ test functions, 89% coverage gate). See Status & Maturity for what that does and does not guarantee.

In plain terms: Cognithor is an AI assistant that lives entirely on your computer. You talk to it through your terminal, a web UI, Telegram, Discord, or any of 18 supported channels — and it talks back, remembers what you said last week, and acts on your behalf. It can search the web, write and edit files, run shell commands, control your browser, automate your desktop (clicking, typing, reading windows), manage your calendar, and learn new skills over time. Think of it as a local, private, self-improving Jarvis.

Unlike cloud-based assistants, Cognithor keeps all your data on your machine. Your conversations, memories, documents, and credentials never leave your hardware unless you explicitly configure a cloud LLM provider. It works fully offline with Ollama or LM Studio, and it encrypts everything at rest with SQLCipher (AES-256). If privacy matters to you — and it should — this is the architecture you want.

What makes it different from other local AI tools is that Cognithor is not just a chatbot. It is an agent operating system: it plans multi-step tasks, evaluates its own results, learns from mistakes, and improves autonomously. It can control your desktop through Computer Use (screenshots, clicks, keystrokes, window automation), compete in ARC-AGI-3 reasoning benchmarks, and manage a marketplace of community-contributed skills. It is built to grow with you.

Cognithor is Beta / Experimental software. It is under rapid, active development.

What the test suite covers: Unit tests, integration tests, property-based tests (Hypothesis), audit-completeness burn-ins (nightly CI), real-life scenario tests, and live Ollama tests for all modules. The 17,000+ test functions verify code correctness in controlled environments.

What the test suite does NOT cover: Real-world deployment scenarios, network edge cases, long-running stability, multi-user load, hardware-specific voice/GPU issues, or actual LLM response quality.

Important notes for users:

• This project is developed by a solo developer with AI assistance. Code is human-reviewed, but the pace is fast.
• Breaking changes may occur between minor versions. Pin your version if stability matters.
• The default language is German, switchable to English via the Flutter Command Center or config.yaml. See Language & Internationalization.
• For production use, thorough testing in your specific environment is strongly recommended.
• Bug reports and contributions are welcome — see Issues.

Cognithor is a fully local, Ollama/LM Studio-powered, autonomous agent operating system that acts as your personal AI assistant. All data stays on your machine — no cloud, no mandatory API keys, full GDPR compliance. It supports tasks ranging from research, project management, and knowledge organization to file management and automated workflows. Optional cloud LLM providers (OpenAI, Anthropic, Gemini, and 12 more) can be enabled with a single API key. Users can add custom skills and rules to tailor the agent to their needs.

• Why Cognithor?
• Status & Maturity
• Highlights
• Architecture
• LLM Providers
• Channels
• Quick Start (under 5 minutes)
• Configuration
• Security
• Operational Trust (TRUST-1..10)
• MCP Tools
• Tests
• Deployment
• Language & Internationalization
• License
• What's New

• 19 LLM Providers — Ollama (local), LM Studio (local), vLLM (local), llama-cpp-python (local), OpenAI, Anthropic, Google Gemini, Groq, DeepSeek, Mistral, Together AI, OpenRouter, xAI (Grok), Cerebras, GitHub Models, AWS Bedrock, Hugging Face, Moonshot/Kimi, Claude Code — plus any custom OpenAI-compatible endpoint
• 17 Communication Channels — CLI, Web UI, REST API, Telegram, Discord, Slack, WhatsApp, Signal, iMessage, Microsoft Teams, Matrix, Google Chat, Mattermost, Feishu/Lark, IRC, Twitch, Voice (STT/TTS)
• 6-Tier Cognitive Memory — Core identity, episodic logs, semantic knowledge graph, procedural skills, working memory, tactical memory
• 4-Channel Hybrid Search — BM25 full-text + vector embeddings + knowledge graph traversal + hierarchical document reasoning with score fusion
• PGE Architecture — Planner (LLM) -> Gatekeeper (deterministic policy engine) -> Executor (sandboxed)
• Crew-Layer (v0.93.0) — high-level Multi-Agent API (from cognithor import Crew, CrewAgent, CrewTask) on top of PGE-Trinity. Async kickoff with parallel fan-out, idempotent replay, guardrails, 5 templates (research, customer-support, data-analyst, content, versicherungs-vergleich).
• 8-page Quickstart — docs/quickstart/ — from pip install to your first Crew in under 10 minutes, bilingual (DE primary + EN). Includes runnable examples for first-crew, first-tool, first-skill, guardrails, and PKV report.
• Security — Platform-adaptive sandbox (bubblewrap on Linux, subprocess+timeout fallback), AST-based Python/Shell code analysis (Python ast.NodeVisitor + bashlex parser), SHA-256 audit chain, credential vault, runtime token encryption (Fernet AES-256), Gatekeeper policy engine with GREEN/YELLOW/ORANGE/RED risk classification (not independently audited — see Status & Maturity)
• Knowledge Vault — Obsidian-compatible Markdown vault with YAML frontmatter, tags, [[backlinks]], full-text search
• Document Analysis — LLM-powered structured analysis of PDF/DOCX/HTML (summary, risks, action items, decisions)
• Video Input — Attach local videos (.mp4 / .webm / .mov / .mkv / .avi) or paste direct video URLs; Qwen3.6-27B (or any video-capable VLM) analyzes them end-to-end via vLLM's native video_url content type. Adaptive frame sampling (fps=3 for short clips, num_frames=32 for long) via ffprobe. Single video per turn, served from a 127.0.0.1-only HTTP file server, 24h auto-cleanup. Requires vLLM backend — Windows installer bundles LGPL ffmpeg. See docs/vllm-user-guide.md.
• VLM Router — Three-tier profile system (fast / balanced / premium) for video and image understanding. Heuristic classifier inspects the user's prompt and clip metadata, picks the right VLM (Qwen3-VL-8B-Instruct → Qwen3-VL-8B-Thinking → Qwen3.6-27B-NVFP4), exposes the routing decision via TRUST-2 (rule_id + matched_pattern) so a receipt reviewer can replay why a particular model was chosen. Override via with router.quality_scope("premium"): or config.vllm.quality_default. ContextVar-isolated for async safety. See src/cognithor/core/vlm_router.py.
• Video Composition & Rendering — Cognithor doesn't only read videos, it can make them. The cognithor.video package ships a thin RendererABC abstraction with HyperFrames (Apache-2.0) as the default backend; future renderers (Remotion, cloud) can be swapped without touching the MCP-tool layer. Five MCP tools cover the workflow: video_compose (GREEN — pure-function, builds a self-contained HTML composition from a structured spec, no subprocess, no FS write), video_compose_explainer (16:9 title-card + body + CTA preset), video_compose_social_cut (vertical 9:16 hook + fast-cut beats + outro), video_caption_overlay (parallel caption track), and video_render (ORANGE — renders composition HTML to MP4 / MOV / WebM under ~/.cognithor/render/<run_id>/). Raw user-supplied HTML is RED at the Gatekeeper — only structured specs reach the renderer. Render output is linked to the agent run via the same run_id the streaming EventEmitter and TRUST-1 receipt API use, so every frame is provenance-attributable end-to-end. Composer prompts + reusable templates live in cognithor.video.skills (HF-5). VLM video-read can drive composition directly — see VLM-4 smoke at tests/test_video/.
• Model Context Protocol (MCP) — 141 tools across 30 modules (filesystem, shell, memory, web, browser, media, vault, synthesis, code, skills, documents, social, kanban, identity, evolution, computer-use, sevDesk, A2A and more) — counts auto-generated into docs/integrations/catalog.json on every release
• Computer Use — Complete desktop automation: screenshots, clicking, typing, scrolling, dragging, Windows UI Automation via pywinauto for exact element coordinates, 3-layer security, adaptive wait
• ARC-AGI-3 Benchmark Agent — Compete in ARC Prize 2026: 13/25 games solved (24 levels), 4 solver families (ClusterClick, SequenceClick+SimA*, KeyboardDFS, SmartExplorer), persistent game profiles, multimodal vision (qwen3-vl)
• Distributed Locking — Redis-backed (with file-based fallback) locks for multi-instance deployments
• Durable Message Queue — SQLite-backed persistent queue with priorities, DLQ, and automatic retry
• Prometheus Metrics — /metrics endpoint with Grafana dashboard for production observability
• Skill Marketplace — SQLite-persisted skill marketplace with ratings, search, and REST API
• Community Skill Marketplace — GitHub-hosted registry with publisher verification (4 trust levels), 5-check validation pipeline, ToolEnforcer runtime sandboxing, async install/search/report
• Telegram Webhook — Polling + webhook mode with sub-100ms latency
• Auto-Dependency Loading — Missing optional packages detected and installed at startup
• Agent-to-Agent Protocol (A2A) — Linux Foundation RC v1.0 with full JSON-RPC 2.0 server/client, Planner-level delegation via MCP tools, auto-discovery, SSE streaming
• Integrated Chat — Full chat page in the Flutter Command Center with WebSocket streaming, tool indicators, canvas panel, approval banners, and voice mode
• Flutter Command Center — Cross-platform UI (Flutter 3.41, Web/Desktop/Mobile) with Sci-Fi aesthetic, GEPA pipeline visualization, Robot Office pathfinding, 20+ editable config pages, Observe panel, Knowledge Graph, Voice Mode, Learning Dashboard, Light/Dark theme, 4-language i18n
• Active Learning & Curiosity — CuriosityEngine detects knowledge gaps, KnowledgeConfidenceManager with time decay and feedback, ActiveLearner processes files in background during idle time
• Human Feel — Personality Engine (warmth, humor, greetings), sentiment detection (frustrated/urgent/confused/positive), user preference learning, real-time status callbacks, user-friendly German error messages
• Auto-Detect Channels — Channels activate automatically when tokens are present in .env — no manual config flags needed
• Knowledge Synthesis — Meta-analysis across Memory + Vault + Web with LLM fusion: knowledge_synthesize (full synthesis with confidence ratings), knowledge_contradictions (fact-checking), knowledge_timeline (causal chains), knowledge_gaps (completeness score + research suggestions)
• Adaptive Context Pipeline — Automatic context enrichment before every Planner call: BM25 memory search + vault full-text search + recent episodes, injected into WorkingMemory in <50ms
• Security Hardening — Runtime token encryption (Fernet AES-256) across all channels, TLS support for webhook servers, file-size limits on all upload/processing paths, persistent session mappings in SQLite
• One-Click Start — Double-click start_cognithor.bat -> browser opens -> click Power On -> done
• Enhanced Web Research — 4-provider search fallback (SearXNG -> Brave -> Google CSE -> DuckDuckGo), Jina AI Reader for JS-heavy sites, domain filtering, source cross-checking
• Procedural Learning — Reflector auto-synthesizes reusable skills from successful sessions
• DAG Workflow Engine — Directed acyclic graph execution with parallel branches, conditional edges, cycle detection, automatic retry. Now wired into the Executor for parallel tool execution
• Distributed Workers — Capability-based job routing, health monitoring, failover, dead-letter queue
• Multi-Agent Collaboration — Debate, voting, and pipeline patterns for agent teams
• i18n Language Packs — JSON-based internationalization with SHA-256 integrity verification, German and English included, extensible to any language
• Tool Sandbox Hardening — Per-tool resource limits, network guards, escape detection (8 attack categories)
• GDPR Compliance Toolkit — Data processing logs (Art. 30), retention enforcement, right-to-erasure (Art. 17), audit export
• ARC-AGI-3 Benchmark — Compete in the ARC Prize 2026 ($2M+ prize pool) via the arc/ module: hybrid agent (algorithmic + LLM + CNN), 3 MCP tools (arc_play, arc_status, arc_replay), CLI mode, swarm mode. pip install cognithor[arc]
• Multi-Agent Benchmark Scaffold — Reproducible benchmark harness in the top-level cognithor_bench/ package: own pyproject.toml, cognithor-bench CLI, Cognithor + AutoGen adapters
• Deterministic Replay — Record and replay agent executions with what-if analysis and diff comparison
• Agent SDK — Decorator-based agent registration (@agent, @tool, @hook), project scaffolding
• Plugin Remote Registry — Remote manifests with SHA-256 checksums, dependency resolution, install/update/rollback
• Cross-Platform Social Listening — Reddit + Hacker News (Firebase/Algolia) + Discord (REST API v10) scanning with LLM lead scoring. 2 unified MCP tools (social_scan, social_leads), configurable per-platform intervals, auto-post opt-in
• Hierarchical Document Reasoning — Tree-based retrieval (4th search channel): 5 parsers (Markdown, PDF, DOCX, HTML, plaintext) build heading trees, LLM-navigated section selection, no embedding required
• CAG Layer (Cache-Augmented Generation) — Deterministic prefix generation for LLM KV-cache reuse. Prefix builder (Ollama-compatible) + native llama.cpp builder, content normalization, hit-rate metrics
• CLI Config TUI — Interactive terminal config editor (rich + prompt_toolkit): model discovery from live providers, section navigation, validation, save/discard
• AST-Based Security Guards — Python ast.NodeVisitor + bashlex shell parser replacing regex-based code analysis. Detects imports, subprocess calls, eval, network access, file operations at the syntax tree level
• _safe_call() Pattern — Unified error handling replacing silent except Exception: pass. Failure registry with per-function tracking, async variant, circuit-breaker integration
• uv Installer Support — Automatic uv detection for 10x faster installs, transparent pip fallback
• Observer Audit Layer — Every response audited against 4 quality dimensions (hallucination, sycophancy, laziness, tool-ignorance) with differentiated retry strategies. Runs locally with qwen3:32b.
• Operational Trust (TRUST-1..10) — Signed run-receipts (cognithor receipt show / verify / list / export-all / diff), structured Gatekeeper decision-explanations (rule_id + rule_source + matched_pattern), 15-value FailureMode taxonomy, pack rollback, six append-only ledgers (Provenance, Permission-Scopes, Tool-Fingerprints, Cloud-Escalation, Cost in micro-USD, Migration). REST: GET /api/crew/trace/{trace_id}/receipt. See Operational Trust and docs/operational_trust.md
• Resilient Workflow Engine (CRWE, v0.99.0) — cognithor task <manifest.json> runs declarative batch workflows with JSONL streaming, per-task flush() + os.fsync(), atomic modulo-N .checkpoint.json, .checkpoint.lock (POSIX fcntl.flock / Windows msvcrt.locking), SIGINT/SIGTERM emergency-checkpoint between tasks, manifest-tamper detection on --resume, and audit-chain integration (workflow_resumed / system_checkpoint_created events). Crash-recovery uses results.jsonl line count as source-of-truth.
• Hash-Chained Audit Log — Every audit event is HMAC-SHA-256 chained to the previous via prev_hash over canonical NFC-normalized JSON. The Reflector ships through a dedicated AuditCategory.REFLECTION channel (Compliance-Spring v0.98.0) with nine event types covering causal sequences, weight snapshots, episodic appends, semantic facts, and procedure auto-creation — verifiable end-to-end with cognithor audit verify. Property-based Hypothesis tests + a nightly burn-in CI keep the chain intact.
• TUF-Light Pack Registry Signing — Community pack registry uses Ed25519 + SHA-256 (offline Root key, online Targets key) per the TUF-Light spec. Operator runbook: docs/runbooks/registry_key_rotation.md. Marketplace dormant by default until owner mints Root keypair.
• Integrations Catalog — Auto-generated from src/cognithor/mcp/ — see docs/integrations/catalog.json. DACH-specific: sevDesk REST connector (accounting).
• Competitive Analysis — docs/competitive-analysis/ — Cognithor vs AutoGen / MAF / LangGraph / CrewAI.
• 17,000+ test functions · 89% coverage gate · 0 lint errors · 0 CodeQL alerts · mypy --strict clean across the new TRUST + CRWE surface

flowchart TB

%% ===== ENTRY POINTS =====
subgraph ENTRY["Entry Points"]
    UI["Flutter UI / Web UI
(Mobile app available)"]
    API["REST API"]
    CHANNELS["Messaging Channels
(17 channels — chat, voice, REST)"]
end

%% ===== CORE LOOP =====
subgraph CORE["Autonomous Agent Loop"]
    GATEWAY["Gateway
Session + Routing + State"]

    CONTEXT["Context Pipeline
Memory + Vault + Episodes"]

    subgraph PGE["PGE Core Engine"]
        direction LR
        PLANNER["Planner 
Task Understanding
+ Planning"]

        GATEKEEPER["Gatekeeper
Policy + Risk Validation"]

        EXECUTOR["Executer
Sandbox + Action"]
    end

    MEMORY["Cognitive Memory
6-Tier Learning System"]
end

%% ===== EXECUTION =====
subgraph EXEC["Execution Stack"]
    WORKFLOW["DAG Engine
Parallel + Retry Logic"]

    TOOLS["MCP Tools
(141 tools / 30 modules)"]

    LLM["LLM Backends
Local + Cloud Models
(22 API endpoints)"]
end

%% ===== FLOW =====

%% Entry into loop
UI --> GATEWAY
API --> GATEWAY
CHANNELS --> GATEWAY

%% Core loop
GATEWAY --> CONTEXT
CONTEXT --> PLANNER
PLANNER --> GATEKEEPER --> EXECUTOR

%% Execution path
EXECUTOR --> WORKFLOW --> TOOLS --> LLM --> MEMORY

%% Feedback loop
MEMORY --> CONTEXT

%% Visibility (important fix)
EXECUTOR --> UI

Every user request passes through three stages:

1. Planner — LLM-based understanding and planning. Analyzes the request, searches memory for relevant context, creates structured action plans with tool calls. Supports re-planning on failures.
2. Gatekeeper — Deterministic policy engine. Validates every planned tool call against security rules (risk levels GREEN/YELLOW/ORANGE/RED, sandbox policy, parameter validation). No LLM, no hallucinations, no exceptions.
3. Executor — Executes approved actions via DAG-based parallel scheduling (independent actions run concurrently in waves). Shell commands run isolated (Process -> Namespace -> Container), file access restricted to allowed paths.

Memory search uses a 4-channel hybrid approach: BM25 (full-text search with FTS5, optimized for German compound words) + Vector Search (Ollama embeddings, cosine similarity) + Graph Traversal (entity relations) + Hierarchical Document Reasoning (tree-based section navigation for long documents). Score fusion with configurable weights and recency decay.

In addition to the 6-tier memory, Cognithor includes an Obsidian-compatible Knowledge Vault (~/.cognithor/vault/) for persistent, human-readable notes:

• Folder structure: recherchen/, meetings/, wissen/, projekte/, daily/
• Obsidian format: YAML frontmatter (title, tags, sources, dates), [[backlinks]]
• 6 tools: vault_save, vault_search, vault_list, vault_read, vault_update, vault_link
• Open the vault folder directly in Obsidian for graph visualization

After completed sessions, the Reflector evaluates results, extracts facts for semantic memory, and identifies repeatable patterns as procedure candidates. Learned procedures are automatically suggested for future similar requests.