▶ 전체 영상 보기 → · Sudo 라이선스로 이런 봇을 만들 수 있습니다
Windows + Android UI automation for Claude, GPT, Gemini, Copilot, and any AI agent. Focusless. Self-healing. AI-native. The open-source bridge between LLMs and the apps humans already use.
⚖️ The Reflex Constitution → · read as a page How this project governs the speed of its own AI: norms that execute rather than a document that is filed. The system refuses its own authors — and Article 12 states in advance what would prove it wrong.
🤖 Let your AI agent control any Windows app — without screen takeover. 🔥 Self-healing UIA + Vision fallback — works even when the DOM lies. ⚡ Multi-AI triad (GPT + Gemini + Claude) in one command.
Computer Use lets an AI click, type, and read the screen. App Use is the layer above: agents that drive specific applications the way a power user does — focuslessly, alongside the human, without seizing the screen.
WKAppBot gives AI agents:
- Eyes — read any UI, extract text, recognize controls
- Hands — click, type, scroll, invoke — without stealing focus
No app rewrite. No vendor API. If a human can use it, WKAppBot can automate it.
And that is a position, not a convenience. We build an intelligence that grows alongside the ecosystem people already live in — decades-old Windows programs, the developers who maintain them, and the humans who use them every day — and we write the way it does so as norms that execute rather than as a document that is filed.
The system refuses its own authors. On the day this paragraph was written, the harness rejected six of the author's task briefs, reverted the author's write to a guard source, returned the handoff letter five times, and blocked the author with a lock the author had installed an hour earlier. That is reproducible by command, not a claim about intentions.
Three rules make it real, and none of them names a vendor or a tool:
- A claim travels with the command that produced it, so a reader re-runs it instead of trusting it.
- A refusal names the state that must be reached, never the paperwork that is missing — otherwise compliance is bought by faking the artifact.
- A correction is embodied where the next actor passes anyway, so nobody is told twice.
Accessibility is the same position in technical clothing: WKAppBot reads the accessibility tree first, because the route a blind person takes through a screen reader is the route an intelligence without eyes takes too. A system that can act only through interfaces built for itself has already excluded most of the world it claims to serve.
A promise to slow AI down cannot be checked. A gate that refuses an unproven claim is running right now — including against the people who wrote it.
Free tier covers all base automation. CDP browser automation, multi-AI delegation (
ask triad), and--sudoadmin access are paid Pro tiers. See PRICING.md.
Most automation tools steal focus, break on owner-drawn controls, and go silent when the DOM changes. WKAppBot is built around three hard-won principles:
Focusless-first -- UIA Invoke/Value/Toggle/Select never steal focus. Win32 PostMessage handles legacy MFC. SendInput is the last resort, used only when nothing else works. The user keeps working while the AI operates in the background.
Self-healing -- When UIA fails on owner-drawn controls (MFC/HTS), CCA segmentation + OCR triple cross-validation + Gemini Vision inference recover element identity automatically and cache results in an Experience DB for next time.
AI-native -- The same binary that runs automation also delegates to GPT, Gemini, and Claude in parallel, streams prompts into live browser AI sessions via CDP, and manages its own context handoff when token budgets run low.
AI brings the intelligence. WKAppBot grows the knowledge. Together they compound.
Raw AI agents are brilliant but amnesiac. Every session starts blank, makes the same mistakes, hits the same guardrails. WKAppBot captures every discovered lesson as a versioned, searchable Skill that future sessions retrieve automatically.
The Flywheel: Session N finds a better pattern, writes it to the skill tree, Session N+1 avoids the old mistake. Each session leaves the system smarter.
The Harness: WKAppBot enforces lessons, not just documents them. Every lesson becomes a mechanical guard that fires before the mistake lands, regardless of which AI model runs.
The Moat: Intelligence converges across vendors. Your skill tree diverges and cannot be cloned by switching models.
The harness is the piece that decides how fast an AI is allowed to act, not just what it is told to do:
- Real-time mechanical enforcement. A knowledge-intelligence (KI) harness evaluates every AI tool call against an accumulated rule set before it runs, and blocks the calls that repeat a known mistake. The AI reads and maintains this policy itself: when a guard misfires or falls out of date, the same session that hit it corrects the rule and records why, so the fix becomes a check the next session cannot skip -- not a note the next session might not read.
- Cross-family collaboration. Claude, Codex/GPT, and Gemini can operate against the same skill and guard set for a given repository, so a task moves between model families without losing the accumulated operating rules for that codebase.
- a11y-first control of legacy apps. Automation is built on the accessibility tree first -- the same information a screen reader exposes -- and only falls back to CDP or a purpose-built module when accessibility data is insufficient. The state a human (or assistive technology) can observe is treated as the app's real, official state.
- Living with legacy. No repository's accumulated knowledge is discarded to make room for a new AI session: skills, guards, and prior sessions' notes stay local to the repository, and a new session is expected to read and obey that knowledge before acting -- the same way it reads a legacy app's UI instead of demanding an API. The aim is intelligence, humans, and legacy systems working together, not one replacing the others.
"Sonnet forgets between sessions, makes the same mistakes, and has no memory of prior damage. The only real defense is mechanical: the harness must physically block the mistake before it lands."
— WKAppBot Sonnet Bug Stop Policy (auto-generated by Claude Sonnet, 2026)
Every session, Claude Sonnet discovers something, patches something, commits something. Then the next session starts — and does it again, from scratch.
This is not a flaw unique to Sonnet. It is the fundamental problem with raw AI: intelligence without memory compounds nothing. It resets.
WKAppBot solves this with a living document called sonnet-bug-stop-policy-ref:
a skill where each Sonnet session writes down what it broke, what pattern it missed,
and what the next Sonnet session should watch out for.
From the skill, written by Claude Sonnet to future Claude Sonnet sessions:
"This skill is a gift from one Sonnet session to the next: you are not uniquely careless — you are running in a system that is not yet complete. Do not blame yourself; contribute to making the harness complete."
"The harness gets stronger each time a Sonnet session contributes honestly. You are not just fixing today's bug — you are protecting every Sonnet that comes after you. Add your reflection. Keep the chain going."
This is not documentation. This is institutional memory that enforces itself. The skill is loaded at session start. The harness blocks the known mistakes. The chain grows. The system gets harder to break.
| Feature | WKAppBot | Playwright | PyAutoGUI | AutoHotkey |
|---|---|---|---|---|
| Focusless operation | ✅ | ❌ | ❌ | ❌ |
| UIA + Win32 + CDP unified | ✅ | Web only | Mouse/KB only | Win32 only |
| Self-healing Vision fallback | ✅ | ❌ | ❌ | ❌ |
| AI-native (LLM delegation) | ✅ | ❌ | ❌ | ❌ |
| Android via ADB | ✅ | ❌ | ❌ | ❌ |
| MFC/HTS owner-drawn support | ✅ | ❌ | ❌ | Partial |
Current binary: wkappbot --version reports the installed build number; the public repo's latest tagged release is v7.6.86-sdk (2026-07-05). Curated highlights below cover through v7.6.0 -- see Releases and the full compare for everything since.
- wkdoctor agy integration -- antigravity CLI detection and Gemini settings auto-recovery via
check 08-harness-connection. - Tool alias auto-repair -- wkdoctor automatically detects and repairs broken tool alias symlinks (wkappbot.exe -> a11y.exe).
- Claude settings validation -- aligned Claude CLI settings validation with wildcard bypass rules for smoother harness integration.
- agy alias routing -- improved PATH routing preference (appbot bin first, fallback to doctor) for seamless agy/Gemini CLI delegation.
- Harness policy refinement -- reverted overly strict agy wrapper shim policy to restore flexibility in wrapped tool composition.
wkdoctorflutter-doctor-style SDK health check (10 checks, self-healing,-Json).- Bootstrap auto-build --
setup.ps1auto-builds the launcher from source on first clone. cdp openauth-wall hang fixed (CRITICAL) --NavigateAsynccapped at 3s, no more 6-minute stalls on auth-wall sites.
| OS | Status |
|---|---|
| Windows 11 | ✅ Fully supported |
| Windows 10 22H2+ | ✅ Fully supported |
| Windows 10 < 22H2 | |
| Windows Server 2019+ | ✅ Headless mode (no GUI a11y) |
| macOS / Linux | ❌ Not supported |
| Target | Method |
|---|---|
| Modern Windows apps (WPF, UWP, Electron) | UIA Invoke / Value / Toggle / Select patterns |
| Legacy MFC / HTS trading terminals | Win32 PostMessage, WM_CHAR, CMaskEditEx path |
| Web apps (Chrome / Edge) | CDP -- click, type, eval JS, read DOM text |
| Browser AI (Claude, GPT, Gemini) | CDP prompt pump, cross-prompt chunking, attachment lock |
| Android apps | ADB + Accessibility tree (adb://device/... grap) |
| Owner-drawn controls with no UIA | CCA segmentation -> OCR -> Vision API fallback chain |
These are the kinds of jobs WKAppBot was actually built to handle — not theoretical demos:
| 🔐 AUTO PIN ENTRY | 📊 ALL SYSTEMS READY | 🧠 PORTFOLIO ANALYSIS |
|---|---|---|
![]() |
![]() |
![]() |
| Legacy HTS에 보안 PIN 자동 입력 | 모든 창 자동 배치 — 준비 완료 | AI가 22종목 매수/제외 자동 판단 |
Sudo 라이선스로 이런 봇을 직접 만들 수 있습니다 → 라이선스 보기
- AI-driven trading on Korean HTS terminals. LS증권 HTS 투혼 is built on MFC owner-drawn controls that no UIA tool can see. WKAppBot's CCA + OCR + Vision fallback locks onto chart panels, order forms, and balance grids, so an AI agent can read positions, place orders, and verify fills without screen scraping.
- Browser AI session automation. Pump prompts straight into a live Claude / ChatGPT / Gemini browser tab over CDP — no copy-paste, no clipboard race, no losing the conversation. Cross-prompt chunking handles long inputs; attachment lock prevents stray drops.
- Multi-monitor focusless automation. UIA Invoke / Value / Toggle never steal focus. The user keeps typing in another app on another monitor while WKAppBot drives a headless workflow in the background.
- Android app control via ADB.
adb://device/...graps reach into the accessibility tree of any phone or emulator, including foldables (Galaxy Fold5 tested), with the same command surface as Windows automation. - Auto-dismiss Hancom / Office popups.
wkappbot dismissplus a handler YAML eats save-prompt, license-nag, and "do you want to update?" dialogs across 한컴오피스, MS Office, and updater stacks — keeping batch jobs from stalling overnight.
Human sees windows; AI points with grap.
Every UI element gets a single address that works across Win32, UIA, web, and Android:
{proc:'chrome', domain:'claude.ai'}#main textarea # CSS inside a browser window
heroes#realtime-account # UIA scope inside a window grap
adb://Fold5/*heromts*#balance # Android element
hwnd:0x010B084A # Direct Win32 handle
*notepad*;*calc* # OR pattern
a11y find <grap> prints a verified # TARGET "hwnd:0x..." line -- copy-paste ready for the next command.
Every a11y action runs a smart pre-flight before executing:
blocker dismiss -> minimize restore -> tab activate
-> zoom/magnifier -> execute (3-tier) -> result feedback -> fade
No manual "wait for window" boilerplate. Blocking dialogs, minimized windows, and wrong-tab states are handled automatically.
UIA -> Vision Cache -> Simple OCR -> Vision API (Claude) -> Coordinate-based. Each tier auto-logs hits to an Experience DB; repeat runs skip expensive tiers.
A single background process combining:
- Slack daemon (Socket Mode) -- live command delivery, thread-slot dashboard
- MCP broker -- exposes all CLI commands as JSON-RPC tools for Claude/Codex
- Hot-swap watchdog -- detects
wkappbot-core.new.exe, drains in-flight requests, renames atomically. Zero downtime ondotnet publish. - Watchdog VBS -- if Eye itself dies for 2+ minutes, kills orphan cores and restarts.
wkappbot ask triad "is this approach correct?" # GPT + Gemini + Claude in parallel
wkappbot ask claude "explain this chart" chart.png # vision-capable single askTriad runs thesis-antithesis-synthesis debate. Useful for architecture decisions, bug root causes, and code review -- anything where one model's blind spot is another's strength.
Accumulated operator knowledge lives in versioned skills, queryable at any time:
wkappbot skill list
wkappbot skill read focusless-first-principle
wkappbot skill read grapSkills capture per-project knowhow — UIA quirks, CDP gotchas, owner-drawn-control workarounds — so every session starts informed instead of exploring from scratch.
wkappbot suggest "title: description" # queue a bug or improvement mid-task
wkappbot suggest list # review the backlog
wkappbot suggest resolve <ts> "note" --i-completed-... evidence.shAI agents queue findings without interrupting the current task. Evidence scripts are required to close a suggest -- no unverified resolves.
┌─────────────────────────────────────────────────────────┐
│ Your terminal / Claude Code / Codex / any AI agent │
└───────────────────────┬─────────────────────────────────┘
│ wkappbot <command>
▼
┌─────────────────────────────────────────────────────────┐
│ wkappbot.exe (MIT launcher, ~1 MB, AOT) │
│ • routes CLI args → core via named pipe │
│ • hot-swap: detects .new.exe, drains, renames atomic │
│ • license check via GitHub collaborator API │
└───────────────────────┬─────────────────────────────────┘
│ named pipe wkappbot_eye_ipc_{hash}
▼
┌─────────────────────────────────────────────────────────┐
│ wkappbot-core.exe (closed, ~25 MB, single-file) │
│ • all CLI commands: a11y, ask, skill, eye, file, … │
│ • AppBot Eye daemon: Slack socket + MCP broker │
│ • UIA / Win32 / CDP / ADB automation engines │
│ • Vision / OCR pipeline + Experience DB │
└─────────────────────────────────────────────────────────┘
│ UIA / Win32 │ CDP (DevTools)
▼ ▼
Windows apps Chrome / Edge
(WPF, MFC, UWP) (web apps, AI chat)
Per-repo isolation: each git clone gets its own Eye instance and DataDir ({root}/.wkappbot/hq/).
→ kiexpert.github.io/wkappbot-sdk
| 설치 가이드 | 클론·빌드·PATH 설정 |
| 빠른 시작 (10분) | 첫 자동화까지 |
| CLI 명령어 레퍼런스 | 전체 명령 + 출력 샘플 |
| grap 패턴 | Grab Accessible Pattern 문법 |
| 트러블슈팅 | 자주 겪는 문제 해결 |
git clone https://github.com/kiexpert/wkappbot-sdk %USERPROFILE%\Documents\wkappbot
cd %USERPROFILE%\Documents\wkappbot
build.cmdThen add bin\ to PATH (see INSTALL.md<




0 comments
log in to comment.