AIF

Risk-tiered governance for AI-assisted tools at scale
Score risk across 7 dimensions, route to the right review track, and run a uniform
5-model agent pipeline that produces structured findings no human could replicate alone.

AIF Demo

Universities are drowning in AI-built tools. Faculty spin up chatbots, students ship dashboards with FERPA data, departments deploy scripts that talk to third-party APIs with no DPA. Nobody knows what's running, what data it touches, or who's responsible when it breaks.

Traditional IT governance doesn't fit: a faculty member's internal grading helper shouldn't require the same review as a student-facing AI that handles HIPAA data. But ignoring it isn't an option either.

AIF is proportional governance. Low-risk tools register and go. High-risk tools get formal review. Everything in between gets exactly the scrutiny its risk profile demands — scored automatically, analyzed by five independent AI models, and documented for compliance.

The portal is the enforcement layer: builders submit tools via a 21-question intake form, the system scores them on seven weighted dimensions, routes them to a Track (1-4), runs a uniform 5-model agent pipeline, and produces structured reports with auto-generated documentation. Reviewers approve or request changes; admins manage users and audit activity.

Built by the CIO's office at the University of Montana. Designed for portability to other institutions.

Submit tool  -->  Score 7 dimensions  -->  Route to Track  -->  5-model pipeline  -->  Review

1. Intake: Builder answers 21 questions about the tool — what it does, who uses it, what data it touches, how it authenticates, whether users know it's AI
2. Scoring: Seven dimensions scored 0-3, weighted by artifact type (public site, internal app, AI agent, etc.), producing a risk percentage
3. Track routing: Risk percentage maps to a governance track. Nine escalation conditions can force Track 4 regardless of score
4. Agent pipeline: Five independent AI models analyze the codebase using the same prompt. Deterministic tools (Semgrep, ESLint, npm audit) run in parallel. Claude synthesizes everything with dispute resolution
5. Review: Track 1 auto-activates. Track 2 lets builders self-certify. Tracks 3-4 require reviewer approval. All decisions are audit-logged

Each artifact type has a different weight profile. A public website weights accessibility and security heavily; an AI agent weights autonomy and blast radius.

Weighted % = Sum(score x weight) / (3 x Sum(weight)) x 100

Nine conditions force Track 4 regardless of weighted percentage:

1. Regulated data (HIPAA/IRB/export-controlled/tribal) present
2. FERPA data in a public-facing tool: unauthenticated, or authenticated without campus SSO
3. Institutional data in personal accounts
4. AI model without approved DPA
5. Authentication outside campus SSO
6. No version control
7. Students unaware they're interacting with AI
8. Payment card data (PCI DSS)
9. Autonomous decisions without human review

FERPA behind campus SSO on an internet-reachable deployment doesn't escalate to Track 4 — it applies a Track 3 floor instead, guaranteeing IT review without forcing formal-project governance on every SSO-protected campus web app.

AI models are good at reasoning about architecture, intent, and context. They're bad at exhaustive mechanical checking — verifying that every <input> has a <label>, that every dependency is free of known CVEs, that no file contains a SQL injection pattern.

The pipeline exploits both strengths:

• Layer 0 — Deterministic Tooling: SAST scanners, linters, and dependency auditors that mechanically check every element against known rule sets. High precision, exhaustive coverage, zero hallucination. Runs in parallel with model passes — no added latency.
• Layer 1 — Multi-Model AI: Five AI models reason about what tools can't — business logic flaws, architecture concerns, auth flow correctness, and "does this actually make sense?" judgment calls. Claude synthesizes with filesystem access for dispute resolution.

Three confidence tiers in output:

Five different AI models receive the same prompt and independently analyze the entire codebase. No model reviews its own work. Claude only synthesizes — it never runs a pass.

Pass 1 (Codex) has full filesystem access and explores the codebase autonomously. Passes 2-5 receive a pre-bundled codebase (deterministic file selection, 400K char budget) with structured JSON output enforcement. Synthesis uses Claude Code CLI so it can read source files to resolve disputes.

All tracks run all agents. The pipeline is uniform — track determines governance requirements, not analysis depth.

• 21-question form with live scoring sidebar showing dimension scores and track assignment as you answer
• localStorage auto-save (5s debounce) + server draft auto-save (60s) with recovery prompt on return
• Progress bar with field validation hints and focus-to-first-error on submit
• 6 artifact types with distinct weight profiles driving proportional governance

• SSE live streaming of agent progress, pass completions, and CLI output in real time
• Cancel button with AbortController propagation to all child processes (SIGTERM + SIGKILL fallback)
• Retry with dead letter queue (max 2 total attempts before permanent failure)
• Per-model timeouts tuned to observed performance (Codex 15min, MiniMax 8min, etc.)
• Partial results synthesis — synthesis proceeds with whatever passes completed (minimum 1 of 5); such runs are flagged partial_analysis and Track 1 auto-activation is blocked for them
• Pass metrics recorded per-pass: timing, JSON parse status, output size, error category

• Structured findings with severity, file:line evidence, and confidence tier (tool-verified / confirmed / potential)
• 7-dimension scores with weighted percentage and track recommendation
• Tabbed agent results — Code & Security, Accessibility, QA, Documentation
• Findings review — triage findings as open/resolved/won't fix with debounced persistence
• File tree view — browse findings organized by source file
• HECVAT 4.15 XLSX — official EDUCAUSE template pre-filled (~65% of 87 questions answerable from code analysis)
• Document export — User Guide, Admin Guide, Compliance Summary as .docx

• Track-appropriate governance — Track 1 auto-activates, Track 2 self-certifies, Track 3-4 requires reviewer
• Approve / request changes with comment thread
• Track override — reviewers/admins can escalate or de-escalate with documented reason
• Sandbox mode — builder-only visibility until ready for review

• Dashboard — submission stats, active pipelines, review queue depth
• Pipeline analytics — per-model performance (avg/median/max timing, success rate, parse failures), cost tracking, daily/weekly trends
• User management — role assignment, activation/deactivation
• Audit log — every status change, review decision, and admin action with actor, timestamp, and IP
• Data retention — configurable cleanup (pass results 90d, notifications 30d, audit log report-only)

• Dark mode — instant theme switching via [data-theme="dark"] CSS selectors
• WCAG 2.2 AA — 4.5:1 contrast ratios, keyboard navigation, ARIA labels, focus indicators
• In-app + email notifications — pipeline completion, review needed, status changes, per-user preferences
• Pluggable SSO — CAS, header-based (Shibboleth), OIDC/SAML stubs. Set AUTH_PROVIDER env var.
• Institution portability — INSTITUTION_NAME, INSTITUTION_DOMAIN, AUTH_PROVIDER env vars, one config to switch

                        +-----------------+
                        |    Browser      |
                        |  React + Vite   |
                        +--------+--------+
                                 |
                        +--------+--------+
                        |  Nginx Proxy    |
                        |  (reverse proxy)|
                        +--------+--------+
                                 |
                    +------------+------------+
                    |                         |
           +--------+--------+      +--------+--------+
           |   Express API    |      |  Agent Pipeline  |
           |   /auth          |      |                  |
           |   /intake        |      |  Codex CLI       |
           |   /registry      |      |  OpenRouter API  |
           |   /pipeline      |      |  Claude Code CLI |
           |   /review        |      |  Semgrep         |
           |   /reports       |      |  ESLint          |
           |   /admin         |      |  npm audit       |
           |   /analytics     |      |  Snyk            |
           +--------+--------+      +--------+--------+
                    |                         |
                    +------------+------------+
                                 |
                    +------------+------------+
                    |                         |
           +--------+--------+      +--------+--------+
           |   PostgreSQL     |      |   File System    |
           |   16-alpine      |      |   /data/output   |
           |   14 migrations  |      |   /data/codebases|
           +-----------------+      +-----------------+

Browser  --->  Nginx  --->  Express (JWT verify + RBAC)  --->  Route handler
                                      |
                                      +---> PostgreSQL (data)
                                      +---> Pipeline queue (SSE streaming)
                                               |
                                               +---> Agent 1 (5 models + tools + synthesis)
                                               +---> Agent 2 (5 models + tools + synthesis)
                                               +---> Agent 3 (5 models + tools + synthesis)
                                               +---> Agent 4 (3 parallel passes)
                                               |
                                               +---> Results to DB + filesystem

AIF/
├── docker-compose.yml                  # App + PostgreSQL (2 containers)
├── Dockerfile                          # Multi-stage build (frontend + backend + CLI tools)
├── hecvat415.xlsx                      # HECVAT 4.15 official template
├── backend/
│   ├── migrations/                     # 14 SQL migrations
│   ├── .env.example                    # All env vars documented
│   └── src/
│       ├── server.js                   # Express API (helmet, CSRF, rate limiting, routes)
│       ├── index.js                    # CLI entry point (node src/index.js <path> [track])
│       ├── config.js                   # Institution-specific configuration
│       ├── scoring.js                  # 7-dimension scoring engine (weights, tracks, escalations)
│       ├── validation.js               # Zod schemas for all state-changing routes
│       ├── logger.js      

Read the rest on GitHub