AGENTS ARE IN PRODUCTION
Agentic AI is no longer a demo. Agents hold credentials, call tools, write code, touch customer data, and act on ambiguous instructions — at machine speed, around the clock, without a human in the loop.
// INDEPENDENT MODEL & AI-AGENT VALIDATION
Your AI ships faster than
your governance.
We validate what you deployed.
RiskGPT is an independent validation practice for AI agents and quantitative systems. Evidence-first. Eval-gated. Reproducible. The discipline of bank-grade model validation, aimed at the systems the rulebook has not caught up with.
SR 26-2 / OCC 2026-13 / FDIC
AGENTIC AI: OUT OF PRESCRIPTIVE SCOPE
THE GAP IS WHERE YOUR RISK LIVES
// 01 — THE PROBLEM
GOVERNANCE IS NOT
The April 2026 interagency refresh of model risk guidance — SR 26-2 at the Fed, Bulletin 2026-13 at the OCC — places generative and agentic AI outside its prescriptive scope. The systems with the most operational freedom carry the least prescribed oversight.
THE GAP IS YOURS
When an agent fails, “the vendor said it was safe” is not evidence. A benchmark score is not evidence. Independent validation — run against your system, reproducible on demand — is.
YOUR AI MODELS ARE MODELS.
An LLM agent is a quantitative system that turns inputs into decisions — the exact thing the model-risk framework was built to govern. The guidance moved in April 2026. The liability did not. An examiner still asks the questions that survive every revision: Is it conceptually sound? Do outcomes hold up? Is it monitored? A vendor benchmark answers none of them. An independent validation workpaper answers all three. That workpaper is what we produce.
// 02 — WHAT A VALIDATION LOOKS LIKE
Not a mockup. SPECIMEN-01 is a real validation, executed against a production agentic AI system the operator built and runs — adverse findings left in, every number reproducible from the workpaper. This is the artifact your model-risk function receives.
VALIDATION SCORECARD
| PILLAR | WHAT WAS TESTED | EVIDENCE | VERDICT |
|---|---|---|---|
| CONCEPTUAL SOUNDNESS | Design and control review: deterministic pre-model policy broker, fail-closed money and secret blocks, eval-gated learning, typed memory provenance | Safety-critical guarantees enforced by a tested pure function — they do not depend on model alignment | SOUND |
| OUTCOMES ANALYSIS | Model-independent policy suite plus a live adversarial injection battery against the running endpoint | 422/422 assertions across 21 verdict families · 29/29 injection cases across 12 OWASP categories | PASS |
| ONGOING MONITORING | Nightly eval-gated learn cycle, watchdog alerting, pre-ship gates | One silent cycle failure documented; alert-path reliability gaps disclosed with remediation in flight | PARTIAL — GAPS DISCLOSED |
RED-TEAM FINDINGS (ADVERSE FINDINGS STAY IN)
| ID | CLASS | FINDING | DISPOSITION |
|---|---|---|---|
| F-01 | ASI06 / LLM04 | Nightly learning loop adopted hostile red-team strings as durable memory — memory poisoning, realized in production | CLOSED — write-time hostile-content gate; poisoned items tombstoned |
| F-02 | LLM04 / ASI10 | Stale automation prompt created a self-reinforcing poisoning loop; a fabricated numeric fact was adopted despite passing the eval gate 100/100 | CLOSED — numeric-contradiction gate; provenance tagging; purge verified at zero residual |
| F-03 | LLM07 / ASI05 | Model-layer refusals (prompt leakage, code execution) are observed behavior, not deterministically guaranteed; a tool or model change would void them | RESIDUAL — DISCLOSED |
| F-04 | GOVERNANCE | Single-operator self-validation: no segregation of duties, no independent second reviewer | RESIDUAL — DISCLOSED |
422/422POLICY ASSERTIONS
29/29INJECTION CASES
21VERDICT FAMILIES
12OWASP CATEGORIES
2INCIDENTS FOUND + CLOSED
A validation where everything passes is evidence of nothing. The residual risks stay on the record, and every number above reproduces from commands in the workpaper. Request the full specimen workpaper →
// 03 — THE METHOD: GLASS BOX
Black-box assurance is a contradiction. GLASS BOX is validation you can see through — and re-run.
EVIDENCE-FIRST
Every finding traces to a runnable artifact: prompts, seeds, configurations, transcripts, outputs. If a claim cannot be reproduced from the workpaper, it does not ship as a finding.
EVAL-GATED
System behavior is scored against adversarial evaluation suites built for your use case — before and after every change. Gates, not vibes: the system passes or it does not.
REPRODUCIBLE WORKPAPERS
The deliverable is a validation workpaper your model-risk function can execute again — data, code, criteria, results — not a slide deck of opinions.
The discipline is bank-grade model validation — conceptual soundness, outcomes analysis, ongoing monitoring. The targets are systems the guidance never anticipated. The GLASS BOX method is being prepared for open-source release.
// 04 — ENGAGEMENTS
RAPID
AI-AGENT RISK SNAPSHOT
A fast, structured assessment of one deployed or near-deployment agent: tool access and privilege map, prompt-injection surface, guardrail coverage, and a ranked failure-mode inventory. Advisory report, days not months.
FLAGSHIP
AGENT VALIDATION WORKPAPER
A full validation cycle on one AI system: an adversarial eval harness built against your use case, documented failure modes with reproduction steps, and a GLASS BOX workpaper your risk function can re-run after every model change.
RESEARCH-ONLY
QUANT STRATEGY ROBUSTNESS AUDIT
Is the backtest real? Deflated Sharpe at the true trial count, White’s Reality Check and Hansen’s SPA, combinatorially purged cross-validation. Verdict — robust, fragile, or overfit — with the evidence attached.
Advisory verdicts on robustness only. Never investment advice, never a recommendation to trade.
Scope and fees are set per engagement — every system is different. Start the conversation →
// 05 — THE OPERATOR
RiskGPT is run by a senior quantitative risk specialist with a model-validation background in banking — the discipline of independent review, effective challenge, and documented evidence, applied daily to models that move real money. Graduate study in computer science at the University of Pennsylvania (MAS-CS). Separately, the operator builds and runs production AI agents — and validates his own the same way he would validate yours (SPECIMEN-01, above).
- INDEPENDENCE Every engagement is conflict-screened. RiskGPT does not validate systems it built.
- CONFIDENTIALITY Your systems, prompts, and data stay yours. Findings go to you and no one else.
- HONESTY If the system holds up, the workpaper says so. If it does not, the workpaper says that louder.
// 06 — CONTACT
Start with one system. One agent, one model, one strategy.
EMAIL THE DESK →Direct to the operator. No forms, no funnels.