Threat modeling for AI systems

Traditional threat modeling assumes static systems and fixed attacker goals; AI breaks both, which is why agentic methods like MAESTRO exist.

Threat modeling is the discipline you run before attacking or defending - and it’s where traditional security most visibly breaks on AI. You cannot bolt AI threats onto a data-flow diagram and call it done, and your instinct about that is correct.

Up front

• Stop trusting a static boundary. Instructions and data share one channel, so the boundary runs through the model, not around it - which is why STRIDE-style data-flow diagrams mislead.
• The two AI-specific lenses that replace it - the two failure points and the lethal trifecta - are defined below and drive the whole method.

Why STRIDE - and “STRIDE-AI” - fall short

STRIDE, PASTA, LINDDUN, OCTAVE and VAST were built for static, predictable systems: deterministic logic, fixed data flows, clear trust boundaries, and a pre-determined attacker goal. AI breaks every one of those assumptions. The model is probabilistic and can be socially engineered; instructions and data share one channel (II.1 · How LLMs work), so the critical trust boundary runs through the model rather than around it; agents are autonomous and show emergent behavior; multi-agent systems add collusion and sybil dynamics; and the “component” itself learns and shifts. The deeper problem is that these methods assume attacker goals are fixed and data flows are static - which falls apart on a black-box, semantically-driven agent. “STRIDE-AI” merely appends AI threat categories to the same static DFD; it’s a useful checklist but it inherits the deterministic-boundary assumption that is the actual problem. That’s the precise reason it disappoints in practice.

MAESTRO - the current agentic method

The Cloud Security Alliance introduced MAESTRO (Multi-Agent Environment, Security, Threat, Risk & Outcome) in 2025 as a threat-modeling framework purpose-built for agentic AI. It decomposes a system into seven interrelated layers, threat-models each, and then hunts cross-layer paths - the compromises that traditional methods miss because they don’t span the stack.

flowchart TB
  ATK["Attacker / untrusted content"] -->|"enters context (failure point 1)"| L3
  L7["L7 · Agent Ecosystem<br/>impersonation · collusion · sybil"]
  L5["L5 · Evaluation & Observability<br/>blind spots · metric tampering"]
  L3["L3 · Agent Frameworks<br/>prompt injection · tool misuse"]
  L4["L4 · Deployment Infrastructure<br/>serving · container · SSRF"]
  L2["L2 · Data Operations<br/>poisoning · RAG · embedding inversion"]
  L1["L1 · Foundation Models<br/>adversarial · extraction · jailbreak"]
  L6["L6 · Security & Compliance, cross-cutting<br/>identity / NHI · access · regulatory"]
  L7 --> L5 --> L3 --> L4 --> L2 --> L1
  L3 -->|"consequential action exits (failure point 2)"| OUT["External effect"]
  L4 -.->|"cross-layer compromise path"| L1
  L6 -.- L3
  classDef l fill:#0f1a18,stroke:#5bd1c5,color:#bdeee2;
  classDef r fill:#241310,stroke:#ff5b4d,color:#ffc4bb;
  class L1,L2,L3,L4,L5,L7,L6 l; class ATK,OUT r;

The seven layers, with the AI-specific lens overlaid: where untrusted content enters (failure point 1) and where a consequential action exits (failure point 2). Cross-layer is where real compromises live - infrastructure → data → model, then surfaced through the agent.

The seven layers and their characteristic threats:

Layer	Characteristic threats	Cross-refs
L1 Foundation Models	adversarial examples, extraction, jailbreaks	I.4, II.3
L2 Data Operations	poisoning, backdoors, RAG and vector-store exposure, embedding inversion	I.3, II.4, V.3
L3 Agent Frameworks	prompt injection, tool misuse, logic manipulation	II.2, III.4
L4 Deployment Infrastructure	serving exposure, container escape, SSRF, pipelines	V.2, I.5
L5 Evaluation & Observability	monitoring blind spots, metric tampering	VII.3
L6 Security & Compliance (cross-cutting)	identity / NHI (non-human identity), access control, regulatory	IV.6, VIII.5
L7 Agent Ecosystem	impersonation, collusion, sybil, rogue agents over A2A (agent-to-agent)	IV.4, III.4

MAESTRO extends rather than discards STRIDE - it adds the AI-specific threat classes, the multi-agent context, and a lifecycle (continuous) emphasis that the static methods lack.

The AI-specific lenses any method must add

• The two failure points - map first where untrusted content enters the context and where consequential actions exit (II.1 · How LLMs work, VI.1 · Security, safety & who is actually attacking you); the trust boundary runs through the model.
• The lethal trifecta as triage - private data + untrusted content + external comms = exploitable (II.2 · Prompt injection & the LLM attack surface).
• Autonomy & blast radius - what can the agent do, and the worst per action equals its identity/permissions (IV.6 · Agent identity & access (NHI)).
• Persistence - memory/RAG poisoning survives a restart (VII.3 · Detection, IR & forensics for AI).
• Non-determinism - threats are probabilistic; model attack-success-rate, not pass/fail.
• Emergence - multi-agent collusion, cascading failures, delegation escalation.

A practical modern methodology

AI threat-modeling workflow (with tooling)

# 1. CHARACTERIZE - inventory model-backed features, tools, autonomy, data sources.
#    Capture as threat-model-as-code so the model is diffable in CI:
pip install pytm
#    author tm.py: Boundaries=[UntrustedInput, ModelContext, ExternalComms]; Dataflows tagged
python3 tm.py --dfd | dot -Tpng -o dfd.png       # data + control flow diagram
python3 tm.py --report tm-template.md > threat-model.md

# 2-3. DECOMPOSE by MAESTRO L1-L7 and MARK the two failure points
#      (untrusted-content IN, consequential-action OUT) as boundary tags in tm.py

# 4. ENUMERATE + CROSS-LAYER, then map to catalogs. Pull ATLAS as a machine-readable layer:
git clone https://github.com/mitre-atlas/atlas-data   # atlas-navigator-data is deprecated
#      map each dataflow threat to AML.Txxxx + OWASP ASI01-ASI10

# 5-6. ASSESS trifecta / autonomy / persistence, then CONTROL + TEST the top threat.
#      Indirect prompt-injection -> tool-exfil path, authorized target only:
pip install promptfoo || npm i -g promptfoo
promptfoo redteam generate --plugins indirect-prompt-injection,pii,ssrf --purpose "support agent, email tool"
promptfoo redteam run --target <your-agent-endpoint> --max-concurrency 4
promptfoo redteam report        # exfil attack-success-rate, per-plugin ASR

# 7. ITERATE - re-run tm.py + promptfoo in CI on every model/prompt/tool change

Worked mini-model - support agent (RAG + email tool)

Characterize: agent over customer tickets, with an email-send tool and PII access. Failure points: inbound email body enters the context (untrusted-in); email-send is the action-out. Top threats: indirect injection at L3 (ATLAS AML.T0051 LLM Prompt Injection) → exfil via the send tool (ATLAS AML.T0057 LLM Data Leakage; OWASP Agentic ASI02 Tool Misuse); RAG poisoning at L2 (ASI06 Memory & Context Poisoning); permission-stripping leak at L2/L6. Trifecta verdict: PII + untrusted email + external send = data-theft path present. Control + test: approval gate on send, role-aware retrieval; test with an indirect-injection probe (VI.4 · AI red-team playbook Ch3) and measure exfil ASR.

Cross-reference

Applied offensively as recon in VI.4 · AI red-team playbook Ch10; turned into concrete tests in VI.5 · Running the engagement; controls in II.5 · Guardrails - what holds, and how to prove it / IV.6 · Agent identity & access (NHI) / VII.3 · Detection, IR & forensics for AI. A network-monitoring agent worked example using MAESTRO end-to-end is in the literature.

Threat libraries & risk references

A threat model is only as complete as the catalog behind it, and no single taxonomy is sufficient - cross-reference several so coverage isn’t bounded by one author’s lens:

• MITRE ATLAS - adversary tactics/techniques for AI, ATT&CK-style (the operational kill-chain; VIII.1 · Frameworks & standards); agentic techniques added across the 2026 releases - “Publish Poisoned AI Agent Tool” and “Escape to Host” both landed in v5.4.0 (6 Feb 2026), and v2026.06 (30 Jun 2026) added “Steal Web Session Cookie”, “Use Alternate Authentication Material: Web Session Cookie” and “AI Service Web Interface”.
• OWASP Top 10 for LLM Apps (2025) - the priority risk checklist for LLM systems (II.2 · Prompt injection & the LLM attack surface), extended by the OWASP Top 10 for Agentic Applications 2026 (ASI01 Agent Goal Hijack … ASI10 Rogue Agents, released Dec 2025 - the successor to the Feb-2025 “Agentic Threats” draft) and the NHI list for agents.
• BIML Architectural Risk Analysis - the Berryville Institute’s design-level risk catalogs (the BIML-78 for generic ML, and an LLM ARA / “23 black-box risks”, IEEE Computer, Apr 2024). Its premise is useful: many ML risks are design-level and don’t require an adversary to be real.
• MIT AI Risk Repository - a living database of 1,700+ risks classified by cause and domain; good for breadth and governance conversations.
• AI Incident Database - real-world AI failures and harms; grounds a threat model in what has actually gone wrong.
• AVID - the AI Vulnerability Database, cataloguing model/data/infrastructure/governance weaknesses with referenceable IDs.

How to use them

Drive the STRIDE/MAESTRO pass (above) with these as prompts: ATLAS and OWASP for attacker techniques, BIML for design-level risk, the MIT repository for breadth, and the incident database + AVID for “has this actually happened?” evidence. The goal is a model that survives the question “what did you not consider?”