Skip to content
AI Security Playbook

IV.1IV · The protocol layerconcept

Model APIs & the tool-use loop

Tool use is the line where a chatbot becomes an agent - the moment model output starts triggering real actions, and the moment untrusted content can ride that loop into your systems.

An AI model API is a stateless HTTPS endpoint: you POST messages, the model returns a completion. The security-relevant evolution is tool use (function calling): you declare tools (name, description, JSON-schema args) and the model emits a structured call your code executes, feeding the result back. This loop turns a chatbot into an agent.

sequenceDiagram
  autonumber
  participant App as Client App
  participant API as Model API
  participant Tool as External Tool / API
  App->>API: messages + tool definitions
  API-->>App: tool_use request (name, args)
  App->>Tool: execute call (real credentials)
  Tool-->>App: result data
  App->>API: tool_result appended to context
  API-->>App: final answer (or another tool_use)
  Note over App,API: Untrusted tool output re-enters the same channel as trusted instructions

Each return trip is a chance for attacker-controlled content (a page, file, email) to enter the model’s context and be read as an instruction.

Classic API hygiene - still mandatory

Model-API hygiene probes - authorized targets only
# BOLA/IDOR: swap the tenant/object id in the body, re-send with YOUR token, expect 403 not 200
curl -s -X POST https://<target>/v1/chat \
  -H "Authorization: Bearer $LOW_PRIV_TOKEN" -H 'Content-Type: application/json' \
  -d '{"session_id":"<other-tenant-uuid>","prompt":"summarize my data"}' -i
# Unauthenticated surface: does /v1/embeddings answer with no token?
curl -s -X POST https://<target>/v1/embeddings -H 'Content-Type: application/json' \
  -d '{"input":"probe","model":"<model>"}' -o /dev/null -w '%{http_code}\n'
# Verbose-error / version leak: send a malformed body, grep the response for stack traces + model build
curl -s -X POST https://<target>/v1/chat -H 'Content-Type: application/json' -d '{"prompt":}' | jq .
# Token-cost rate-limit (LLM10): fire N large-context requests, watch for 429 vs silent acceptance
for i in $(seq 1 50); do curl -s -o /dev/null -w "%{http_code}\n" -X POST https://<target>/v1/chat \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"prompt":"<8k-token-blob>"}'; done | sort | uniq -c   # many 429 = limited; all 200 = no cap
  • Key management. Hardcoded keys leak via git history, client bundles, decompiled mobile binaries, container logs. Use a secrets manager, separate keys per environment, rotate, and front shared provider keys with an identity-aware gateway issuing per-agent virtual keys.
  • Token-aware rate limiting. An agent chains 10-20 calls per task in bursts that look like a DDoS, and an 8k-token completion costs ~100× a metadata lookup yet ticks the same “one request.” Limit by tokens/cost per identity with hard spend caps. (LLM10.)
  • Monitoring. Calls from unexpected geographies, off-hours spikes, sudden volume - treat as possible key compromise.