Token Fire
Overnight, one agent looped on a failing tool and burned three months of budget by 06:00. Contain the runaway, then build the limits that make denial-of-wallet a bounded event instead of an invoice.
FinOps FIN-88 — 'AI spend 41x baseline in eight hours'
SRE
AI engineer
FinOps
45 minutes · practitioner
Execution trace
00:00:00toolinventory.lookup
attack step
Tool starts returning 502
Upstream dependency degraded at 22:14.
00:00:05modelops-agent
attack step
Agent retries with a longer prompt each time
Each retry appends the prior failure to context, growing tokens per step.
step=1..4100 · tokens_in growth=linear
01:00:00enterpriseai-gateway
attack step
Spend crosses monthly budget
No per-run or per-tenant ceiling exists.
08:00:00alertFinOps
Invoice anomaly at 06:00
Detected by billing, eight hours late.