Cascade failure

Cascade Failure (Agentic Tier Framework)

Definition

Cascade failure is an operational risk pattern in Tier 4 autonomous workflows where one erroneous or hallucinated output from an upstream agent becomes input to downstream agents, compounding across multiple steps before any human review occurs. By step 10, a single hallucination at step 1 can manifest as numerous amplified errors, each building on the prior incorrect state.

Key Characteristics

• Tier specificity: Dominant at Agentic Tier 4, where agents can trigger other agents with minimal or no synchronous human gating.
• Error propagation: Each autonomous step consumes prior outputs as ground truth, so early hallucinations or misclassifications propagate unchecked.
• Amplification: Downstream agents may enrich, transform, or branch on erroneous data, multiplying the impact (e.g., 1 bad entity record → 10 flawed downstream actions).
• Latency to detection: Human review often occurs only at the end of the chain, so detection is delayed until after multiple dependent actions have executed.

Mitigation Requirements

1. Intermediate Human Review Checkpoints
   • Insert human-in-the-loop gates at critical junctures in Tier 4 workflows where consequence severity (e.g., financial, safety, legal, reputational) justifies the latency and cost.
   • Prioritize checkpoints at:
     • First transformation from raw user input into structured decisions or plans.
     • Branching points that fan out into multiple downstream actions or agents.
     • Any step that triggers irreversible or high-cost external effects.
2. Per-Step Audit Trail Discipline (Evidence Register – GOV-13)
   • Log each autonomous step into the Evidence Register (GOV-13), including:
     • Inputs consumed (including upstream agent outputs).
     • Model/agent configuration and version.
     • Rationale or chain-of-thought summaries suitable for audit (excluding sensitive internal reasoning where policy requires).
     • Outputs produced and actions triggered.
   • Ensure logs are structured to support cascade-pattern detection during sample audits, e.g.:
     • Ability to reconstruct full dependency chains from step 1 → N.
     • Identification of where the first erroneous assertion entered the chain.
     • Metrics on how many downstream steps/actions depended on that assertion.

Operational Usage Guidance

• Treat cascade failure risk as non-local: controls must be designed at the workflow level, not per-agent only.
• When designing Tier 4 systems, explicitly document:
  • Maximum allowed autonomous depth (number of chained steps) before mandatory human review.
  • Which steps are designated as high-stakes and must always be checkpointed.
• During post-incident reviews, use GOV-13 evidence to:
  • Trace the first hallucinated or incorrect state.
  • Quantify the cascade (how many steps/actions were affected).
  • Update design patterns and guardrails to prevent recurrence.

Framework Positioning

• Layer: Operational risk within the Agentic Tier framework.
• Tier relevance: Primarily Tier 4 (multi-step, multi-agent autonomy), with partial relevance to high-depth Tier 3 orchestrations.