AI Agent Architecture Patterns

What it is

A single LLM loop that decides whether to call a tool, observe the result, and iterate until it has an answer. Tools are functions the model invokes by emitting structured calls.

Components

• • LLM
• • Tool registry (5-10 functions)
• • Memory (last N turns)
• • Output parser

When to use

You have one well-scoped task (answer support questions, run a SQL query, draft a doc). Tool count under 10. Latency matters. You want simple debugging.

When to avoid

You need parallel work, multiple specialties, or long-running plans across days. The model gets confused once tool count crosses 15-20.

Real-world examples

• • Customer support agent
• • SQL analyst
• • Code review bot

Common failure modes

• • Tool selection errors when tool count grows
• • Token bloat from tool descriptions
• • Single point of failure

What it is

A lightweight router LLM (or classifier) inspects the input and routes to one of N specialist agents, each with its own tools and prompt. Specialists do not talk to each other.

Components

• • Router (LLM or classifier)
• • 3-7 specialist agents
• • Per-specialist tool sets
• • Shared session memory

When to use

You have 3-7 distinct domains (billing, technical, sales). Each domain has different tools or knowledge. You want clean separation and per-domain accuracy metrics.

When to avoid

Domains overlap heavily, or a single user request typically spans multiple specialties. Then you get expensive ping-pong handoffs.

Real-world examples

• • Multi-product support
• • Insurance triage
• • Internal helpdesk across HR/IT/Finance

Common failure modes

• • Misrouted requests
• • Lost context on handoff
• • Specialist scope creep over time

What it is

The supervisor decomposes a request into subtasks, dispatches each to a worker agent, collects outputs, and synthesizes the final answer. Workers can run in parallel.

Components

• • Supervisor LLM
• • Plan/task decomposer
• • Worker pool
• • Result aggregator
• • Shared scratchpad

When to use

Tasks need to be broken into independent subtasks (research a topic from 5 angles, generate a report from multiple data sources). You want parallelism for speed.

When to avoid

Subtasks have heavy dependencies on each other. You will spend more on coordination tokens than on actual work. Also avoid for sub-second latency needs.

Real-world examples

• • Deep research agent
• • Competitive analysis report
• • Multi-source due diligence

Common failure modes

• • Supervisor token cost explodes
• • Worker outputs that do not compose cleanly
• • Hard to debug across many turns

What it is

A tree of agents: top-level supervisor delegates to mid-level team leads, each leading their own pool of workers. Mirrors a real org chart.

Components

• • Top supervisor
• • Sub-supervisors (per team)
• • Workers per team
• • Cross-team event bus
• • Shared knowledge base

When to use

Very large workflows with clear divisions (engineering / marketing / legal each own a slice). 20+ tools that would overwhelm a single supervisor. Stable, repeated workflows.

When to avoid

You are still iterating on the workflow. Every layer multiplies token cost and debug surface area. Most teams should not start here.

Real-world examples

• • End-to-end product launch agent
• • Enterprise back-office automation
• • Multi-team DevOps assistant

Common failure modes

• • Coordination overhead dominates cost
• • Failures cascade across layers
• • Slow iteration once in production

What it is

A planner LLM produces an explicit step-by-step plan. An executor runs each step (often calling tools or sub-agents). After each step, a replanner decides whether to continue, replan, or stop.

Components

• • Planner LLM
• • Step executor
• • Replanner
• • Plan memory
• • Tool layer

When to use

Tasks span 5-30 steps with branching logic (file a bug, find duplicates, propose a fix, open a PR). You want a visible plan for human oversight or audit.

When to avoid

Tasks under 3 steps (overhead is wasted). High-frequency, low-latency tasks (planning round-trips add seconds). Highly creative tasks where a fixed plan hurts.

Real-world examples

• • Bug-fix agent
• • Travel booking
• • Data migration agent

Common failure modes

• • Plan drift after replans
• • Plans too rigid for surprises
• • Costly when replans cascade

What it is

Two LLM roles in a loop. The generator produces a draft (code, copy, plan). The critic evaluates against criteria. The generator revises until the critic passes or a max iteration is hit.

Components

• • Generator agent
• • Critic agent (different prompt or model)
• • Iteration controller
• • Stopping criterion

When to use

Quality matters more than latency (legal drafts, code, marketing copy). You can encode "good" in checkable criteria. You have token budget for 2-4x the naive call.

When to avoid

Truly subjective tasks where the critic will not converge. Latency-sensitive UX. Tasks where the first draft is usually correct.

Real-world examples

• • Code generation with tests
• • Legal contract drafting
• • Long-form copy editor

Common failure modes

• • Loop never converges
• • 2-4x token cost vs single shot
• • Critic biased to its own training distribution

What it is

The LLM emits a Thought, then an Action (tool call), reads the Observation, then more Thought, more Action, until it answers. The pattern that started the modern agent wave.

Components

• • Single LLM
• • Tool registry
• • Trace logger (Thought / Action / Observation)
• • Stopping rule

When to use

You want a single, transparent reasoning trace. Tasks need 2-8 tool calls. You value debuggability and the ability to inject human review at each step.

When to avoid

You need parallel tool calls, multi-agent specialization, or sub-second responses. Modern frameworks have largely replaced raw ReAct with parallel tool use.

Real-world examples

• • Search-augmented Q&A
• • Stepwise data exploration
• • Beginner agent prototypes

Common failure modes

• • Slow due to serial tool calls
• • Hallucinated observations
• • Verbose traces inflate cost

What it is

Agents subscribe to a shared blackboard (or message bus). Each agent reads relevant updates, contributes its piece, and writes back. No central orchestrator. Coordination emerges.

Components

• • Shared blackboard / message bus
• • N specialist agents
• • Termination detector
• • Conflict resolver

When to use

Highly parallel, weakly coupled work (creative brainstorming, simulations, distributed scraping). You want emergent behavior and graceful degradation when one agent fails.

When to avoid

Strict ordering matters. Auditability is required. You want predictable cost and latency. Most production B2B use cases.

Real-world examples

• • Idea generation swarm
• • Multi-agent simulations
• • Research with self-organizing roles

Common failure modes

• • Non-deterministic outputs
• • Hard to debug
• • Cost is unpredictable