One agent, one blind spot
Ask a single LLM "should we roll back this deploy?" and you get one confident answer shaped by whatever context you happened to paste into the prompt. Paste the error logs and it blames the code. Paste the latency graph and it blames the database. The model isn't reasoning about your incident — it's pattern-matching on the slice of evidence in its context window, and it will sound equally sure either way.
A multi-agent panel attacks that blind spot directly: several agents, each given a different slice of the evidence and a different mandate, analyze the same decision independently, then a synthesizer reconciles them. Done right, the panel catches what any single agent misses. Done wrong, it's an expensive way to average three copies of the same mistake. This guide shows the difference, with runnable orchestration code.
If you want the single-agent, tool-calling approach to incidents first, see AI Agents for SRE: Autonomous Incident Response — this article is about the panel pattern layered on top.
The one rule that makes it work: information asymmetry
Recent research is blunt about naive multi-agent debate. Two findings you need to internalize before writing a line of code:
- Majority voting explains most of the gains. A lot of the accuracy attributed to elaborate "debate" comes from simply sampling several answers and voting — not from the agents talking to each other.
- Identical-input debate is a martingale. If every agent sees the same prompt, expected correctness does not improve as you add debate rounds. You're paying more tokens to converge on the answer you'd already have gotten.
The lever that does move accuracy is information asymmetry: give each agent genuinely different evidence or a different analytical lens. In a forecasting-under-asymmetry setup, agents that each hold a distinct piece of the picture and deliberate beat both single agents and same-input panels. For DevOps this is a gift, because your telemetry is already siloed:
- one agent reads metrics (latency, error rate, saturation),
- one reads logs and traces,
- one reads the deploy diff and change history,
- one reads the runbook and recent incidents.
That is real asymmetry, not theater. Each expert can reach a defensible conclusion the others literally cannot, because they're looking at different data.
The architecture
Three roles:
- Experts — N specialized agents, each with its own system prompt, its own tools, and its own data source. They never see each other's raw context, only the decision to make.
- Orchestrator — fans the decision out to the experts in parallel, collects structured verdicts.
- Synthesizer — a final agent (or a deterministic rule) that reconciles the verdicts into one recommendation, weighted by each expert's self-reported confidence and, crucially, by whether its evidence is relevant to this incident.
Structured output is non-negotiable — you want to reconcile data, not prose. Force every expert to return the same schema.
from dataclasses import dataclass
from concurrent.futures import ThreadPoolExecutor
import json
@dataclass
class Verdict:
expert: str
recommendation: str # "rollback" | "hold" | "investigate"
confidence: float # 0.0 - 1.0, self-reported
evidence: str # the concrete signal it saw
EXPERTS = {
"metrics": {
"system": "You are an SRE metrics analyst. You ONLY reason about the "
"metrics provided (latency, error rate, saturation). If the "
"metrics are clean, say so and set low confidence for rollback.",
"fetch": lambda ctx: query_prometheus(ctx["service"], ctx["window"]),
},
"logs": {
"system": "You are a log/trace analyst. Reason ONLY about the logs and "
"exception traces provided. Quote the specific error you rely on.",
"fetch": lambda ctx: query_loki(ctx["service"], ctx["window"]),
},
"deploy": {
"system": "You are a release engineer. Reason ONLY about the deploy diff "
"and change timeline. Correlate the incident start with changes.",
"fetch": lambda ctx: git_diff_since(ctx["last_good_sha"]),
},
}
Each expert is a thin function: fetch its private evidence, call the model with its narrow system prompt, parse a Verdict.
def run_expert(name, cfg, decision, ctx):
evidence = cfg["fetch"](ctx)
resp = llm(
system=cfg["system"],
user=f"Decision: {decision}\n\nYour evidence:\n{evidence}\n\n"
f"Return JSON: {{recommendation, confidence, evidence}}.",
response_format="json",
temperature=0.2,
)
d = json.loads(resp)
return Verdict(name, d["recommendation"], float(d["confidence"]), d["evidence"])
def convene(decision, ctx):
with ThreadPoolExecutor(max_workers=len(EXPERTS)) as pool:
verdicts = list(pool.map(
lambda kv: run_expert(kv[0], kv[1], decision, ctx),
EXPERTS.items()))
return verdicts
Fan-out is parallel, so wall-clock is the slowest single expert, not the sum. On a real incident that matters — you want the panel's answer in seconds, not a minute.
Synthesis: weight by confidence and relevance
The naive synthesizer takes a majority vote. That is already a solid baseline (see rule #1 above) — but it throws away the confidence signal and treats an expert staring at clean, irrelevant metrics the same as one holding a smoking-gun trace. Weight the votes:
def synthesize(verdicts):
weight = {}
for v in verdicts:
# a "hold/clean" verdict from an expert with low confidence should not
# cancel a high-confidence "rollback" from the expert that sees the cause
weight.setdefault(v.recommendation, 0.0)
weight[v.recommendation] += v.confidence
winner = max(weight, key=weight.get)
total = sum(weight.values()) or 1.0
return {
"decision": winner,
"score": round(weight[winner] / total, 2),
"dissent": [f"{v.expert}: {v.recommendation} ({v.confidence:.0%}) — {v.evidence}"
for v in verdicts if v.recommendation != winner],
}
Keep the dissent in the output. The single most useful thing a panel produces is not the winning vote — it's the minority report. "Deploy agent says rollback (90%): the incident started 40s after commit a1b2c3, which touched the connection pool" is exactly the lead a human on-call wants, even if the metrics and logs agents outvoted it. Pipe this into your incident runbook so the panel augments the responder instead of silently overruling them.
When to use a real debate round (and when not to)
Independent verdicts + weighted synthesis covers most cases and is cheap. Add an actual debate round — where experts see each other's verdicts and revise — only when the experts disagree and their evidence overlaps, so one can correct another's interpretation. A useful gate:
def needs_debate(verdicts):
recs = {v.recommendation for v in verdicts}
high_conf_conflict = sum(v.confidence > 0.7 for v in verdicts) >= 2
return len(recs) > 1 and high_conf_conflict
If the experts already agree, a debate round is pure token cost with no expected gain — the martingale result. Only convene the second round on genuine high-confidence conflict, and cap it at one or two rounds; deliberation that runs until "consensus" tends to converge on the loudest agent, not the correct one.
Cost, latency, and failure modes
- Token cost scales linearly with panel size. Four experts + a synthesizer is ~5x a single call. Reserve the panel for high-stakes, ambiguous decisions (rollback, data-loss risk, capacity cutover), not routine alerts.
- Self-hosting the experts changes the math entirely — if you're running open models for this, the per-call cost approaches zero and a panel becomes cheap. See Kubernetes LLM Inference: Deploy and Scale Open-Source LLMs.
- Correlated errors. If all experts use the same base model, they share failure modes and a "panel" can be falsely confident. Mix models, or at minimum mix prompts and temperature.
- Garbage evidence in, garbage verdict out. The panel is only as good as the telemetry each expert reads. This pattern rewards teams who already invested in structured observability — see the OpenTelemetry setup guide and AI-powered observability.
A concrete run
For a latency incident right after a deploy, the panel might return:
{
"decision": "rollback",
"score": 0.61,
"dissent": [
"metrics: hold (55%) — p99 elevated but within error budget",
"logs: investigate (60%) — pool-timeout errors, cause unclear"
]
}
The deploy expert's high-confidence rollback (it correlated the incident to a connection-pool change) carried the weighted vote, but the dissent tells the on-call engineer exactly what to verify before pulling the trigger — and gives them the metrics and logs context for free. That is the whole point: not to replace the human's judgment, but to hand them three expert briefings and a defensible recommendation in one shot.
Takeaways
- A multi-agent panel beats a single agent only when each agent holds different evidence. Same input, more agents = the martingale trap.
- Weighted synthesis over majority vote, and always surface the dissent — the minority report is the highest-value output.
- Gate real debate rounds behind genuine high-confidence conflict; otherwise you're paying tokens to re-derive the vote.
- Reserve the pattern for high-stakes, ambiguous decisions, and mix models to avoid correlated errors.
Multi-agent orchestration is not magic and it is not free — but pointed at the right decision, with asymmetric evidence, it turns a confident guess into a reviewed one. In production, that difference is the incident that stays small.