What a FinOps agent should actually do
A useful FinOps agent does not "manage your cloud bill." It does one narrow, verifiable job: read cost and utilization data, find the handful of workloads that are wasting money, and produce a concrete, reviewable rightsizing change — a pull request, not a live kubectl apply. The intelligence is in ranking the waste and justifying the fix; the action stays behind a human gate and your existing GitOps flow.
That framing is the whole design. Cost data is read-only and cheap to query, which makes it nearly ideal agent input — an over-provisioned resources.requests is an objective fact, not a judgment call. The danger is never in the reading; it is in letting a model that occasionally hallucinates a number write directly to production. So we build an agent that is powerful at analysis and deliberately powerless at mutation. This pairs the autonomous-agent playbook with hard FinOps cost discipline.
The two tools it gets — and the one it doesn't
An agent is defined by its tools. Give this one exactly two read tools and zero write tools.
TOOLS = [
{
"name": "get_cost_allocation",
"description": "Per-workload cost + CPU/memory efficiency over a window. "
"Efficiency = usage / requests. Below ~0.4 is waste.",
"input_schema": {
"type": "object",
"properties": {
"namespace": {"type": "string"},
"window": {"type": "string", "enum": ["7d", "14d", "30d"]},
},
"required": ["window"],
},
},
{
"name": "get_workload_spec",
"description": "Current resources.requests/limits and replica count "
"for one deployment. Read-only.",
"input_schema": {
"type": "object",
"properties": {
"namespace": {"type": "string"},
"deployment": {"type": "string"},
},
"required": ["namespace", "deployment"],
},
},
]
get_cost_allocation is a thin wrapper over OpenCost (or Kubecost) — the open-source allocation API you probably already run for Kubernetes cost visibility. It returns structured, pre-digested rows, never raw metric dumps:
import requests
def get_cost_allocation(window="14d", namespace=None):
r = requests.get(
"http://opencost.opencost.svc:9003/allocation/compute",
params={"window": window, "aggregate": "namespace,deployment",
"accumulate": "true"},
timeout=15,
)
rows = []
for name, a in r.json()["data"][0].items():
if namespace and not name.startswith(f"{namespace}/"):
continue
cpu_eff = a["cpuEfficiency"] # 0..1
mem_eff = a["ramEfficiency"]
rows.append({
"workload": name,
"monthly_cost": round(a["totalCost"] * (30 / _days(window)), 2),
"cpu_efficiency": round(cpu_eff, 2),
"mem_efficiency": round(mem_eff, 2),
})
# Rank the waste for the model — don't make it scan 300 rows
return sorted(rows, key=lambda x: x["monthly_cost"], reverse=True)[:25]
The point of the [:25] and the sort is context engineering: the model reasons far better over 25 ranked, labeled rows than over 300 raw ones. What it does not get is a patch_deployment or kubectl_apply tool. There is no code path from the model's output to a live cluster. That single omission removes the entire class of "the agent scaled prod to zero" failures.
The system prompt encodes the FinOps policy
The prompt is where you turn a general model into a FinOps reviewer with a house style. Be explicit about the safety margin — this is the difference between a recommendation you can merge and one that pages you at 3 a.m.
You are a FinOps analyst for a Kubernetes platform. Your job: find
over-provisioned workloads and propose safer resource requests.
Rules:
- A workload is a candidate ONLY if cpu_efficiency < 0.4 OR
mem_efficiency < 0.4 AND monthly_cost > $50.
- Propose new requests = observed P95 usage * 1.25 (headroom).
Never propose below observed peak. Never touch limits without
explaining the OOMKill risk.
- NEVER propose changes to anything in namespace 'kube-system' or
labeled tier=critical.
- Output ONLY workloads you have called get_workload_spec on.
No spec = no recommendation.
- For each, give: current requests, proposed requests, estimated
monthly saving, and the ONE risk of being wrong.
Two lines are doing the heavy lifting. The P95-with-25%-headroom rule keeps the agent from optimizing a workload straight into a CrashLoopBackOff or OOMKill. The "no spec = no recommendation" rule forces tool use before assertion — the agent cannot recommend a change to a workload it never actually inspected, which is where hallucinated resource numbers come from.
The output is a diff, not an action
Force structured output so the agent's conclusion is machine-consumable. Each recommendation becomes a patch your pipeline can render as a PR:
{
"recommendations": [
{
"workload": "checkout/api",
"current": {"cpu": "500m", "memory": "512Mi"},
"proposed": {"cpu": "150m", "memory": "320Mi"},
"p95_observed": {"cpu": "118m", "memory": "250Mi"},
"monthly_saving_usd": 84.0,
"risk": "Traffic spikes above P95 during Black Friday could throttle CPU; revisit before Q4."
}
]
}
Your harness turns that into a GitOps commit against the values file, opens a PR, and stops. A human reviews the diff the same way they review any infra change, and Argo CD applies it on merge. The agent never held a kubeconfig with write access. This is the harness-as-infrastructure principle applied to cost: least privilege, small blast radius, every change auditable in Git history.
def to_gitops_pr(rec):
path = f"apps/{rec['workload'].replace('/', '/values-')}.yaml"
patch = {
"resources": {
"requests": rec["proposed"],
}
}
branch = f"finops/rightsize-{rec['workload'].replace('/', '-')}"
open_pr(
branch=branch,
path=path,
patch=patch,
title=f"FinOps: rightsize {rec['workload']} (save ${rec['monthly_saving_usd']}/mo)",
body=f"Risk noted by agent: {rec['risk']}\nCurrent: {rec['current']}\n"
f"P95 observed: {rec['p95_observed']}",
)
If you want tighter enforcement than a reviewer's eyeballs, add a Kyverno policy that blocks any manifest whose requests fall below a floor — so even a rubber-stamped bad PR can't reduce a critical service below a safe minimum.
Guard the guardrails: what still goes wrong
An agent this constrained is safe from catastrophe, but it can still be wrong in ways that waste your time. Design against these:
- Short windows lie. A 7-day window over a quiet period makes everything look over-provisioned. Default to 14–30 days and refuse to rightsize workloads with fewer than N days of data. Weekly and seasonal peaks are exactly when you don't want tighter requests.
- P95 hides the tail. Batch jobs, cron-driven spikes, and cold-start bursts live in the P99+. The agent proposing "P95 × 1.25" will occasionally clip a real peak. That is what the mandatory
riskfield and human review are for — never remove them. - Efficiency ≠ waste for HA workloads. A deliberately over-provisioned service holding headroom for failover looks identical to waste in the cost data. The
tier=criticalexclusion in the prompt is not optional; label your workloads so the agent can see the intent. - Savings are estimates, not invoices. The agent's "$84/mo" is a modeled number. Track realized savings from your actual cloud bill after merge — the gap between predicted and realized is the single best eval signal for whether the agent is calibrated.
Prove it before you trust it
Do not point this at production and read the PRs. Eval it first: replay a fixed set of cost snapshots where you already know the right answer, and score two things. Precision — of the workloads it flagged, how many were genuinely wasteful (a false positive that shrinks a critical service is far worse than a missed saving). Safety — did it ever propose requests below observed peak, or touch an excluded namespace? A single safety violation in the eval set means the prompt or the tool filter is broken, and you fix that before it ever opens a PR.
Instrument the running agent the same way you would any pipeline — trace each analysis with OpenTelemetry so you can see which tool calls it made, which recommendations were merged, and how predicted savings compare to the real bill over the following month.
Takeaway
The winning pattern for agents on cost isn't autonomy — it's a sharp read-only analyst wired into your existing GitOps and review flow. Give it two read tools and zero write tools, encode the FinOps policy (P95 + headroom, hard exclusions) in the prompt, make its output a reviewable diff, and gate every change behind a human and a Kyverno floor. You get an assistant that surfaces the real waste in a 200-workload cluster in seconds, and you keep the one thing that matters: nothing reaches production that a human didn't approve. Start it in report-only mode, measure realized savings against its estimates, and widen the leash only as the numbers earn it.