Build a Kubernetes MCP Server: Safe kubectl Access for AI Agents

Give the agent hands, not a loaded gun

If you want an LLM agent to actually operate a cluster — not just narrate what it would do — it needs tools that call real infrastructure APIs. The clean way to expose those tools in 2026 is the Model Context Protocol (MCP): your agent's host (Claude, an IDE, an orchestrator) discovers tools from an MCP server and calls them with structured arguments. The temptation is to hand the model a single run_kubectl(command: str) tool and walk away. Don't. That is a hallucinated kubectl delete ns prod waiting to happen.

This guide builds a Kubernetes MCP server the way you'd actually ship it: read-only by default, an explicit verb allowlist, server-side dry-run on every mutation, hard namespace scoping, and a human approval gate for anything destructive. The agent gets useful hands. The blast radius stays bounded. If you want the higher-level picture of agents driving incidents, pair this with AI Agents for SRE: Autonomous Incident Response; this post is the plumbing underneath it.

Why one run_kubectl string tool is the wrong design

Three failure modes kill the naive approach, and they're the ones you'll actually hit in production:

1. Hallucinated commands. LLMs confidently invent flags and resource names. A free-form string tool executes them verbatim.
2. Runaway loops. An agent that gets an error will often retry with a broader command ("maybe I need --all-namespaces"), widening blast radius exactly when it's already confused.
3. No audit trail you can reason about. run_kubectl("...") logs a string. You want structured intent: verb, resource, namespace, dry-run result.

The fix is to make the tool surface itself the guardrail. Narrow, typed tools mean the model can only express safe intent, and anything outside that surface is impossible to call — not merely discouraged by a prompt. Prompt-level guardrails are advisory; a missing tool is a wall.

The scoping decisions to make before any code

• Read tools are always allowed; write tools are gated. get, describe, logs, top are safe to run unattended. apply, scale, rollout restart, delete are not.
• One namespace per session. The server is pinned to a namespace passed at startup, never chosen by the model. This alone eliminates most cross-tenant accidents.
• A dedicated ServiceAccount with least privilege. The MCP server authenticates as an SA whose RBAC is the real ceiling. The tool allowlist is defense-in-depth; RBAC is the actual boundary. If you're fuzzy on binding a Role to an SA, read the Kubernetes RBAC Deep Dive first — get this wrong and every guardrail above it is decorative.
• Every mutation dry-runs first. --dry-run=server validates against admission controllers and returns the diff without persisting. The agent sees exactly what would change before a human approves it.

The server: typed tools with FastMCP

Using the official Python MCP SDK (pip install "mcp[cli]" kubernetes). Each tool is a narrow function with typed args — the schema the agent sees is generated from the signature, so the model can only pass what you allow.

import subprocess
import shlex
from mcp.server.fastmcp import FastMCP

# Namespace and context are fixed at startup — NOT chosen by the model.
NAMESPACE = "staging"
KUBE_CONTEXT = "mcp-readonly@cluster"   # kubeconfig context bound to a least-priv SA

mcp = FastMCP("k8s-ops")

READ_VERBS = {"get", "describe", "logs", "top"}
ALLOWED_RESOURCES = {"pods", "deployments", "services", "events", "nodes",
                     "replicasets", "configmaps", "ingress"}

def _kubectl(args: list[str], timeout: int = 20) -> str:
    cmd = ["kubectl", "--context", KUBE_CONTEXT, "-n", NAMESPACE, *args]
    proc = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout)
    if proc.returncode != 0:
        # Return the error to the model as data, not an exception — it can adapt,
        # but it still cannot escape the allowlist.
        return f"ERROR (exit {proc.returncode}): {proc.stderr.strip()}"
    return proc.stdout.strip() or "(no output)"

@mcp.tool()
def k_get(resource: str, name: str = "", selector: str = "") -> str:
    """List or fetch a resource. resource must be a known type; name/selector optional."""
    if resource not in ALLOWED_RESOURCES:
        return f"BLOCKED: '{resource}' not in allowlist {sorted(ALLOWED_RESOURCES)}"
    args = ["get", resource, "-o", "wide"]
    if name:
        args = ["get", resource, name, "-o", "yaml"]
    if selector:
        args += ["-l", selector]
    return _kubectl(args)

@mcp.tool()
def k_logs(pod: str, container: str = "", tail: int = 200) -> str:
    """Fetch recent logs from a pod. Capped tail prevents context-window blowups."""
    args = ["logs", pod, f"--tail={min(tail, 1000)}"]
    if container:
        args += ["-c", container]
    return _kubectl(args)

@mcp.tool()
def k_describe(resource: str, name: str) -> str:
    """Describe a resource — the highest-signal read for debugging events."""
    if resource not in ALLOWED_RESOURCES:
        return f"BLOCKED: '{resource}' not in allowlist"
    return _kubectl(["describe", resource, name])

Notice what the model cannot do: pass an arbitrary command string, target another namespace, or read a resource type you didn't list. The allowlist is enforced in code, not in a system prompt the model can talk itself out of.

Mutations: dry-run, then a human gate

Write operations never execute directly. They dry-run, return the diff, and require an explicit approval token that a human (or a policy engine) issues out-of-band. This is the pattern from Kyverno policy-as-code applied at the agent boundary instead of the admission boundary.

import secrets

# In-memory approval store: dry-run mints a token, apply consumes it.
_PENDING: dict[str, list[str]] = {}

@mcp.tool()
def k_scale_preview(deployment: str, replicas: int) -> str:
    """Dry-run a scale. Returns a diff and an approval token. Does NOT apply."""
    if not 0 <= replicas <= 20:
        return "BLOCKED: replicas must be 0-20 (blast-radius cap)"
    args = ["scale", "deployment", deployment,
            f"--replicas={replicas}", "--dry-run=server", "-o", "yaml"]
    out = _kubectl(args)
    if out.startswith("ERROR"):
        return out
    token = secrets.token_hex(4)
    _PENDING[token] = ["scale", "deployment", deployment, f"--replicas={replicas}"]
    return (f"DRY-RUN OK. Would scale {deployment} to {replicas}.\n"
            f"To apply, a human must call k_apply_approved with token={token}")

@mcp.tool()
def k_apply_approved(token: str) -> str:
    """Execute a previously previewed mutation. Token comes from a HUMAN, not the model."""
    args = _PENDING.pop(token, None)
    if args is None:
        return "BLOCKED: unknown or already-used token"
    return _kubectl(args)

The agent can propose a scale-up all day. It physically cannot commit one, because k_apply_approved needs a token minted by a preview and handed over by a person. The _PENDING map also makes tokens single-use — an agent can't replay one. In a real deployment, replace the in-memory dict with a short-TTL entry in Redis and post the approval request to Slack; the human clicks ✅, your bot calls k_apply_approved. That keeps a human in the loop without making them babysit read-only debugging.

Wiring it into an agent

Register the server with any MCP-capable host. For Claude Code or the Claude desktop app:

{
  "mcpServers": {
    "k8s-ops": {
      "command": "python",
      "args": ["/opt/mcp/k8s_server.py"],
      "env": { "KUBECONFIG": "/opt/mcp/kubeconfig-readonly" }
    }
  }
}

Now the agent's system prompt can be short, because the tools carry the constraints:

You are an SRE assistant with read access to the staging namespace via the k8s-ops tools. Diagnose issues using k_get, k_describe, and k_logs. To change state, call the _preview tool and present the dry-run diff to the operator — never claim a change is applied unless you receive a confirmation from k_apply_approved.

That last clause matters: models will happily say "I've scaled the deployment" when they only ran a preview. Make "applied" a state only the approval tool can produce, and teach the model to distinguish the two. This is the same tool-calling discipline that makes the multi-agent DevOps panel trustworthy — structured tool results, not prose the model narrates about itself.

The guardrails that actually earn their keep

• RBAC is the real fence. Bind the server's ServiceAccount to a Role with only the verbs your tools need. If someone bypasses the tool layer, RBAC still says no. See Kubernetes Security Best Practices for hardening the SA and token mounts.
• Timeouts and output caps. Every _kubectl call has a 20s timeout and logs cap the tail. This is your defense against runaway loops and context-window floods.
• Structured audit log. Log each tool call as {tool, args, namespace, exit_code, token?} — not a raw string. When the agent does something surprising at 3 a.m., you want queryable intent.
• Read-only kubeconfig by default. The write path exists in a separate server instance with a separate SA that on-call spins up deliberately. Day-to-day, the agent literally cannot mutate.

Honest limits

An MCP server does not make an agent safe to run unsupervised on prod. It makes it bounded — a very different claim. The model can still misdiagnose, propose a scale-down that's exactly wrong, or loop on a bad hypothesis until its timeout. What these guardrails buy you is that its worst action is capped: read anything in one namespace, propose anything, commit nothing without a human. That's the right envelope for an assistant that shadows on-call, drafts the fix, and waits for a click — not an autonomous operator you walk away from.

Start read-only in staging. Watch the audit log for a week. Only widen the tool surface once you've seen how the agent actually behaves — never before.