Build an EC2 Right-Sizing Agent: Compute Optimizer, CloudWatch Agent Memory, Previous-Generation Families, and Graviton Moves Without a Stop/Start Surprise

What This Agent Does

An EC2 right-sizing agent reads every running instance in the account, joins it with 30 days of CloudWatch CPU, memory, and network data plus Compute Optimizer's recommendation, and produces four kinds of findings: over-provisioned instances that should drop a size, previous-generation families (m4, c4, r4, t2) that cost more than the current generation for less performance, x86 instances with a cheaper Graviton equivalent, and burstable instances paying surplus credit charges because they are really always-on workloads. Every number is deterministic. The LLM's only job is to read the instance's Terraform or launch template plus the files that build its AMI, write the one-line diff, and list the reasons the change might fail a stop/start. The agent never calls modify-instance-attribute, never stops an instance, and never touches anything owned by an Auto Scaling group, EKS, EMR, or Beanstalk directly.

It has the same shape as the Lambda cost agent: read-only collector, fixed finding vocabulary with margins, LLM that drafts but cannot apply. EC2 needs its own agent because it is usually the largest line on the bill and the one where a wrong change is most visible. A Lambda memory change rolls back in seconds. An EC2 instance type change is a stop/start, and a stop/start on an instance built in 2021 can lose its instance-store data, change its public IP, and fail to boot if the AMI never got NVMe drivers.

Where EC2 Spend Hides

Hourly list prices in us-east-1 for a Linux xlarge, as a feel for the gaps the agent is chasing: m4.xlarge is $0.20, m6i.xlarge is $0.192, m7i.xlarge is $0.2016, m6g.xlarge is $0.154, and m7g.xlarge is $0.1632. Moving one m4.xlarge to m7g.xlarge saves about $27 a month on a single instance, and the m7g is a much faster machine. On a fleet of 300 the number is why this agent exists. Keep prices in a config file the collector loads, or pull them from the Pricing API once a week. The agent will outlive the next price change.

Step 1: Give Compute Optimizer Memory to Work With

Compute Optimizer is the second opinion for every finding and the first opinion for Graviton, and it is free. Enroll the account and ask for arm64 options:

aws compute-optimizer update-enrollment-status --status Active

aws compute-optimizer put-recommendation-preferences \
  --resource-type Ec2Instance \
  --scope name=AccountId,value=123456789012 \
  --preferred-resources '[{"name":"Ec2InstanceTypes","includeList":["m7i","m7g","m6i","m6g","c7i","c7g","c6i","r7i","r7g","r6i","t3","t4g"]}]'

The includeList matters. Without it Compute Optimizer happily recommends an m5zn or a bare-metal type because the math works, and the agent then proposes a family the team has never run. The list should be the families your AMIs and Savings Plans already cover.

By default Compute Optimizer sees CPU, network, and EBS throughput, and nothing about memory. An instance at 15% CPU with 92% memory in use looks over-provisioned until you drop it a size and it starts swapping. The fix is the CloudWatch agent publishing mem_used_percent in the CWAgent namespace with an InstanceId dimension, which is the exact shape Compute Optimizer ingests automatically:

{
  "metrics": {
    "namespace": "CWAgent",
    "append_dimensions": {"InstanceId": "${aws:InstanceId}"},
    "metrics_collected": {
      "mem": {"measurement": ["mem_used_percent"], "metrics_collection_interval": 60},
      "swap": {"measurement": ["swap_used_percent"], "metrics_collection_interval": 60}
    }
  }
}

Ship that through SSM State Manager with AmazonCloudWatch-ManageAgent so every instance gets it, then wait. Compute Optimizer needs 30 consecutive hours of metrics before it will produce a finding and looks back 14 days by default. Enhanced infrastructure metrics extend the lookback to 93 days for a small per-resource fee, and that fee is worth paying for anything with a monthly batch peak.

Step 2: Collect Utilization, Recommendations, and Ownership

The role needs ec2:DescribeInstances, ec2:DescribeInstanceAttribute, ec2:DescribeImages, ec2:DescribeLaunchTemplateVersions, cloudwatch:GetMetricData, compute-optimizer:GetEC2InstanceRecommendations, and pricing:GetProducts. Attach an explicit Deny on ec2:ModifyInstanceAttribute, ec2:StopInstances, ec2:TerminateInstances, and ec2:ModifyLaunchTemplate. The IAM right-sizing agent post explains why the deny is worth ten extra lines even on a role that was never granted those actions.

Ownership comes first, because the most common way an EC2 cost agent causes an incident is by proposing a change to an instance that something else manages. An Auto Scaling group replaces a hand-modified instance on the next scale event. A Karpenter node gets consolidated under you. The collector reads tags and routes each instance to the thing that actually owns its size:

# collect_ec2.py — read-only inventory, one dict per running instance
import boto3
from datetime import datetime, timedelta, timezone

ec2, cw, co = (boto3.client(s) for s in ("ec2", "cloudwatch", "compute-optimizer"))
NOW = datetime.now(timezone.utc)
PREV_GEN = {"m4", "c4", "r4", "t2", "m3", "c3", "r3", "i2", "d2"}

def owner(tags):
    if "karpenter.sh/nodepool" in tags or "eks:cluster-name" in tags:
        return "eks"                      # Karpenter or managed node group sizes these
    if "aws:autoscaling:groupName" in tags:
        return "asg"                      # change the launch template, not the instance
    if "aws:elasticmapreduce:job-flow-id" in tags or "elasticbeanstalk:environment-name" in tags:
        return "managed"
    return "standalone"

def metric(instance_id, name, stat, namespace="AWS/EC2", days=30):
    r = cw.get_metric_data(MetricDataQueries=[{"Id": "m", "MetricStat": {
        "Metric": {"Namespace": namespace, "MetricName": name,
                   "Dimensions": [{"Name": "InstanceId", "Value": instance_id}]},
        "Period": 3600, "Stat": stat}}],
        StartTime=NOW - timedelta(days=days), EndTime=NOW)
    return r["MetricDataResults"][0]["Values"]        # empty list = no data

def facts(inst):
    iid, itype = inst["InstanceId"], inst["InstanceType"]
    tags = {t["Key"]: t["Value"] for t in inst.get("Tags", [])}
    cpu = metric(iid, "CPUUtilization", "p99")
    mem = metric(iid, "mem_used_percent", "p99", namespace="CWAgent")
    surplus = metric(iid, "CPUSurplusCreditsCharged", "Sum")
    ena = ec2.describe_instance_attribute(InstanceId=iid, Attribute="enaSupport")["EnaSupport"].get("Value", False)
    return {
        "id": iid, "type": itype, "family": itype.split(".")[0], "size": itype.split(".")[1],
        "arch": inst["Architecture"], "ami": inst["ImageId"], "owner": owner(tags), "tags": tags,
        "launch_template": inst.get("LaunchTemplate"), "ena": ena,
        "root_only": all(b["DeviceName"] == inst["RootDeviceName"] for b in inst["BlockDeviceMappings"]),
        "cpu_p99_max": max(cpu) if cpu else None, "mem_p99_max": max(mem) if mem else None,
        "surplus_credits_30d": sum(surplus) if surplus else 0.0,
        "prev_gen": itype.split(".")[0] in PREV_GEN,
    }

def co_recommendation(iid):
    r = co.get_ec2_instance_recommendations(
        instanceArns=[f"arn:aws:ec2:us-east-1:123456789012:instance/{iid}"],
        recommendationPreferences={"cpuVendorArchitectures": ["AWS_ARM64", "CURRENT"]})
    recs = r.get("instanceRecommendations", [])
    if not recs:
        return None
    rec = recs[0]
    opts = [{"type": o["instanceType"], "risk": o["performanceRisk"],
             "effort": o.get("migrationEffort"), "diffs": o.get("platformDifferences", []),
             "savings": o.get("savingsOpportunity", {}).get("estimatedMonthlySavings", {}).get("value", 0)}
            for o in rec["recommendationOptions"]]
    return {"finding": rec["finding"], "options": sorted(opts, key=lambda o: (o["risk"], -o["savings"]))}

The max over hourly p99 values is deliberate. An instance that touches 85% CPU for one hour every night is not over-provisioned, and an average would hide that hour. The root_only flag records whether the instance has instance-store or extra volumes, because that is the first thing a stop/start can destroy. The ENA attribute records whether the AMI can boot on a Nitro family at all.

Compute Optimizer's performanceRisk runs from 0 to 4, where 0 means the option is predicted to always meet the workload's needs. The platformDifferences list is the field most people never read. It names exactly what changes between the current and recommended type: Hypervisor when moving from Xen to Nitro, StorageInterface when EBS volumes will appear as NVMe devices, Architecture for a Graviton move, InstanceStoreAvailability when the new type drops the local disk. Every one of those is a reason a stop/start might not come back, and the classifier passes them straight to the LLM.

Step 3: Classify with Margins a Reviewer Can Defend

# classify.py — fixed vocabulary, each rule names its evidence
GRAVITON = {"m": "m7g", "c": "c7g", "r": "r7g", "t": "t4g"}
CURRENT  = {"m4": "m6i", "c4": "c6i", "r4": "r6i", "t2": "t3", "m3": "m6i", "c3": "c6i", "r3": "r6i"}
SIZES = ["nano", "micro", "small", "medium", "large", "xlarge", "2xlarge", "4xlarge", "8xlarge", "12xlarge", "16xlarge", "24xlarge"]

def findings(f, co, alarms_7d=0, days_of_data=30):
    out = []
    if f["owner"] != "standalone" or days_of_data < 30 or alarms_7d > 0:
        return out                                  # someone else owns it, or not enough evidence
    if f["cpu_p99_max"] is None:
        return out                                  # no metrics at all: do not guess
    mem_ok = f["mem_p99_max"] is not None and f["mem_p99_max"] < 50
    cpu_ok = f["cpu_p99_max"] < 40
    co_over = co and co["finding"] == "Overprovisioned"
    if cpu_ok and mem_ok and co_over and f["size"] != "nano":
        smaller = f"{f['family']}.{SIZES[SIZES.index(f['size']) - 1]}"
        out.append(("downsize", smaller, "cpu_p99<40 mem_p99<50 co=Overprovisioned"))
    if f["cpu_p99_max"] < 40 and f["mem_p99_max"] is None:
        out.append(("needs_memory_metric", None, "no CWAgent mem_used_percent"))
    if f["prev_gen"]:
        out.append(("previous_generation", f"{CURRENT[f['family']]}.{f['size']}", "family in PREV_GEN"))
    if f["arch"] == "x86_64" and co:
        arm = [o for o in co["options"] if o["type"].split(".")[0] in GRAVITON.values() and o["risk"] <= 1]
        if arm:
            out.append(("graviton_candidate", arm[0]["type"], f"co option risk={arm[0]['risk']} effort={arm[0]['effort']}"))
    if f["family"].startswith("t") and f["surplus_credits_30d"] > 0:
        out.append(("burst_surplus", f"{'m7g' if f['arch']=='arm64' else 'm6i'}.{f['size']}",
                    f"{f['surplus_credits_30d']:.0f} surplus vCPU-hours charged in 30d"))
    return out

Three margins do the real work. The downsize rule needs all three signals to agree: own CPU p99 under 40%, own memory p99 under 50%, and Compute Optimizer saying Overprovisioned. One size down is the maximum per PR, because a 4xlarge that is 10% busy might be a 2xlarge or might be an xlarge, and the second step is cheap to take after the first one has run for a week. Graviton candidates only come from Compute Optimizer options at performance risk 0 or 1. Instances with a CloudWatch alarm in the last 7 days are skipped entirely, because the first question after an incident is whether the cost agent changed anything, and the answer must be no.

The burst_surplus finding is the one teams never find by hand. A t3.xlarge in unlimited mode costs $0.1664 an hour plus $0.05 per vCPU-hour of surplus credits. Four vCPUs pinned above the 40% baseline around the clock bill roughly 2.4 surplus vCPU-hours per hour, about $0.12 extra, which makes that "cheap" t3 cost more than an m6i.xlarge at $0.192. The metric that proves it is CPUSurplusCreditsCharged, and it is zero on every instance where the t-family was the right call.

Step 4: The LLM Writes the Diff and the Stop/Start Checklist

The model receives one finding, the numbers behind it, the instance's Terraform block or launch template, and whatever builds the AMI: the Packer template, the user-data script, and the /etc/fstab baked into the image if the repo has it. It returns a structured change, and the harness opens the PR through the GitOps path so nothing reaches AWS without a human merge and a Terraform plan.

{
  "name": "propose_ec2_change",
  "description": "Produce a minimal diff for one EC2 finding. Change only instance_type (or the AMI reference when the finding is graviton_candidate). Never change more than one instance per call.",
  "input_schema": {
    "type": "object",
    "properties": {
      "finding": {"type": "string", "enum": ["downsize", "previous_generation", "graviton_candidate", "burst_surplus"]},
      "iac_file": {"type": "string"},
      "diff": {"type": "string", "description": "unified diff touching only instance_type and, for arm64, the ami data source filter"},
      "stop_start_risks": {"type": "array", "items": {"type": "string"}, "description": "things that break on stop/start for THIS instance: instance store, no EIP, missing ENA/NVMe, fstab device names"},
      "reviewer_check": {"type": "string", "description": "one concrete verification before merge"},
      "blockers": {"type": "array", "items": {"type": "string"}}
    },
    "required": ["finding", "iac_file", "diff", "stop_start_risks", "reviewer_check", "blockers"]
  }
}

The prompt is short and prefers a blocker to a guess:

You are reviewing one EC2 instance for an instance type change. You have the finding,
the measured metrics, Compute Optimizer platformDifferences, the IaC block, and the AMI
build files. Output a propose_ec2_change call.
Rules:
- platformDifferences contains Hypervisor or StorageInterface and the AMI build files do
  not show ENA and NVMe support (ena_support = true in Packer, nvme module, UUID or
  LABEL entries in fstab rather than /dev/xvd*): add a blocker, emit an empty diff.
- graviton_candidate: if there is no arm64 AMI build in the repo, the blocker is
  "no arm64 AMI pipeline", not a guessed AMI id. If a Dockerfile or requirements file
  pins x86-only binaries, name them.
- The instance has instance-store volumes or no Elastic IP: list it in stop_start_risks
  even when the change is otherwise safe.
- reviewer_check names the maintenance window and the dashboard to watch for 24h:
  CPUUtilization p99, mem_used_percent p99, and StatusCheckFailed.

The Nitro check is where the model earns its tokens. Deterministic code can read the enaSupport attribute, but an attribute set to true on an instance does not prove the AMI has the NVMe driver or that the image's fstab mounts by UUID. A previous-generation m4 moving to m6i goes from Xen to Nitro, and the symptom of a missing NVMe module is an instance that stops cleanly and never passes its status check. The model reads the Packer template and fstab and says so before anyone presses merge.

A merged PR for a previous-generation move is a one-line diff:

 resource "aws_instance" "reports_worker" {
-  instance_type = "m4.xlarge"
+  instance_type = "m6i.xlarge"   # prev-gen; cpu p99 31%, mem p99 44%; ENA+NVMe confirmed in packer/base.pkr.hcl
   ami           = data.aws_ami.base_x86.id
   ebs_optimized = true
 }

Terraform applies instance_type in place with a stop/start, so the maintenance window is the apply, and the PR description says what time that is. For instances behind an Auto Scaling group the agent instead edits the launch template and the PR notes that an instance refresh with a 90% minimum healthy percentage will replace instances one at a time, which is the canary.

What It Found, and What It Got Wrong

On an account with 418 running instances, 146 were standalone after ownership routing, and the first pass produced 71 findings. Previous-generation families were 29 of them, which surprised nobody who had read the console and everybody who had not. The deterministic monthly estimate for those 29 was within 6% of the bill change after they merged across three weeks of maintenance windows.

Graviton candidates were the largest savings line and the slowest to land. The agent found 22 at performance risk 0 or 1, but only 9 had an arm64 AMI pipeline, so 13 shipped as blockers with the exact missing piece named. Two weeks later the platform team added an arm64 build to the shared Packer repo and the next run turned 11 of those blockers into diffs. That is the right outcome: the agent surfaced the prerequisite instead of pretending it did not exist.

The misses were instructive. A downsize on a memory-heavy r6i looked clean on both CPU and memory, but the instance was a self-hosted CI runner whose load was entirely in a Docker build cache on an instance-store volume, and Compute Optimizer's option had InstanceStoreAvailability in its platform differences. The model listed it in stop_start_risks; the reviewer merged anyway and the cache was gone after the apply. The collector now hard-blocks any downsize where the current type has instance store and the target does not, rather than trusting a risk list to be read. Separately, two burst_surplus findings turned out to be instances in standard mode that had briefly been switched to unlimited for a migration; the fix was reading the CreditSpecification and only firing when unlimited is the current setting.

Limits Worth Stating in the PR Template

A family change touches commitments. Standard Reserved Instances are bound to a family, so moving m4 to m6i can strand RI hours that keep billing. Compute Savings Plans cover any family and any architecture, so Graviton moves under a Compute Savings Plan just lower the covered spend. Run the Savings Plans coverage agent after a batch of EC2 changes, not before, or the coverage math will be stale. Idle instances, the ones under 2% CPU with no network traffic, belong to the waste reclamation agent, not this one, because the fix is a stop and a snapshot rather than a size.

EKS nodes are excluded on purpose. Their size is a NodePool or node group decision, and the Karpenter versus Cluster Autoscaler comparison covers why consolidation there is a different problem with a different tool. Windows instances publish memory as Memory % Committed Bytes In Use rather than mem_used_percent, so the collector needs a second metric name before the memory rule fires for them. And the 30-day window has the same blind spot every right-sizing tool has: the quarter-end job that pins an instance for six hours in March is invisible in October. Enhanced infrastructure metrics give Compute Optimizer 93 days, the agent reads 90 days for peak memory and CPU and 30 for cost, and the PR template says so in its first line.