Back to blog

Which workloads in your cluster can run on Graviton? An open source tool to find out

"Move to Graviton" is common FinOps advice on AWS. arm64 nodes tend to cost less per unit of work, and Karpenter knows how to mix the two architectures. What almost nobody says is how to find out which workloads in your cluster can go, and a wrong guess has a familiar face: a pod that restarts in a loop with exec format error.

graviton-readiness is an open source command line tool that turns that question into an objective check. It looks at the two things that decide whether a pod runs on arm64: whether the scheduling configuration lets the pod reach an arm64 node, and whether each image was published for linux/arm64.

What it checks

It reads Deployments, StatefulSets, DaemonSets, Jobs, CronJobs and Pods from YAML or JSON files, and applies eleven rules. Each workload gets a score and a verdict.

RuleSeverityWhat it checks
GR001BlockerPinned to amd64 by kubernetes.io/arch in a node selector or required node affinity
GR002BlockerOnly x86 instance types allowed (instance-type selector or Karpenter instance family)
GR003BlockerThe image reference names amd64 (app:1.4-amd64, amd64/ubuntu)
GR004BlockerThe image has no linux/arm64 manifest (needs --check-images)
GR005InfoThe image architecture could not be verified (needs --check-images)
GR006CaveatCommand, args or environment variables mention amd64 or x86_64
GR007CaveatRequests a GPU
GR008InfoDaemonSet: runs on every node, so its images must be multi-arch first
GR009BlockerTargets Windows nodes
GR010BlockerSelects a Karpenter NodePool that only provisions amd64
GR011InfoSelects a NodePool that is not among the manifests that were read

Init containers are checked like the others. graviton-readiness explain GR004 documents any rule, with a problem and a fix.

Two rules deserve a comment. GR001 and GR002 look at scheduling: a nodeSelector fixed to kubernetes.io/arch: amd64 stops the scheduler from placing the pod on an arm64 node, even if the image is multi-arch. In affinities, required terms are ORed, so a workload that lists amd64 in one term and arm64 in another can run on both. GR002 recognizes the Graviton families by name (m7g, c6gn, x2gd, im4gn, is4gen, g5g, hpc7g and the first generation, a1) and only blocks when every allowed option is x86.

The NodePool decides too

A pod that picks its nodes by karpenter.sh/nodepool, or by a label that its NodePool puts on the nodes (role: web), has the architecture of that pool, and the pod itself does not say which one. So the image can be multi-arch and the workload still never reaches a Graviton node.

When the folder includes the Karpenter NodePool manifests, GR010 links each workload to the pools it can land on (through its nodeSelector and its required affinity) and reports it when none of them provisions arm64. GR011 says so when a workload selects a pool that is not among the manifests that were read. A pool only counts as amd64 when the manifest proves it: a kubernetes.io/arch In [amd64], or instance types and families that are all x86.

graviton-readiness scan ./k8s ./karpenter

A pool that states no architecture is not assumed to be amd64: the Karpenter documentation does not promise a default, so the rule only speaks when there is proof.

What if there is no NodePool in the scan? The rule does not run, and the tool says so: the summary counts the workloads that choose nodes by a label that no pool it read defines. That is the case when the pools are not Karpenter's (an AWS managed node group, a GKE or AKS pool) or live outside the folder. Then you declare the architecture with --node-pool key=value:arch, which can be repeated:

graviton-readiness scan ./k8s \
  --node-pool eks.amazonaws.com/nodegroup=general:amd64 \
  --node-pool cloud.google.com/gke-nodepool=arm-pool:arm64

The EC2NodeClass counts too: a NodePool whose class uses a Windows AMI family (amiFamily: Windows2022, or an alias such as windows2022@latest) only provisions amd64, whatever the kubernetes.io/arch of the pool itself says, and GR010 explains why. The architecture of a custom AMI (Custom, or one chosen by id or tags) is in AWS and not in the manifest, so it is not read.

Images are the part that decides

The configuration can be clean and the pod can still fail, because the image only exists for amd64. That is why the tool has the --check-images flag.

graviton-readiness scan --check-images ./k8s

With it, the tool reads the manifest of each image from the registry, never the layers, and looks for a linux/arm64 entry. A few details matter:

  • Credentials. It uses the ones in your Docker config and credential helpers, which is how ECR, GCR and private registries work. Each distinct image is asked about once.
  • Attestations. BuildKit adds unknown/unknown entries next to the real images. They are not platforms, and the tool ignores them.
  • What the registry does not answer. No credentials, no network or a templated reference (${REGISTRY}/app:1): the image is reported as not verified (GR005), and the workload stays unverified. The tool never assumes everything is fine.

Without --check-images, nothing goes to the network, and no workload is called `ready`. A clean configuration is unverified, because nothing confirms the images run on arm64. This is a design choice: a "ready" that nobody verified would be worse than no verdict.

How to use it

# files, Helm or kustomize
graviton-readiness scan ./k8s
helm template my-release ./chart | graviton-readiness scan -

# reading the images from the registry
graviton-readiness scan --check-images ./k8s

# the workloads and the Karpenter NodePools together
graviton-readiness scan ./k8s ./karpenter

# the plan, the live cluster and the SARIF
graviton-readiness plan --check-images ./k8s ./karpenter
graviton-readiness scan --cluster
graviton-readiness scan -f sarif ./k8s > gr.sarif

# output for CI and pull requests
graviton-readiness scan --check-images --fail-on not-ready ./k8s
graviton-readiness scan -f markdown ./k8s
graviton-readiness scan -f json ./k8s

--fail-on takes not-ready, unverified or caveats, and each level includes the ones before it. The exit codes are 0 (success), 1 (the requested level was reached) and 2 (a usage or runtime error).

An example of the output

This is the real result on the repository's testdata, which is a made-up namespace, without --check-images:

VERDICT     SCORE  WORKLOAD                         REPLICAS  CPU    FINDINGS
unverified  100    kube-system/daemonset/log-agent  -         -      GR008
unverified  100    shop/cronjob/report              -         -      -
not-ready   60     shop/deployment/api              2         1000m  GR001
unverified  100    shop/deployment/web              3         500m   -
not-ready   50     shop/deployment/worker           2         250m   GR003,GR006

5 workloads: 0 ready, 0 with caveats, 2 not ready, 3 unverified
Graviton potential: 0% of requested CPU (workloads ready or with caveats).
Unverified: 37% of requested CPU could move if the images support arm64.
Images were not checked: run with --check-images to read their platforms from the registry.

The reading is straightforward. api is pinned to amd64 by a nodeSelector (GR001), and worker has a -amd64 tag (GR003) and downloads a linux_amd64 binary in its args (GR006). web has no blocker, but nobody verified its image, so it is unverified, which is why the summary shows 0% confirmed potential and 37% of the CPU that still depends on the images.

The migration plan

graviton-readiness plan turns the verdicts into an order of work, with the CPU of each group:

  • Ready now: no blocker, no caveat, and every image verified for arm64. The first move, and a good pilot.
  • Ready after a check: ready, with a caveat that a person has to look at (an architecture written in a command, a GPU).
  • Ready after changes: blockers that a change in the manifest or in the image build removes. They are ordered by the CPU that would move divided by the number of changes: a big workload with one change goes before a small one with four. It is a way to order the work, not a forecast of savings.
  • Verify first: nothing blocks it in the configuration, but its images were not checked, or the registry did not answer. Never counted in the potential.
  • Keep on amd64: needs an architecture that Graviton does not have (Windows).
  • Already on Graviton: can only land on arm64 nodes (it selects arm64, or a pool that only has arm64 nodes) and has no blocker. It is done, so it is left out of what remains to move: without this group, the workloads that were already migrated would show up as "ready now".

Each change is the first sentence of the fix of the rule, with what the rule found. This is the real output on the repository's testdata, without --check-images:

GRAVITON MIGRATION PLAN

Total workloads:          5
Ready now:                0
Ready after a check:      0
Ready after changes:      2
Verify first:             3
Keep on amd64:            0

Requested CPU:            4.0 vCPU
Movable now:              0.0 vCPU (0%)
Potential after changes:  2.5 vCPU (62%)
To verify first:          1.5 vCPU (37%), not counted in the potential

Phase 1: Ready after changes
  blockers that a change in the manifest or in the image build removes. The best return for the effort first.
  1. shop/deployment/api (2.0 vCPU)
       GR001  Remove the kubernetes.io/arch constraint, or allow both architectures.
  2. shop/deployment/worker (0.5 vCPU)
       GR003  Use the multi-arch tag, the one without the architecture in it, or the arm64 build, and keep the tag identical across architectures so one manifest serves both.

Phase 2: Verify first
  nothing blocks it in the configuration, but its images were not checked, or the registry did not answer.
  shop/deployment/web (1.5 vCPU)
            Run with --check-images, to read the platforms of its images from the registry.
  kube-system/daemonset/log-agent
            Run with --check-images, to read the platforms of its images from the registry.
  shop/cronjob/report
            Run with --check-images, to read the platforms of its images from the registry.

Images were not checked: run with --check-images to read their platforms from the registry.

Live cluster, read-only

With --cluster, the tool reads Deployments, StatefulSets, DaemonSets, CronJobs, Argo Rollouts, Karpenter NodePools, EC2NodeClasses and the nodes straight from the cluster instead of from files. Only list calls are sent (the tests check that no other verb is used), paged by 500 and with a few requests a second, one resource at a time. A resource that is not installed is skipped with a note.

graviton-readiness scan --cluster
graviton-readiness plan --cluster --check-images

The labels of the nodes solve what manifests cannot: the pools that are not Karpenter's. An AWS managed node group, a GKE or AKS pool gets the architecture that its own nodes report, and the message says it was observed (observed on the cluster: all 2 node(s) labeled eks.amazonaws.com/nodegroup=core are amd64). It is a picture of today's nodes: a pool scaled to zero says nothing, and a pool with both architectures does not block.

SARIF and a GitHub Action

-f sarif writes a SARIF 2.1.0 log for GitHub code scanning. Each finding points at the line of the field that caused it (the image:, the node selector), and the fingerprint ignores the line, so an alert survives an edit that moves the manifest. The log was validated against the official SARIF schema.

The repository is also a GitHub Action. It installs the release with a checksum check and runs one scan that writes every report, so the registries are asked once. It comments the report and, if you ask, the plan on the pull request: each comment is found by a marker, and the Action never edits the comment of another tool. It uploads the SARIF and only fails the pipeline at the fail-on level after the comments.

- uses: robertasolimandonofreo/graviton-readiness@v0.3.2
  with:
    path: ./k8s ./karpenter
    check-images: "true"
    comment: "true"
    comment-plan: "true"
    fail-on: not-ready

How to read the numbers

The summary weighs the verdicts by the CPU the workloads request. Two points so you are not misled:

  • Only long-running services count. The figures cover Deployments, StatefulSets and Pods. A DaemonSet has no replica count to weigh by, and a Job requests CPU only while it runs.
  • Unverified stays apart. Graviton potential is the CPU of workloads that are ready or ready with caveats, with their images verified. unverified CPU is shown separately and never added to it.

A common mistake in mixed clusters

When the cluster has both x86 and arm64 nodes, a workload with no architecture constraint and an amd64-only image can be scheduled on an arm64 node and fail. GR004 says so when that is the case: the workload is not pinned to amd64, so it can land on an arm64 node. Until the image is multi-arch, pin kubernetes.io/arch: amd64 on the workload.

To build an image that serves both architectures:

docker buildx build --platform linux/amd64,linux/arm64 -t registry.example.com/api:1.4 --push .

Keep the same tag on both architectures, so one manifest serves both, and drop the -amd64 from the name.

What it does not do

  • Ready is not safe. The tool reads configuration and the registry. It does not know whether the application has native dependencies, JIT or assembly that behave differently on arm64. Test a ready workload on a Graviton node before moving production.
  • An image with `linux/arm64` was published for arm64, which does not guarantee it works.
  • It does not measure cost or performance. It says whether the workload can run on Graviton, not whether it is worth it for that workload.
  • No Helm values. Render charts with helm template. Karpenter NodePools are read from the manifests or the cluster, and the other pools come from the nodes (with --cluster) or are declared with --node-pool. The EC2NodeClass only proves Windows: the architecture of a custom AMI is in AWS and is not read.
  • The rules are heuristics. GR006 looks for the text amd64 or x86_64, so it can miss a hidden architecture or flag a harmless mention.
  • AWS first. The instance-type rules know the Graviton families. Ampere on Azure and Axion on Google Cloud are not recognized by GR002.

The tool was tested with an in-memory registry, which covers multi-arch indexes, single-platform images, attestations and invalid references, and also against real public images.

Next steps for your cluster

  1. Run graviton-readiness scan --check-images and look at what blocks first.
  2. Fix the cheap things: drop the amd64 nodeSelector, point the workload at an arm64 NodePool, replace the -amd64 tag with the multi-arch one, publish the image with buildx.
  3. Resolve the unverified ones: log in to the registry and run again.
  4. Move stateless, ready workloads first, in a staging environment, and observe before widening.
  5. Use the Kubernetes savings calculator, which has a Graviton lever, to estimate the gain, and the Karpenter NodePool generator to create a pool that accepts both architectures.

Graviton and Spot combine, and spot-readiness answers the equivalent question for Spot. Before either, it is worth cutting inflated requests: with honest requests, Karpenter consolidation has something to pack.

The tool's page, with install and example, is at /en/tools/graviton-readiness, and the code is at github.com/robertasolimandonofreo/graviton-readiness, under Apache-2.0. If you want me to review your cluster and build a prioritized plan, the efficiency assessment is the starting point.

Want to apply this to your environment?

Book a free 30-minute call. I will look at your setup and tell you where to start.

Book a call

Get new articles by email

One email when a new article on infrastructure, backend and AI is published. No spam, unsubscribe anytime.