Which workloads in your cluster can run on Spot? An open source tool to find out

"Use Spot for interruption-tolerant workloads" is common FinOps advice. What is rarely said is how to find out which workloads those are. In practice the answer is usually a guess, and a wrong guess becomes an incident.

spot-readiness is an open source command-line tool that turns that question into an objective check on workload configuration.

What it checks

It reads Deployments, StatefulSets, Argo Rollouts, Jobs, CronJobs, KEDA ScaledJobs and Pods, from YAML files or straight from the cluster, and applies nineteen rules. Each workload gets a score and a verdict: ready, ready with caveats or not ready.

Rules SR001, SR004, SR005, SR006, SR009, SR014 and SR015 apply to long-running services. Rules SR010 to SR013 apply to Jobs, CronJobs and ScaledJobs. The rest apply to every workload. A KEDA ScaledObject counts as the minimum replica count, like an HPA.

The 120-second limit in SR003 comes from the AWS Spot interruption notice, which is two minutes. A longer shutdown is cut short.

One detail that often causes confusion: a PodDisruptionBudget does not stop Spot capacity from being reclaimed. It only limits how many replicas a drain removes at the same time. That is why SR014 exists: a budget with minAvailable equal to the replicas allows zero evictions, and a drain, a Karpenter consolidation or a node upgrade hangs on it.

How to use it

# files, Helm or kustomize
spot-readiness scan ./k8s
helm template my-release ./chart | spot-readiness scan -

# the current cluster (read-only)
spot-readiness scan --context prod -n shop

# output for spreadsheets or CI
spot-readiness scan -f csv ./k8s > workloads.csv
spot-readiness scan --fail-on not-ready ./k8s

# the fix for each finding, and the order to move things
spot-readiness scan --suggest ./k8s
spot-readiness plan ./k8s

At the end, the tool shows the share of requested CPU that sits in ready or ready-with-caveats workloads, with a link to the Kubernetes savings calculator already filled in with that number.

Can the workload move between capacity pools?

Surviving a restart is not enough. Spot works when Karpenter can choose among many capacity pools, and AWS advises being flexible across instance types and zones. Rules SR016 to SR019 read nodeSelector and required nodeAffinity and flag a workload pinned to one zone, to one instance type or to a handful of families. Each workload also gets a flexibility score, in its own table:

SPOT FLEXIBILITY

WORKLOAD                   SCORE  AZS  FAMILIES  CONSTRAINTS
ml/deployment/ml-training  10     1    g5        high

What is the cluster doing right now?

In cluster mode (spot-readiness scan with no paths) the tool also reads the running pods and nodes, and answers a different question: if this Spot node disappeared now, what would happen?

• Runtime risk follows each pod to its node, zone, capacity type and instance type. It warns when all replicas share one node or one zone, and shows the blast radius of losing one node.
• Interruption handling checks whether Karpenter runs with an interruption queue, whether the AWS Node Termination Handler runs, and how many workloads have PodDisruptionBudget problems or need more than 120 s to terminate. The result is a cluster score out of 100.
• With --karpenter, it reads NodePools, NodeClaims and EC2NodeClasses and applies seven KR rules: a single zone, too few instance families or types, over-restrictive requirements, disruption budgets, a Spot-only pool with no on-demand alternative and a missing minValues.

INTERRUPTION HANDLING

Karpenter detected             ✓
Interruption queue configured  ✓
Node Termination Handler       -
PDB issues                     2
Workloads >120s termination    1

Cluster interruption readiness: 86/100

The values above are illustrative. The runtime view is a snapshot and does not change the workload score.

From findings to a plan

spot-readiness plan turns the findings into an order of work, in four groups, with the CPU of each:

SPOT MIGRATION PLAN

Total workloads:             3
Ready now:                   1
Ready after changes:         1
Manual review:               1
Keep on on-demand:           0

Requested CPU:               5.5 vCPU
Immediately movable to Spot: 1.5 vCPU (27%)
Potential after changes:     3.5 vCPU (64%)
Waiting for a review:        2 vCPU (not in the potential)

• Ready now: no blocker and no caveat. Listed safest first, a good order for a pilot.
• Ready after changes: blockers or caveats that a configuration change fixes. They are ranked by priority: the vCPU that would move, times the score, times the confidence, divided by the number of changes, so a big workload that needs one change comes before a small one that needs four.
• Manual review: persistent state, such as a StatefulSet with volumes, or a database run as a Deployment with one replica and a volume. Some of these can use Spot (Kafka with a replication factor of 3, a PostgreSQL cluster spread over zones) and the manifest does not show which. The plan does not decide: it lists what to look at (storage topology, replication and quorum, recovery time, zone dependency), and keeps their CPU out of the potential.
• Keep on on-demand: data that lives on the node (hostPath), which no setting fixes.

With --suggest, each finding also prints a snippet to adapt, already using the names of your workload, and a link to the documentation. With --pricing, the plan adds a monthly estimate (what the requested CPU costs now, after phase 1 and after phase 2). The tool has no prices of its own: you give them, or it reads the AWS Price List and the Spot Instance Advisor. It is an estimate from requested CPU, not a billing forecast. The tool never changes your files.

How far can you trust a verdict?

The score counts what the manifest shows. Each workload also has a confidence (high, medium or low) that counts what it cannot show: whether an application replicates and recovers its own state, whether a job checkpoints, which nodes sit behind a scheduling constraint. A stateless Deployment with nothing hidden has high confidence. A database with a volume has low confidence, whatever its score. Confidence never changes the score: it tells you which "ready" to test first.

spot-readiness explain SR014 documents any rule, with a problematic and a recommended configuration. spot-readiness explain shop/api ./k8s shows how a workload got its score: the 100 points, each finding and what it cost, why the verdict is what it is, and the confidence and its reasons.

Using it in a team

• A config file. .spot-readiness.yaml sets thresholds, excludes namespaces, changes the severity of a rule or turns it off, and ignores a finding for one workload with a written reason, so a team can adopt the tool without accepting every heuristic.
• Fail only on new problems. A project that already has 30 findings cannot fail its build on them. scan --baseline base.json --fail-on-regression compares with an earlier report and fails only when a blocker or a caveat is new. The GitHub Action does this on its own on pull requests: it scans the base branch, shows the score change and the new and resolved findings in the comment, and fails with fail-on-regression: "true".
• On the manifest line. -f sarif writes a report for GitHub Code Scanning, so each finding shows on the line it is about, such as the container without a readinessProbe. In the Action, upload-sarif: "true" does the upload in the same step.
• A JSON you can depend on. The JSON starts with schemaVersion and toolVersion, and --baseline refuses a report with a newer schema instead of comparing against a layout it cannot read.

A real case

I ran the tool on the manifests of a real environment, with production and staging, first with the thirteen original rules and again with the current version (0.8.0). The figures below are from the current version. It analyzed 114 long-running services, plus 120 Jobs, CronJobs and ScaledJobs (and ignored a few objects, such as DaemonSets). The services first:

• No workload ready and 3 ready with caveats. The other 111 were not ready. Two of the three have a PodDisruptionBudget that lacks unhealthyPodEvictionPolicy: AlwaysAllow (SR015), a rule that did not exist in the first run, when one of them counted as ready.
• 111 workloads (97%) ran with a single replica, which blocks them under SR001. It was the dominant problem, and every other finding came after it.
• 75 had no readinessProbe (SR006), and 101 had no preStop (SR009).
• 16 used persistent volumes (SR007), 12 had a shutdown period above 120 s (SR003) and 14 were pinned to a single zone (SR016).
• Spot potential came out at 12% of requested CPU of the services. Jobs, CronJobs and ScaledJobs are not part of that number, because they request CPU only while they run.

Jobs, CronJobs and ScaledJobs tell a different story: of the 120, 41 were ready, 78 ready with caveats and 1 not ready (backoffLimit: 0). The caveats came almost entirely from a shutdown period above 120 s (SR003). More than half had activeDeadlineSeconds (SR011) and none had a podFailurePolicy to ignore interruptions (SR012). Batch workloads are often the best Spot candidates, as long as they checkpoint, and that is the one thing the tool cannot see.

The plan puts those numbers in order. Of the 234 workloads, 41 (all Jobs) are ready now, 177 are ready after changes and 16 go to manual review: 14 Deployments with a single replica and a volume (MongoDB, PostgreSQL, RabbitMQ, n8n and SFTP, among others) and 2 StatefulSets. The 16 request 4.9 of the 12.3 vCPU of the services, so they are the part nobody can decide from the manifest. With the review group left out, moving the services after the changes would put 60% of the requested CPU on Spot (7.3 vCPU), against 12% without the changes. Before the review group existed, the plan counted those workloads as "ready after changes" and showed 94%, which was too optimistic.

The practical reading for the services: in this environment, the first gain for anyone who wants to use Spot is not configuring Karpenter. It is giving small services a second replica and adding a readinessProbe, cheap changes that remove the main blocker.

A few fixes came out of that test, and they are in the tool: workloads scaled to zero are now ignored, and Deployments that an HPA or KEDA keeps above spec.replicas now use that minimum. Pods that only run a pause image, used to reserve capacity for Karpenter, are also ignored, because they are not applications.

What it does not do

• Ready does not mean safe. The tool only sees configuration. It does not know whether the application handles SIGTERM, keeps state in memory or survives a restart.
• The rules are heuristics. A workload can pass every rule and still be a poor fit for Spot. Read the findings instead of following the score alone.
• It does not measure real cost: OpenCost and Kubecost exist for that. The --pricing estimate uses requested CPU and one reference instance type, and leaves out memory, storage, network and discounts. spot-readiness answers a different question, which workloads can move.
• DaemonSets are not analyzed, and the tool cannot see whether a Job checkpoints its progress.
• For NodePools it counts the instance types and families that are listed. It has no instance catalog, so a pool limited only by category or size shows "any". The interruption queue is read from the Karpenter controller, not from your AWS account.
• The plan and its groups are heuristics from the manifests. They do not know how much a move saves in real money, or how your application behaves.
• It is AWS first. The workload rules apply to any cluster, but the capacity type, the interruption handling and the NodePool rules read AWS and Karpenter labels. GKE and AKS Spot are not recognized yet.

Next steps for your cluster

1. Run spot-readiness scan and look first at what blocks, usually SR001.
2. Fix the cheap things: replicas, readinessProbe, preStop and topology spread.
3. Run spot-readiness plan to see what can move now and what to keep on On-Demand.
4. Open the manual review group with whoever owns the data: replication, quorum and recovery time decide it.
5. Use the savings calculator to estimate the gain, and the Karpenter NodePool generator to create a Spot NodePool with an On-Demand fallback.
6. Move the high-scoring workloads first, in staging, and watch the interruptions before you expand.

The code is at github.com/robertasolimandonofreo/spot-readiness, licensed Apache-2.0. If you want me to review your cluster and build a prioritized plan, the efficiency assessment is the starting point.