Open source

rightsizing-check: which requests in your cluster are inflated

A Go CLI, Apache-2.0, that compares the CPU and memory requests in your manifests with observed usage and tells you what to set instead. By default it works offline: it reads files and never connects to the cluster. If you have a Prometheus, --prometheus and --window 7d fetch the usage for you.

What it does

  • Offline by default

    Reads manifests and an exported usage file (Prometheus JSON, kubectl top or CSV). Fits CI, reviewing a client's cluster from files, and isolated environments. If you have a Prometheus, --prometheus and --window fetch the usage, with GET requests only.

  • 11 rules

    Inflated CPU and memory requests, usage above the request, memory close to the limit, a missing request, missing data, a warning when an HPA or KEDA scaler would scale out at the recommended request and another when a VPA will overwrite the change. Each workload is oversized, undersized, ok or without data.

  • A recommendation with headroom

    CPU: p95 usage plus 15%. Memory: the maximum plus 20%. A request is inflated above 1.5 times the recommendation. Everything is configurable by flag.

  • What can be reclaimed

    Adds up the vCPU and GiB reclaimable in requests, with an optional cost line using your prices. It only becomes savings once nodes consolidate.

  • Live cluster and a GitHub Action

    With --cluster, it reads workloads, HPAs, KEDA ScaledObjects, VPAs and Argo Rollouts straight from the cluster, with list calls only, and the usage from metrics-server. The Action comments on the pull request and fails the pipeline.

  • The busiest replica decides

    With several replicas, the hottest pod sets the request: one busy replica needs the reservation even if the others idle.

Install

Binaries for Linux, macOS and Windows (amd64 and arm64), with checksums.

brew install robertasolimandonofreo/tap/rightsizing-check

The Docker image only reads the files you mount.

How to use it

Pass the manifests and a usage file. Nothing in your files is changed and nothing connects to the cluster.

rightsizing-check scan ./k8s --prometheus https://prometheus.example.com --window 7d
rightsizing-check scan --cluster --prometheus https://prometheus.example.com   # the cluster itself, read-only
rightsizing-check scan ./k8s --csv usage.csv
rightsizing-check scan ./k8s --prom-cpu cpu.json --prom-mem mem.json
rightsizing-check scan ./k8s --top top.txt -n shop
rightsizing-check scan ./k8s --csv usage.csv -v            # every finding
rightsizing-check scan ./k8s --csv usage.csv --cpu-price 25 --mem-price 4   # a cost line
rightsizing-check scan ./k8s --csv usage.csv --fail-on undersized          # exit 1 in CI
rightsizing-check explain RC001                            # what a rule means

Getting the usage

With a Prometheus, --prometheus and --window (for example 7d) fetch the usage for you, one query at a time. The window must include your busiest period. A long window is heavy: the CPU query is a subquery over the whole window, and above 7d the tool refuses unless you pass --prom-allow-long-window. Without a Prometheus it also takes the two exports by hand, a CSV and the output of kubectl top, which is the weakest: one instant.

# let the tool read it, over a window you choose (up to 7d):
rightsizing-check scan ./k8s --prometheus http://prometheus:9090 --window 7d

# or export it by hand, with the same two queries:
PROM=http://prometheus:9090

# CPU: the 95th percentile of the per-container usage, in cores
curl -sG "$PROM/api/v1/query" --data-urlencode \
  'query=max by (namespace, pod, container) (quantile_over_time(0.95, rate(container_cpu_usage_seconds_total{container!="",container!="POD"}[5m])[7d:5m]))' > cpu.json

# Memory: the maximum working set, in bytes
curl -sG "$PROM/api/v1/query" --data-urlencode \
  'query=max by (namespace, pod, container) (max_over_time(container_memory_working_set_bytes{container!="",container!="POD"}[7d]))' > mem.json

Example output

$ rightsizing-check scan ./k8s --prometheus https://prometheus.example.com --window 7d
STATUS      WORKLOAD                     CONTAINER  REPLICAS  CPU REQ  CPU USE  CPU REC  MEM REQ  MEM USE  MEM REC  FINDINGS
oversized   shop/deployment/api          app        3         1000m    120m     140m     2Gi      600Mi    720Mi    RC001,RC002
oversized   kube-system/daemonset/agent  agent      -         500m     40m      50m      512Mi    120Mi    144Mi    RC001,RC002
undersized  shop/deployment/worker       worker     2         100m     320m     370m     256Mi    576Mi    692Mi    RC003,RC004,RC005
undersized  shop/statefulset/db          db         1         -        450m     520m     -        1.5Gi    1844Mi   RC006
ok          shop/deployment/web          web        2         150m     120m     140m     200Mi    144Mi    176Mi    -
no-data     shop/cronjob/report          report     -         -        ?        ?        -        ?        ?        RC007

6 workloads: 2 oversized, 2 undersized, 1 ok, 1 without data
Reclaimable requests: 2.6 vCPU (73% of the 3.5 requested) and 3.9 GiB (56% of 6.9).

Real output from the repository's testdata.

What it does not do

  • A recommendation is a starting point: it is as good as the window of your data. A monthly job or a spike you did not capture makes a request look inflated.
  • HPA: a HorizontalPodAutoscaler that scales on utilization uses the request as the denominator. The tool reads HPAs and KEDA ScaledObjects from the manifests and warns (RC009) when the recommended request would make the HPA scale out. An HPA that exists only in the cluster is not seen: export it and pass the file.
  • Memory is not compressible: a limit below the usage kills the container. Do not shrink a container flagged RC005 before you know why it is close to its limit.
  • A long window is heavy: the CPU query is a subquery over the whole window, and on a small Prometheus a 14-day window got the pod evicted for lack of memory. Above 7d the tool refuses unless you pass --prom-allow-long-window.
  • Pods are matched to workloads by name. Jobs, CronJobs and CRDs that create pods are left out.

After the diagnosis

Want help reading the result on your cluster?

I review your environment, validate the recommendations against your data and build a cost reduction plan.