How to size Kubernetes networking for ephemeral AI agent sandboxes

Consider an enterprise platform where artificial intelligence agents run tests, call APIs, and process documents in disposable environments. Each task creates a sandbox, downloads dependencies, opens connections, and finishes minutes later. Capacity planning is not just about how many environments remain active. It must also account for how many environments start, communicate, and disappear every second.

This article complements Designing ephemeral sandboxes for AI agents on Kubernetes, which covers isolation, Jobs, and expiration. Here the subject is network capacity. It develops a sizing model for that scenario. All numerical examples are hypothetical, not benchmarks. The goal is to translate concurrency, workload turnover, and external dependencies into explicit budgets for addressing, DNS, connections, and traffic.

1. Define what you are sizing

An ephemeral sandbox is a time-bounded execution environment with controlled disposal. In this design, each execution uses a Pod, usually managed by a Job. Containers within a Pod share networking, so adding a sidecar does not inherently require another IP address. An ordinary Pod is not a sufficient security boundary for every hostile-code scenario; runtime, node, or cluster isolation may also be necessary. (kubernetes.io)

The main networking components are:

• CNI — Container Network Interface: integrates Pod network configuration with the execution environment.
• IPAM — IP Address Management: manages address allocation and release.
• Service: provides stable discovery and access to endpoints; its address space is distinct from the Pod address pool.
• NetworkPolicy: declares communication permissions enforced by a compatible networking implementation.
• DNS — Domain Name System: resolves service names and external destinations. (kubernetes.io)

For planning purposes, separate six dimensions: available IPs, sandbox starts per second, DNS queries, concurrent connections, new connections per second, and bytes/packets per second. Do not assume that spare CPU implies spare network capacity.

2. Reference architecture

A proposed production architecture is:

Client → platform API → queue → execution controller → Job → isolated Pod

Outbound communication follows a separate path:

Pod → network policy → egress proxy → NAT or private endpoint → authorized destination

The controller tracks execution state, enforces tenant limits, and admits work against available capacity. The proxy centralizes authentication, auditing, and destination authorization. Prefer a small set of reusable policies for execution profiles rather than generating a unique policy for every task.

In this design, sandboxes do not receive individual load balancers. Interactive access goes through a shared, authenticated gateway. Sandboxes also do not connect directly to the platform's operational database; they submit results through a restricted interface.

Important design decision: the proxy is a security boundary, not merely a forwarding service. Its configuration must reject unauthorized internal destinations, metadata services, unsafe redirects, and attempts to bypass authorization through DNS resolution. These controls need dedicated negative tests.

3. Sizing: from arrival rate to an IP budget

Concurrency and turnover

Start with a steady-state approximation:

N_average ≈ lambda_average × T_average
N_design = ceil(lambda_sustained_peak × T_planned × H)

lambda is the number of admitted executions per second, T is their occupancy time, and H is an explicit planning allowance. The second expression is a heuristic: validate it against the actual duration distribution and the shape of the arrival burst. Multiplying unrelated percentiles does not automatically yield a concurrency percentile.

Hypothetical inputs:

Estimated average concurrency is 8 × 120 = 960. The design budget is 16 × 120 × 1.25 = 2,400 concurrent executions.

Also measure the interval between application completion and actual IP reuse. Do not confuse it with Job object retention: an object remaining in the API does not prove that its address is still allocated. Observe Pod lifecycle and runtime cleanup separately. (kubernetes.io)

Per-zone IP capacity, including failure conditions

Build a budget for each address pool:

required_IPs = Pods_holding_IPs
               + IPs_pending_release
               + idle_preallocated_IPs
               + infrastructure
               + upgrade_overlap

Assume one Pod per execution and a measured post-execution address retention time of 30 seconds:

IPs_pending_release = 16 × 30 = 480

To tolerate losing one of three zones, assume the workload is redistributed evenly across the two survivors. Each surviving zone would need:

(2,400 + 480) / 2 = 1,440 IPs

Now add a hypothetical 256 idle preallocated addresses and 150 addresses for infrastructure and maintenance overlap: 1,440 + 256 + 150 = 1,846 addresses per surviving zone. These allowances must come from inventory and measured IPAM behavior; they are not recommended defaults.

A conventional AWS IPv4 subnet reserves five addresses. A /21 therefore provides 2,043 usable addresses, leaving only 197 in this example. A /20, with 4,091 usable addresses, would be a more comfortable candidate, subject to the organization's addressing plan. Do not generalize these numbers to every provider or CNI mode. (docs.aws.amazon.com)

Density is more than available addresses

Also calculate:

sandbox_Pods_per_node = min(
  Pod_limit_minus_system_Pods,
  CPU_capacity,
  memory_capacity,
  validated_network_capacity
)

If the validated operating ceiling is 72 sandboxes per node, 1,200 active executions in one zone require at least ceil(1,200 / 72) = 17 nodes. This is an illustrative floor; reservations and other consumers can raise the requirement. CPU and memory requests influence scheduling, but they do not provide a universal network bandwidth reservation. (kubernetes.io)

4. Amazon EKS considerations

On Amazon Elastic Kubernetes Service, Amazon VPC CNI IPv4 prefix mode delegates /28 prefixes containing 16 addresses. Allocation requires contiguous space; a subnet with many scattered free addresses may still be unable to supply a prefix. When configured, WARM_IP_TARGET and MINIMUM_IP_TARGET take precedence over WARM_PREFIX_TARGET. (docs.aws.amazon.com)

My operational recommendation is to test instance type, Pod limits, and preallocation policy together in a canary node group. Compare startup time, reserved addresses, and allocation errors before expanding. Prefix delegation does not remove the need to size the subnet.

Overlay networking changes infrastructure address consumption, but Pod address blocks still require planning. Encapsulation also requires attention to the maximum transmission unit, or MTU. Native routing is an alternative that places different requirements on the underlying network. (docs.cilium.io)

5. Bandwidth, DNS, and connections

Bandwidth: model execution startup

Assume each new execution causes 60 decimal MB of effective network transfer, accounting for node image caching and application downloads:

B_startup = 16 × 60 MB/s = 960 MB/s = 7.68 Gbit/s
B_with_allowance = 7.68 × 1.30 = 9.984 Gbit/s

This excludes ongoing application traffic, logs, and replication. Allocate the estimate to the actual paths: registry to node, Pod to proxy, and proxy to the internet. Do not attribute every image pull to the Pod's IP or charge every execution for a full image download when layers are cached.

Validate bytes per second and packets per second independently. Test large downloads, short API calls, streaming, and idle connections. A single bulk-throughput test should not be treated as a complete capacity certification.

DNS: count queries, not just sandboxes

A proposed estimate is:

DNS_QPS = starts_per_second × queries_per_start
          + active_Pods × queries_per_second_per_Pod

With 40 queries per start and two queries per second per sandbox, the example produces 16 × 40 + 2,400 × 2 = 5,440 queries/s. Specify whether the inputs represent application lookups or actual queries on the wire: search domains, ndots, A/AAAA lookups, caching, and retries change that relationship. (kubernetes.io)

NodeLocal DNSCache can reduce DNS traffic traversing the conventional Service and conntrack path. However, it introduces a per-node component that also needs capacity planning and monitoring. Match policy rules to the DNS path actually in use. (kubernetes.io)

Connections: budget every hop

At 12 concurrent connections per execution, the initial budget is 2,400 × 12 = 28,800 connections. That does not imply 28,800 external connections if the proxy pools upstream traffic. Nor does it mean the infrastructure as a whole tracks only that many states.

For each hop, estimate open connections and lingering states separately:

lingering_states ≈ connection_closures_per_second × average_retention_time

On Linux, nf_conntrack_count reports allocated flow entries and nf_conntrack_max sets the table limit. Do not raise the limit without considering memory, traffic patterns, and timeouts. Alternative dataplanes may also require monitoring their implementation-specific tables. (docs.kernel.org)

AWS NAT Gateway supports up to 55,000 simultaneous connections per IPv4 address to each unique destination, identified by destination IP, port, and protocol. This is not a cluster-wide connection limit. When using zonal NAT gateways, preserve zone-local egress and contingency capacity. (docs.aws.amazon.com)

Prefer bounded pools, connection reuse, deadlines, retries with backoff and jitter, and per-destination concurrency limits. Treat these as load controls to validate, not substitutes for measuring the external dependency.

6. Configuration: restricted outbound access

The following example creates a namespace and allows sandbox-labeled Pods to reach only DNS and an internal proxy. This is a teaching baseline, not a complete environment: the proxy, its Service, and its authorization rules must already exist.

apiVersion: v1
kind: Namespace
metadata:
  name: agent-sandboxes
  labels:
    pod-security.kubernetes.io/enforce: restricted
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns-and-egress-proxy
  namespace: agent-sandboxes
spec:
  podSelector:
    matchLabels:
      workload: agent-sandbox
  policyTypes:
    - Egress
  egress:
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
          podSelector:
            matchLabels:
              k8s-app: kube-dns
      ports:
        - protocol: UDP
          port: 53
        - protocol: TCP
          port: 53
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: platform-egress
          podSelector:
            matchLabels:
              app: egress-proxy
      ports:
        - protocol: TCP
          port: 3128

This policy assumes a default-deny is already applied to the namespace, as in Designing ephemeral sandboxes for AI agents on Kubernetes, which also covers runtime, identity, and lifecycle. The focus here is what that design does not cover: how much DNS and egress capacity the proxy needs. Namespace and Pod selectors within the same peer are combined, and the CNI must enforce NetworkPolicy. (kubernetes.io)

Adapt labels, proxy port, and DNS routing. This rule assumes CoreDNS is reached through the selected Pods, not NodeLocal DNSCache. If the proxy namespace also restricts ingress, add the corresponding permission there. Test UDP/TCP DNS and attempts at direct outbound access before admitting real tasks.

A complete Job for DNS validation

apiVersion: batch/v1
kind: Job
metadata:
  name: sandbox-dns-check
  namespace: agent-sandboxes
spec:
  backoffLimit: 0
  activeDeadlineSeconds: 60
  ttlSecondsAfterFinished: 300
  template:
    metadata:
      labels:
        workload: agent-sandbox
    spec:
      automountServiceAccountToken: false
      restartPolicy: Never
      securityContext:
        runAsNonRoot: true
        runAsUser: 10000
        runAsGroup: 10000
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: check
          image: busybox:1.37.0
          command:
            - sh
            - -c
            - nslookup kubernetes.default.svc.cluster.local
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop:
                - ALL
          resources:
            requests:
              cpu: 100m
              memory: 32Mi
            limits:
              cpu: 200m
              memory: 64Mi

This Job performs one DNS lookup. It neither runs an agent nor proves proxy connectivity. backoffLimit prevents automatic retries for this test, activeDeadlineSeconds bounds execution, and ttlSecondsAfterFinished enables subsequent Job cleanup. (kubernetes.io)

Adjust the cluster domain, resources, and image. In production, pin the approved image by digest. The example does not set runtimeClassName, so it adds no isolation beyond the default runtime. Select an installed and validated RuntimeClass when the threat model requires one.

Probes are intentionally omitted for this short-lived check. Persistent APIs and proxies should have readiness, liveness, and, where appropriate, startup probes. Avoid making liveness depend on an external provider; that converts an upstream outage into local restart churn. (kubernetes.io)

7. Production: admission, availability, and security

I recommend limiting execution admissions by tenant and zone before creating Pods. The admission budget should reflect the lowest available capacity across addressing, DNS, egress, and compute. At the limit, keep work in a queue with an explicit deadline or reject new requests rather than relying on retries alone.

Spread persistent component replicas across zones using topology spread or anti-affinity. Use a PodDisruptionBudget to manage voluntary disruptions, recognizing that it cannot prevent node or zone failure. For interruptible sandboxes, prefer checkpoints and idempotent re-execution. (kubernetes.io)

Separate the privileged controller from execution environments. Role-based access control, or RBAC, should prevent sandboxes from creating workloads or modifying policies. For untrusted code, evaluate sandboxed runtimes, dedicated nodes, or separate clusters according to criticality and compliance requirements. (kubernetes.io)

For AWS access, use workload-specific Identity and Access Management, or IAM, identities through EKS Pod Identity or IAM Roles for Service Accounts, subject to compatibility. Do not distribute node credentials or shared static access keys. (docs.aws.amazon.com)

As an operating policy, require TLS, short-lived credentials, Secret rotation, and token-free logs. Disabling the default token in the manifest does not configure an AWS identity; any identity integration must be added and tested explicitly.

8. Observability and capacity testing

Define a service-level objective, or SLO, for the time from admission to the first successful authorized network operation. The proposed breakdown is queue wait, scheduling, network setup, download, DNS, TCP connection, and TLS negotiation.

Organize dashboards by zone, execution profile, and destination:

For NAT Gateway, pay particular attention to ErrorPortAllocation, PacketsDropCount, ActiveConnectionCount, and IdleTimeoutCount. Correlate them with destination concentration and application behavior. (docs.aws.amazon.com)

One implementation could use Prometheus and Grafana for metrics, centralized logs, and platform-level tracing. Keep individual sandbox identifiers in logs and traces rather than adding them indiscriminately to metric labels.

The recommended test plan covers ramps, bursts, sustained churn, cold caches, slow dependencies, zone failure, and upgrades with overlapping capacity. Include denied-traffic tests. Acceptance should combine latency, errors, isolation, and recovery, not just the number of Pods created.

9. Symptom-driven troubleshooting

Use events, status, and logs to distinguish scheduling problems from Pod setup and application failures. (kubernetes.io)

kubectl get nodes
kubectl get pods -n agent-sandboxes -o wide
kubectl get events -n agent-sandboxes --sort-by=.metadata.creationTimestamp
kubectl describe job sandbox-dns-check -n agent-sandboxes
kubectl logs job/sandbox-dns-check -n agent-sandboxes
kubectl get networkpolicy -n agent-sandboxes
kubectl get pods -n kube-system -l k8s-app=kube-dns -o wide

On Amazon EKS using the standard Amazon VPC CNI DaemonSet, inspect an affected instance:

kubectl get pods -n kube-system -l k8s-app=aws-node -o wide
kubectl logs -n kube-system <aws-node-pod> -c aws-node --since=10m

Replace <aws-node-pod> with the Pod on the affected node. Investigate explicit hypotheses:

• Pod stuck during creation → possible IP or prefix shortage → inspect events and CNI logs → resolve address availability or fragmentation before adding nodes. (docs.aws.amazon.com)
• Name resolution times out → possible saturation or blocked DNS path → correlate queries, policies, and the actual resolver → fix the identified rule or capacity constraint. (kubernetes.io)
• Outbound connections fail under load → possible NAT port exhaustion → inspect `ErrorPortAllocation` → review pooling and egress distribution. (docs.aws.amazon.com)
• Large transfers fail over an overlay → possible MTU mismatch → test packet sizes and the path → correct MTU and the handling of messages required for path MTU discovery. (docs.cilium.io)

Avoid automatically restarting every sandbox. First constrain new admissions so that recovery traffic does not amplify the incident.

10. Cost, development, and maintenance

NAT Gateway charges include availability and processed data. Evaluate private endpoints for supported services and compare the complete cost, including endpoints and cross-zone traffic. Include caching and download reduction in FinOps experiments rather than assuming universal savings. (docs.aws.amazon.com)

For this architecture, measure cost per completed execution, including retries, logs, storage, and idle capacity. Reserve Spot capacity for recoverable tasks, while keeping critical DNS, proxy, and control capacity on a stable foundation. Do not lower resource requests merely to make utilization look better.

A development environment may use one replica and a smaller workload, but it should retain the same isolation rules. Production needs validated multi-zone behavior, health checks, alerts, recovery procedures, and tenant limits. Do not duplicate tasks with external side effects unless execution is idempotent.

Version CNI configuration, policies, DNS, and proxy settings through GitOps. Upgrade through a canary and retain previous node groups until connectivity and isolation checks pass. The proposed rollback procedure is to stop admissions, drain the new group, and redirect executions to the validated configuration. Do not assume every CNI change can be undone with a simple rollout command.

Back up and restore-test durable state: queues, metadata, configuration, and artifacts. Disposable Pods are not the recovery unit.

11. When to use this approach—and what to consider instead

This approach fits concurrent execution platforms with multiple tenants, audit requirements, and a team equipped to operate networking and isolation. A small worker pool may be simpler for a few trusted internal tasks.

This comparison is an evaluation framework, not a ranking. Run a proof of concept against the same traffic profile and threat model before choosing an implementation.

Practical summary

1. Size concurrency and startup rate independently.
2. Include IP release delays, preallocation, zone failure, and upgrades.
3. Model startup bandwidth, DNS, and connections by path and destination.
4. Restrict egress and test bypass attempts.
5. Control admissions before consuming the final unit of capacity.
6. Validate recovery and cost per execution, not throughput alone.

Conclusion

For ephemeral sandboxes, the platform's product is not merely a Pod. It is an execution with predictable preparation time, authorized connectivity, and verifiable disposal.

This design connects DevOps, SRE, Cloud, and Platform Engineering through measurable operating contracts. Networking becomes an explicit budget with tests, admission limits, and recovery procedures rather than an implicit prerequisite. That discipline allows agent workloads to scale without turning every demand spike into an unplanned capacity experiment.