GKE Agent Sandbox for Reinforcement Learning: gVisor, warm pools and cold-starts

Introduction

Reinforcement Learning (RL) and AI agent workloads pose a different problem from traditional applications.

A conventional API runs known code that was packaged and validated in the CI/CD pipeline. On an agent platform, the workload may run dynamically produced scripts, libraries chosen at runtime, simulation environments and even code generated by a Large Language Model (LLM).

That changes the threat model.

Running this kind of code directly in a regular Pod means sharing the Kubernetes node's kernel with other workloads. Even with Linux namespaces, seccomp and capabilities, there is still a significant attack surface between the container and the kernel.

Google Kubernetes Engine (GKE) Agent Sandbox was created for exactly this kind of environment. It adds sandbox lifecycle management on top of Kubernetes and uses gVisor-based isolation, making it possible to create ephemeral or persistent environments for agents, notebooks, untrusted code and AI applications.

Agent Sandbox is different from the traditional GKE Sandbox feature:

GKE Sandbox
    ↓
gVisor / runtime isolation

GKE Agent Sandbox
    ↓
Controller + CRDs + lifecycle management
    ↓
GKE Sandbox / gVisor

Agent Sandbox currently uses Kubernetes resources such as Sandbox, SandboxTemplate, SandboxWarmPool and SandboxClaim. The warm pool model keeps pre-initialized environments around to significantly reduce perceived provisioning time.

In this lab we will build an architecture like the following:

                        ┌──────────────────────┐
                        │ RL Orchestrator      │
                        │ API / Trainer        │
                        └──────────┬───────────┘
                                   │
                           SandboxClaim
                                   │
                                   ▼
                       ┌──────────────────────┐
                       │ Agent Sandbox        │
                       │ Controller           │
                       └──────────┬───────────┘
                                  │
                    ┌─────────────┴──────────────┐
                    │ SandboxWarmPool            │
                    │ SandboxTemplate            │
                    └─────────────┬──────────────┘
                                  │
                                  ▼
                     ┌─────────────────────────┐
                     │ Pod / RuntimeClass      │
                     │ gVisor                  │
                     └─────────────┬───────────┘
                                   │
                         GKE sandbox node pool
                                   │
                       ┌───────────┴───────────┐
                       │ Gymnasium Environment │
                       │ RL Agent / Python     │
                       └───────────────────────┘

The goal is not just to run CartPole. It is to build a model that can later host evaluation environments, agent-generated code and distributed RL workloads.

What is GKE Agent Sandbox

GKE Agent Sandbox is a management layer for isolated environments running on GKE.

A Sandbox is essentially an isolated Kubernetes environment managed by a controller. It lets you control identity, storage, lifecycle and network exposure without forcing the application requesting the environment to manipulate Pods directly.

The project uses Kubernetes Custom Resource Definitions (CRDs). The main resources are:

Sandbox

The isolated environment itself. It owns the Pod that will run the workload.

SandboxTemplate

A blueprint. Instead of repeating the full Pod definition every time an agent needs an environment, we can create python-rl-template and reuse it for hundreds of sandboxes.

SandboxWarmPool

Keeps a fixed number of pre-initialized environments. For example, replicas: 5 means the controller tries to keep five sandboxes available for immediate use.

SandboxClaim

A request for an environment. The application asks "I need a sandbox" and the controller binds an available Sandbox from the warm pool.

The current Agent Sandbox documentation recommends this model when fast startup matters.

Why this matters for Reinforcement Learning

A simplified RL pipeline looks like this:

Policy
  │
  ▼
Action
  │
  ▼
Environment
  │
  ▼
Observation + Reward
  │
  ▼
Trainer
  │
  ▼
New Policy

The problem appears when we want to run thousands of independent environments:

environment-0001
environment-0002
environment-0003
...
environment-5000

Each environment may run:

• Python;
• simulators;
• external tools;
• code produced by the agent;
• different dependencies;
• potentially untrusted scripts.

Agent Sandbox turns those environments into isolated units of infrastructure:

Trainer
   │
   ├── SandboxClaim → Gym Env 001
   ├── SandboxClaim → Gym Env 002
   ├── SandboxClaim → Gym Env 003
   └── SandboxClaim → Gym Env N

This cleanly separates the training control plane from environment execution.

gVisor and the sandboxed runtime

Traditional containers share the host's Linux kernel:

Container
   ↓
Linux namespaces
   ↓
Host Kernel

With gVisor:

Container
   ↓
gVisor user-space kernel
   ↓
seccomp / namespaces
   ↓
Host Kernel

gVisor intercepts most system calls and reduces direct interaction between the container process and the host kernel.

In Kubernetes, it is selected with:

runtimeClassName: gvisor

On GKE, Sandbox-enabled nodes automatically get the label:

sandbox.gke.io/runtime: gvisor

and the taint:

sandbox.gke.io/runtime=gvisor:NoSchedule

This prevents regular workloads from being accidentally scheduled onto the nodes meant for sandboxes.

Lab architecture

We will use GKE Standard to make the node pool separation explicit:

GKE Cluster
│
├── system-pool
│     ├── controllers
│     ├── observability
│     └── RL orchestrator
│
└── rl-sandbox-pool
      ├── gVisor
      ├── taint sandbox.gke.io/runtime=gvisor
      ├── taint workload=rl-sandbox
      │
      ├── Sandbox 1
      ├── Sandbox 2
      ├── Sandbox 3
      └── Sandbox N

This separation matters because untrusted code should not compete for the same node pool used by controllers or critical applications.

GKE's own documentation warns that untrusted workloads must be factored into your isolation strategy even when gVisor is in use.

Creating the GKE cluster

At the time of writing, the GKE documentation uses the Agent Sandbox v1beta1 API on clusters running 1.36.3-gke.1767000 or later. Minimum versions may change, so check the documentation before creating permanent environments.

Define:

export PROJECT_ID="$(gcloud config get-value project)"
export CLUSTER_NAME="rl-agent-sandbox"
export LOCATION="us-central1-a"
export CLUSTER_VERSION="1.36.3-gke.1767000"

Create the cluster:

gcloud beta container clusters create "${CLUSTER_NAME}" \
  --location="${LOCATION}" \
  --cluster-version="${CLUSTER_VERSION}" \
  --num-nodes=1 \
  --machine-type=e2-standard-2 \
  --workload-pool="${PROJECT_ID}.svc.id.goog"

In this lab we keep a single initial node to reduce cost. In production we would normally use a regional cluster across multiple zones.

Creating the dedicated sandbox node pool

Now we create the pool used for RL:

gcloud container node-pools create rl-sandbox-pool \
  --cluster="${CLUSTER_NAME}" \
  --location="${LOCATION}" \
  --machine-type=e2-standard-4 \
  --image-type=cos_containerd \
  --sandbox=type=gvisor \
  --enable-autoscaling \
  --min-nodes=0 \
  --max-nodes=10 \
  --node-labels=workload=rl-sandbox \
  --node-taints=workload=rl-sandbox:NoSchedule

The key flag is --sandbox=type=gvisor. GKE Sandbox requires the cos_containerd node image.

We also add our own taint, workload=rl-sandbox:NoSchedule, so there are two scheduling barriers:

sandbox.gke.io/runtime=gvisor:NoSchedule
workload=rl-sandbox:NoSchedule

This makes the intent of the node pool explicit.

Enabling Agent Sandbox

After the gVisor pool exists:

gcloud beta container clusters update "${CLUSTER_NAME}" \
  --location="${LOCATION}" \
  --enable-agent-sandbox

Validate:

gcloud beta container clusters describe "${CLUSTER_NAME}" \
  --location="${LOCATION}" \
  --format="value(addonsConfig.agentSandboxConfig.enabled)"

The expected result is True. This is the procedure Google currently documents for GKE Standard.

Creating the namespace

apiVersion: v1
kind: Namespace
metadata:
  name: rl-sandbox
  labels:
    workload-type: agent-sandbox

kubectl apply -f namespace.yaml

In production, this namespace should also get specific RBAC, NetworkPolicy, ResourceQuota, LimitRange and Pod Security policies.

SandboxTemplate for RL

Now we create the Python runtime:

apiVersion: extensions.agents.x-k8s.io/v1beta1
kind: SandboxTemplate
metadata:
  name: rl-python-runtime
  namespace: rl-sandbox
spec:
  podTemplate:
    metadata:
      labels:
        app: rl-environment
        workload: rl-sandbox
    spec:
      runtimeClassName: gvisor

      automountServiceAccountToken: false

      securityContext:
        runAsNonRoot: true
        runAsUser: 1000

      nodeSelector:
        sandbox.gke.io/runtime: gvisor
        workload: rl-sandbox

      tolerations:
        - key: sandbox.gke.io/runtime
          operator: Equal
          value: gvisor
          effect: NoSchedule

        - key: workload
          operator: Equal
          value: rl-sandbox
          effect: NoSchedule

      containers:
        - name: rl-runtime

          image: us-central1-docker.pkg.dev/PROJECT_ID/rl/rl-runtime:1.0.0

          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop:
                - ALL

          resources:
            requests:
              cpu: "500m"
              memory: "512Mi"

            limits:
              cpu: "2"
              memory: "2Gi"

          volumeMounts:
            - name: tmp
              mountPath: /tmp

      volumes:
        - name: tmp
          emptyDir:
            sizeLimit: 1Gi

      restartPolicy: OnFailure

This configuration does four particularly important things:

1. runtimeClassName: gvisor makes the workload use the sandboxed runtime;
2. the nodeSelector with sandbox.gke.io/runtime: gvisor ensures the Pod lands in the right pool;
3. the tolerations let the workload pass the existing taints;
4. the securityContext blocks several forms of privilege escalation.

Agent Sandbox has admission policies that require settings such as runAsNonRoot, capabilities.drop: ["ALL"], resource limits, gVisor, node selector and toleration. Settings such as privileged, hostNetwork, hostPID, hostIPC, hostPath and extra capabilities are forbidden.

Creating a Warm Pool

Now we can keep ready-to-use environments around:

apiVersion: extensions.agents.x-k8s.io/v1beta1
kind: SandboxWarmPool
metadata:
  name: rl-python-warmpool
  namespace: rl-sandbox
spec:
  replicas: 3

  sandboxTemplateRef:
    name: rl-python-runtime

kubectl apply -f rl-warmpool.yaml
kubectl get sandboxwarmpool -n rl-sandbox

Conceptually:

Warm Pool
│
├── ready sandbox
├── ready sandbox
└── ready sandbox

When an application requests an environment, it no longer has to go through:

schedule
→ image pull
→ container creation
→ Python startup
→ dependency initialization

That is exactly where the cold-start gain comes from.

Requesting a sandbox

An application can request an environment through a SandboxClaim:

apiVersion: extensions.agents.x-k8s.io/v1beta1
kind: SandboxClaim
metadata:
  name: rl-session-001
  namespace: rl-sandbox
spec:
  warmPoolRef:
    name: rl-python-warmpool

kubectl apply -f claim.yaml

kubectl get \
  sandboxwarmpool,sandboxclaim,sandbox,pod \
  -n rl-sandbox

The controller binds the claim to an environment from the warm pool and rebuilds the available capacity in the background. GKE recommends this model for startup-latency-sensitive workloads.

Runtime image

A simple requirements.txt:

gymnasium[classic-control]
numpy
openai
opentelemetry-api
opentelemetry-sdk
opentelemetry-exporter-otlp-proto-grpc

Dockerfile:

FROM python:3.12-slim

RUN useradd \
    --create-home \
    --uid 1000 \
    sandbox

WORKDIR /app

COPY requirements.txt .

RUN pip install \
    --no-cache-dir \
    -r requirements.txt

COPY app.py .

RUN chown -R sandbox:sandbox /app

USER sandbox

CMD ["python", "app.py"]

Avoid running pip install ... at container startup. It dramatically hurts cold-start, makes builds harder to reproduce and increases dependence on external connectivity.

Gymnasium example

The old OpenAI Gym project evolved, and the ecosystem now mainly uses Gymnasium, maintained by the Farama Foundation.

The core interface is still:

observation, info = env.reset()

observation, reward, terminated, truncated, info = env.step(action)

Since the modern API versions, terminated and truncated replace the old single done concept.

A simple example:

import gymnasium as gym


env = gym.make("CartPole-v1")

observation, info = env.reset(seed=42)

total_reward = 0.0


for step in range(500):

    action = env.action_space.sample()

    observation, reward, terminated, truncated, info = env.step(action)

    total_reward += reward

    if terminated or truncated:
        break


print({
    "steps": step + 1,
    "reward": total_reward,
})

env.close()

This code can run entirely inside the Sandbox.

OpenAI + Gymnasium example

We can also use a model as an experimental policy.

This example is deliberately didactic: it is not a cost-effective way to train CartPole. Querying an LLM on every timestep adds far more latency and cost than a local neural policy.

Still, the pattern is useful when the environment represents software tasks, browsers, shells or complex tools.

import os

import gymnasium as gym

from openai import OpenAI


client = OpenAI(
    api_key=os.environ["OPENAI_API_KEY"]
)


env = gym.make("CartPole-v1")

observation, info = env.reset(seed=42)

total_reward = 0


for step in range(50):

    prompt = f"""
You control a CartPole environment.

Observation:
{observation.tolist()}

The possible actions are:

0 = move left
1 = move right

Choose a single action.

Answer only:
0
or
1
"""

    response = client.responses.create(
        model="gpt-5.6-luna",
        input=prompt,
    )

    result = response.output_text.strip()

    if result not in {"0", "1"}:
        action = env.action_space.sample()
    else:
        action = int(result)

    observation, reward, terminated, truncated, info = env.step(action)

    total_reward += reward

    print(
        {
            "step": step,
            "action": action,
            "reward": reward,
            "total_reward": total_reward,
        }
    )

    if terminated or truncated:
        break


env.close()

The current OpenAI API uses the client.responses.create(...) pattern through the Responses API. The architecture looks like this:

Sandbox
   │
   ├── Gymnasium environment
   │
   ├── observation
   │
   ▼
OpenAI API
   │
   ├── decision
   ▼
action
   │
   ▼
Gymnasium
   │
   └── reward

For real RL workloads, a more efficient architecture usually looks like:

Local Policy Model
        ↓
Environment
        ↓
Rollout Buffer
        ↓
Trainer

and reserves external LLMs for tasks such as reward evaluation, planning, reasoning, code generation and environment generation.

A realistic enterprise scenario

Imagine an internal platform building agents that can automatically fix applications. Each training run receives a broken application. The agent can:

read code
→ run tests
→ change files
→ run commands
→ observe the error
→ try a new solution

Running all of this in the trainer's own Pod would be dangerous. With Agent Sandbox:

RL Trainer
     │
     ├── Claim #001
     │      └── repository clone
     │      └── generated code
     │      └── tests
     │
     ├── Claim #002
     │
     └── Claim #003

Each episode runs in its own sandbox. A reward function could use:

tests passed            +10
lint passed              +2
build passed             +5
timeout                  -3
crash                    -5
security test failed    -10

When the episode ends:

SandboxClaim deleted
         ↓
environment discarded
         ↓
new clean sandbox

This reduces contamination between episodes and improves reproducibility.

Production environment

A design closer to production would be:

                    ┌─────────────────────┐
                    │ RL Control Plane    │
                    │ Trainer / API       │
                    └──────────┬──────────┘
                               │
                ┌──────────────┴───────────────┐
                │                              │
                ▼                              ▼
         CPU Sandbox Pool               GPU Sandbox Pool
        e2 / n2 / c3 etc.             L4 / A100 / H100
                │                              │
          gVisor Runtime                  gVisor Runtime
                │                              │
             Env Pods                      Training Pods
                │                              │
                └──────────────┬───────────────┘
                               │
                        Observability
                               │
             ┌─────────────────┼─────────────────┐
             ▼                 ▼                 ▼
        Prometheus            Loki        OpenTelemetry

GKE Sandbox also supports certain GPU and TPU types with compatible GKE versions, so isolation and hardware acceleration are not necessarily mutually exclusive.

A mature architecture would typically split environment execution, policy inference, training, reward model, orchestrator and observability into independent workloads.

Observability for RL

CPU and memory are not enough to observe RL.

Google currently documents instrumenting RL workloads on GKE with OpenTelemetry, including metrics specific to sampling, training, reward and synchronization. Some important metrics:

Simple instrumentation:

import time

from opentelemetry import metrics


meter = metrics.get_meter("rl-sandbox")


step_duration = meter.create_histogram(
    "rl.step.duration",
    unit="ms",
)


reward_counter = meter.create_counter(
    "rl.reward.total",
)


start = time.perf_counter()

observation, reward, terminated, truncated, info = env.step(action)

elapsed_ms = (
    time.perf_counter() - start
) * 1000


step_duration.record(
    elapsed_ms,
    attributes={
        "rl.environment.name": "CartPole-v1",
        "rl.run.id": "experiment-001",
    },
)


reward_counter.add(
    reward,
    attributes={
        "rl.environment.name": "CartPole-v1",
    },
)

Prometheus, Grafana, Loki and OpenTelemetry

In a stack independent of Cloud Monitoring:

Sandbox
   │
   ├── stdout
   │      ↓
   │     Loki
   │
   ├── /metrics
   │      ↓
   │   Prometheus
   │
   └── OTLP
          ↓
     OpenTelemetry Collector
          ↓
   Tempo / Cloud Trace

Dashboards should correlate infrastructure and RL: mean reward versus CPU usage, step latency, sandbox cold-start and node provisioning latency. That helps tell a bad algorithm apart from slow infrastructure.

Measuring cold-start

Cold-start should not be treated as a single metric. We can break it down:

T0 Sandbox requested
 │
 ├── scheduler latency
 ├── node provisioning
 ├── image pull
 ├── container creation
 ├── Python startup
 ├── environment initialization
 │
 ▼
T1 Sandbox ready

GKE directly provides the kubernetes.io/pod/latencies/pod_first_ready metric, which measures the time between Pod creation and its first Ready state, including image pull.

There is also kubernetes.io/node/latencies/startup, which measures from VM creation until the node is available in Kubernetes.

These two metrics are essential because they let you separate Pod cold-start from Node cold-start.

Cold-start comparison

The lab should compare at least four scenarios:

A. Regular Pod + containerd
B. gVisor Pod without warm pool
C. Agent Sandbox without a pre-warmed instance
D. Agent Sandbox + WarmPool

It is important not to publish numbers as universal. Cold-start varies with image size, node cache, VM type, autoscaler, registry, application dependencies, readiness probe, region and cluster pressure.

A hypothetical lab result, only to illustrate the format of the analysis, could look like this:

These numbers are not official GKE benchmarks. They must be replaced with real measurements from your environment.

The expected behavior, however, explains the architecture well. The biggest cost is usually not containerd versus gVisor, but:

no node available
     ↓
autoscaler
     ↓
Compute Engine VM
     ↓
node Ready
     ↓
image pull
     ↓
sandbox

That is why warm pools and minimum capacity can have a much larger impact on user experience than small runtime tweaks.

Recommended benchmark

Run at least 30 or 50 samples per scenario. Collect:

request_timestamp
sandbox_assigned_timestamp
pod_created_timestamp
pod_ready_timestamp
environment_ready_timestamp
first_action_timestamp

Compute P50, P90, P95 and P99. Never compare a single run.

Also run two distinct experiments: image cached and image not cached. Otherwise the result may only measure Artifact Registry/image pull rather than the runtime itself.

Warm Pool vs scale to zero

There is a direct trade-off:

• Warm Pool: lower latency, higher idle cost;
• Scale-to-zero: lower cost, higher cold-start.

A common strategy is:

peak hours:     warm pool = 20
normal hours:   warm pool = 5
overnight:      warm pool = 0

Capacity should be determined by observing the session creation rate. If the arrival rate is 10 sandboxes/s and replenishment takes 5 s, a warm pool of only three environments will be consumed almost instantly.

Security

Agent Sandbox does not remove the need for additional controls. The recommendation is to combine:

gVisor
+
NetworkPolicy
+
RBAC
+
resource limits
+
admission policies
+
secret isolation

Agent Sandbox has a security model with admission policies that restrict dangerous configurations and establishes a network isolation posture for sandboxes.

Service Accounts

A particularly important setting is automountServiceAccountToken: false. Code running inside the sandbox should not automatically receive Kubernetes credentials.

API keys

Avoid putting a privileged OpenAI key directly into the environment when the executed code can be modified by the agent. A safer architecture:

Sandbox
   │
   ▼
Internal AI Gateway
   │
   ├── authentication
   ├── quota
   ├── model allowlist
   ├── rate limit
   └── auditing
   │
   ▼
OpenAI API

This way the sandbox never receives the primary credential.

NetworkPolicy

A reasonable posture is to start with deny all and open only the necessary destinations. For workloads that run AI-generated code, unrestricted egress is a particularly important risk.

sandbox
   ├─X→ metadata
   ├─X→ internal databases
   ├─X→ Kubernetes API
   ├─X→ other namespaces
   │
   └──→ AI Gateway

This model turns external access into an explicitly granted capability.

Requests and Limits

Another common mistake is creating sandboxes with requests far below their limits:

requests:
  cpu: 10m
limits:
  cpu: 4

This allows a theoretical density much higher than real capacity. Imagine 100 environments with request = 100m and limit = 2 CPU:

Scheduler:  100 × 100m = 10 CPU
Potential:  100 × 2 CPU = 200 CPU

If they all start processing at once, severe contention follows. For RL, monitor requested CPU, actual CPU, CPU throttling, working set memory, OOMKills, episodes/s and steps/s, and size based on real execution.

FinOps

The biggest cost drivers are usually:

number of sandboxes
lifetime
warm pool
GPU/TPU
idle nodes
oversized requests
excessive telemetry
external model calls

For interruption-tolerant environments, Spot VMs can be very attractive. One possible design:

Critical trainer                  → On-Demand
RL evaluation environments        → Spot
Low-latency warm pool             → minimum On-Demand
Burst capacity                    → Spot

This split avoids using premium capacity for workloads that are easy to rebuild.

Development vs production

Troubleshooting

Sandbox is not scheduled

kubectl get pods -n rl-sandbox -o wide
kubectl describe pod <pod> -n rl-sandbox

Look for messages such as untolerated taint or node affinity conflict. Validate the nodes and their taints:

kubectl get nodes --show-labels
kubectl describe node <node> | grep -A5 Taints

Runtime is not using gVisor

Check from the outside:

kubectl get pod <pod> \
  -n rl-sandbox \
  -o jsonpath='{.spec.runtimeClassName}'

Expected: gvisor. The GKE documentation recommends verifying the RuntimeClass from Kubernetes, rather than trusting information emitted by the potentially untrusted workload.

SandboxTemplate rejected

kubectl get events \
  -n rl-sandbox \
  --sort-by=.metadata.creationTimestamp

Possible causes:

automountServiceAccountToken missing
runAsNonRoot missing
limits missing
capabilities not dropped
hostPath configured
privileged=true
runtimeClass other than gvisor

Warm pool is not available

kubectl get sandboxwarmpool -n rl-sandbox -o yaml
kubectl get sandbox -n rl-sandbox
kubectl get pods -n rl-sandbox

If the Pods are Pending, investigate capacity and autoscaling.

High cold-start

First check whether it is Node cold-start or Pod cold-start by analyzing kubernetes.io/node/latencies/startup and kubernetes.io/pod/latencies/pod_first_ready. Then investigate image size, image pull, init containers, application initialization, autoscaler, resource pressure and readiness probe.

What to avoid

Do not run agent-produced code directly in privileged Pods. Avoid privileged: true. Do not use hostNetwork: true, hostPID: true or hostIPC: true.

Do not mount via hostPath:

/var/run/docker.sock
/
/proc
/sys

Also avoid using the same node pool for untrusted agents, databases, controllers and critical observability. Even with gVisor, blast-radius isolation remains a fundamental part of the architecture.

Best practices

For workloads like this, a reasonable baseline would be:

1. a node pool exclusive to sandboxes;
2. runtimeClassName: gvisor;
3. taints and node selectors;
4. runAsNonRoot;
5. dropped capabilities;
6. read-only filesystem where possible;
7. automountServiceAccountToken: false;
8. explicit CPU and memory limits;
9. default-deny NetworkPolicy;
10. a warm pool for interactive workloads;
11. small, pre-built images;
12. RL observability separate from infrastructure observability;
13. P50/P95/P99 startup metrics;
14. Spot only for interruption-tolerant workloads;
15. automatic expiry of abandoned sandboxes.

When to use it (and when not to)

Agent Sandbox is especially interesting for:

AI coding agents
RL environments
browser agents
terminal agents
automated code evaluation
isolated notebooks
CI running untrusted code
multi-tenant developer environments
agent evaluation farms

The model also makes sense when environments need to be created and destroyed programmatically at scale.

On the other hand, it adds operational complexity. If your application only runs a Java API, a Go worker or a Node.js frontend with known, controlled code, a regular Kubernetes Deployment is probably enough. It also makes no sense to turn every HTTP call into a Sandbox. For trusted workloads, Deployment, Job, CronJob and StatefulSet remain simpler abstractions.

Practical summary

RL Orchestrator
      │
      ▼
SandboxClaim
      │
      ▼
SandboxWarmPool
      │
      ▼
Sandbox
      │
      ▼
Pod
      │
runtimeClassName: gvisor
      │
      ▼
Dedicated GKE Node Pool
      │
      ▼
Gymnasium / Agent Code

The main value of Agent Sandbox is not simply running a container with Python. It is turning potentially untrusted environments into a managed platform resource.

The main performance lesson is just as important: cold-start is not just container startup. It is the result of capacity, scheduler, node provisioning, image pull, runtime and application initialization. Warm pools attack exactly the critical path by removing several of those steps from the user's request.

Conclusion

Reinforcement Learning and autonomous agents introduce an operating model different from traditional applications. Kubernetes remains an excellent scheduler, but Pods isolated only by containers are not necessarily the right security boundary when workloads can run dynamically generated code.

The combination of Agent Sandbox, gVisor, dedicated node pools, taints/tolerations, NetworkPolicy, resource isolation, Warm Pools and OpenTelemetry creates a much better foundation for modern agent execution platforms.

From a Platform Engineering standpoint the result is especially interesting, because the AI team doesn't need to know the details of nodes, RuntimeClasses or security policies. The application simply requests a SandboxClaim, and the platform handles scheduling, isolation, capacity, runtime, security and observability.

For RL workloads at scale, the natural evolution is to separate CPU environments from GPU/TPU trainers, introduce queues and quotas, use vectorized environments when per-episode isolation isn't needed, and build autoscaling driven by the real rate of SandboxClaims.

At that point, Kubernetes stops being just the place where the model runs and becomes the control plane of an agent and Reinforcement Learning platform.

Technical references

• Google Kubernetes Engine — GKE Agent Sandbox: architecture and concepts.
• Google Kubernetes Engine — enabling and configuring Agent Sandbox.
• Google Kubernetes Engine — Sandbox, SandboxTemplate, SandboxClaim and SandboxWarmPool CRDs.
• Google Kubernetes Engine — GKE Sandbox and gVisor.
• Google Kubernetes Engine — Reinforcement Learning observability with OpenTelemetry.
• Google Kubernetes Engine — Pod and node startup metrics.
• Gymnasium — environments API and vectorized environments.
• OpenAI — Responses API for current integrations.