Temporal on Cloud Run: architecture and criteria for on-demand workers

An integration platform may wait hours for a partner file and then receive thousands of records to validate and persist. Keeping peak capacity running looks wasteful. Turning every consumer off, however, raises a more important question: what brings them back when work becomes available?

Traditional Temporal workers retrieve tasks through long polling. They do not automatically receive an HTTP invocation for every task. Putting the container on Cloud Run therefore does not, by itself, provide demand-driven scaling: the execution model and activation mechanism must be designed together. (docs.temporal.io)

The central recommendation is to separate durability, execution, and capacity control. Consider scale-to-zero only after proving how capacity returns, including after timers, retries, and failures.

1. What survives a worker failure

A Workflow defines orchestration. Its event history supports state reconstruction through replay of deterministic code. External calls normally belong in Activities, rather than deterministic Workflow code. An execution can outlive any individual container. (docs.temporal.io)

The relevant building blocks are:

• Temporal Service: maintains durable execution state and coordination; this can be Temporal Cloud or a deployment operated by your team.
• Worker: an application process built with a Software Development Kit, or SDK, that executes registered code.
• Task Queue: distributes tasks that workers consume.
• Namespace: provides a logical scope for executions.

Running workers is not the same as hosting the Temporal Service. This article assumes the durable service already exists outside the processing containers. (github.com)

Activities can be retried. External effects should therefore be idempotent: repeating an operation with the same business key should not duplicate a payment or import. Heartbeats can carry checkpoints, but application code must read those details and implement recovery. Temporal does not restore an Activity's process memory. (docs.temporal.io)

2. Services, Jobs, and Worker Pools are different execution models

Cloud Run Services

Services are an option for HTTP applications. To keep polling outside requests, use instance-based billing, which allocates CPU throughout the instance lifetime. Configuring minimum instances alone does not replace this setting. (docs.cloud.google.com)

Temporal backlog is not a native input to the HTTP autoscaler. A Service at zero needs a request to start again; CPU utilization cannot trigger anything when no instance exists. For a continuously available worker using this model, configure a positive minimum and address capacity expansion separately. (docs.cloud.google.com)

The container must also satisfy the Service's HTTP contract by listening on its configured port. Adding a dummy endpoint does not turn Temporal tasks into HTTP traffic or solve activation. (docs.cloud.google.com)

Cloud Run Jobs

Jobs suit finite execution. Each task has a timeout; documentation allows up to 168 hours for tasks without GPUs, but that does not make a Job a permanent process. (docs.cloud.google.com)

My recommendation is to use Jobs only with an explicit batch boundary or execution budget. An indefinitely polling worker does not know when the batch is complete. Define when polling stops, how active work finishes or gets interrupted, and what starts the next execution. A Cloud Run task and a Temporal Task are not the same object.

Cloud Run Worker Pools

Worker Pools are a more direct fit for continuously polling processes. Their instance count can be set manually, including to zero, or adjusted by an external controller. Creating a pool does not automatically install a Temporal-aware autoscaler. Requested instances remain billable even without useful work. (docs.cloud.google.com)

Temporal Serverless Workers integration

In the documentation reviewed for this article, Temporal support for Cloud Run is in Public Preview. A Worker Controller Instance, or WCI, adjusts Worker Pools through the Cloud Run administrative API; it does not invoke a container per task. The integration requires Worker Versioning. Use a pool for each version and do not share its queues with independently operated permanent workers. (docs.temporal.io)

Before adoption, verify availability, compatible versions, permissions, and support requirements. The official deployment procedure covers identities, pool creation, association with a Worker Deployment Version, and promotion of the appropriate version. Adding an environment variable to a worker is not sufficient. (docs.temporal.io)

3. Proposed architecture

For an enterprise import platform, I recommend separating responsive orchestration from elastic Activity execution:

Client → API → Temporal Service
                    ↓
              Workflow queue
                    ↑ polling
          Orchestration workers
                    ↓ schedules Activities
               Import queue
                    ↑ polling
          Activity Worker Pool
                    ↓
        Object Storage + PostgreSQL

Controller → Cloud Run administrative API → pool capacity

This design uses the ability to route Activities to a different Task Queue from the Workflow. Workers establish connections to Temporal; they do not need a public inbound endpoint to receive tasks. (docs.temporal.io)

Proposed decision: keep positive orchestration capacity when responsive queries and fast continuation matter. Allow the import queue to reach zero only when its additional latency is acceptable. Give each queue a capacity policy based on criticality rather than applying one platform-wide setting.

There are two control paths:

1. Temporal integration: the WCI participates in capacity control and can reactivate workers when tasks become available. This is a Temporal mechanism, not Cloud Run HTTP autoscaling. (docs.temporal.io)
2. Custom controller: an external component reads demand signals and changes the instance count. Cloud Run documents external scaling patterns; adapting signals and reconciliation to Temporal remains your team's responsibility. (docs.cloud.google.com)

For the custom-controller path, I propose these engineering requirements:

• run the controller outside the capacity it can reduce to zero;
• reconcile periodically rather than relying on a single arrival notification;
• account for work becoming available after timers, retries, and Activity completions;
• enforce limits, cooldown, and explicit handling of unavailable metrics;
• assign one owner to the desired instance count;
• test the race between observing an empty queue and receiving new work.

These are architectural acceptance criteria, not capabilities automatically installed by the following example.

4. Deploying an initial Worker Pool

The following example establishes fixed capacity, without WCI or automatic scale-to-zero. It assumes a published image whose entrypoint starts the worker and reads the listed variables. The application must configure TLS, authentication, and Activity registration through its SDK; environment variable names do not implement those behaviors themselves.

Prepare the project, region, Service Account, repository, and secret. Replace the illustrative values:

export PROJECT_ID='your-project'
export REGION='us-central1'
export POOL='temporal-import-v1'
export IMAGE="${REGION}-docker.pkg.dev/${PROJECT_ID}/workers/import-worker:1.0.0"
export RUNNER_SA="temporal-worker@${PROJECT_ID}.iam.gserviceaccount.com"
export TEMPORAL_ADDRESS='your-temporal-endpoint:7233'
export TEMPORAL_NAMESPACE='your-namespace'

gcloud run worker-pools deploy "$POOL" \
  --project="$PROJECT_ID" \
  --region="$REGION" \
  --image="$IMAGE" \
  --instances=2 \
  --cpu=1 \
  --memory=1Gi \
  --service-account="$RUNNER_SA" \
  --set-env-vars="TEMPORAL_ADDRESS=${TEMPORAL_ADDRESS},TEMPORAL_NAMESPACE=${TEMPORAL_NAMESPACE},TEMPORAL_TASK_QUEUE=imports" \
  --set-secrets='TEMPORAL_API_KEY=temporal-api-key:1'

These are supported Google Cloud CLI options. Two instances, one CPU, and 1 GiB are teaching values, not a universal sizing recommendation. For production, prefer an image digest and determine resource allocation through testing. (docs.cloud.google.com)

The secret must exist, and the runtime identity must be allowed to access it. A numeric secret version makes credential rollout explicit. Rotate credentials through a controlled deployment without placing their values in the repository. (docs.cloud.google.com)

Do not reduce this pool to zero before implementing and testing reactivation. To adopt WCI, follow the integration procedure and remove competing controllers. This deployment command alone does not configure Serverless Workers. (docs.temporal.io)

5. Criteria for on-demand execution

Use this table as a decision checklist, not a performance guarantee:

For initial capacity planning, I propose this approximation:

instances ≈ ceil((arrival rate × average duration) /
                 (concurrency per instance × target utilization))

This is a planning estimate, not the product's contractual scaling algorithm. In a hypothetical example, 10 Activities/s, a 2-second average duration, 5 concurrent executions per instance, and a 70% utilization target suggest 6 instances. Validate the estimate against duration distributions, retries, and database limits.

Execution slots, pollers, and instances are separate dimensions. More pollers do not create CPU capacity or remove downstream saturation. Measure useful throughput per worker and configure concurrency in the SDK. (docs.temporal.io)

6. Failure handling, timeouts, and shutdown

Distinguish the Activity timeout settings:

• Start-to-Close: maximum duration of one attempt.
• Schedule-to-Close: total budget, including waiting and retries.
• Heartbeat Timeout: detects missing progress reports.
• Schedule-to-Start: time waiting for a worker; it is not a substitute for fixing insufficient capacity.

Include activation time in the waiting budget. Overly aggressive timeouts can turn expected cold starts into failures. (docs.temporal.io)

The documented Worker Pool contract sends SIGTERM and allows 10 seconds before SIGKILL. A longer SDK graceful-shutdown setting does not extend that platform window. (docs.cloud.google.com)

On termination, initiate worker shutdown so polling stops and active tasks are handled according to the SDK's behavior. Test with actual signals rather than relying only on local application exits. (docs.temporal.io)

In the proposed design, long Activities should persist partial results and tolerate repetition. A heartbeat reports progress; it is not a distributed transaction with PostgreSQL. I recommend combining idempotency keys, uniqueness constraints, and checkpoints with the destination system's semantics. (docs.temporal.io)

7. An everyday implementation scenario

Problem: a team imports partner catalogs. Long gaps between files alternate with concentrated bursts. The goal is to reduce idle capacity without losing execution traceability.

Proposed solution: the API starts one Workflow per import. The Workflow schedules Activities on a dedicated queue. Elastic workers process file partitions and persist results using a key composed of the import identifier and item identifier.

Proposed implementation: keep files outside Workflow history and pass references and identifiers; limit PostgreSQL connections; record checkpoints; reserve an activation allowance. In staging, terminate containers after a write but before Activity completion is acknowledged.

Expected outcome, not a measured result: recovery without duplicate effects, traceable execution, and less time spent running idle capacity. Publish savings or latency improvements only after comparable measurements exist.

8. Kubernetes as an operational alternative

A Kubernetes Deployment is an alternative when the team needs more direct control over draining, zone distribution, and execution policies. The manifest below is an illustrative baseline containing complete definitions for the objects shown. It does not install Temporal or create credentials. (kubernetes.io)

Image prerequisites: an environment-configured worker, TLS enabled, and a health server on 8080. The health paths are an application contract, not built-in Temporal endpoints. Provision the temporal-credentials Secret with an api-key entry beforehand.

apiVersion: v1
kind: Namespace
metadata:
  name: temporal-workers
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: import-worker
  namespace: temporal-workers
automountServiceAccountToken: false
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: import-worker
  namespace: temporal-workers
spec:
  replicas: 3
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 0
      maxSurge: 1
  selector:
    matchLabels:
      app: import-worker
  template:
    metadata:
      labels:
        app: import-worker
    spec:
      serviceAccountName: import-worker
      terminationGracePeriodSeconds: 60
      securityContext:
        runAsNonRoot: true
        runAsUser: 10001
        runAsGroup: 10001
        seccompProfile:
          type: RuntimeDefault
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: topology.kubernetes.io/zone
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
            matchLabels:
              app: import-worker
      containers:
        - name: worker
          image: us-central1-docker.pkg.dev/PROJECT_ID/workers/import-worker:1.0.0
          ports:
            - name: health
              containerPort: 8080
          env:
            - name: TEMPORAL_ADDRESS
              value: "REPLACE_WITH_TEMPORAL_ENDPOINT:7233"
            - name: TEMPORAL_NAMESPACE
              value: "REPLACE_WITH_NAMESPACE"
            - name: TEMPORAL_TASK_QUEUE
              value: "imports"
            - name: TEMPORAL_API_KEY
              valueFrom:
                secretKeyRef:
                  name: temporal-credentials
                  key: api-key
          resources:
            requests:
              cpu: 500m
              memory: 512Mi
            limits:
              cpu: "1"
              memory: 1Gi
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop: [ALL]
          startupProbe:
            httpGet:
              path: /health/startup
              port: health
            periodSeconds: 5
            failureThreshold: 24
          readinessProbe:
            httpGet:
              path: /health/ready
              port: health
            periodSeconds: 5
          livenessProbe:
            httpGet:
              path: /health/live
              port: health
            periodSeconds: 10
          volumeMounts:
            - name: tmp
              mountPath: /tmp
      volumes:
        - name: tmp
          emptyDir:
            sizeLimit: 256Mi
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: import-worker
  namespace: temporal-workers
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: import-worker

Replace the image, endpoints, secret references, and resource values. Requests influence scheduling; limits constrain consumption. Insufficient CPU can increase latency, while undersized memory can cause termination. (kubernetes.io)

Zone distribution depends on eligible nodes. DoNotSchedule can leave Pods pending when compatible capacity is unavailable. The PodDisruptionBudget limits simultaneous voluntary disruptions but does not prevent a zone failure. (kubernetes.io)

Liveness should evaluate local health rather than trigger cascading restarts during a temporary Temporal outage. Readiness does not automatically stop polling: the application must implement that behavior. A Kubernetes Service is not required to consume the queue. (kubernetes.io)

9. Security, observability, and maintenance

Security

Separate the identity executing Activities from the identity changing capacity. In the integration, the controller needs permissions such as run.workerPools.get and run.workerPools.update; the runtime identity needs access to its secrets. Do not make the worker a project administrator. (docs.temporal.io)

Configure Transport Layer Security, or TLS, and authentication supported by the Temporal endpoint. Reuse client connections rather than reconnecting for every task. (docs.temporal.io)

As a security baseline, I recommend narrowly scoped database and storage access, rehearsed rotation, environment isolation, and exclusion of credentials from logs. On Kubernetes, review role-based access control, or RBAC, and network policies against actual dependencies.

Observability

Monitor schedule-to-start latency, backlog, slot capacity, Activity failures, and execution duration. Low CPU usage does not establish health while tasks are waiting. Correlate worker metrics with available capacity and queue behavior. (docs.temporal.io)

For an operational dashboard, I propose tracking:

• time from demand arrival to the first processed task;
• desired versus observed instances;
• controller errors and age of its last reconciliation;
• retries and duplicate effects prevented through idempotency;
• database connections and downstream saturation;
• cost per completed import.

Define a Service Level Objective, or SLO, for work starting and finishing. Test alerts with zero workers too: metrics emitted only by live processes cannot fully verify the activation mechanism.

Upgrades and rollback

Replay representative histories before changing Workflows. Worker Versioning supports routing executions by version; rollback must respect compatibility rather than merely restore an image. (docs.temporal.io)

With the Cloud Run integration, retain older version pools while executions remain associated with them. Do not silently replace the image behind a pool associated with a live version. (docs.temporal.io)

For the continuous integration and delivery pipeline, or CI/CD, I recommend replay, interruption, rotation, cold-start, and recovery tests. For self-hosted Temporal, require persistence backup and restoration procedures; restarting workers is not a replacement for recovering the durable service.

10. Troubleshooting

These checks follow the integration's troubleshooting flow. (docs.temporal.io)

gcloud run worker-pools describe "$POOL" \
  --project="$PROJECT_ID" --region="$REGION" --format=yaml

kubectl get pods -n temporal-workers
kubectl describe deployment import-worker -n temporal-workers
kubectl logs deployment/import-worker -n temporal-workers
kubectl get events -n temporal-workers --sort-by=.metadata.creationTimestamp

For a restarted Pod, use kubectl logs POD_NAME -n temporal-workers --previous. A successful rollout is not proof that workers can process tasks.

11. Costs, development, and alternatives

Include instance runtime, Temporal, networking, storage, logs, and scaling control in the financial model. Instance-based billing covers the instance lifetime; scale-to-zero does not eliminate the other components' costs. Do not treat hypothetical pricing as a budget. (cloud.google.com)

My FinOps recommendation is to compare three experiments: small fixed capacity, minimum capacity with expansion, and zero with reactivation. Measure cost per completed execution, including retries and repeated work.

For development, one instance and simplified observability may be sufficient for learning. For production, I recommend additional gates: capacity limits, health checks, alerts, isolated credentials, tested restoration, and documented rollback. Production does not require zero; it requires predictable behavior.

Avoid scale-to-zero when activation compromises the SLO, utilization is continuous, or recovery remains unproven. Also avoid polling on request-dependent CPU, hardcoded secrets, arbitrary capacity limits, and competing autoscalers.

Practical summary

• Separate the Temporal Service, workers, and capacity controller.
• Choose Worker Pools for continuous consumption; do not assume an HTTP Service observes Temporal queues.
• Require a verifiable path back from zero.
• Address interruption, idempotency, and checkpoints before optimizing cost.
• Make the decision per queue, SLO, and operational maturity.

Conclusion

Temporal on Cloud Run is an execution-architecture decision, not merely a container-hosting choice. A useful Platform Engineering approach is to offer standardized permanent, elastic, and finite execution profiles with explicit adoption criteria. DevOps automates delivery, SRE validates recovery and service objectives, and FinOps establishes whether elasticity improves cost per outcome. The strongest design is the one whose activation, interruption, and recovery behavior the team can demonstrate.