LLMJacking: how compromised credentials drive unexpected LLM costs

An application can remain healthy while someone else uses its credentials to invoke language models directly through the provider. Unauthorized traffic does not have to cross your Ingress, application API, or gateway: a compromised identity may be enough to authorize inference billed to your account. That is the core problem behind the LLMJacking activity documented by Sysdig's research team. (sysdig.com)

For platform teams, the operational question extends beyond the price of a response. Who can authorize that spending, where can the identity be used, and how quickly can its consumption be stopped?

This article develops a defensive architecture using Amazon Bedrock and Amazon Elastic Kubernetes Service, or Amazon EKS. The focus is identity compromise, consumption outside the approved request path, and financial containment. For content policies and sensitive-data handling, see the complementary article on AI proxies and sensitive-data redaction.

What LLMJacking means

LLMJacking is the unauthorized use of large language model services through compromised access. In research published on May 6, 2024, Sysdig described stolen cloud credentials, discovery of accessible services, and evidence suggesting access resale. The victim paid for the consumption. (sysdig.com)

This differs from manipulating a model's instructions. In the scenario covered here, an attacker uses a valid identity to obtain inference capacity. Neither changing the model nor compromising the provider is required. (sysdig.com)

A useful threat-analysis sequence is:

Vulnerable application → exposed credential → permission discovery → unauthorized inference → charges

This summarizes the incident described by Sysdig; it is not a claim that every incident follows the same sequence. (sysdig.com)

For your own threat model, also consider credentials in continuous integration and delivery pipelines, development workstations, and compromised workloads. Treat these as hypotheses to investigate through inventory and testing—not as automatic evidence of compromise.

Separate the responsibilities of each control

Use this separation as a design model. Do not expect one layer to provide guarantees that belong to another.

Bedrock quotas can be specific to a model and endpoint. They should not automatically be interpreted as a global monetary budget. Inspect the effective quotas in your account and inventory every enabled inference path. (docs.aws.amazon.com)

Likewise, an AWS Budgets notification is not a synchronous per-request spending block. AWS documents delays between resource consumption, billing, and notification, during which additional charges may occur. (docs.aws.amazon.com)

Estimate the financial exposure

For a simplified text-inference exercise, use:

estimated cost = input tokens / 1,000,000 × input price + output tokens / 1,000,000 × output price

Assume entirely hypothetical values, unrelated to a commercial price list:

• 10,000 requests per hour;
• 8,000 input tokens per request;
• 2,000 output tokens per request;
• $2 per million input tokens;
• $8 per million output tokens.

Input costs $160 per hour and output costs another $160 per hour. The total is $320 per hour, or $7,680 over 24 hours of constant consumption.

These are not current prices, benchmark results, or an incident forecast. The exercise illustrates why response time matters. For planning, also consider:

approximate exposure = spending rate × time to effective containment + consumption already admitted

This is not a guaranteed ceiling because the spending rate can change. Include the billable components of your actual contract and a margin for in-flight requests, asynchronous processing, and measurement uncertainty.

Architecture: close the unauthorized request path

For legitimate calls, I recommend this request path:

Authenticated client → application → financial admission → inference executor → Bedrock

Alongside it:

Cloud audit + provider metrics + platform records → correlation → incident response

The architectural rule is to keep inference permissions on the authorized executor. Frontends and services that only forward requests should not receive the same cloud identity.

Centralizing requests does not, by itself, address theft of the executor's identity. The executor also needs restricted permissions and additional origin controls. Bedrock supports interface endpoints through AWS PrivateLink and endpoint policies for private connectivity. (docs.aws.amazon.com)

However, creating a private endpoint does not force a stolen credential to use it. To enforce that requirement, evaluate perimeter controls using conditions such as aws:SourceVpce, including legitimate service integrations and exceptions. Test both approved and unauthorized access paths before deploying the policy. (docs.aws.amazon.com)

Keep workload network isolation as a separate layer. The egress architecture is covered in the article on networking for agent sandboxes on Kubernetes.

Implementation: dedicated workload identity on EKS

1. Create a Namespace and ServiceAccount

The following manifest is complete for the objects it declares. It does not install a gateway or configure an IAM association:

apiVersion: v1
kind: Namespace
metadata:
  name: llm-runtime
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: llm-inference
  namespace: llm-runtime
automountServiceAccountToken: false

The Namespace separates application objects. The ServiceAccount identifies the workload. Setting automountServiceAccountToken: false disables automatic mounting of the default Kubernetes API token; a Pod-level setting can override it. (kubernetes.io)

Set serviceAccountName: llm-inference in the authorized workload's Pod template. Adapt the names to your conventions. Do not create a RoleBinding merely to enable AWS requests: Kubernetes authorization and cloud authorization are separate controls.

Use role-based access control, or RBAC, to restrict who can create Pods in this Namespace, modify workloads, or access Secrets. Permission to create workloads can provide access to identities and credentials available within the Namespace. (kubernetes.io)

2. Associate an IAM role through EKS Pod Identity

With the EKS Pod Identity Agent installed on supported nodes, a prepared IAM role, and a compatible AWS SDK, create the association:

aws eks create-pod-identity-association \
  --cluster-name production-eks \
  --namespace llm-runtime \
  --service-account llm-inference \
  --role-arn arn:aws:iam::111122223333:role/llm-inference-runtime

Replace the cluster, account, and role. EKS manages the association without a ServiceAccount annotation. The role's trust policy must allow pods.eks.amazonaws.com to use sts:AssumeRole and sts:TagSession; further constrain the trust context for your environment. (docs.aws.amazon.com)

Pod Identity uses its own projected token and supplies temporary credentials to compatible SDKs. Disabling the default Kubernetes token does not remove that mechanism. Verify the effective identity and remove static credentials that the application's credential chain might select instead. (docs.aws.amazon.com)

Temporary credentials still require protection. If exposed, they may need to be revoked before expiration. (docs.aws.amazon.com)

3. Authorize only the required resources

This Terraform configuration creates an inline policy on an existing role. Its scope is direct inference against a foundation model:

variable "runtime_role_name" {
  type = string
}

variable "approved_model_arn" {
  type = string
}

resource "aws_iam_role_policy" "llm_inference" {
  name = "approved-llm-inference"
  role = var.runtime_role_name

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Sid    = "InvokeApprovedModel"
      Effect = "Allow"
      Action = [
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream"
      ]
      Resource = var.approved_model_arn
    }]
  })
}

Supply the actual Amazon Resource Name, or ARN, of the approved model. Verify that direct inference is supported in the selected region. The Converse API requires bedrock:InvokeModel; ConverseStream requires bedrock:InvokeModelWithResponseStream. (docs.aws.amazon.com)

This is not a universal policy for inference profiles, cross-region inference, or other endpoints. Review the resources required by the selected request path. A narrow Allow also does not remove permissions granted by other policies: inspect the complete identity and organizational controls. (docs.aws.amazon.com)

Financial admission: reserve before forwarding

For production, I recommend an explicit admission process:

1. Authenticate the caller and derive its tenant on the server.
2. Validate the model, input size, maximum output, and concurrency.
3. Calculate a conservative reservation for the request.
4. Reserve funds atomically before forwarding inference.
5. Reconcile the reservation against observed consumption.
6. Keep a reservation pending when the financial outcome is unknown.

Do not implement the balance as an independent counter in each replica. Design a shared ledger, idempotent operations, and recovery after failures. Transactional PostgreSQL is one implementation option. Redis can support rate limiting, but an isolated volatile counter should not be the sole financial record.

Keep capacity and cost separate. Bedrock can reserve quota based on input and maximum output while billing for actual usage. Adjusting the output maximum therefore also affects available concurrency. (docs.aws.amazon.com)

Define limits by tenant, workload, model, and environment. Size a global ceiling as well: several acceptable local limits can add up to an unacceptable organizational exposure.

This design governs requests admitted by the platform. It does not constrain requests sent directly to the provider with stolen credentials. IAM, perimeter enforcement, and external reconciliation remain necessary.

A production-oriented scenario

Problem: a hypothetical company runs document-summary workers on EKS. Its initial design shares an API credential between development and production and permits direct provider calls.

Solution: separate environments, replace the shared credential with dedicated workload identity, and introduce financial admission with reconciliation.

Proposed implementation:

• Run multiple executor replicas across availability zones.
• Set requests and limits from load tests without treating them as spending limits.
• Use readiness to withdraw unavailable replicas and keep liveness focused on local process health.
• Operate a highly available ledger with backups and restore tests.
• Preserve global limits during autoscaling.
• Bound concurrency and retries, with backoff and an attempt budget.
• Reject new inference when funds cannot be reserved safely.
• Degrade functionality—for example, postpone summaries—instead of silently switching to another credential.

Expected outcome: reduce the impact of a compromised identity and make containment testable. No performance or savings measurements are claimed for this fictional scenario.

The trade-off is intentional: failing closed protects the budget but can make a feature unavailable. Agree on that behavior with the business and establish a service-level objective, or SLO, for the LLM-backed capability.

Observability: detect consumption outside the application

Organize investigation into three layers:

Identity audit

Bedrock operations including InvokeModel, InvokeModelWithResponseStream, Converse, and ConverseStream are recorded by CloudTrail as management events. Other operations require data-event coverage. Do not assume a single configuration covers every enabled feature. Additional stream-completion events also require care to avoid counting one invocation twice. (docs.aws.amazon.com)

Provider consumption

For bedrock-runtime, monitor Invocations, InputTokenCount, OutputTokenCount, InvocationLatency, and InvocationThrottles in CloudWatch. Check dimensions to avoid summing overlapping series. These metrics do not replace the platform's tenant attribution. (docs.aws.amazon.com)

Platform records

I recommend recording the internal identity, model, request identifier, reservation, reconciled usage, and admission decision. Correlate these with logs and traces without adding prompts or credentials to attributes by default.

Bedrock model invocation logging is configured separately and can record inputs and outputs. Before enabling it, establish retention, encryption, access restrictions, and sensitive-data handling. (docs.aws.amazon.com)

As a detection rule, look for provider consumption without corresponding platform requests, allowing for delays, retries, administrative calls, and coverage differences. A discrepancy is an investigation signal, not standalone proof of LLMJacking.

Alert on estimated spending per minute, unexpected identities, unapproved models, backlog, reconciliation failures, and missing telemetry. Use billed cost as subsequent confirmation rather than the only operational sensor.

Incident response: contain access and preserve control

Prepare an approved runbook before an incident:

1. Preserve evidence alongside containment: identities, timestamps, events, and recent changes.
2. Block the compromised identity: disable keys and use the appropriate procedure for temporary sessions.
3. Prevent renewed access: fix the entry point and review trust relationships, pipelines, and permissions.
4. Inventory work already started: do not assume that denying new requests cancels active inference or jobs.
5. Restore with a clean identity: validate permissions and reintroduce traffic gradually.
6. Reconcile usage and impact: document the timeline and contact provider support when necessary, without assuming reimbursement.

For a role dedicated to the compromised executor, a deliberately broad emergency policy is:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "IncidentContainment",
      "Effect": "Deny",
      "Action": "*",
      "Resource": "*"
    }
  ]
}

This blocks all AWS actions authorized through that role, not only inference. It is a disruptive containment measure, not a permanent configuration. AWS also uses deny policies for session revocation; the official procedure can target sessions by their issuance time. (docs.aws.amazon.com)

Do not promise instantaneous blocking or retroactive cancellation. Verify effectiveness through tests and telemetry. A safe rollback removes containment only after the root cause is fixed, without restoring the exposed credential.

Troubleshooting

Inspection commands for the example Namespace:

kubectl get pods -n llm-runtime
kubectl get serviceaccount llm-inference -n llm-runtime -o yaml
kubectl get events -n llm-runtime --sort-by=.metadata.creationTimestamp
aws eks list-pod-identity-associations --cluster-name production-eks

Listing associations helps verify the cluster, Namespace, and ServiceAccount. It does not prove that the application is using the intended credential. (docs.aws.amazon.com)

Avoid copying tokens, sensitive environment variables, or complete Secret dumps into troubleshooting tickets. Kubernetes Secrets require access controls and protection at rest; base64 encoding is not encryption. (kubernetes.io)

Development, production, and maintenance

For development, I recommend an isolated account, approved models, a low financial ceiling, and less redundancy. Never reuse the production identity to simplify testing.

In production, require named identity and budget owners, high availability, TLS, tested backups, actionable alerts, and recurring containment exercises. Version IAM and admission policies through GitOps. Before upgrading an SDK, model, or endpoint, test authentication, usage accounting, limits, expected denials, and recovery.

Capacity planning should include requests, memory, concurrency, queues, and provider quotas. The total operating budget should also account for private endpoints, networking, ledger storage, load balancers, and observability. Evaluate Spot capacity for the executor as an interruption-tolerance decision, not as a way to reduce managed-inference prices.

Avoid hardcoded credentials, shared identities, in-memory-only counters, unlimited retries, missing probes, and untested emergency procedures. Do not let the inference workload modify its own permissions or the controls responsible for detecting its misuse.

When to centralize—and when to keep it simple

A central layer is a useful architectural choice when multiple teams share providers, several tenants require coordinated limits, or potential financial exposure justifies dedicated controls.

For a single low-volume application, a restricted identity, account isolation, backend limits, and alerts may be more proportionate. Operating Kubernetes is not a prerequisite for addressing LLMJacking.

Practical summary

• Identify who can turn a request into a charge.
• Narrow permissions and eliminate shared credentials.
• Reserve budget before admitting new requests.
• Compare cloud consumption with known platform traffic.
• Test revocation, recovery, and rollback before an incident.

Conclusion

Defending against LLMJacking should be a shared responsibility across DevSecOps, FinOps, and site reliability engineering, or SRE. A platform needs an explicit contract for authorizing consumption: identity, scope, limits, evidence, and a tested interruption procedure.

The goal is not merely to discover a large bill. It is to demonstrate which requests were authorized, constrain exposure during a failure, and restore service without restoring the vulnerability.