AI FinOps in BigQuery: control generative function costs with daily token quotas

A support-ticket classification pipeline can start with a small sample and accidentally process the entire archive after someone removes a filter. When that transformation calls a language model, reviewing scanned bytes alone leaves an important part of the cost model unexamined.

In an October 14, 2026 publication, Google highlighted the general availability of daily token controls for BigQuery generative functions, with consumption limits at project and user level. (cloud.google.com)

This guide turns that capability into an operating model: establish a budget, configure limits, run recoverable batches, and decide in advance what should happen when the day's financial capacity is exhausted.

1. What the quotas actually control

Four quotas separate input and output consumption at project and user scope:

These controls do not replace model request-rate or throughput limits. Inspect the project's effective values rather than treating published defaults as your budget. (docs.cloud.google.com)

The documented scope covers Gemini-backed generative inference, including AI.CLASSIFY and AI.GENERATE_TEXT, but excludes embeddings such as AI.EMBED. Usage is aggregated globally across regions and resets at midnight Pacific time. Cached tokens are excluded; aggregation in millions makes very small limits less precise. (docs.cloud.google.com)

My recommendation is to treat a quota as an operational boundary, not an exact financial meter. Leave headroom between planned spending and the maximum budget approved for production.

Quotas, alerts, and query limits serve different purposes

An alerts-only Cloud Billing budget does not automatically block consumption. Spend cap budgets are a separate mechanism, and their coverage must be checked for the services involved. (docs.cloud.google.com)

Likewise, maximum_bytes_billed and dry runs address query-processing costs. Do not interpret a scanned-byte estimate as a complete generative inference estimate. (docs.cloud.google.com)

Structure governance around three layers: data processing, inference consumption, and consolidated spending.

2. A proposed production architecture

For asynchronous ticket enrichment, I recommend this flow:

Scheduler → pending-record selection → batch table → generative function → attempt table → validation → published results

Alongside it:

FinOps policy → quotas → pipeline admission control → alerts → financial reconciliation

Admission control is application logic in this design, not an additional native BigQuery feature. Its job is to decide whether another batch can start, considering estimated consumption, concurrency, and priority.

Assign three clear responsibilities:

• Platform team: quotas, identities, and execution policies.
• Data team: SQL, prompts, validation, and recovery.
• Product owner: budget, acceptable processing delay, and quality criteria.

For critical processing and experimentation, I recommend separate execution projects. The goal is to establish explicit budget boundaries, not to create more projects to evade limits.

This architecture uses managed services and SQL jobs. Kubernetes is not required; introducing a cluster solely to submit queries would expand the operational scope without solving token governance.

3. Translating a budget into token limits

Start with a representative sample. Include short and long documents, different languages, categories, and cases requiring longer responses. Do not size the pipeline from the average of a convenient sample.

A simplified planning model is:

Inference_cost = (Input_tokens / 1,000,000) × Input_price
               + (Output_tokens / 1,000,000) × Output_price

Use rates for the actual model, processing mode, and commercial agreement. Calculate caching, cache storage, and other applicable charges separately; exclusion from a token quota does not imply free usage. (cloud.google.com)

A fully hypothetical example

The following assumptions are illustrative and do not represent current Google Cloud prices:

• Daily inference budget: US$90.
• Chosen operating reserve: 20%.
• Amount allocated for planning: US$72.
• Fictional input rate: US$1 per million tokens.
• Fictional output rate: US$4 per million tokens.
• Expected ratio: five input tokens for each output token.

With I and O measured in millions of tokens:

I = 5 × O
I + 4 × O = 72
O = 8
I = 40

The starting point would therefore be 40 million input tokens and 8 million output tokens per day. These are neither universal recommendations nor a guaranteed maximum bill.

A hypothetical workload of 50,000 tickets averaging 800 input tokens and 160 output tokens would consume that entire allocation. This exposes a capacity-planning issue: there would be no room within the quota for retries or larger documents. Reduce admitted work or revisit the allocation before rollout.

For multiple models, I prefer separate estimates. A blended rate should guide limits only when the model mix is controlled. Recalculate the policy whenever prompts, models, document sizes, or reprocessing rates change.

4. Configuring the quotas

Through the console

Select the correct project and open IAM & Admin → Quotas & System Limits. Filter for BigQuery API, locate the four quotas, and edit their desired values. Lowering quotas restricts consumption; increases may require review. (cloud.google.com)

Before applying a change, record the following in a versioned change request:

• Current and proposed values.
• Financial justification.
• Affected workloads and identities.
• Approver and operational owner.
• Treatment of work already running.
• Conditions for restoring the previous limit.

Discover the configuration through the CLI

With an authenticated Google Cloud CLI and access to the Cloud Quotas API, inspect metadata before automating changes:

PROJECT_ID='your-execution-project'

gcloud quotas info list --service=bigquery.googleapis.com --project="$PROJECT_ID" --billing-project="$PROJECT_ID" --format=json

Check quotaId, units, dimensions, and effective values. The project consuming the Cloud Quotas API can differ from the project being administered; this example uses the same project for both. (docs.cloud.google.com)

Example input-quota preference

The following command creates a 40-million-token preference for the project's input quota. Run it only after confirming the API-returned identifier and checking that an existing preference does not need updating instead:

gcloud quotas preferences create --project="$PROJECT_ID" --billing-project="$PROJECT_ID" --service=bigquery.googleapis.com --quota-id=GenAiInputTokensPerDay --preferred-value=40000000 --preference-id=finops-genai-input-daily --allow-high-percentage-quota-decrease

--allow-high-percentage-quota-decrease permits a reduction greater than 10%. Do not automatically add --allow-quota-decrease-below-usage: lowering a limit below current consumption requires an explicit interruption decision. (docs.cloud.google.com)

Configure output and per-user limits as well. Verify the effective value after each change. Build automation as desired-state reconciliation, not a series of blind create operations.

5. SQL example: bound the input before generating

This example assumes an existing Gemini remote model named gemini_model, an authorized connection, and datasets compatible with the selected location. Creating the connection and authorizing its service account are environment prerequisites. (docs.cloud.google.com)

Assume a tickets table containing ticket_id, body, and created_at. This instructional batch is intended to run once with the names shown:

CREATE TABLE `your-project.support.ai_batch_20261014_001` AS
SELECT
  ticket_id,
  CONCAT(
    'Classify the ticket as billing, technical, or other. ',
    'Return only the category. Ticket: ',
    body
  ) AS prompt
FROM `your-project.support.tickets`
WHERE DATE(created_at) = DATE '2026-10-13'
  AND body IS NOT NULL
ORDER BY ticket_id
LIMIT 1000;

CREATE TABLE `your-project.support.ai_attempt_20261014_001` AS
SELECT *
FROM AI.GENERATE_TEXT(
  MODEL `your-project.ai.gemini_model`,
  TABLE `your-project.support.ai_batch_20261014_001`,
  STRUCT(128 AS max_output_tokens)
);

The materialized table bounds inference input. The function returns fields including result, status, and statistics; an empty status indicates a successful call. max_output_tokens bounds response generation. (docs.cloud.google.com)

Here, LIMIT belongs to preparation rather than an outer query over generated results. Fixed names and CREATE TABLE without replacement are intentional: a repeated execution should not silently overwrite previous evidence.

For production, I recommend these extensions:

1. Generate unique batch and attempt identifiers.
2. Select pending records and exclude already accepted results.
3. Associate each result with document identity, content version, prompt, and model.
4. Separate technical success from semantic acceptance.
5. Publish only allowed category values.
6. Set bounded retention for prompts and raw responses.

A technically successful response with an invalid category should remain unresolved. Do not silently turn missing output into a default classification.

6. What happens when the quota is exhausted

If a required quota is already exhausted, BigQuery rejects the query with QuotaExceeded. If exhaustion occurs during execution, new model calls are refused. (docs.cloud.google.com)

For supported functions such as AI.IF, max_error_ratio affects whether partial results can be returned. Do not assume every function accepts that argument. (cloud.google.com)

My recommended recovery policy distinguishes four cases:

• Daily quota exhaustion: stop new submissions until reset or an approved adjustment.
• Transient capacity failure: use bounded retries with exponential backoff and jitter.
• Permission or configuration error: stop retrying and fix the underlying issue.
• Invalid output: route the record to validation or a previously approved fallback.

Configure scheduling with America/Los_Angeles rather than a fixed UTC offset. Resume gradually after replenishment instead of releasing the entire backlog at once.

7. Production design: reliability before speed

Scenario: problem → solution → implementation → expected outcome

Problem: a team must classify tickets daily while analysts experiment against the same data domain.

Proposed solution: separate experimental and operational execution, assign a budget to each workload, and keep enrichment outside the synchronous ticket-submission path.

Implementation: durable batches, reviewed quotas, a dedicated execution identity, category validation, and checkpoints for accepted results.

Expected outcome: inference can pause without preventing customers from opening tickets, and processing can resume from outstanding records. This is an architectural expectation, not an observed performance claim.

Availability and recovery

Define a service-level objective, or SLO, for maximum backlog age. In this scenario, ticket intake availability and classification freshness should have separate objectives.

I recommend storing pipeline state outside the process that submits jobs. If multiple executors are used, coordinate batch ownership to prevent duplicate submissions.

Document how to restore results, retain source input, and recover checkpoints. Rehearse failures with a small budget before permitting large-scale reprocessing.

Rollback and upgrades

Version SQL, prompts, model references, and quota policies through code review. For each change, run a canary batch and compare estimated cost, category distribution, and rejection rate.

Rollback should stop new batches and restore a validated configuration. Do not design it as a way to reverse consumption that has already occurred. When changing models, revisit output contracts and quality acceptance criteria as well.

8. Observability and security

Monitor three separate planes

My proposed dashboard brings together:

Use the console's quota history as one source of consumption data. (cloud.google.com)

The following query helps investigate project jobs. Run it in the matching location with permission to list those jobs:

SELECT
  creation_time,
  job_id,
  user_email,
  state,
  error_result.reason AS error_reason,
  total_bytes_billed,
  total_slot_ms
FROM `your-project.region-us.INFORMATION_SCHEMA.JOBS_BY_PROJECT`
WHERE creation_time >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 1 DAY)
  AND job_type = 'QUERY'
ORDER BY creation_time DESC
LIMIT 100;

These fields support SQL-processing investigation; total_bytes_billed is not a model-token counter. Replace region-us with the appropriate location and correlate jobs with inference telemetry. (docs.cloud.google.com)

As an initial policy, consider alerts at 60%, 80%, and 90% consumption, together with projected exhaustion time. These are example thresholds: tune them to the team's response time.

Least privilege

Separate the query-execution identity from the quota-administration identity. Cloud Quotas provides dedicated reader and administrator roles, including roles/cloudquotas.viewer and roles/cloudquotas.admin. (docs.cloud.google.com)

For remote-model execution, distinguish the principal submitting the job from the connection service account accessing the model. The documented setup includes model-service authorization for that connection. (docs.cloud.google.com)

I recommend removing unnecessary personal data before inference, avoiding full-prompt logging by default, and keeping service account keys out of repositories. Include connections, intermediate tables, and observability access in the security review.

9. Development, production, and troubleshooting

For development, use small samples, sanitized data, and conservative limits. For production, require checkpoints, a budget owner, alerts, change approval, and a documented recovery procedure.

The first two conditions reflect documented enforcement behavior; the remaining entries are operational hypotheses to guide diagnosis. (docs.cloud.google.com)

Avoid four shortcuts: automatically raising quotas after every error, rerunning entire queries without inspecting previous results, leaving permissive limits without an owner, and relying on sensitive logs as the only execution record.

10. When to use this approach—and what to consider instead

I recommend this pattern when generative processing belongs close to analytical data, volumes are meaningful, and the business accepts asynchronous execution with explicit backlog management.

For a small set of stable classification rules, evaluate deterministic SQL first. For applications needing admission decisions by customer, session, or request, consider an application layer with its own accounting while retaining provider controls as additional protection.

Provisioned Throughput offers contracted capacity at a fixed cost. Evaluate utilization and contractual commitments rather than token volume alone; an overly restrictive daily limit can block capacity that has already been reserved. (docs.cloud.google.com)

Compare operational complexity, predictability, isolation requirements, and dependence on BigQuery-specific functions. Do not introduce another platform solely because it offers more controls if the team cannot operate those controls consistently.

Practical summary

• Establish the budget before choosing technical limits.
• Configure input and output controls at project and user level.
• Materialize batches and preserve attempt evidence.
• Separate successful calls from acceptable results.
• Plan suspension, recovery, and rollback before the first incident.
• Measure cost per accepted result, not just processing volume.

Conclusion

FinOps for generative functions should be an operating agreement between product, data, and platform teams. A quota establishes a boundary; the pipeline must operate within it and fail predictably when that boundary is reached.

For DevOps, this means reviewed and reproducible changes. For SRE, it means treating budget and backlog as reliability concerns. For Platform Engineering, it means providing a standard path where experimentation is easy but expanding financial exposure requires an explicit, accountable decision.