Use Groq with a scoped key and a spend cap

Spend caps on Groq

Keylayer prices six Groq models: openai/gpt-oss-120b, openai/gpt-oss-20b, openai/gpt-oss-safeguard-20b, qwen/qwen3.8-27b, meta-llama/llama-prompt-guard-2-22m and meta-llama/llama-prompt-guard-2-86m. What a request reserves depends on its route:

The worst case counts about one token per byte of the request body, plus max_completion_tokens or max_tokens on chat completions, or max_output_tokens on responses. Without one, Keylayer reserves the model's maximum output: for openai/gpt-oss-120b that's 65,536 tokens, about $0.039 per request in flight, where the example above reserves about $0.0006. When the response ends, Keylayer charges the cost from the usage Groq reports and frees the rest. A cancelled request, or a 504 or 524 timeout with no usage, is charged the full reservation; any other 5xx error with no usage is free.

Under a hard cap, Keylayer refuses with 403 unpriced_operation any request it can't bound up front: any model outside the six above, images, built-in tools, n other than 1, and a service_tier other than on_demand or flex on chat completions, or default or flex on responses. It also sets service_tier to on_demand on chat completions and default on responses when the request names none, since auto can pick Groq's performance tier. If Groq still reports a tier it can't price, Keylayer suspends the key.

Under an estimated cap, those requests go through, except a model outside the six and a body Keylayer can't read, such as one with a duplicate key, which get 403 unpriced_operation under either cap. Keylayer reserves a quarter token per byte of the body at the model's input price, plus $0.01, then charges the reported usage when it can price it, or else that reservation, so spend can pass the cap. Built-in tool requests, such as browser search, are charged their reported usage plus the reservation, which stands in for the tool fees. With escape hatches on, a model outside the six goes through at the highest input price among them, $0.80 per million tokens.

Groq reports usage in each stream's x_groq field, so Keylayer leaves streamed requests as they are.

Speech and transcripts

Orpheus speech costs $22 per million characters on canopylabs/orpheus-v1-english and $40 on canopylabs/orpheus-arabic-saudi. Keylayer counts the input text in UTF-8 bytes, never fewer than its characters, and charges that cost when the audio returns, so 200 English characters cost $0.0044. An error costs nothing unless it's a 504 or 524 timeout. Any other speech model, or a field Groq doesn't document for speech, such as stream, gets 403 unpriced_operation, unless the cap is estimated and the key has escape hatches, which let it through for $0.01 plus a quarter token per byte of the body at $0.80 per million tokens.

Groq's API reference documents no audio length or usage in a transcript or a translation, so Keylayer can't settle one and refuses it with 403 unpriced_operation under either cap. A key with an estimated cap and escape hatches lets it through and charges an estimate: the upload read as 16 kbit/s audio, at least 10 seconds, or at least an hour when a form field or query parameter name contains url in any case, at $0.111 per hour on whisper-large-v3 and any other model, or $0.04 on whisper-large-v3-turbo unless another form field or query parameter name also contains model. A 10 MB upload counts as about 83 minutes, $0.15 on whisper-large-v3. An error from Groq costs nothing unless it's a 504 or 524 timeout. A body or form Keylayer can't read is always refused, escape hatches or not.

Blocked routes

Keylayer blocks no Groq routes, since Groq's API has no key or organization management. Because a Groq key carries no scopes, the project that only Keylayer uses is what limits it. You can still allow or block methods and paths on each key; the key limits page explains how.

Errors

When Keylayer refuses a request, it answers in the OpenAI-style error format Groq uses, with the code in error.code and in a keylayer-error header:

HTTP/1.1 403 Forbidden
content-type: application/json
cache-control: no-store
keylayer-error: budget_exceeded
retry-after: 3600

{
  "error": {
    "message": "Keylayer budget_exceeded: this key's spend cap is reached; retry after 60m.",
    "type": "invalid_request_error",
    "param": null,
    "code": "budget_exceeded"
  }
}

Refusals before Keylayer knows the provider, such as 401 missing_token and invalid_token, have a plain body like {"error": "invalid_token"} instead, with the same keylayer-error header.

The SDKs raise their usual errors: PermissionDeniedError on 403 codes such as budget_exceeded and unpriced_operation, AuthenticationError on 401 codes such as invalid_token, and RateLimitError on 429. Errors from Groq itself pass through with no keylayer-error header, so check for it:

import groq

try:
    completion = client.chat.completions.create(
        model="openai/gpt-oss-120b",
        max_completion_tokens=1024,
        messages=[{"role": "user", "content": "Write a haiku about rate limits."}],
    )
except groq.APIStatusError as error:
    reason = error.response.headers.get("keylayer-error")
    if reason is None:
        raise
    print(f"Keylayer refused the request: {reason} (HTTP {error.status_code})")

The SDKs retry a 429 twice by default. When the wait is over 60 seconds or unknown, Keylayer adds x-should-retry: false, so the SDK stops at once. The error reference lists every code.

What doesn't work

• Images and built-in tools on a key with a hard cap. Use an estimated cap for those.
• Models outside the six above on a key with a spend cap, unless the cap is estimated and the key has escape hatches.
• Transcripts, translations, batches, files and speech models other than Orpheus on a key with a spend cap, since Keylayer can't price them up front. With escape hatches, an estimated cap lets transcripts, translations and other speech models through at their estimate.
• Request bodies over 16 MiB on a key with a spend cap. Keylayer reads the whole body to price it, so it answers 413 body_too_large_for_budget, and 415 content_encoding_unsupported to a compressed body. Keys without a spend cap stream the body straight through.
• Requests that run past 900 seconds. Keylayer ends them with 504 deadline_exceeded, or cuts off a started response.