Short answer: ThrottlingException means you hit a Bedrock quota for that model in that Region, usually requests per minute or tokens per minute. Retry with backoff, spread load with cross-Region inference, and request a quota increase for sustained traffic.
The error
terminal
botocore.errorfactory.ThrottlingException: An error occurred (ThrottlingException) when calling the InvokeModel operation: Too many requests, please wait before trying again.Why it happens
| Cause | How to tell |
|---|---|
| Requests per minute quota reached | Many small calls in bursts; errors come in spikes |
| Tokens per minute quota reached | Few calls with very large prompts or outputs |
| Several apps share the same account and Region | Errors appear when another workload is busy |
| No retry logic, so every throttle becomes a failure | Errors surface directly to users |
Fix 1: retry with backoff
The AWS SDKs can retry throttled calls for you. Adaptive mode also slows down the client when it sees throttling.
client.py
import boto3
from botocore.config import Config
bedrock = boto3.client(
"bedrock-runtime",
region_name="us-east-1",
config=Config(retries={"max_attempts": 10, "mode": "adaptive"}),
)Fix 2: cross-Region inference
Inference profiles route requests to the model in several Regions of the same geography, which gives you more throughput without code changes beyond the model ID. Check which models support it in your Region.
Fix 3: the right capacity model
- Quota increase in Service Quotas for sustained on-demand traffic.
- Batch inference for offline jobs, so they stop competing with user traffic.
- Provisioned throughput for steady, high and predictable volume.
Retries multiply token usage when the failure is not a throttle. Retry only on throttling and transient errors, and always set maxTokens.
Prevent it
- Put a queue (SQS) in front of non-interactive calls and control concurrency.
- Create CloudWatch alarms on the Bedrock throttling metrics for each model you use.
- Separate production and experiments in different accounts, so tests do not consume production quota.