Rate limits and throttling¶
cloudg 0.6.0 adds cloudg.resilience, a package that paces every cloud API call cloudg makes, reacts when a provider throttles, and records what happened in the run's coverage instead of failing the map. This document is for programmers: what the package does, how each provider is wired into it, every configuration key, what a throttled run looks like, the Python API, and what it does not cover yet.
Related documents: the feature reference (configuration and Python API chapters), the MCP guide (section 15 covers the live tools) and the MCP tool reference.
Unless a section says otherwise, every number below was read from the code in cloudg/resilience/ and cloudg/config.py, and every example output was produced by running the code shown.
Sections¶
-
A single cloudg map -p aws --org --regions all runs 133 deep collectors, a Cloud Control sweep and a tagging sweep in every region of every member account,...
-
That is the full sequence of call_with_resilience. The SDK integrations in section 3 reuse the same parts but let the SDK do the retrying, so not every step...
-
install_aws_hooks(session, account_id, region, default_config=...) (resilience/aws.py) registers four handlers on a boto3 or aioboto3 session's event...
-
The defaults sit at or just below each provider's documented limits, so they bind only where the provider would throttle anyway. They live in BUILTIN_LIMITS...
-
Everything sits under ratelimit in config.yaml (CloudGConfig.ratelimit, a RateLimitConfig). The section is additive: a config without it behaves exactly as...
-
6. What you see when a provider throttles
Throttling never fails a map. A service that stayed throttled after its retries is recorded in coverage, the rest of the run continues, and the run's...
-
7. Live operations and the MCP layer
LiveOperationGuard (resilience/guard.py) is pure asyncio with no MCP or provider imports. It decides whether a live operation may start:
-
Everything below is exported from cloudg.resilience. cloudg.retry re-exports call_with_resilience, call_with_resilience_sync, RetryPolicy, Scope, classify,...
-
Start from the defaults and read the run's throttling block before changing anything. A few patterns:
-
tests/test_resilience.py has 120 tests: classification across SDK shapes (real botocore errors included), Retry-After parsing, token buckets and AIMD on a...