Live tools and the credential preflight

Section 15 · From docs/MCP.md on GitHub

The live category has five tools. Four touch cloud APIs: map_inventory (deep inventory mapping, timeout 3600 s), collect_assets (the standard collection, 3600 s), run_scanners (external scanners plus reachability analysis, 7200 s) and run_pipeline (collect, scan, analyze, normalize, report; 7200 s). They are annotated readOnlyHint: false, idempotentHint: false, openWorldHint: true, report progress per pipeline phase, put their result in the workspace as a new active dataset, and return a summary with resource links instead of the full data. The profiles strict, read_only, airgapped, audit and soc-analyst hide them, and --read-only drops them.

The fifth, rate_limit_status, makes no cloud call. It reports the live-operation guard and the cloud API throttling state; the resource cloudg://ratelimit serves the same data. Every profile lists it.

15.1 What happens before a cloud call

The collecting tools run four checks, in this order, and stop at the first failure, so a call that cannot succeed fails before it spends any API quota:

1. Dataset name. name is validated, and an existing dataset with that name is an error unless replace=true (example in 11.1). An empty name generates one from the tool and the time, inventory-YYYYmmdd-HHMMSS for map_inventory. run_scanners adds findings to an existing dataset and has no name.
2. force. force=true asks to skip the cooldown (step 4). Only principals with the admin or operator role may pass it; anyone else gets -31001 at this point.
3. Credential preflight (15.2), unless preflight=false.
4. The live operation guard (15.3).

run_scanners also checks its iac_dir against the allowed roots before any of this, and only runs steps 3 and 4 for cloud scopes when Prowler or ScoutSuite is among the scanners.

15.2 The credential preflight

Provider	Check
AWS	boto3 importable; static keys in the config pass; a role ARN plus a web-identity token file (aws.web_identity_token_file or $AWS_WEB_IDENTITY_TOKEN_FILE) passes if the file exists; otherwise a boto3 session (with aws.profile if set) must find credentials. No API call is made
Azure	builds cloudg's Azure credential and requests one token for https://management.azure.com/.default
GCP	builds cloudg's GCP credentials (application default credentials or the configured key)

The whole check runs in a thread and is limited to 25 seconds. A failed check returns an error result like:

Cannot collect live data. aws: no credentials found (set aws.profile / access keys in the
cloudg config, AWS_PROFILE / AWS_ACCESS_KEY_ID in the server's environment, or run on a host
with an instance role). Fix the server's credentials, choose other providers, or work offline
with load_dataset. Pass preflight=false to skip this check.

with cloudg/error_code -32603 and cloudg/error_data {"providers": {"aws": "..."}}. (That message is quoted from the code; I did not run a live tool against real cloud accounts.)

15.3 The live operation guard

Every workspace has one LiveOperationGuard (workspace.live_guard), built from the ratelimit section of the cloudg config on first use. Each call is keyed by its operation and its scopes. A scope is a provider plus whatever the config names for it: AWS accounts or the profile, Azure subscription ids, GCP project ids or the organization id, or just the provider when the config names none. The guard does three things:

• Single flight: a call identical to one already running waits for it and returns the same result with "joined": true and a note, instead of collecting a second time.
• Cooldown: after a live operation on a scope, new ones on that scope are refused for live_cooldown_seconds (120 by default; per-provider overrides under ratelimit.<provider>.live_cooldown_seconds), also after a failed run.
• Concurrency and quotas: at most live_max_concurrent (1) operations per scope and live_max_concurrent_total (2) overall, and optionally live_caller_max_operations per caller per live_caller_window_seconds.

# cloudg config.yaml
ratelimit:
  live_cooldown_seconds: 120       # min time between live collections of one scope
  live_max_concurrent: 1           # concurrent live collections per scope
  live_max_concurrent_total: 2     # concurrent live collections overall
  live_caller_max_operations: 0    # per caller per window (0 = unlimited)
  live_caller_window_seconds: 3600

A refusal is LiveOperationRefused, a RateLimitedError (-31029). Its message names the dataset from the last live run so the model can query that instead. With the engine replaced by the test suite's fake engine and the preflight stubbed out, a second map_inventory for aws right after the first returned:

{
  "isError": true,
  "text": "live collection of aws ran recently; cooling down, retry in 120s or use the cached dataset. The last live result is already loaded as dataset 'live-1'; query it instead of collecting again. An admin or operator can pass force=true to skip the cooldown.",
  "_meta": {
    "cloudg/error_code": -31029,
    "cloudg/error_data": {
      "reason": "cooldown",
      "message": "live collection of aws ran recently; cooling down, retry in 120s or use the cached dataset",
      "retry_after_seconds": 120.0,
      "scopes": ["aws"],
      "caller": "local",
      "use_dataset": "live-1"
    }
  }
}

(Shape condensed from the ToolResult.) In the same run, force=true from the local user (roles default, local) got

force=true (skip the live cooldown) needs the admin or operator role. Use the dataset from the
last collection, or wait for the cooldown to end.

with -31001, and the same call from Principal(id="ops", roles={"operator"}) ran and produced dataset live-3. rate_limit_status afterwards:

{
  "live_guard": {
    "in_flight": 0,
    "active_total": 0,
    "active_by_scope": {},
    "cooling_down": {"aws": 120.0},
    "started": 2,
    "joined": 0,
    "cooldown_seconds": 120.0,
    "provider_cooldowns": {},
    "max_concurrent_per_scope": 1,
    "max_concurrent_total": 2,
    "caller_max_operations": 0
  },
  "throttling": {
    "totals": {"calls": 0, "throttled": 0, "transient_errors": 0, "retries": 0, "gave_up": 0,
               "rejected": 0, "breaker_trips": 0, "wait_seconds": 0},
    "messages": [],
    "skipped": {},
    "scopes": {},
    "breakers": {},
    "slowed_buckets": {}
  }
}

The throttling half comes from the cloud API rate-limiting layer in cloudg.resilience, which is documented with the rest of cloudg's configuration, not here. Live results also carry a throttling summary when the run hit provider throttling.

The credentials are the server's, whoever calls. MCP principals do not get their own cloud identities, so restrict the live tools to trusted roles in the policy.