GCP

Section 9 · From docs/INVENTORY_INTERNALS.md on GitHub

9.1 Collection

GCPCollector(project_id, credentials=None, **options) (GCPCollectorOptions: organization_id, scope, project_filter, skip_asset_types, page_size, include_iam, client, coverage, timeout, link_locally):

1. ListAssets with content_type=RESOURCE at the scope (projects/<id> or organizations/<id>), full resource JSON and ancestors, skip_asset_types filtered, blocking gRPC calls in worker threads with retries. At organization scope project_filter keeps only the listed projects (org- and folder-level resources are always kept).
2. If ListAssets is denied: SearchAllResources (summary fields only, summary_only: true), recorded PARTIAL.
3. Each record → MappedRecord → extract(GCPContext) → build_cloud_asset() (base metadata, extractor metadata / relations / aliases, the selfLink as alias, exposure entries).
4. Firewall evaluation (apply_firewalls): ingress rules matched to instances by network, target tags and target service accounts, priority order; rules open to the internet mark the instance exposed.
5. _enrich() (deep collector): SearchAllIamPolicies → apply_iam_policies().
6. collect_edges(): internet_edges(), one edge per exposure entry.

GCPDeepInventoryCollector adds the deep type map (gcp_asset_types.yaml deep over base) and IAM, and does not link locally: the mapper's linker does it for every provider at once.

The deep collector also takes GCPDeepOptions: services, exclude_services, rest_client (an injected REST client for tests), deep_sources, rest_timeout, max_images_per_repository and source_options (GCPSourceOptions or a dict of its fields). The mapper binds them from inventory.* and gcp.* with deep_options_from_config(). Its _enrich() runs the API-backed sources and the IAM policy search (9.6), and its _finalize() the cross-asset passes (9.7).

9.2 The extraction framework (gcp_relations/context.py)

@_extractor("compute.googleapis.com/Instance")
def _instance(ctx: GCPContext, out: Extracted) -> None: ...

GCPContext: name (full resource name), asset_type, data (resource JSON), project_id, project_number, location, ancestors; ctx.region (a zone reduced to its region) and ctx.default_sa() (the project's default compute service account).

Extracted methods:

Method	Purpose
out.add(target, edge, relationship=None, **kwargs)	Declare a relation (reverse, description, properties as for rel); empty targets and repeats are dropped.
out.sa(ref, ctx, description=None, **props)	The asset runs as a service account ("default" → the default compute SA); ASSUMES_ROLE / RUNS_ON to serviceAccount:<email>. Returns the email.
out.in_subnet(ref)	Subnet (or network) CONTAINS the asset.
out.alias(*values)	Extra identifiers.
out.expose(reason, protocol=None, ports=(), kind="INTERNET_EXPOSED")	Mark internet exposure; becomes exposure_reasons, internet_ingress and an internet edge.
out.metadata	Metadata to merge.

extract(ctx) runs the registered extractor (an exception is caught and recorded as extraction_error), then the generic CMEK scan: every Cloud KMS key name anywhere in the JSON becomes kms_keys and an ENCRYPTED_BY_KMS relation.

9.3 Name normalisation (gcp_relations/names.py)

GCP references come as compute selfLinks (https://www.googleapis.com/compute/v1/projects/p/...), other API URLs, relative names (projects/p/locations/l/...), short names and emails. full_name(value, service=None) normalises all of them to the Cloud Asset Inventory form //<service>.googleapis.com/<relative name>, which is what assets are indexed under; it returns None for values it cannot normalise (and rel() / Extracted.add() drop empty targets, so a bad reference simply declares nothing). Service accounts are referenced as serviceAccount:<email> (sa_ref), an alias of every service account asset. url_alias() keeps only scheme://host of URLs, because paths and queries can carry tokens.

9.4 Registered asset types

Module	Area	Cloud Asset Inventory types
gcp_relations/compute.py	instances, disks, snapshots, images, templates, instance groups, autoscalers	compute.googleapis.com/Autoscaler, compute.googleapis.com/Disk, compute.googleapis.com/Image, compute.googleapis.com/Instance, compute.googleapis.com/InstanceGroup, compute.googleapis.com/InstanceGroupManager, compute.googleapis.com/InstanceTemplate, compute.googleapis.com/MachineImage, compute.googleapis.com/RegionAutoscaler, compute.googleapis.com/RegionDisk, compute.googleapis.com/RegionInstanceGroup, compute.googleapis.com/RegionInstanceGroupManager, compute.googleapis.com/RegionInstanceTemplate, compute.googleapis.com/Snapshot
gcp_relations/network.py	networks, subnets, firewalls and firewall policies, routes, routers / NAT, VPN, interconnect, PSC, addresses, Shared VPC, DNS, Serverless VPC Access, NCC, Managed Kafka	compute.googleapis.com/Address, compute.googleapis.com/Firewall, compute.googleapis.com/FirewallPolicy, compute.googleapis.com/GlobalAddress, compute.googleapis.com/InterconnectAttachment, compute.googleapis.com/Network, compute.googleapis.com/NetworkFirewallPolicy, compute.googleapis.com/Project, compute.googleapis.com/RegionNetworkFirewallPolicy, compute.googleapis.com/Route, compute.googleapis.com/Router, compute.googleapis.com/ServiceAttachment, compute.googleapis.com/Subnetwork, compute.googleapis.com/TargetVpnGateway, compute.googleapis.com/VpnGateway, compute.googleapis.com/VpnTunnel, dns.googleapis.com/ManagedZone, dns.googleapis.com/Policy, dns.googleapis.com/ResponsePolicy, managedkafka.googleapis.com/Cluster, networkconnectivity.googleapis.com/Spoke, vpcaccess.googleapis.com/Connector
gcp_relations/load_balancing.py	forwarding rules, target proxies, URL maps, backend services and buckets, target pools, NEGs, Cloud Armor, SSL certificates	compute.googleapis.com/BackendBucket, compute.googleapis.com/BackendService, compute.googleapis.com/ForwardingRule, compute.googleapis.com/GlobalForwardingRule, compute.googleapis.com/GlobalNetworkEndpointGroup, compute.googleapis.com/NetworkEndpointGroup, compute.googleapis.com/RegionBackendService, compute.googleapis.com/RegionNetworkEndpointGroup, compute.googleapis.com/RegionSecurityPolicy, compute.googleapis.com/RegionSslCertificate, compute.googleapis.com/RegionTargetHttpProxy, compute.googleapis.com/RegionTargetHttpsProxy, compute.googleapis.com/RegionTargetTcpProxy, compute.googleapis.com/RegionUrlMap, compute.googleapis.com/SecurityPolicy, compute.googleapis.com/SslCertificate, compute.googleapis.com/TargetGrpcProxy, compute.googleapis.com/TargetHttpProxy, compute.googleapis.com/TargetHttpsProxy, compute.googleapis.com/TargetInstance, compute.googleapis.com/TargetPool, compute.googleapis.com/TargetSslProxy, compute.googleapis.com/TargetTcpProxy, compute.googleapis.com/UrlMap
gcp_relations/gke.py	GKE clusters and node pools, Kubernetes workloads, services, ingresses, service accounts, RBAC bindings	apps.k8s.io/DaemonSet, apps.k8s.io/Deployment, apps.k8s.io/StatefulSet, batch.k8s.io/CronJob, batch.k8s.io/Job, container.googleapis.com/Cluster, container.googleapis.com/NodePool, extensions.k8s.io/Ingress, k8s.io/Pod, k8s.io/Service, k8s.io/ServiceAccount, networking.k8s.io/Ingress, rbac.authorization.k8s.io/ClusterRoleBinding, rbac.authorization.k8s.io/RoleBinding
gcp_relations/serverless.py	Cloud Run, Cloud Functions, App Engine, Workflows, Cloud Tasks, API Gateway, Eventarc	apigateway.googleapis.com/Api, apigateway.googleapis.com/ApiConfig, apigateway.googleapis.com/Gateway, appengine.googleapis.com/Application, appengine.googleapis.com/Service, cloudfunctions.googleapis.com/CloudFunction, cloudfunctions.googleapis.com/Function, cloudtasks.googleapis.com/Queue, eventarc.googleapis.com/Trigger, run.googleapis.com/Job, run.googleapis.com/Service, workflows.googleapis.com/Workflow
gcp_relations/messaging.py	Pub/Sub topics and subscriptions	pubsub.googleapis.com/Subscription, pubsub.googleapis.com/Topic
gcp_relations/data_logging.py	log sinks and buckets, Cloud SQL, Redis, Memcache, Filestore, AlloyDB, BigQuery, Cloud Storage, Cloud KMS keys, Secret Manager, Artifact Registry	alloydb.googleapis.com/Cluster, alloydb.googleapis.com/Instance, artifactregistry.googleapis.com/Repository, bigquery.googleapis.com/Dataset, cloudkms.googleapis.com/CryptoKey, file.googleapis.com/Instance, logging.googleapis.com/LogBucket, logging.googleapis.com/LogSink, memcache.googleapis.com/Instance, redis.googleapis.com/Cluster, redis.googleapis.com/Instance, secretmanager.googleapis.com/Secret, sqladmin.googleapis.com/Instance, storage.googleapis.com/Bucket
gcp_relations/iam.py	service accounts and keys, workload / workforce identity providers, custom roles, API keys	apikeys.googleapis.com/Key, iam.googleapis.com/Role, iam.googleapis.com/ServiceAccount, iam.googleapis.com/ServiceAccountKey, iam.googleapis.com/WorkforcePoolProvider, iam.googleapis.com/WorkloadIdentityPoolProvider
gcp_relations/pipelines.py	Composer, Dataproc, Dataflow, Vertex AI endpoints and models, notebooks, Cloud Build triggers, Cloud Deploy	aiplatform.googleapis.com/Endpoint, aiplatform.googleapis.com/Model, cloudbuild.googleapis.com/BuildTrigger, clouddeploy.googleapis.com/DeliveryPipeline, clouddeploy.googleapis.com/Target, composer.googleapis.com/Environment, dataflow.googleapis.com/Job, dataproc.googleapis.com/Cluster, notebooks.googleapis.com/Instance, notebooks.googleapis.com/Runtime
gcp_relations/resource_manager.py	organization, folders, projects, organization policies, VPC Service Controls perimeters	accesscontextmanager.googleapis.com/ServicePerimeter, cloudresourcemanager.googleapis.com/Folder, cloudresourcemanager.googleapis.com/Organization, cloudresourcemanager.googleapis.com/Project, orgpolicy.googleapis.com/Policy

123 asset types have dedicated extractors; every other type still gets the generic CMEK scan, parent containment and a catalog classification.

GCP parity modules (added for depth parity with AWS). Several of them register a second extractor for a type an older module already handles, and the registry runs both (it keeps a list per type):

Module	Area	Cloud Asset Inventory and API types
gcp_relations/apis.py	Apigee organizations, instances, environments, proxies, target servers; API Gateway JWT providers and backends	apigateway.googleapis.com/ApiConfig, apigateway.googleapis.com/Authorizer, apigee.googleapis.com/ApiProxy, apigee.googleapis.com/ApiProxyRevision, apigee.googleapis.com/Environment, apigee.googleapis.com/Instance, apigee.googleapis.com/Organization, apigee.googleapis.com/TargetServer
gcp_relations/cicd.py	Cloud Build worker pools, 2nd gen connections and repositories, Developer Connect, trigger sources and pools	cloudbuild.googleapis.com/BitbucketServerConfig, cloudbuild.googleapis.com/BuildTrigger, cloudbuild.googleapis.com/Connection, cloudbuild.googleapis.com/GithubEnterpriseConfig, cloudbuild.googleapis.com/Repository, cloudbuild.googleapis.com/WorkerPool, developerconnect.googleapis.com/Connection, developerconnect.googleapis.com/GitRepositoryLink, developerconnect.googleapis.com/InsightsConfig
gcp_relations/containers_ext.py	Artifact Registry images and upstreams, Binary Authorization, GKE fleets, Backup for GKE, GKE namespaces and RBAC roles	artifactregistry.googleapis.com/DockerImage, artifactregistry.googleapis.com/Repository, binaryauthorization.googleapis.com/Attestor, binaryauthorization.googleapis.com/PlatformPolicy, binaryauthorization.googleapis.com/Policy, gkebackup.googleapis.com/Backup, gkebackup.googleapis.com/BackupPlan, gkebackup.googleapis.com/RestorePlan, gkehub.googleapis.com/Feature, gkehub.googleapis.com/Fleet, gkehub.googleapis.com/Membership, gkehub.googleapis.com/MembershipBinding, gkehub.googleapis.com/Namespace, gkehub.googleapis.com/RBACRoleBinding, gkehub.googleapis.com/Scope, k8s.io/Namespace, rbac.authorization.k8s.io/ClusterRole, rbac.authorization.k8s.io/Role
gcp_relations/data_ext.py	Spanner, Bigtable, Firestore, BigQuery tables / routines / connections / transfers, Dataplex, Data Catalog, Metastore, Data Fusion, Datastream, Storage Transfer, NetApp Volumes, Filestore backups, secret rotation, Parameter Manager, key rings	bigquery.googleapis.com/Model, bigquery.googleapis.com/Routine, bigquery.googleapis.com/RowAccessPolicy, bigquery.googleapis.com/Table, bigqueryconnection.googleapis.com/Connection, bigquerydatatransfer.googleapis.com/TransferConfig, bigtableadmin.googleapis.com/AppProfile, bigtableadmin.googleapis.com/AuthorizedView, bigtableadmin.googleapis.com/Backup, bigtableadmin.googleapis.com/Cluster, bigtableadmin.googleapis.com/Instance, bigtableadmin.googleapis.com/LogicalView, bigtableadmin.googleapis.com/MaterializedView, bigtableadmin.googleapis.com/Table, cloudkms.googleapis.com/KeyRing, datacatalog.googleapis.com/EntryGroup, datacatalog.googleapis.com/TagTemplate, datacatalog.googleapis.com/Taxonomy, datafusion.googleapis.com/DnsPeering, datafusion.googleapis.com/Instance, dataplex.googleapis.com/Asset, dataplex.googleapis.com/DataScan, dataplex.googleapis.com/EntryGroup, dataplex.googleapis.com/Lake, dataplex.googleapis.com/Task, dataplex.googleapis.com/Zone, datastream.googleapis.com/ConnectionProfile, datastream.googleapis.com/PrivateConnection, datastream.googleapis.com/Stream, file.googleapis.com/Backup, firestore.googleapis.com/Backup, firestore.googleapis.com/BackupSchedule, firestore.googleapis.com/Database, metastore.googleapis.com/Service, netapp.googleapis.com/ActiveDirectory, netapp.googleapis.com/Backup, netapp.googleapis.com/BackupPolicy, netapp.googleapis.com/BackupVault, netapp.googleapis.com/KmsConfig, netapp.googleapis.com/Replication, netapp.googleapis.com/Snapshot, netapp.googleapis.com/StoragePool, netapp.googleapis.com/Volume, parametermanager.googleapis.com/Parameter, parametermanager.googleapis.com/ParameterVersion, secretmanager.googleapis.com/Secret, spanner.googleapis.com/Backup, spanner.googleapis.com/Database, spanner.googleapis.com/Instance, spanner.googleapis.com/InstancePartition, storagetransfer.googleapis.com/TransferJob
gcp_relations/edge.py	Cloud DNS record sets, Certificate Manager, CA Service, Cloud Run domain mappings, Identity Platform	certificatemanager.googleapis.com/Certificate, certificatemanager.googleapis.com/CertificateIssuanceConfig, certificatemanager.googleapis.com/CertificateMap, certificatemanager.googleapis.com/CertificateMapEntry, certificatemanager.googleapis.com/DnsAuthorization, certificatemanager.googleapis.com/TrustConfig, dns.googleapis.com/ResourceRecordSet, identitytoolkit.googleapis.com/Config, identitytoolkit.googleapis.com/DefaultSupportedIdpConfig, identitytoolkit.googleapis.com/InboundSamlConfig, identitytoolkit.googleapis.com/OauthIdpConfig, identitytoolkit.googleapis.com/Tenant, privateca.googleapis.com/CaPool, privateca.googleapis.com/CertificateAuthority, run.googleapis.com/DomainMapping
gcp_relations/governance.py	tags and tag bindings, liens, custom constraints, Assured Workloads, authorized orgs, access levels, Infrastructure Manager, Deployment Manager	accesscontextmanager.googleapis.com/AccessLevel, accesscontextmanager.googleapis.com/AuthorizedOrgsDesc, accesscontextmanager.googleapis.com/ServicePerimeter, assuredworkloads.googleapis.com/Workload, cloudresourcemanager.googleapis.com/Lien, cloudresourcemanager.googleapis.com/TagBinding, cloudresourcemanager.googleapis.com/TagKey, cloudresourcemanager.googleapis.com/TagValue, config.googleapis.com/Deployment, config.googleapis.com/Preview, deploymentmanager.googleapis.com/Deployment, orgpolicy.googleapis.com/CustomConstraint, orgpolicy.googleapis.com/Policy
gcp_relations/hybrid.py	NCC hubs, route tables, groups, internal ranges, policy-based routes, regional endpoints; Interconnect; external VPN gateways; PSC accept lists; packet mirroring; sole-tenant nodes; resource policies	compute.googleapis.com/Disk, compute.googleapis.com/ExternalVpnGateway, compute.googleapis.com/Instance, compute.googleapis.com/Interconnect, compute.googleapis.com/NodeGroup, compute.googleapis.com/NodeTemplate, compute.googleapis.com/PacketMirroring, compute.googleapis.com/RegionDisk, compute.googleapis.com/ResourcePolicy, compute.googleapis.com/ServiceAttachment, compute.googleapis.com/VpnTunnel, networkconnectivity.googleapis.com/Group, networkconnectivity.googleapis.com/Hub, networkconnectivity.googleapis.com/InternalRange, networkconnectivity.googleapis.com/PolicyBasedRoute, networkconnectivity.googleapis.com/RegionalEndpoint, networkconnectivity.googleapis.com/RouteTable
gcp_relations/identity_ext.py	workforce / workload identity pools, OAuth clients, role permissions, deny policies, Principal Access Boundary, policy bindings, PAM entitlements, Cloud Identity groups	cloudidentity.googleapis.com/Group, iam.googleapis.com/DenyPolicy, iam.googleapis.com/OauthClient, iam.googleapis.com/OauthClientCredential, iam.googleapis.com/PolicyBinding, iam.googleapis.com/PolicyV2, iam.googleapis.com/PrincipalAccessBoundaryPolicy, iam.googleapis.com/Role, iam.googleapis.com/WorkforcePool, iam.googleapis.com/WorkforcePoolProviderKey, iam.googleapis.com/WorkloadIdentityPool, iam.googleapis.com/WorkloadIdentityPoolManagedIdentity, iam.googleapis.com/WorkloadIdentityPoolNamespace, iam.googleapis.com/WorkloadIdentityPoolProviderKey, privilegedaccessmanager.googleapis.com/Entitlement
gcp_relations/integration.py	Cloud Scheduler, Eventarc channels and Eventarc Advanced, Managed Kafka topics / consumer groups / Connect, Pub/Sub snapshots and schemas, Workflows definitions, Batch	batch.googleapis.com/Job, cloudscheduler.googleapis.com/Job, eventarc.googleapis.com/Channel, eventarc.googleapis.com/ChannelConnection, eventarc.googleapis.com/Enrollment, eventarc.googleapis.com/GoogleApiSource, eventarc.googleapis.com/MessageBus, eventarc.googleapis.com/Pipeline, eventarc.googleapis.com/Trigger, managedkafka.googleapis.com/ConnectCluster, managedkafka.googleapis.com/Connector, managedkafka.googleapis.com/ConsumerGroup, managedkafka.googleapis.com/Topic, pubsub.googleapis.com/Schema, pubsub.googleapis.com/Snapshot, workflows.googleapis.com/Workflow
gcp_relations/ml.py	Agent Engine, Vector Search, Discovery Engine, Dialogflow CX, Model Armor, pipelines and training jobs, Feature Store, Colab Enterprise runtimes	aiplatform.googleapis.com/BatchPredictionJob, aiplatform.googleapis.com/CustomJob, aiplatform.googleapis.com/Dataset, aiplatform.googleapis.com/FeatureGroup, aiplatform.googleapis.com/FeatureOnlineStore, aiplatform.googleapis.com/Featurestore, aiplatform.googleapis.com/HyperparameterTuningJob, aiplatform.googleapis.com/Index, aiplatform.googleapis.com/IndexEndpoint, aiplatform.googleapis.com/NasJob, aiplatform.googleapis.com/NotebookRuntime, aiplatform.googleapis.com/NotebookRuntimeTemplate, aiplatform.googleapis.com/PipelineJob, aiplatform.googleapis.com/ReasoningEngine, aiplatform.googleapis.com/TrainingPipeline, aiplatform.googleapis.com/TuningJob, dialogflow.googleapis.com/Agent, dialogflow.googleapis.com/KnowledgeBase, discoveryengine.googleapis.com/Assistant, discoveryengine.googleapis.com/Collection, discoveryengine.googleapis.com/DataStore, discoveryengine.googleapis.com/Engine, modelarmor.googleapis.com/Template
gcp_relations/netsec.py	Cloud NGFW, security profiles, TLS policies, authorization policies, address groups, Cloud IDS, Network Services gateways / meshes / routes, Media CDN	compute.googleapis.com/FirewallPolicy, compute.googleapis.com/NetworkEdgeSecurityService, compute.googleapis.com/NetworkFirewallPolicy, compute.googleapis.com/RegionNetworkFirewallPolicy, ids.googleapis.com/Endpoint, networksecurity.googleapis.com/AddressGroup, networksecurity.googleapis.com/AuthorizationPolicy, networksecurity.googleapis.com/AuthzPolicy, networksecurity.googleapis.com/ClientTlsPolicy, networksecurity.googleapis.com/FirewallEndpoint, networksecurity.googleapis.com/FirewallEndpointAssociation, networksecurity.googleapis.com/SecurityProfile, networksecurity.googleapis.com/SecurityProfileGroup, networksecurity.googleapis.com/ServerTlsPolicy, networksecurity.googleapis.com/TlsInspectionPolicy, networkservices.googleapis.com/EdgeCacheKeyset, networkservices.googleapis.com/EdgeCacheOrigin, networkservices.googleapis.com/EdgeCacheService, networkservices.googleapis.com/EndpointPolicy, networkservices.googleapis.com/Gateway, networkservices.googleapis.com/GrpcRoute, networkservices.googleapis.com/HttpRoute, networkservices.googleapis.com/Mesh, networkservices.googleapis.com/ServiceBinding, networkservices.googleapis.com/ServiceLbPolicy, networkservices.googleapis.com/TcpRoute, networkservices.googleapis.com/TlsRoute
gcp_relations/ops.py	Monitoring alert policies / channels / uptime checks, OS Config, Backup and DR	backupdr.googleapis.com/Backup, backupdr.googleapis.com/BackupPlan, backupdr.googleapis.com/BackupPlanAssociation, backupdr.googleapis.com/BackupVault, backupdr.googleapis.com/DataSource, backupdr.googleapis.com/ManagementServer, monitoring.googleapis.com/AlertPolicy, monitoring.googleapis.com/NotificationChannel, monitoring.googleapis.com/UptimeCheckConfig, osconfig.googleapis.com/OSPolicyAssignment, osconfig.googleapis.com/PatchDeployment, osconfig.googleapis.com/VulnerabilityReport
gcp_relations/security_ext.py	Security Command Center, Sensitive Data Protection, Data Access audit configs, IAM recommender, Artifact Analysis summaries	containeranalysis.googleapis.com/VulnerabilitySummary, dlp.googleapis.com/DeidentifyTemplate, dlp.googleapis.com/DiscoveryConfig, dlp.googleapis.com/InspectTemplate, dlp.googleapis.com/JobTrigger, dlp.googleapis.com/StoredInfoType, logging.googleapis.com/AuditConfig, recommender.googleapis.com/IamPolicyRecommender, securitycenter.googleapis.com/BigQueryExport, securitycenter.googleapis.com/ContainerThreatDetectionSettings, securitycenter.googleapis.com/EventThreatDetectionSettings, securitycenter.googleapis.com/MuteConfig, securitycenter.googleapis.com/NotificationConfig, securitycenter.googleapis.com/ResourceValueConfig, securitycenter.googleapis.com/SecurityCenter, securitycenter.googleapis.com/SecurityHealthAnalyticsSettings, securitycenter.googleapis.com/Source, securitycenter.googleapis.com/VirtualMachineThreatDetectionSettings, securitycenter.googleapis.com/WebSecurityScannerSettings, securitycentermanagement.googleapis.com/SecurityCenterService

With them, 350 asset types have dedicated extractors. test_every_new_catalog_type_has_an_extractor keeps every type in the catalog's deep section covered.

9.5 IAM bindings and principals (gcp_relations/iam.py)

apply_iam_policies(assets, policies) turns each binding into relations on the principal:

• service accounts that were collected carry the relation themselves;
• other members become principal assets: gcp-principal:<member> (users, groups, domains, workforce / workload identity principals) or k8s-gke://<pool project>/<ns>/<ksa> for GKE Workload Identity, typed IDENTITY_USER, IDENTITY_GROUP, SERVICE_PRINCIPAL or K8S_SERVICE_ACCOUNT;
• roles that let the member act as a service account (token creator, service account user, workload identity user) become ASSUMES_ROLE / ROLE_ASSUMES_ROLE; everything else GRANTS_ACCESS / POLICY_ALLOWS_ACTION with the role;
• allUsers / allAuthenticatedUsers mark the resource internet-exposed and create no principal.

merge_gcp_principals(assets, edges) runs in the mapper after deduplication: per-project placeholders created for a service account referenced before its project was seen are folded into the real service-account asset, and edges are re-pointed.

9.6 API-backed sources (gcp_sources/)

Cloud Asset Inventory does not return everything the AWS collector maps. GCPDeepInventoryCollector._enrich() runs the sources in cloudg/inventory/gcp_sources/ for the rest. Each one is a function registered with @source(name, family, phase=2, needs_rest=True); it gets a SourceContext (the collector, the assets so far, the REST client, GCPSourceOptions) and a SourceRun (call bookkeeping) and returns new assets.

from cloudg.inventory.gcp_sources.base import SourceContext, SourceRun, source


@source("widgets", "compute")
def widgets(ctx: SourceContext, run: SourceRun) -> list:
    """Widgets of every project in scope."""
    out = []
    for project, _number in ctx.projects():
        url = f"https://widgets.googleapis.com/v1/projects/{project}/widgets"
        for item in ctx.items(run, url, "widgets"):
            out.append(
                ctx.build(
                    f"//widgets.googleapis.com/{item['name']}",
                    "widgets.googleapis.com/Widget",
                    item,
                    project=project,
                    source_name="widgets",
                )
            )
    return [a for a in out if a is not None]

ctx.build() runs the resource through the collector's own record pipeline (_build_asset()): the same scrubbing, aliases, project resolution and @_extractor relations as a listed resource. It returns None when the name (or its project number / ID variant) is already in the inventory, so a source never duplicates what the listing found. Asset types that only an API returns (dns.googleapis.com/ResourceRecordSet, securitycenter.googleapis.com/Source, ...) are named like CAI types and mapped in gcp_asset_types.yaml (deep, marked (API)).

Phases: 0 runs cloud_identity (groups are principals), then the IAM policy search runs (it reuses the group assets instead of creating placeholders), then phases 2 and 3 (iam_roles and dlp_discovery, which read what earlier sources built). Inside a phase, sources run concurrently in worker threads (gcp.source_concurrency).

Source	Family	API	Maps
cloud_identity	identity	Cloud Identity groups, memberships	IDENTITY_GROUP per Google group with aliases group:<email> and gcp-principal:group:<email>; group CONTAINS each member
iam_deny	identity	IAM v2 denypolicies	IAM_POLICY that GOVERNS / POLICY_DENIES_ACTION its attachment point and every denied principal
principal_access_boundary	identity	IAM v3 principalAccessBoundaryPolicies, policyBindings	IAM_POLICY that GOVERNS / PERMISSION_BOUNDARY_LIMITS the allowed resources; the bound principal set REFERENCES / PERMISSION_BOUNDARY_LIMITS the policy
pam	identity	Privileged Access Manager entitlements	PERMISSION_SET: eligible principals ASSUMES_ROLE / ROLE_ASSUMES_ROLE it, it GRANTS_ACCESS the resource and ROLE_HAS_POLICY each role
iam_roles	identity	IAM roles.get	IAM_ROLE //iam.googleapis.com/roles/<name> per bound predefined role, with included_permissions
scc	security	Security Command Center management securityCenterServices, SCC v1 sources, notificationConfigs, bigQueryExports, v1beta2 subscription	SECURITY_HUB per activated parent (tier), one asset per service (ETD / CTD / VMTD as THREAT_DETECTOR, SHA as CONFIG_RECORDER, WSS as VULNERABILITY_SCANNER) that MONITORS the parent; placeholders where SCC is not activated
artifact_analysis	security	Service Usage, Container Analysis occurrences	VULNERABILITY_SCANNER per project that MONITORS / MONITORED_BY each image with severity counts; placeholder when scanning is off
iam_recommender	security	Recommender google.iam.policy.Recommender	ACCESS_ANALYZER per project; MONITORS / FINDING_AFFECTS each principal with an excess role
dlp_discovery	security	none (reads the listing)	Placeholder per project no Sensitive Data Protection discovery config covers
audit_configs	logging	Resource Manager v3 getIamPolicy	CLOUDTRAIL per organization, folder and project with Data Access logging, MONITORS it and LOGS_TO the _Default bucket; placeholder where Data Access logs are off
monitoring	logging	Monitoring v3 alertPolicies, notificationChannels, uptimeCheckConfigs; v1 metricsScopes	ALARM / NOTIFICATION_TOPIC; the scoping project MONITORS the projects in its metrics scope
dns_records	dns	Cloud DNS rrsets	DNS_RECORD per record set, contained by its zone, ROUTE / DNS_RESOLVED to the IPs and hosts it answers
os_config	ops	OS Config patchDeployments, osPolicyAssignments, vulnerabilityReports	SCHEDULE, RUNBOOK and VULNERABILITY_SCANNER
cloud_scheduler	integration	Cloud Scheduler jobs	SCHEDULE that INVOKES / SCHEDULED_BY its target
backup_dr	backup	Backup and DR backupVaults, backupPlans, backupPlanAssociations, managementServers	BACKUP_VAULT / BACKUP_PLAN
firestore_backups	backup	Firestore backupSchedules, backups	schedule MANAGES / BACKUP_TO the database
apigee	serverless	Apigee environments, targetservers, deployments	API_DESTINATION per target server; proxy ATTACHED_TO the environments it is deployed to
infra_manager	iac	Infrastructure Manager revision resources	deployment MANAGES / OWNED_BY every resource of its latest revision
deployment_manager	iac	Deployment Manager deployments, resources	IAC_STACK that MANAGES / OWNED_BY its resources

Several sources also fill types that Cloud Asset Inventory can return (alert policies, OS Config, Backup and DR, PAM, deny and boundary policies, SCC notification configs). For those the listing wins and the source only adds what the listing missed.

The REST client is GCPRestClient(credentials, *, session=None, timeout=60.0, max_retries=4, sleep=time.sleep) in gcp_sources/rest.py. It wraps google-auth's AuthorizedSession, so it needs nothing beyond the gcp extra. get(), post() (read-only RPCs such as getIamPolicy) and list() (follows nextPageToken, optional max_items) go through the resilience governor under gcp / <parent> / <api host>, and retry 429 and 5xx answers with backoff (honouring Retry-After). Errors are GCPApiError (a CollectionError) with status_code, error_code, disabled (the API is not enabled), not_found and precondition; HTTP 401 raises CredentialError. Without credentials and without an injected rest_client, the client is not built and the sources that need it are SKIPPED.

Every source is best effort. SourceRun.call() turns every API error into bookkeeping: a disabled API or a failed precondition counts as "not in use", 404 as an empty answer, anything else as an error. The source's coverage record gcp_<name> is then SUCCESS, PARTIAL (some calls failed), FAILED (nothing answered, or the source raised) or SKIPPED (missing prerequisite, or the API is off everywhere it was asked). A failing source never stops the run.

For services that are off, ctx.placeholder(service, parent, asset_type, label) builds the GCP counterpart of _disabled_asset(): arn = cloudg:gcp:<service>:<parent>:not-enabled, enabled: false, and the alias gcp-security:<service>:<parent> that the enabled asset of the same service also carries (gcp-security: is a globally unique scheme for the linker).

9.7 Cross-asset passes (gcp_sources/derived.py)

_finalize() runs these before the base finalization (firewalls, project node, principal merge):

Pass	Result
fold_tag_bindings	Each TagBinding becomes a REFERENCES / TAGGED_WITH relation from the bound resource to the tag value, plus resource_manager_tags metadata; the binding asset is dropped
link_instance_policies	Instance resource policies by kind: instance schedules MANAGES / SCHEDULED_BY the VM, snapshot schedules MANAGES / BACKUP_TO, placement policies are a DEPENDS_ON
link_secret_rotation	A secret with a rotation period INVOKES / ROTATES_SECRET every function, service or push endpoint that consumes one of its rotation topics
link_managed_labels	goog-dm=<deployment> labels: the Deployment Manager deployment MANAGES / OWNED_BY the resource; managed-by-cnrm=true: the project's one Config Connector cluster does
fold_policy_bindings	Principal Access Boundary bindings: the principal set REFERENCES / PERMISSION_BOUNDARY_LIMITS the policy
link_os_config_filters	Patch deployments and OS policy assignments MANAGES / SCHEDULED_BY the VMs their label, zone and name-prefix filters select
authorizer_assets	An AUTHORIZER per JWT provider in an API Gateway config (OpenAPI x-google-issuer, gRPC authentication.providers), contained by the config, DEPENDS_ON its issuer
ensure_principal_placeholders	Placeholders for gcp-principal:<member> relation targets (deny rules, PAM, group members) that are not assets

cap_images keeps the newest inventory.max_images_per_repository Docker images per repository before the records are built.

9.8 Service families and coverage

gcp_tasks.py gives every asset type a family: type_families in gcp_asset_types.yaml first, then the API service in families (compute.googleapis.com/... is compute), else other. inventory.services / inventory.exclude_services (--services / --exclude-services) accept families, API services (pubsub), full asset types and source names, with the AWS rules: include defaults to all, exclude always wins. Listed records are filtered in _keep(); sources and the IAM policy search (iam_policies, family identity) by is_selected(). The hierarchy family (organization, folders, projects) is kept unless it is excluded by name.

Coverage per run, besides gcp_list_assets and gcp_full:

Record	Meaning
gcp_assets_<family>	Assets of the family built from the listing; status follows the listing (PARTIAL when it was truncated or fell back to SearchAllResources). One record per selected family, also when empty
gcp_<source>	One per selected API-backed source (see 9.6)
gcp_iam_policies	SearchAllIamPolicies
gcp_derived_<pass>	Only when a cross-asset pass raised (FAILED)

9.9 Configuration

Key	Default	Effect
gcp.deep_sources	true	Run the API-backed sources at all
gcp.rest_timeout_seconds	60	Per-request timeout of the REST client
gcp.cloud_identity_customer_id	none	Cloud Identity customer (C0xxxxx); otherwise read from the organization
gcp.max_dns_records_per_zone	5000	Record sets read per managed zone
gcp.max_vulnerability_occurrences	5000	Artifact Analysis occurrences read per project
gcp.max_groups, gcp.max_group_members	2000, 5000	Cloud Identity groups, and members per group
gcp.expand_predefined_roles, gcp.max_role_expansions	true, 500	Read the permissions of bound predefined roles
gcp.max_scheduler_locations	25	Regions asked for Cloud Scheduler jobs (where the project has resources, plus the App Engine location)
gcp.source_concurrency	4	Sources run at the same time
inventory.services, inventory.exclude_services	all, none	Families, services, types and sources to map or skip
inventory.max_images_per_repository	20	Newest Docker images kept per Artifact Registry repository

Permissions the sources need, read-only: roles/securitycenter.adminViewer, roles/cloudidentity.groups.viewer (or a Workspace groups reader), roles/iam.denyReviewer, roles/iam.principalAccessBoundaryViewer, roles/privilegedaccessmanager.viewer, roles/monitoring.viewer, roles/dns.reader, roles/osconfig.viewer, roles/containeranalysis.occurrences.viewer, roles/backupdr.viewer, roles/cloudscheduler.viewer, roles/apigee.readOnlyAdmin, roles/config.viewer, roles/deploymentmanager.viewer, roles/recommender.iamViewer and roles/resourcemanager.organizationViewer / folderViewer. A missing one shows as that source's FAILED record.