Google Cloud
Connect Google Cloud so Ace can map your projects, read Cloud Monitoring, Logging, audit logs and traces during investigations, open incidents from Cloud Monitoring alerts, scan for savings and, if you allow it, make approved changes.
Ace reads Google Cloud to see what is failing, what changed and how your services connect. You connect once, at the organization, a folder or a single project, and setup runs as one command in Cloud Shell or as Terraform. Write access comes from a separate, optional service account, so you can grant it later or never.
| Category | Infrastructure, telemetry and change context |
| Ace reads | Cloud Asset Inventory, Cloud Monitoring metrics and alerts, Cloud Logging, Admin Activity audit logs, Cloud Trace, Error Reporting, Service Health, Recommender findings |
| Ace can change | Nothing with the read account. With the optional actions account, the services you choose, after approval |
| Opens incidents | Yes, from Cloud Monitoring alerts |
| Setup | One command in Cloud Shell, or Terraform. Keyless by default |
What it reads
During an investigation, Ace reads:
- Cloud Monitoring time series for the incident window, and the alerts open on the affected resources.
- Cloud Logging entries, such as the first errors from a Cloud Run service or a VM.
- Admin Activity audit logs, which show who changed what, and when.
- Cloud Trace spans and Error Reporting groups for the failing service.
- Google Cloud Service Health, so an incident on Google's side is visible next to your own signals.
- Cloud SQL Query Insights for the affected databases.
Ace reads only the resources near the failing workload on the System Map, and every read is bounded and audited.
For the System Map, Ace reads Cloud Asset Inventory across your connected scope when you connect, once a day and when you choose Sync: projects, networks and subnets, Cloud Run services and functions, VMs and instance groups, GKE clusters and node pools, load balancers, Cloud SQL, Memorystore, Spanner, Pub/Sub, Cloud Tasks, Cloud Scheduler, Cloud DNS and more. It draws edges from each resource's configuration, such as a load balancer to its backends, a Cloud Run service to its database and secrets, or a Pub/Sub subscription to its push endpoint. AWS and Google Cloud share one map, so a Cloud Run service that calls an endpoint in AWS shows as a cross-cloud dependency.
The read account uses Google's predefined viewer roles. It can't read Secret Manager values, Cloud Storage objects, database rows, queue messages or Data Access audit logs. Three of its roles return more than metadata, and Ace says so beside each one in setup:
- Cloud Asset Inventory returns each resource's configuration, which includes Cloud Run and Cloud Run functions environment variable values and Compute Engine metadata. Ace uses them in memory to find hostnames for the System Map and keeps only the variable names.
- Cloud Logging returns your application logs.
- Cloud Trace's narrowest role can also manage trace scopes. Ace never uses that.
Ace also scans Google's Recommender for savings: idle and oversized VMs and instance groups, idle disks and IP addresses, idle and overprovisioned Cloud SQL instances, Cloud Run cost, and commitments. It checks daily, and findings appear under Cost opportunities on the Google Cloud page in Ace.
What it can change
The read account can't create, update or delete anything. To let Ace make changes, add the separate actions account with at least one of these groups:
| Group | Services |
|---|---|
| Compute | Cloud Run, Cloud Run functions, instance groups, VMs, GKE node pools |
| Data | Cloud SQL, Memorystore, Spanner, Pub/Sub, Cloud Tasks, Cloud Scheduler |
| Network and edge | Load balancers, Cloud DNS, Cloud Armor, firewall rules |
| Deployments | Cloud Deploy, Workflows |
| Runbooks | Workflows and Cloud Run jobs you allow |
| Alert snoozing | Snooze an alert policy during a fix |
Each group is one custom role that holds only the permissions its actions use. None of the roles can read data, change IAM, act as another identity or get onto a machine through SSH, OS Login, metadata or the serial console.
A change has to clear three checks:
- The actions account holds the group's role. Ace asks Google which permissions it holds when you turn actions on, when resources are confirmed, and again before every plan and every write.
- Ace proposes changes only on the Google Cloud resources an admin confirms in each workload's Cloud resources settings (
https://app.tryace.dev/workloads/<workload-id>/settings/resources). - A person approves the exact plan, and Ace reads the resource again before it writes. If anything changed since the plan, it doesn't write.
Ace also narrows the riskier operations itself:
- Cloud DNS changes take A, AAAA, CNAME and SRV records only, so Ace can't change mail routing or domain verification records.
- Turning a firewall rule on or off says whether access opens or closes, and for which ranges.
- Cloud Run scaling and restart deploy a new revision, so Ace refuses them while traffic is pinned to a specific revision, such as after a rollback.
- Memorystore failover, removing a Cloud SQL replica and changing a Cloud Armor rule's action always need a person.
Changes to data stores, such as Cloud SQL, Memorystore and Spanner, never run automatically. Every Google Cloud action is listed in the action catalog.
Ace changes resources through Google's API. If Terraform, Pulumi or Config Connector manages a resource, its next apply puts the change back. The plan says so when the resource's labels show a tool manages it, or when the workload's mapped repository declares it. It's a caution and never blocks the change, so your team can follow up in code.
Actions need keyless access. A connection that reads with a service account key can't turn actions on, because Ace never makes changes with a stored key.
What opens an incident
Ace checks Cloud Monitoring alerts every 2 minutes with the read account, so you don't need a webhook or notification channel. With more than 25 projects, each check covers 25, those with open alerts first and the rest in turn.
- An alert that opens on a project in the connection opens an incident or joins an open one.
- The incident resolves when Google closes the alert.
An alert opens an incident only for a workload that routes Google Cloud. To route it, open the workload's settings, go to Alerts (https://app.tryace.dev/workloads/<workload-id>/settings/alerts), turn on Investigate alerts and tick Google Cloud under Sources.
Admin Activity audit log entries, checked every 2 minutes, join the change timeline on the resource they changed, whether or not a workload routes Google Cloud, so Ace can line up a config or infrastructure change with an incident. Ace keeps who changed what and when, never a value. Changes Google makes on its own, such as autoscaler resizes and host maintenance, aren't on the timeline, though Ace can still search for them in the logs during an investigation.
Before you start
You need the admin role in Ace (see Roles and permissions) and a project to hold Ace's service accounts, called the host project. Any project in the organization works, and Ace adds nothing else to it. For a single-project connection, it's that project. A connection covers up to 1,000 projects. Manage resources lists every organization, folder and project with its ID, as do gcloud organizations list and gcloud projects list.
Whoever runs setup in Google Cloud needs these roles:
| Where | Role | For |
|---|---|---|
| Host project | Owner, or Service Usage Admin, Service Account Admin and IAM Workload Identity Pool Admin | Enabling APIs, creating the service accounts and the keyless trust |
| Organization or folder Ace covers | Organization Administrator, or Folder IAM Admin for a folder | Granting the read roles |
| Organization, only with actions | Organization Role Administrator | Creating the custom action roles. A folder's roles live in its organization |
For a single project, Owner on that project covers all of it.
An IAM admin grants these roles in IAM for the host project, folder or organization.
Owning an organization doesn't include Organization Role Administrator, so even the person who created it usually needs it granted. The setup script checks every role before it changes anything. If one is missing, it stops with nothing changed and prints the command that grants it, for example:
gcloud organizations add-iam-policy-binding ORGANIZATION_ID \
--member user:you@example.com --role roles/iam.organizationRoleAdmin --condition NoneIf you can already set IAM on the organization, the script offers to grant you the role, asks first, and prints the command that removes it afterwards.
If your organization blocks workload identity pools, use a service account key for read-only access.
Review before you run it
Setup is one public script, the same for every customer: https://app.tryace.dev/setup/gcp.sh. Your security team can read it before anyone runs it. To see exactly what it would do in your organization, with every role and permission, add --plan to the command Ace shows. It changes nothing:
curl -fsSL https://app.tryace.dev/setup/gcp.sh | bash -s -- --identity ACE_IDENTITY --actions compute --planWhat Ace gets is listed under Credentials and permissions. Neither service account holds Owner, Editor or any other broad role, and the read account can't read Secret Manager values, Cloud Storage objects, database rows, queue messages or Data Access audit logs.
Split setup between two people
Often the engineer connecting Ace doesn't administer the organization's IAM. Setup runs in two parts:
- The engineer adds
--part projectto the command and runs it. It creates the service accounts and the keyless trust in the host project, then prints the command for the second part with every value filled in. - Someone who administers IAM on the organization or folder runs that command. It grants the read roles and, with actions, creates and grants the action roles. It needs no access to the host project.
- The engineer pastes the connection code into Ace. Until the second part runs, Test connection reports the read roles as missing.
Credentials and permissions
With keyless access, which is the default, Ace stores no Google credential. Setup creates a workload identity pool in the host project that trusts exactly one identity: the Ace session made for your organization. Ace exchanges that identity for a short-lived token of the read service account. There is no key to leak or rotate.
If your organization can't create a pool, you can connect with a service account key instead. Ace stores the key encrypted and can only read with it.
| Account | Where it lives | What it allows | Why Ace needs it |
|---|---|---|---|
ace-read | The host project, with roles on the scope you connect | Google's viewer roles for assets, monitoring, logging, trace, error reporting, recommendations, Service Health and service usage | Investigations, the System Map, alerts, the change timeline and cost scans |
ace-actions | The host project, only if you choose action groups | One custom role per group you choose | Approved changes and alert snoozing |
Roles bound on an organization or folder reach every project under it, including projects created later. On connect and on every check, Ace also asks whether the account holds permissions it never uses, and the Google Cloud page in Ace names the broader role they come from so you can remove it.
Set up
Connect the read account
- Open Settings > Integrations > Google Cloud.
- Under Choose what Ace covers, choose a Scope:
- Organization (recommended) covers every project in the organization.
- One folder covers the projects in a folder. Enter its Folder ID, which is a number.
- One project covers one project.
- Choose Access: Keyless (recommended) or Service account key.
- Under Run the setup in Google Cloud, choose Open Cloud Shell. In Cloud Shell, select the host project in the project picker, then paste and run the one-line command Ace shows. To name another host project, add
--project PROJECT_IDto the end of the command. - The script finds the project and its organization and checks your roles. It shows what it will do and asks before it changes anything. When it finishes, it prints a connection code that starts with
ace-gcp:. - Under Paste the code it printed, paste the code into Connection code. For a service account key, also paste the contents of
ace-read-key.jsoninto Service account key, then delete the file. Choose Connect Google Cloud.
It's safe to run again.
To use Terraform instead, choose Terraform under Setup format, enter the Organization ID or Project ID and, for an organization or folder, the Host project ID. Apply the Terraform Ace shows with the roles above, then paste its ace_connection_code output as the connection code. Terraform is available with keyless access only.
Ace checks the account's permissions, then reads your inventory for the System Map.
Add actions
Actions need keyless access.
- In Settings > Integrations > Google Cloud, find Actions.
- Under Choose what Ace can change, select the groups.
- Choose Show the setup. Under Run the setup in Google Cloud, run the command in Cloud Shell or apply the Terraform Ace shows. Creating the roles needs Organization Role Administrator, as in Before you start. It adds the
ace-actionsaccount and leaves read access as it is. - When the setup finishes, choose Verify and turn on. Ace checks which groups the account holds and stores no credential.
- For each workload, go to its settings, open Cloud resources (
https://app.tryace.dev/workloads/<workload-id>/settings/resources) and choose Google Cloud. Tick the resources Ace may change and choose Confirm resources.
Resources linked to the workload start ticked, and its direct dependencies start unticked. You can search the System Map for others or add one by its full resource name, such as //run.googleapis.com/projects/my-project/locations/us-central1/services/checkout. Removing a resource cancels the plans waiting for approval on it, and other plans stay.
Set up with an agent
An agent can do the Google Cloud side from the terminal. An admin connects it in Ace. See how agent setup works. The agent runs setup in place of Cloud Shell, with the gcloud CLI signed in as someone who holds the roles above.
-
Admin: copy the setup command. Open Settings > Integrations > Google Cloud and choose the Scope, Access and any groups under Allow actions (optional). Give the agent the command under Run the setup in Google Cloud. It carries the one Ace identity Google will trust, so the agent never writes its own.
-
Agent: run the setup. Find the IDs, preview the plan, then run it:
gcloud auth login gcloud organizations list gcloud projects list curl -fsSL https://app.tryace.dev/setup/gcp.sh | bash -s -- <flags from the command> --project <host-project-id> --plan curl -fsSL https://app.tryace.dev/setup/gcp.sh | bash -s -- <flags from the command> --project <host-project-id> --yes- The agent runs the
--yescommand only after you approve the plan. An agent's shell usually can't answer the script's prompt, so it needs--yes. - If a role is missing, the script stops with nothing changed. It prints the
gcloudcommand that grants the role, for an IAM admin to run. - To split setup, add
--part project. The script prints the second command for the IAM admin. - With a service account key, the agent moves
ace-read-key.jsonto~/.ace-setup/gcp-read-key.
- The agent runs the
-
Admin: connect Google Cloud. Under Paste the code it printed, enter:
- Connection code: the
ace-gcp:line the script printed last - Service account key, for key access only: contents of
~/.ace-setup/gcp-read-key
Choose Connect Google Cloud, then delete the key file.
- Connection code: the
-
Agent: verify. Check the accounts, the keyless trust and the read account's roles:
gcloud iam service-accounts describe ace-read@<host-project-id>.iam.gserviceaccount.com --project <host-project-id> gcloud iam workload-identity-pools describe ace-ai --location global --project <host-project-id> gcloud organizations get-iam-policy <organization-id> \ --flatten 'bindings[].members' \ --filter 'bindings.members:serviceAccount:ace-read@<host-project-id>.iam.gserviceaccount.com' \ --format 'value(bindings.role)'For a folder or project, use
gcloud resource-manager folders get-iam-policy <folder-id>orgcloud projects get-iam-policy <project-id>with the same flags. The last command lists the read account's roles. Then the admin chooses Test connection, and Verify and turn on under Actions if they chose groups.
Confirm it's working
- The Connection section of the Google Cloud page shows the organization, folder or project, and Keyless access or Service account key with the service account.
- Test connection passes. If Read roles missing appears, the read roles aren't granted yet. If you split setup, the second part hasn't run. Otherwise run the setup again, then test the connection.
- System Map shows the last sync, and your projects appear on the System Map within a few minutes.
- With actions on, Actions shows the actions account with the groups it holds and when it was verified.
Disconnect
To turn off actions only, choose Turn off in Actions. Ace stops making Google Cloud changes, and the service account and its roles stay in your project until you delete them.
To disconnect Google Cloud, choose Disconnect on the Google Cloud page. Ace stops reading your Google Cloud resources, but it doesn't delete anything in Google Cloud. To remove Ace's access:
- Delete the
ace-readandace-actionsservice accounts and theace-aiworkload identity pool from the host project. - Remove their role bindings from the organization, folder or project in IAM.
- Delete the custom roles named
aceActions*in the organization's Roles.
To delete the accounts and the pool with gcloud:
gcloud iam service-accounts delete ace-read@<host-project-id>.iam.gserviceaccount.com --project <host-project-id>
gcloud iam service-accounts delete ace-actions@<host-project-id>.iam.gserviceaccount.com --project <host-project-id>
gcloud iam workload-identity-pools delete ace-ai --location global --project <host-project-id>Next steps
- Actions and approvals: approval, automatic policies and undo.
- AWS: connect AWS to the same System Map.
- Security and data: what Ace stores and how it is protected.
AWS
Connect AWS so Ace can read CloudWatch, CloudTrail, X-Ray and your infrastructure during investigations, open incidents from CloudWatch alarms, scan for savings and, if you allow it, make approved changes.
Kubernetes
Connect a Kubernetes cluster so Ace can read resources, events and logs across the cluster or in the namespaces you choose, open incidents from failing pods and nodes and, if you allow it, make approved changes.