Cloud acceptance runbook

Use this runbook before the first dbt run against a real BigQuery or Snowflake target for release or customer acceptance. It starts with a local preflight that checks the target identity and configuration offline. Run it, fix every NEEDS ACTION line, and only then follow the manual steps.

Human gate

Cloud acceptance is a human-gated action. Provisioning, running dbt against a real warehouse, spending money, and destroying resources are performed and signed off by a named human operator, not by an automated agent or CI job. An agent may run the offline preflight and prepare commands, but must not run steps 1, 4, or 6 below, and must never write acceptance evidence it did not observe. Until a human completes the evidence checklist, cloud acceptance stays blocked, regardless of offline test results.

What the preflight does and does not do

scripts/cloud_acceptance_preflight.py makes no network call. It does not run dbt, bq, gcloud, SnowSQL, or any other warehouse client. It never plans or applies Terraform and never reads Terraform state or .tfvars. It reads only:

• the selected edition's dbt_project.yml, dependency files, and profiles.example.yml
• the profiles.yml in the profiles directory you name
• the environment variables that the selected profile target references

For service-account or key-pair authentication, it may check whether the configured key file exists, but it never opens or reads that file. It never prints environment-variable values, profile values, credentials, or key paths. When a value is wrong, it names the profile key or variable but not the value. It also does not echo identity arguments it rejects. A pass shows only that the local configuration is consistent. It does not prove access, permissions, cost, or that the target is actually disposable. dbt debug is the first command that contacts the warehouse.

Required disposable identity

Pass the target identity explicitly on the command line. The preflight does not take it from the environment alone. You must also repeat it with --confirm-disposable-target. The naming rules match the Terraform variable validations and CI guards:

Edition	Identity flags	Accepted form	Confirmation
BigQuery	--project, --dataset	Valid Google Cloud project ID (not bigquery-public-data) and a starterkit_-prefixed dataset	PROJECT.DATASET
Snowflake	--account, --database, --schema	Snowflake account identifier (organization-account name or legacy locator), plus Terraform-style database and matching schema	ACCOUNT/DATABASE.SCHEMA

The preflight resolves the selected dbt target and compares BigQuery project and dataset with --project and --dataset. For Snowflake it compares account, database, and schema with the corresponding flags; the account comparison is case-insensitive. Snowflake user, warehouse, role, and authentication settings are checked separately and are not part of the confirmation identity string.

The preflight refuses an identity in any of these cases:

• Contains DBT_DEV. The legacy identifier is rejected as a safety guard. Current Snowflake examples require an explicit Terraform-style disposable database and schema.
• Looks production-like. Name segments such as prod, prd, production, live, main, master, primary, release, customer, client, pii, shared, or public are refused. Segments that only start with product are allowed.

The selected profile target must resolve to the explicit identity. For Snowflake, its resolved account value must match --account (case-insensitive); a literal, environment variable, or env_var() default is acceptable when the selected target resolves to that same account. If a required target value is unset or resolves elsewhere, the preflight fails. The Snowflake example profile has no account, database, schema, role, or warehouse fallback.

Other required configuration

All editions need:

• --target, naming the profile target the run will use.
• An explicit --profiles-dir or DBT_PROFILES_DIR, so dbt cannot fall back to a shared ~/.dbt/profiles.yml.
• An existing edition workspace whose dbt_project.yml name and profile match the edition (retail_analytics or reliable_pipeline), plus both dependency files.

Edition-specific settings:

• BigQuery: location must resolve to US. maximum_bytes_billed must be a positive integer no greater than 1 GB. With oauth, credentials are not inspected. With service-account, the preflight checks only that the key file exists. If the target sets execution_project, it must resolve to --project, so queries are not billed to another project.
• Snowflake: the selected target's resolved account must match --account; user and warehouse must be set. role must be explicit and must not be ACCOUNTADMIN, ORGADMIN, SECURITYADMIN, SYSADMIN, USERADMIN, or PUBLIC. Password auth needs a password. SNOWFLAKE_JWT needs an existing private_key_path file or a private_key value; the file is never opened.

A method or authenticator written as env_var() is resolved before the credential check; one that cannot be resolved offline is refused.

Steps

1. Provision the target separately, if needed. A human operator reviews and applies the edition's Terraform module, following Terraform lifecycle, or runs the setup commands in the operator command reference. This runbook and the preflight never do this. Use only a dedicated non-production project, or a dedicated Snowflake database and schema.
2. Configure the shell. Activate the edition's Python 3.11 environment and export the profile variables, including SNOWFLAKE_ACCOUNT for Snowflake. Keep the resolved target variables unchanged between preflight and dbt commands. Keep secrets in a secret manager or the shell, never in Git. Use a profiles directory inside the edition workspace, such as .dbt/, and merge profile keys in rather than overwriting existing files.

3. Run the preflight from the repository root.

   # BigQuery — export profile vars so they match the confirmed disposable identity.
   # profiles.example.yml defaults BQ_DATASET to dbt_retail when unset; that would
   # fail the preflight against a starterkit_*-prefixed dataset.
   export BQ_PROJECT="my-starterkit-accept"
   export BQ_DATASET="starterkit_accept"
   export BQ_LOCATION="US"
   python scripts/cloud_acceptance_preflight.py bigquery \
     --project "${BQ_PROJECT}" --dataset "${BQ_DATASET}" \
     --confirm-disposable-target "${BQ_PROJECT}.${BQ_DATASET}" \
     --target dev --profiles-dir editions/Bigquery/.dbt
   
   # Snowflake
   export SNOWFLAKE_ACCOUNT="myorg-accept_acct"
   python scripts/cloud_acceptance_preflight.py snowflake \
     --account "${SNOWFLAKE_ACCOUNT}" \
     --database DBT_KIT_CI_ACCEPT_DB --schema DBT_KIT_CI_ACCEPT_SCHEMA \
     --confirm-disposable-target "${SNOWFLAKE_ACCOUNT}/DBT_KIT_CI_ACCEPT_DB.DBT_KIT_CI_ACCEPT_SCHEMA" \
     --target dev --profiles-dir editions/Snowflake/kit/.dbt
   

   Exit code 0 means the local preflight passed, 1 means configuration items need action, and 2 means arguments are missing or invalid. A 0 does not prove warehouse access or that the target is disposable. Do not continue until it exits 0.

4. Run dbt against the warehouse manually. Do this only after the preflight passes and someone has reviewed the expected cost. From the edition's dbt project directory, use --profiles-dir .dbt --target dev (the repository-root paths shown above become .dbt after changing directory). Keep the selected target variables unchanged. Run dbt debug first, then dbt build, or dbt seed and then dbt build for Snowflake. These commands are outside the preflight's scope.

5. Record evidence. Complete the evidence checklist and add the entry to Release validation record. Do not record credentials or key paths.
6. Tear down. When the target is no longer needed, a human operator destroys it with the matching Terraform state, following Terraform lifecycle, or with the teardown commands in the operator command reference, and confirms it is gone.

Operator command reference

These are the exact commands for one acceptance run against a dedicated disposable target. A human operator runs them in order; every command after the preflight contacts the cloud and may cost money. Prefer the edition's Terraform module (see Terraform lifecycle) when it is available; the commands below create only the minimum disposable target directly and remove it afterwards. Replace <suffix> with a short run tag such as the UTC date 20261101 (lowercase in BigQuery names, uppercase in Snowflake names); it must not contain production-like words such as prod, live, or shared.

BigQuery: disposable dataset

Run from the repository root with the BigQuery edition's Python 3.11 environment active and gcloud auth application-default login completed for a dedicated non-production project.

export BQ_PROJECT="<dedicated-non-production-project-id>"
export BQ_DATASET="starterkit_accept_<suffix>"
export BQ_LOCATION="US"

# 1. Create the disposable dataset. Tables expire after 24 hours as a safety net.
bq --project_id="${BQ_PROJECT}" mk --dataset --location=US \
  --default_table_expiration=86400 \
  --description="Disposable starter-kit cloud acceptance dataset" \
  "${BQ_PROJECT}:${BQ_DATASET}"

# 2. Edition-local profile; cp -n never overwrites an existing profile.
mkdir -p editions/Bigquery/.dbt
cp -n editions/Bigquery/profiles.example.yml editions/Bigquery/.dbt/profiles.yml

# 3. Offline preflight (must exit 0). It also checks the bytes-billed cap.
python scripts/cloud_acceptance_preflight.py bigquery \
  --project "${BQ_PROJECT}" --dataset "${BQ_DATASET}" \
  --confirm-disposable-target "${BQ_PROJECT}.${BQ_DATASET}" \
  --target dev --profiles-dir editions/Bigquery/.dbt

# 4. Warehouse run.
cd editions/Bigquery
dbt debug --profiles-dir .dbt --target dev
dbt build --profiles-dir .dbt --target dev
cd ../..

# 5. Teardown: deletes the dataset and every table in it.
bq --project_id="${BQ_PROJECT}" rm -r -f -d "${BQ_PROJECT}:${BQ_DATASET}"
bq --project_id="${BQ_PROJECT}" show "${BQ_PROJECT}:${BQ_DATASET}"   # expected: Not found

Bytes-billed cap. The dev target in profiles.example.yml sets maximum_bytes_billed: 1000000000 (1 GB) on every dbt query, and the preflight fails if the selected target has no cap or a cap above 1 GB. A query that would scan more fails instead of being billed. This is a per-query limit, not a total budget; review the expected scan before step 4. For any ad-hoc inspection query, pass the same cap, for example bq query --maximum_bytes_billed=1000000000 --use_legacy_sql=false '...'.

Snowflake: disposable database, schema, and warehouse

A Snowflake administrator runs the setup and teardown SQL in Snowsight or the Snowflake CLI. The dbt run uses a dedicated non-admin role (here DBT_KIT_ACCEPT_ROLE, which must already exist and be granted to the dbt user); the preflight refuses ACCOUNTADMIN, SYSADMIN, and other built-in admin roles.

-- Setup (administrator). Names follow the Terraform DBT_KIT_<DEV|CI>_<suffix>_* pattern.
create database DBT_KIT_CI_ACCEPT_<suffix>_DB comment = 'Disposable starter-kit cloud acceptance';
create schema DBT_KIT_CI_ACCEPT_<suffix>_DB.DBT_KIT_CI_ACCEPT_<suffix>_SCHEMA;
create warehouse DBT_KIT_CI_ACCEPT_<suffix>_WH
  warehouse_size = 'XSMALL' auto_suspend = 60 auto_resume = true initially_suspended = true;
grant usage on database DBT_KIT_CI_ACCEPT_<suffix>_DB to role DBT_KIT_ACCEPT_ROLE;
grant usage, create table, create view on schema DBT_KIT_CI_ACCEPT_<suffix>_DB.DBT_KIT_CI_ACCEPT_<suffix>_SCHEMA
  to role DBT_KIT_ACCEPT_ROLE;
grant usage on warehouse DBT_KIT_CI_ACCEPT_<suffix>_WH to role DBT_KIT_ACCEPT_ROLE;

Optional compute cap: an ACCOUNTADMIN can attach a resource monitor (for example create resource monitor DBT_KIT_CI_ACCEPT_<suffix>_RM with credit_quota = 1 triggers on 100 percent do suspend_immediate; then alter warehouse DBT_KIT_CI_ACCEPT_<suffix>_WH set resource_monitor = DBT_KIT_CI_ACCEPT_<suffix>_RM;).

# Run from the repository root with the Snowflake edition's Python 3.11 environment active.
# Keep SNOWFLAKE_PASSWORD (or a key-pair setup) in the shell or a secret manager, never in Git.
export SNOWFLAKE_ACCOUNT="<orgname-accountname>"
export SNOWFLAKE_USER="<dbt-user>"
export SNOWFLAKE_ROLE="DBT_KIT_ACCEPT_ROLE"
export SNOWFLAKE_DATABASE="DBT_KIT_CI_ACCEPT_<suffix>_DB"
export SNOWFLAKE_SCHEMA="DBT_KIT_CI_ACCEPT_<suffix>_SCHEMA"
export SNOWFLAKE_WAREHOUSE="DBT_KIT_CI_ACCEPT_<suffix>_WH"

mkdir -p editions/Snowflake/kit/.dbt
cp -n editions/Snowflake/kit/profiles.example.yml editions/Snowflake/kit/.dbt/profiles.yml

python scripts/cloud_acceptance_preflight.py snowflake \
  --account "${SNOWFLAKE_ACCOUNT}" \
  --database "${SNOWFLAKE_DATABASE}" --schema "${SNOWFLAKE_SCHEMA}" \
  --confirm-disposable-target "${SNOWFLAKE_ACCOUNT}/${SNOWFLAKE_DATABASE}.${SNOWFLAKE_SCHEMA}" \
  --target dev --profiles-dir editions/Snowflake/kit/.dbt

cd editions/Snowflake/kit
dbt debug --profiles-dir .dbt --target dev
dbt seed --profiles-dir .dbt --target dev
dbt build --profiles-dir .dbt --target dev
cd ../../..

-- Teardown (administrator): removes the database, its schema and tables, and the warehouse.
drop database if exists DBT_KIT_CI_ACCEPT_<suffix>_DB;
drop warehouse if exists DBT_KIT_CI_ACCEPT_<suffix>_WH;
drop resource monitor if exists DBT_KIT_CI_ACCEPT_<suffix>_RM;  -- only if created
show databases like 'DBT_KIT_CI_ACCEPT_<suffix>_DB';             -- expected: no rows
show warehouses like 'DBT_KIT_CI_ACCEPT_<suffix>_WH';            -- expected: no rows

If the target was created with Terraform instead, tear it down only with ./scripts/terraform_lifecycle.sh <edition> destroy and its matching state, as described in Terraform lifecycle.

Evidence checklist

Cloud acceptance counts as done only when a human operator records every item below from a run they performed or directly observed. Leave an item empty, and the gate blocked, rather than estimating or copying output from another run.

• [ ] Operator name and date (UTC) of the run.
• [ ] Edition (bigquery or snowflake) and the commit SHA that was checked out.
• [ ] Explicit disposable identity: PROJECT.DATASET for BigQuery, or DATABASE.SCHEMA for Snowflake. Do not record the Snowflake account identifier, user, role secrets, or key paths.
• [ ] Evidence that the target was disposable and non-production, such as the Terraform workspace or plan reference that created it (not the state file or .tfvars).
• [ ] Preflight command and its final line showing exit code 0.
• [ ] dbt debug result (pass/fail summary only).
• [ ] dbt build (or dbt seed then dbt build) summary: counts of models, tests, and seeds that passed, warned, errored, or were skipped.
• [ ] Cost observed or estimated for the run, and who reviewed it beforehand.
• [ ] Teardown: date and confirmation that the target was destroyed, or the reason it was kept and the owner responsible.
• [ ] Confirmation that no credentials, profile files, or customer data were committed or attached.

Offline checks such as the preflight, its unit tests, and CI dbt parse are not acceptance evidence; they only show the local configuration is consistent.

Offline tests

From the repository root:

CI=true migration/.venv/bin/python -m unittest discover -s scripts/tests -p test_cloud_acceptance_preflight.py -v
CI=true migration/.venv/bin/python -m unittest discover -s scripts/tests -p test_first_run_preflight.py -v

The tests use synthetic fixtures and a temporary HOME. They put fake dbt, terraform, bq, gcloud, snowsql, and snow executables first on PATH, and these fail the test if invoked. A socket tripwire fails the test on any connection or DNS attempt. The tests also check that sentinel secret values never appear in output.