Analytics 8 documented workflows

BigQuery

Dataset, table, query, scripting, and API workflows.

Documented workflows

8 workflows
  • datasets.create/list/delete
  • tables.create/list/delete
  • SELECT, JOIN, window functions, CTEs, PIVOT/UNPIVOT, MERGE
  • DECLARE, SET, IF/WHILE/LOOP, BEGIN/EXCEPTION, CALL, EXECUTE IMMEDIATE
  • CREATE MATERIALIZED VIEW, auto-refresh
  • query Parquet/CSV/JSON files (local + GCS)
  • gRPC Storage Read/Write API (AVRO + Arrow)
  • INFORMATION_SCHEMA views

Service boundaries

  • DuckDB type coercion is permissive where BigQuery is strict — queries accepted locally may error in production.
  • Division by zero returns Inf/NaN instead of error; integer division returns int instead of FLOAT64.
  • GROUP BY ROLLUP/CUBE/GROUPING SETS not supported.
  • SEMI/ANTI JOIN syntax not supported (rely on EXISTS/NOT EXISTS transpile).
  • BQML (CREATE MODEL, ML.PREDICT, ML.EVALUATE, etc.) returns clear error.
  • AEAD encryption functions and KLL quantile functions not implemented.
  • GEOGRAPHY functions use haversine approximation, not true WGS84 geodesic.
  • BIGNUMERIC truncated to DECIMAL(38,9); FARM_FINGERPRINT requires pyfarmhash for correct values.
  • Time travel (FOR SYSTEM_TIME AS OF), continuous queries, graph queries, search/vector indexes not supported.
  • External object tables (OBJ.* functions) not supported.
  • SELECT, JOIN, window functions, CTEs, PIVOT/UNPIVOT, MERGE: DuckDB-backed via SQLGlot transpiler. ~96% BigQuery SQL coverage, 175+ functions. 958 tests passing. Key gaps: GROUP BY ROLLUP/CUBE/GROUPING SETS, SEMI/ANTI JOIN, BQML (clear error), AEAD encryption, KLL quantiles, graph queries, continuous queries. ⚠ Type coercion risk: DuckDB is permissive where BigQuery is strict — queries that work locally may error in production. Division by zero returns Inf instead of error. GEOGRAPHY: 12 ST_* functions with haversine approximation (not true geodesic).
  • DECLARE, SET, IF/WHILE/LOOP, BEGIN/EXCEPTION, CALL, EXECUTE IMMEDIATE: Full scripting interpreter with variable scoping, BREAK/CONTINUE, RAISE, transactions.
  • CREATE MATERIALIZED VIEW, auto-refresh: Supported in emulator with PARTITION BY, CLUSTER BY, OPTIONS.
  • query Parquet/CSV/JSON files (local + GCS): CREATE EXTERNAL TABLE with format/uris options. Object tables not supported.
  • gRPC Storage Read/Write API (AVRO + Arrow): gRPC endpoint on port 24088. ReadRows with streaming support.
  • INFORMATION_SCHEMA views: 11 views: COLUMNS, TABLES, SCHEMATA, PARTITIONS, VIEWS, ROUTINES, TABLE_OPTIONS, TABLE_STORAGE, JOBS_BY_*, COLUMN_FIELD_PATHS.
  • Survives restart with the LocalCloud volume
  • Assembled dependency identity remains release-unverified.

Next steps