Dataproc agent testing

Dataproc local testing for AI agents

Use LocalCloud when an agent needs to create, exercise, and reset Dataproc resources without touching a real Google Cloud project. The same SDK shape points at localhost through DATAPROC_EMULATOR_HOST=localhost:24080.

Read docs Check compatibility SDK examples

Agent prompt CTAs

Configure this repo

Have an agent wire an existing project to LocalCloud safely.

Set up this repository to use LocalCloud for local GCP development. First read https://local.cloud/ai/agents.md, then inspect this repo, identify the GCP services and SDK language, configure emulator environment variables, and run the narrowest integration test against localhost. Do not use real GCP credentials or production endpoints.

Configure this repo

Have an agent wire an existing project to LocalCloud safely.

Set up this repository to use LocalCloud for local GCP development. First read https://local.cloud/ai/agents.md, then inspect this repo, identify the GCP services and SDK language, configure emulator environment variables, and run the narrowest integration test against localhost. Do not use real GCP credentials or production endpoints.

Troubleshoot routing

Diagnose why SDKs or Terraform are still reaching real GCP.

Troubleshoot my LocalCloud setup. Read https://local.cloud/ai/agents.md, check whether Docker and the localcloud container are healthy, verify emulator environment variables are set in this shell/test runner, and identify any SDK or Terraform configuration that could still call real Google Cloud.

Fact

Endpoint: DATAPROC_EMULATOR_HOST=localhost:24080.

Fact

Endpoints: gRPC :24080.

Fact

Evidence state: partial; Local development coverage is partial. Known limits: Autoscaling and Kubernetes cluster mode are not complete., Runtime execution requires Docker socket access; without it, cluster metadata remains available but Spark/Hadoop processes are not started..

Fact

Registry default: on; assembled default: on (verified); minimum tier: community.

Fact

Persistence: metadata (verified). Audited metadata survives restart while the LocalCloud volume is retained

Agent quickstart

Route the SDK before writing code

Start LocalCloud, export DATAPROC_EMULATOR_HOST=localhost:24080, and make the agent perform one Dataproc operation before changing application logic. That catches accidental production routing early.

Validation example

Prefer one representative behavior over broad smoke tests

A useful agent check creates local Dataproc state, reads it back with the project SDK, and records which feature was covered. It should not require a GCP account, service-account key, or billing project.

State setup

Use only documented setup paths

Create deterministic state through a contract-documented seed registrar or through an operation listed on this page. Do not assume every service supports seed data, reset, or persistent state.

Copy-ready setup

Commands and prompts

Use these snippets as starting points, then keep the checks scoped to localhost until you intentionally validate against real Google Cloud.

Environment

eval "$(localcloud env)"
# Verify that the generated environment includes DATAPROC_EMULATOR_HOST; do not replace a CLI-remapped value with a hard-coded port.

Compatibility

Local check versus release validation

Area	LocalCloud local check	Real GCP still needed for
SDK routing	DATAPROC_EMULATOR_HOST=localhost:24080 points clients at localhost.	Production endpoint, auth, IAM, quota, and regional behavior.
Supported features	clusters.create/list/get/delete (verified); jobs submit/list/get (partial) — Cluster jobs run in the Dataproc runtime container; jobs without a cluster use ephemeral serverless containers. Execution requires Docker socket access and a compatible runtime image; otherwise only metadata paths are available.	Autoscaling and Kubernetes cluster mode are not complete.; Runtime execution requires Docker socket access; without it, cluster metadata remains available but Spark/Hadoop processes are not started.
Agent safety	No default cloud account, credentials, or billing project required.	Final release validation in the target GCP project.

Limitations

Use the sandbox deliberately

Permitted workflows use local endpoint values and should stop rather than fall back to real Google Cloud or real credentials. The Public Preview License permits individuals and organizations, including for-profit companies, to use LocalCloud for non-production internal development, testing, CI, evaluation, and pilots.
LocalCloud emulates bounded local development workflows. Validate application behavior against real Google Cloud before production deployment.
Before production deployment, unset LocalCloud emulator environment variables and validate behavior against real Google Cloud.
Local development coverage is partial. Known limits: Autoscaling and Kubernetes cluster mode are not complete., Runtime execution requires Docker socket access; without it, cluster metadata remains available but Spark/Hadoop processes are not started..

Next routes

Keep the agent on the supported path

Dataproc service page — Service-specific supported and unsupported capability list.
Compatibility matrix — Check current support boundaries before relying on a local-only test.
Service catalog — Review every LocalCloud service, endpoint, and limitation.
SDK examples — Use standard Google Cloud SDKs pointed at localhost.
Seed data — Load deterministic fixtures for repeatable agent and CI runs.

Sources and review

Claims are tied to current sources

LocalCloud service metadata — Reviewed 2026-08-26; implementation local-facade.
LocalCloud agentic facts reviewed 2026-08-26.