graphd

graphd is a local-first build graph and task runner for general engineering pipelines. Build the graph explicitly, in the same language as your application — no Makefile or CMake context-switching into an untyped build DSL. Side effects are declared alongside the code and verified against what actually happened, so caching is deterministic and truthful by design: the fastest build that still spends compute only where correctness demands. General purpose; hardware and EDA flows are the flagship example, and nothing needs a daemon, cluster, or control plane.

If you have used Bazel or Argo and wished the interesting parts were smaller, faster to adopt, and honest about what actually happened, start with docs/why-graphd.md.

• Explicit graph, in-line authoring. Build the graph in the same languages as your application — Rust, Python, or TypeScript builders that lower to one frozen plan — instead of migrating to a build DSL.
• Declared vs observed truth. Tasks declare inputs, outputs, tools, runtime data, external artifacts, and network resources. Linux execution observes what the process actually touched (ptrace by default; strace when needed) and reports mismatches — which is what keeps parallel, file-heavy tool stitching honest.
• Explainable, truthful caching. Content or metadata fingerprints, typed cache reason codes, all-or-nothing output restore, and replay instead of re-running: the fastest build that still spends compute only where correctness demands.
• Recorded runs. Provenance indexes every task, inspection renders text/JSON/HTML, and a dashboard plus SSE live service stream state while a graph runs.
• Reach when you need it. SSH fleets (per-task runner hints, graphd runners), cloud cache stores (s3://, gs://), opt-in containerized isolation, hierarchical shards, and pointer lists for oversized inputs.
• Leaveable pieces. graphd mtime, graphd cache, graphd sources, and graphd ops are useful standalone utilities; adopting the full graph framework is optional.

Status: 0.2.0, GPL-3.0-or-later, Linux-first. No 1.0 promise until real-world stories exist; see docs/release-readiness.md for the supported surface and known non-goals.

Install

From a checkout:

cargo build --release
target/release/graphd --help

Or install the graphd binary with Cargo:

cargo install --path . --bin graphd

From the repository remote:

cargo install --git ssh://git@git.underzsea.com:2222/microchipster/graphd --bin graphd

60-second demo

scripts/demo.sh

The script builds the CLI, creates a 9-task, 3-wave graph with real inputs, runs it twice, and asserts the second run is served from cache:

• cold run — every task reports needs_run
• warm run — every task reports verified_clean

Then open the visual proof:

target/debug/graphd serve --provenance-dir target/demo-run/provenance --host 127.0.0.1 --port 8790
# http://127.0.0.1:8790/

For the live service plus the Svelte web app:

target/debug/graphd inspect serve --provenance-dir target/demo-run/provenance --host 127.0.0.1 --port 7770
cd apps/graphd-web && npm install && npm run dev   # http://127.0.0.1:4173/

What works today

• Frozen graph plans with selected-target execution, dry runs, and queries (docs/tutorials/graph-dry-run.md, docs/tutorials/run-inspection.md)
• Declared vs observed manifests, including observed network endpoints (docs/network-resources.md)
• Provenance with a rebuildable SQLite query index
• Hybrid cache with typed reason codes and staged restore
• Rust, Python, and TypeScript authoring that lowers to the same IR (docs/authoring-surface.md)
• Hierarchical shards with lazy resolution and digest verification (docs/hierarchical-shards.md)
• SSH fleet execution with per-task and hierarchy runner hints, --dispatch modes, and graphd runners probes; pointer lists, cloud cache stores, and opt-in containerized isolation (docs/fleet-runners.md, docs/pointer-lists.md, docs/containerized-isolation.md)
• Retention and GC helpers that never delete without an explicit policy (docs/retention.md)
• Inspection, timeline, diffs, HTML/dot exports, a localhost dashboard, an SSE live service, and a browser UI
• Observability: /metrics on both services and opt-in OTLP trace export (docs/observability.md)

Quickstart from a checkout

cargo run -p graphd-build -- --help   # CLI
cargo test --workspace                # core, CLI, and integration suites
python -m unittest python.tests.test_api
cd apps/graphd-web && npm test && npm run check

Gate scripts:

• scripts/tests.sh — fmt check, clippy -D warnings, workspace tests, Python tests, frontend suites
• scripts/v1-smoke.sh — canonical hierarchical flow; exports under target/graphd-v1-smoke/
• scripts/demo.sh — the 60-second cache demo above
• scripts/coverage.sh — coverage reports summarized in docs/coverage.md

Where to start

Standalone utilities: docs/tutorials/hardware-smoke-flow.md, docs/tutorials/cache-aware-preview.md, docs/tutorials/authoring-source-elaboration.md

Python authors: docs/tutorials/python-native-bindings.md, python/examples/basic_pipeline.py, python/examples/hierarchical_pipeline.py

TypeScript authors: docs/tutorials/typescript-node-authoring.md, packages/graphd/examples/basic_pipeline.ts

Rust authors: docs/authoring-surface.md, crates/graphd-core/examples/rust_pipeline.rs, crates/graphd-core/examples/hierarchical_flow.rs

Framework users: docs/tutorials/graph-dry-run.md, docs/tutorials/run-inspection.md, docs/tutorials/live-visualization-service.md, docs/tutorials/visualization-snapshot-and-events.md, docs/tutorials/graphd-methodology-for-cross-domain-engineering.md, docs/tutorials/cross-domain-reference-flows.md

Honest limits

• No universal hermeticity: isolation modes are opt-in hardening, not a promise.
• No Kubernetes dependency and no control plane; workers and services run where you run them.
• Network observation currently requires the strace path (the in-process ptrace backend traces file syscalls only).
• Linux-first; other platforms are not supported targets.

License

GPL-3.0-or-later. See LICENSE. Contributions follow CONTRIBUTING.md; security reports follow SECURITY.md.