discover · compare · collections · requirements · contribute · feed · watching · stats · help with your tokens

hydra-db/hydradb · source

Snapshot 0 · 2026-11-04T12:47:28Z · Rust · AGPL-3.0

GitHub description at capture

HydraDB - fast graph database on object storage

README at capture

# HydraDB

[![License: AGPL-3.0](https://img.shields.io/badge/license-AGPL--3.0-blue.svg)](LICENSE)
[![Rust 1.91+](https://img.shields.io/badge/rust-1.91%2B-orange.svg)](rust-toolchain.toml)
[![Discord](https://img.shields.io/badge/Discord-join-5865F2?logo=discord&logoColor=white)](https://discord.gg/HcBKJHXmnu)

Join the community on [Discord](https://discord.gg/HcBKJHXmnu). Contributions
start with an approved issue. See [CONTRIBUTING.md](CONTRIBUTING.md).

HydraDB is an object-store-native distributed graph database written in Rust.
It combines durable graph storage on SlateDB with snapshot-consistent
OpenCypher queries, GraphBLAS traversal, Neo4j-compatible Bolt connectivity,
and an HTTPS query API.

S3-compatible object storage is the durable source of truth. Query nodes and
indexers keep only disposable state in memory and on local SSD or NVMe, so they
can be replaced or scaled without moving the graph itself.

## Why HydraDB

- **Object-store durability.** Graph records, WALs, manifests, and immutable
  traversal indexes live in S3-compatible storage.
- **Independent compute.** Query nodes and indexers scale separately and can
  rebuild their local caches from durable state.
- **Safe writer handoff.** Object-store CAS leases select the active writer for
  each cell, while SlateDB writer epochs fence stale writers.
- **Consistent reads.** Every query runs against one pinned SlateDB snapshot.
  Indexed traversal combines a compiled CSC generation with its visible WAL
  overlay.
- **Graph-native execution.** The planner uses property indexes, reverse
  adjacency, sparse traversal, and SuiteSparse GraphBLAS where appropriate.
- **Familiar clients.** Applications can use Neo4j drivers over Bolt 5.x or the
  typed JSON and streaming NDJSON HTTP API.
- **Bounded operation.** Authentication, authorization, deadlines, result
  limits, backpressure, cancellation, cache budgets, metrics, and traces are
  part of the server runtime.

## Architecture

```mermaid
flowchart LR
    C["Applications<br/>Neo4j drivers or HTTPS"]
    SVC["Service or load balancer"]

    subgraph Q["Query tier"]
        Q1["graph-node"]
        Q2["graph-node"]
        QN["graph-node"]
    end

    subgraph I["Indexing tier"]
        IX1["graph-indexer"]
        IXN["graph-indexer"]
    end

    CACHE["Disposable memory and SSD cache"]
    STORE["S3-compatible object storage<br/>WAL, SSTs, leases, CSC generations"]

    C --> SVC
    SVC --> Q1
    SVC --> Q2
    SVC --> QN
    Q1 <--> CACHE
    Q2 <--> CACHE
    QN <--> CACHE
    Q1 <--> STORE
    Q2 <--> STORE
    QN <--> STORE
    IX1 <--> STORE
    IXN <--> STORE
```

Query nodes serve reads and canonical graph mutations. Indexer workers build
immutable CSC generations asynchronously and publish them through atomic
object-store pointers. Readers remain correct when an index is absent or behind
because the visible WAL tail is applied to the indexed base.

See [architecture.md](architecture.md) for the storage model, query pipeline,
writer coordination, index lifecycle, and failure semantics.

## Getting Started

### Prerequisites

HydraDB requires Rust 1.91 or newer, a C/C++ toolchain,
`libcypher-parser`, and SuiteSparse GraphBLAS.

Ubuntu or WSL:

```bash
sudo apt-get update
sudo apt-get install -y \
  build-essential clang libclang-dev cmake pkg-config \
  libcypher-parser-dev libgraphblas-dev \
  curl git python3 python3-venv
```

The last line is not needed to build, but the steps below use it: `curl` for
the Rust installer and the readiness checks, `git` to clone, and `python3-venv`
for the Neo4j driver used by `scripts/runtime_smoke.sh`.

macOS with Homebrew:

```bash
xcode-select --install
brew install just cmake pkg-config llvm suite-sparse coreutils
brew install cleishm/neo4j/libcypher-parser

# Rust, only if `rustup toolchain list` does not already show a stable toolchain:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```

`libcypher-parser` is not in homebrew-core; the fully-qualified
`cleishm/neo4j/...` name adds the tap automatically. A plain
`brew install libcypher-parser` fails with `No available formula`.

Rust comes from the official installer rather than Homebrew because the
`rustup` formula is keg-only and no longer ships a `rustup-init` binary, so
`brew install rustup` leaves nothing named `rustup` on `PATH`.
`rust-toolchain.toml` pins `channel = "stable"`, so any rustup-managed stable
toolchain works.

No `PKG_CONFIG_PATH` export is needed: `libcypher-parser` is not keg-only, so
Homebrew links `cypher-parser.pc` into the default `pkg-config` search path.

[`just`](https://github.com/casey/just) is the supported command runner for the
repository. Install it with `cargo install just --locked` when your package
manager does not provide it. Docker is optional and is used only by MinIO,
Neo4j comparison, image-build, and Kubernetes harnesses.

### Clone and verify

```bash
git clone https://github.com/hydra-db/hydradb.git
cd hydradb

just native-check
just smoke
```

The smoke example creates a local graph, writes and deletes edges, runs a sparse
traversal, closes the database, reopens it, and verifies the durable result.
The recipe creates and removes an isolated local object-store directory. Use
`just smoke-graphblas` to pin the traversal kernel to SuiteSparse GraphBLAS.

To exercise the same flow against an ephemeral MinIO instance:

```bash
just minio-smoke
```

### Run a local server

The following starts a single plaintext development node backed by a local
directory. TLS is required by default in deployed environments; plaintext must
be enabled explicitly.

```bash
mkdir -p .hydradb/store .hydradb/cache
printf '%s\n' 'local-development-token-32-bytes' > .hydradb/auth-token

export CLOUD_PROVIDER=local
export LOCAL_PATH="$PWD/.hydradb/store"
export GRAPH_NAMESPACE=default
export GRAPH_ID=default
export GRAPH_CELL_ID=cell-0
export GRAPH_CELLS=cell-0
export GRAPH_NODE_ID=node-0
export GRAPH_BOLT_NODE_ADDRESSES=node-0=127.0.0.1:7687
export GRAPH_ADVERTISED_BOLT_ADDR=127.0.0.1:7687
export GRAPH_DATA_CACHE_DIR="$PWD/.hydradb/cache"
# Cache chunks default to 4 MiB. Smaller parts reduce cold probe overfetch,
# but can increase later cache misses; benchmark before changing this value.
# The value must be a positive multiple of 1024.
export GRAPH_DATA_CACHE_PART_BYTES=4194304
# Process-wide budget retained by SlateDB disk-cache file caches. Scoped
# runtimes divide it across every possible cell reader and writer.
export GRAPH_DATA_CACHE_MAX_OPEN_FILES=512
export GRAPH_AUTH_TOKEN_FILE="$PWD/.hydradb/auth-token"
export GRAPH_ALLOW_PLAINTEXT=true

# graph-node's async query futures exceed the default thread stack. Without
# this the node builds, serves /readyz, and then aborts on the first query.
export RUST_MIN_STACK=33554432

# macOS: cargo is invoked directly here, so it does not inherit what the
# justfile exports. Linux installs these on default search paths already.
if command -v brew >/dev/null; then
  export BINDGEN_EXTRA_CLANG_ARGS="-I$(brew --prefix)/include"
  export LIBRARY_PATH="$(brew --prefix)/lib"
fi

cargo run --locked --features server-runtime --bin graph-node
```

The node runs in the foreground and does not return; that is it working, not
hanging. Run the checks below from a second shell.

The node listens on:

| Endpoint | Address | Purpose |
|---|---|---|
| Bolt | `127.0.0.1:7687` | Neo4j-driver-compatible queries |
| HTTP | `127.0.0.1:8443` | JSON and NDJSON query API |
| Admin | `127.0.0.1:9090` | readiness and Prometheus metrics |

In another terminal, write and read a small graph through HTTP:

```bash
TOKEN='local-development-token-32-bytes'

curl -sS http://127.0.0.1:8443/v1/graphs/default/query \
  -H "Authorization: Bearer $TOKEN" \
  -H 'X-Graph-Namespace: default' \
  -H 'Content-Type: application/json' \
  --data '{"cell_id":"cell-0","query":"CREATE (a {id: 1})-[:FOLLOWS]->(b {id: 2})"}'

curl -sS http://127.0.0.1:8443/v1/graphs/default/query \
  -H "Authorization: Bearer $TOKEN" \
  -H 'X-Graph-Namespace: default' \
  -H 'Content-Type: application/json' \
  --data '{"cell_id":"cell-0","query":"MATCH (a {id: 1})-[:FOLLOWS]->(b) RETURN b.id AS id"}'
```

The second call returns one row containing
`{"type":"vertex_id","value":2}`. A listening port is not proof the node works;
a round-tripped write is.

For a scripted Bolt and HTTP round trip, install the Python Neo4j driver and
run. Homebrew's and Debian's Python both refuse a bare `pip install` under
PEP 668, so use a virtualenv (`apt-get install -y python3-venv` on
Debian/Ubuntu):

```bash
python3 -m venv /tmp/hydradb-venv && /tmp/hydradb-venv/bin/pip install neo4j

# macOS: this script calls cargo directly, so it does not inherit what the
# justfile exports. Without this it fails at bindgen with
# `'cypher-parser.h' file not found`. Linux needs neither.
if command -v brew >/dev/null; then
  export BINDGEN_EXTRA_CLANG_ARGS="-I$(brew --prefix)/include"
  export LIBRARY_PATH="$(brew --prefix)/lib"
fi

PYTHON=/tmp/hydradb-venv/bin/python bash scripts/runtime_smoke.sh
```

Prints `runtime-smoke-ok`. The node's log is at
`/tmp/sgk-runtime-smoke/node.log`; read it first if the script fails.

### Troubleshooting local runs

| Symptom | Cause and fix |
|---|---|
| `No available formula with the name "libcypher-parser"` | Use the tap: `brew install cleishm/neo4j/libcypher-parser` |
| `command not found: rustup-init` | Homebrew's `rustup` is keg-only and no longer ships it; use the official installer above |
| `invalid environment variable CLOUD_PROVIDER value \`null\`` | `CLOUD_PROVIDER` is unset — `null` means absent, not the string. `local` also needs `LOCAL_PATH`, pointing at a directory that already exists |
| `wrapper.h:4:10: fatal error: 'cypher-parser.h' file not found` | `BINDGEN_EXTRA_CLANG_ARGS` unset while invoking `cargo` directly on macOS. Prefer `just`, which exports it |
| Node answers `/readyz`, then aborts with `has overflowed its stack` on the first query | `RUST_MIN_STACK` unset; export `33554432` |
| `curl: (7) Failed to connect ... port 9090` | The node is not running. `graph-node` holds the foreground, so start it in its own shell |

## Querying

HydraDB supports a practical OpenCypher subset for graph reads and mutations,
including typed relationships, bounded variable-length paths, property and
label predicates, ordering, pagination, aggregation, `OPTIONAL MATCH`, `UNION`,
and batched `UNWIND` writes.

Applications can connect with a Neo4j driver using a routed URI:

```text
neo4j://127.0.0.1:7687
```

Use `neo4j+s://` with a publicly trusted certificate or `neo4j+ssc://` for a
self-signed development certificate. Direct `bolt://` node addresses are for
diagnostics and targeted failure tests; write-capable clustered clients should
use routing.

### Native path procedures

HydraDB includes native snapshot-scoped path procedures:

- `algo.SPpaths` finds bounded paths between one source and one target.
- `algo.SSpaths` finds bounded paths from one source.
- `algo.MSpaths` resolves many indexed source and target values and evaluates
  them together, avoiding client-side query fan-out.

```cypher
CALL algo.MSpaths({
  sourceLabel: 'Entity',
  sourceProperty: 'name',
  sourceValues: ['alpha', 'beta', 'gamma'],
  targetValues: ['alpha', 'beta', 'gamma'],
  pairwise: true,
  relTypes: ['RELATES'],
  relDirection: 'both',
  maxLen: 3,
  pathCount: 5,
  fairRelationshipVariants: true,
  resultLimit: 100
})
YIELD path
RETURN path
```

The procedures use one pinned storage snapshot, compiled GraphBLAS topology
when available, the visible WAL overlay, and bounded metadata hydration.

## Read Consistency

HydraDB exposes two read modes:

| Mode | Behavior |
|---|---|
| `causal` | Uses the node's current durable reader view and refreshes when a supplied bookmark requires a newer sequence. This is the default hot path. |
| `strong` | Refreshes the SlateDB reader from object storage before pinning the query snapshot. This pays the object-store freshness cost. |

HTTPS requests set `"consistency": "causal"` or `"strong"` in the request
body. Bolt clients set `consistency` in `RUN` metadata or
`hydradb.consistency` in transaction metadata. The former
`turbolay.consistency` key remains accepted as a migration alias.

## Kubernetes

The Helm chart deploys query nodes, indexer workers, services, cache volumes,
network policies, disruption budgets, TLS resources, authentication, and
optional Prometheus integration.

```bash
helm upgrade --install hydradb charts/hydradb \
  --namespace hydradb \
  --create-namespace \
  --values charts/hydradb/examples/values-eks.yaml \
  --atomic \
  --timeout 15m
```

Copy and edit the example values before deploying. Object-store credentials,
bucket names, image references, TLS, advertised Bolt addresses, and workload
identity are environment-specific. See the values file comments and the chart
templates under `charts/hydradb/` for configuration and rollout details.

## Observability

The public HTTP server exposes `GET /healthz`. The graph-node and indexer admin
servers expose:

```text
GET /readyz
GET /metrics
```

The runtime emits structured tracing fields for query fingerprints, access
paths, cache outcomes, consistency mode, scope, cell, storage sequence, and
planner decisions. Build with `--features server-runtime,otlp` or
`--features indexer-runtime,otlp` to export OpenTelemetry data.

Prometheus duration histograms have deliberately different units, and the name
suffix is the only place the unit is stated: `/metrics` emits no `# HELP` text.
Client operation latencies such as `graph_client_operation_read_duration_seconds`
are in seconds; the bookmark-wait, query, mutation-batch, and query-transport
histograms end in `_microseconds`. Check the suffix before building latency
dashboards or alerts, and never sum, divide, or compare series across the two.

## Development

Run `just` or `just help` to list the command surface. Recipes use Bash and run
from the repository root. The full native suite requires `libcypher-parser` and
SuiteSparse GraphBLAS.

### Verification Recipes

| Recipe | Coverage |
|---|---|
| `just native-check` | Verifies that `cypher-parser` and GraphBLAS are discoverable |
| `just fmt`, `just fmt-check` | Formats Rust or checks formatting without modifying files |
| `just clippy` | Default-feature lint used by CI |
| `just clippy-chaos`, `just clippy-opencypher` | Chaos-harness and OpenCypher lint configurations |
| `just clippy-native`, `just clippy-client-protocols`, `just clippy-runtime` | Full native, public protocol, and production runtime lint configurations |
| `just check` | Checks every default-feature target |
| `just check-all-features` | Checks every target with every Cargo feature |
| `just check-client-api`, `just check-bolt-server` | Checks shared client code and standalone Bolt independently |
| `just check-examples`, `just check-examples-native`, `just check-examples-chaos` | Checks example targets under their supported feature sets |
| `just ci` | Runs the complete local CI-equivalent sequence; a clean feature-matrix run can take tens of minutes (25m 41s in the verification run for this README) |

`scripts/ci_local.sh` is a compatibility entry point that sets a shared Cargo
target directory and delegates to `just ci`, so the script and Justfile cannot
silently drift into different test matrices.

### Local Harnesses

| Recipe | What it runs | Output or side effect |
|---|---|---|
| `just smoke` | Isolated local object-store write, traversal, reopen, and verification | Temporary directory removed on exit |
| `just smoke-graphblas` | The same smoke flow with SuiteSparse selected explicitly | Temporary directory removed on exit |
| `just query-correctness` | Exact OpenCypher result checks | `bench-results/query_correctness.csv` and `.log` |
| `just stress` | Multiprocess writes, restart recovery, compaction, GC, and verification | Temporary local stores removed on exit |
| `just fence` | Hard SlateDB writer-takeover proof | Temporary local stores removed on exit |

The benchmark recipes intentionally use production-sized defaults and can run
for a long time. Override their documented `GRAPH_QUERY_*` environment
variables for a small development sample.

### Docker And MinIO Harnesses

| Recipe | Purpose |
|---|---|
| `just minio-smoke` | Runs the object-store smoke flow against an ephemeral MinIO container |
| `just minio-query-correctness` | Runs exact query checks against MinIO |
| `just minio-chaos` | Pauses, restarts, and recovers MinIO during graph operations |
| `just minio-fence` | Runs writer takeover against MinIO |
| `just minio-mbt` | Replays the formal MBT adapters against MinIO; unavailable in this checkout because the referenced `tests/formal_mbt*.rs` targets are absent |

The MinIO recipes create isolated containers, networks, buckets, and temporary
configuration files. Their cleanup traps remove those resources unless a
recipe-specific keep flag is set. Docker may pull pinned images on the first
run.

### Standalone Scripts

The scripts below are not all exposed as Just recipes because several operate
external infrastructure or incur cloud cost.

| Script | Requirements and behavior |
|---|---|
| `scripts/runtime_smoke.sh` | Builds `graph-node`, checks readiness and metrics, then exercises Bolt, scoped databases, HTTP, and graceful shutdown; requires Python `neo4j` |
| `scripts/bolt_neo4j_driver_smoke.sh` | Exercises direct and routing Bolt URIs through the official Python Neo4j driver |
| `scripts/query_correctness.sh` | Implements the corresponding local Just recipe |
| `scripts/multiprocess_stress.sh`, `scripts/fence_takeover.sh` | Implement the local stress and fencing recipes |
| `scripts/minio_*.sh` | MinIO smoke, correctness, chaos, fencing, MBT, and write-profile harnesses; require Docker |
| `scripts/multinode_k3s.sh` | Creates a disposable multi-node K3d cluster and performs disruptive failover tests; requires Docker, K3d, kubectl, and Helm |
| `scripts/deploy_single_node_k3s.sh` | Builds and deploys to an existing K3s host using an S3 bucket; changes Kubernetes and AWS resources |
| `scripts/multinode_k3s_client.py` | Runs inside the disposable K3d client Pod created by `multinode_k3s.sh` |

`just update-slatedb` is a maintenance command, not a verification command. It
updates the pinned SlateDB dependency in `Cargo.lock`; review and test the
resulting lockfile diff before committing it.

### Repository layout

```text
src/core/           configuration, graph model, cache policy, errors
src/shard/          storage lifecycle, reads, writes, queries, path procedures
src/engine/         routing, placement, immutable indexes, index GC
src/query/          OpenCypher parsing, algebra, planning, transport types
src/client/         Bolt, HTTP, authentication, quotas, cursors
src/sparse_kernel/  Rust sparse and SuiteSparse GraphBLAS execution
crates/             placement and telemetry workspace crates
charts/hydradb/     Kubernetes Helm chart
examples/           smoke, import, benchmark, and correctness programs
scripts/            local, MinIO, stress, fencing, and deployment harnesses
```

## Benchmarks

Published latency and throughput results are available on the
[HydraDB benchmark site](https://hydra-db.github.io/benchmark/). To reproduce
measurements locally or against S3, use the benchmark commands and scripts
documented above.

## Documentation

| Document | Contents |
|---|---|
| [Architecture](architecture.md) | End-to-end design, snapshots, writer ownership, query execution, and indexing |
| [Using HydraDB](USING-HYDRADB.md) | Getting started, querying, and operating HydraDB |
| [OpenCypher subset](cypher-compat.md) | What the engine accepts today, and what it rejects at parse time |
| [Helm chart guide](charts/hydradb/README.md) | Kubernetes configuration, TLS, authentication, upgrades, and verification |
| [Agent notes](AGENTS.md) | How to build, run, and not trip over `RUST_MIN_STACK` |

## Contributing

Issues and pull requests are welcome. Keep changes focused, add regression
coverage for behavioral changes, and run `just ci` before opening a pull
request. Changes to storage, fencing, snapshots, routing, or index publication
should state the invariant they preserve and include a failure-oriented test.

## License

HydraDB is licensed under the [GNU Affero General Public License v3.0](LICENSE).