Architecture

Graviton separates pure domain logic from effectful runtime code.

Diagram scope

The High-Level System View below reflects modules that exist in this repository. The Quasar + Graviton topology is a planned product architecture (control plane, job runner, multi-node CAS): useful for direction, not a guarantee of what is deployed or wired end-to-end today.

High-Level System View

Accuracy notes for this diagram: the CLI is the graviton-cli SBT project (./sbt "cli/run …"), not a separate binary artifact. gRPC is not started by server/run today. RocksDB is a backend module but is not required for the default HTTP + Postgres + fs/S3 flows.

Quasar + Graviton (service topology)

Transducer Algebra

The Transducer algebra is the composable pipeline engine that sits between the pure core and the effectful runtime. Transducers compose via >>> (sequential) and &&& (fanout) with automatic Record-state merging:

bytes → countBytes >>> hashBytes >>> rechunk(blockSize) → CanonicalBlock

Each transducer produces a typed Record summary. After composition, the summary contains all named fields from all stages — accessible by name, never by index. Production stages live in IngestPipeline, Transducers, CasIngest, BombGuard, ThroughputMonitor, and BlockVerify (see Transducer Algebra); compression and aggregate framing in the explorer UI are roadmap visuals, not fully implemented transducer chains yet. Transducers compile to ZSink, ZPipeline, or ZChannel.

See the Transducer Algebra page for the full API, or try the Pipeline Explorer to compose stages interactively.

Core

graviton-core contains purely functional data structures and codecs:

• Hashing utilities (HashAlgo, Digest, MultiHasher).
• Binary identifiers (BinaryKey, KeyBits, ViewTransform).
• Locator abstractions (BlobLocator, LocatorStrategy).
• Range utilities (Bound, Interval, Span, RangeSet).
• Manifest encoders and decoders.
• A small UnionFind helper (graviton.core.uf); replica placement that uses it across the runtime is planned, not wired through ingest today.

Streams

graviton-streams bridges the pure types with ZIO Streams by providing chunkers, hashing sinks, and reusable combinators for fan-out and scanning. These helpers orchestrate the content-defined chunking primitives from the zio-blocks-schema library (published to Maven Central).

Runtime

graviton-runtime defines the service ports consumed by the transports and backends. It holds policies, indexes, constraints, and metrics facades while remaining agnostic to concrete storage drivers.

Protocol

• graviton-proto: protobuf contracts for gRPC (and related HTTP design).
• graviton-grpc: generated stubs and service scaffolding; end-to-end gRPC serving from graviton-server is not at parity with HTTP yet (see gRPC API).
• graviton-http: zio-http routes (blob upload/download, dashboard snapshot/stream, health); see HTTP API.

Backends

Each backend implements the runtime ports using specific technologies:

• graviton-s3: AWS SDK v2 object store with multipart uploads.
• graviton-pg: PostgreSQL based object and metadata stores.
• graviton-rocks: RocksDB based key-value primitives with metrics integration.

Server

graviton-server assembles the runtime into a deployable process. It wires configuration, selects block storage (fs or S3-compatible) with Postgres-backed manifests, starts the HTTP server (/api/blobs, health, metrics), and registers an in-memory metrics registry. gRPC, Shardcake, and multipart entity coordination are planned or partial — they are not the primary path for local demos today.