RFDs/0001
Unified event storage for interactive telemetry
Section, evidence and task links open the public pre-migration snapshot. Current canonical sources require membership.
The canonical discussion remains PR #3. The sectioned design and PoC implementation plans are under review in PR #11.
Architecture summary
Toudai provides Scuba-style recent-data exploration over logs, spans, and structured application events through one logical event model. Native metrics remain separate. The PoC receives OTLP in toudai-ingest, an owned Rust service that writes ClickHouse directly; object-store overflow and catch-up are qualification work. Canonical retention and backup/restore support reconstruction; independent all-event archival remains an explicit decision. Bounded accelerators improve interactive queries without making raw retained fields unqueryable.
GCP-equivalent logical TB/PB milestones and derived capacity/cost projections are planning-only: the 32 TB/month source calibration is unverified. Volume accounting separates these estimates from measured rows, physical bytes, timings, and incurred costs.
Design principles
-
One public-engine deployment serves one customer; keys may restrict datasets. Accounts, workspaces, placement, and shared-cluster isolation belong to the private service (Decision 0002).
-
Preserve accepted source events and disclose approximation, completeness, and freshness independently.
-
Require an authorized dataset and bounded time range for interactive queries.
-
Keep derived storage bounded and retain raw fallback.
-
Treat deployment/dataset policy, privacy, bounded work, durability, and recovery as core correctness; cross-customer isolation is private-service work.
-
Use measured evidence before committing storage, scaling, and latency choices.
Section index
The implementation checklist owns all tasks, grouped by the same six sections below. The former section-checklist paths remain links to those groups.
Dependency order
-
Define the PoC workload, pass/fail thresholds, event contract, and acceptance semantics.
-
Build a deterministic emitter and enforce key validity, optional dataset restrictions, and ingestion policy.
-
Demonstrate OTLP ingestion, raw querying, and canonical-data reconstruction on one node.
-
Demonstrate the operator workflow, then bounded acceleration with raw fallback.
-
Qualification: overflow/catch-up, mixed datasets within one deployment, cancellation, privacy and dataset deletion, and small-topology replicated recovery.
-
Only then expand retention, topology, cold-tier execution, and production qualification.
Experiment results may revise design sections, but every checklist item has exactly one implementation-section owner.
PoC completion gate
The PoC must demonstrate one reproducible vertical slice on a single ClickHouse node, using the physical layout fixed in Decision 0001 as amended by the single-tenant scope of Decision 0002:
OTLP -> toudai-ingest (key auth, validation, mapping) ---------> ClickHouse
|
operator client <- query service (typed contract, diagnostics) <-+
|
reconstruct serving rows <- canonical payload
The slice is: a deterministic fixture emitter; toudai-ingest writing the
serving and canonical tables; a query service that enforces the typed-value
contract and reports plan diagnostics; the minimum aggregate states the
dashboard fixture needs, with raw fallback; exact-event retrieval from the
object-backed payload; reconstruction of serving rows from retained canonical
data; and a recorded operator walkthrough of the compare, group-by, outlier,
and drill-down workflow. Independent health counters must remain visible when
ClickHouse is unavailable.
Use a small deterministic corpus before repeating large retained-volume tests.
Each required milestone must record its fixture, command, expected result,
failure cases, and observed result in an experiment under experiments/.
Performance gates require prospective workload and numeric thresholds; the
historical benchmark medians are characterization, not implicit SLOs.
The implementation companions distinguish PoC-required milestones from later validation and production qualification. Completed historical experiments remain evidence, not evidence that the application milestone is implemented. The acceptance plan requires measured OTLP acknowledgement, loss, duplicate, and recovery behavior. Neither queue admission nor protocol compatibility alone establishes durable acceptance.
PoC completion requires the PoC-required sections of the product,
acceptance, query, and retained public policy plans: the emitter and operator tests,
ingest normal-path and reconstruction tests, raw and accelerated semantic
tests, and key/dataset admission with bounded exploration. Superseded workspace
tasks do not gate the public core (Decision 0002 D6). Each plan’s
Qualification section holds the work that was previously gated as
PoC-required and now follows the slice: overflow and catch-up, acknowledgement
and duplicate injection, backup/restore and the archive decision, policy
registry and quotas, privacy and dataset deletion, admission and mixed-dataset
workload bounds, coverage and live accelerator maintenance, and small-topology
replicated recovery. Qualification items are required before a production
readiness claim and before this RFD moves to committed; a PoC pass does not
mark any of them complete.
Provider comparisons, 100 TB loading, cross-zone qualification, and 24-to-72-hour soaks do not block this PoC or qualification. They remain necessary where applicable before a production readiness claim. Passing the PoC does not establish production SLOs.
Cross-cutting exit criteria
RFD 0001 remains in discussion. The MVP contract is ready to commit only when:
-
the completed Experiment 011 query baseline is incorporated, and separate restart, merge-time, and workload-isolation tests close its explicit gaps;
-
the user workflow and p50/p95 latency, freshness, retention, concurrency, and cost targets are explicit;
-
acceptance, retry, deduplication, archive, and replay semantics are decided (write identity, at-least-once delivery, and payload placement are decided for the PoC in Decision 0001; deduplication and archival remain qualification decisions);
-
cardinality and privacy defaults have concrete per-deployment/dataset budgets and observable rejection behavior;
-
one service can emit the versioned envelope, replay into a clean serving table, and prove event/payload checksums;
-
raw, accelerated, approximate, and incomplete responses expose consistent diagnostics; and
-
the comparison, group-by, histogram, percentile, and exact-event drill-down workflow has been tested with an operator rather than inferred from SQL benchmarks alone.