Project Status

RememberStack is being built design-first: the complete system — requirements, architecture, data model, and every load-bearing decision with its rationale — was designed and reviewed before implementation began. That design is public in the repository and is the authority these docs summarize.

Pre-release software is now installable as v0.2.0. Install the Python distribution from PyPI or pull the public container from GHCR. The interfaces and data model may still change before a stable release.

What exists today

• The complete design — in plan/: requirements, binding per-subsystem designs, the research and analyses behind them, and a build-order roadmap.
• The decision log — decisions.md: every architecture decision (D1, D2, …) with context, rationale, and its later refinements. If you want to know why something is the way it is, start there.
• Design-conformance evals — a suite of acceptance checks (in plan/implementation_evals/) that judge the implementation against the binding design as it lands.
• Phase 0, complete — the foundations are merged: the full PostgreSQL schema and migrations, the work ledger and worker runtime (idempotent work tracking, retries, dead-lettering), the self-host delivery shell, the deterministic blockizer with its golden regression corpus, and the eval-harness skeleton.
• Phase 2, complete — the truth machinery: the full entity-resolution cascade (exact, fuzzy, phonetic, embedding, and LLM tiers with versioned per-type thresholds and golden-set precision/recall curves), order-independent clustering with fully reversible merges, ontology governance (the Work extension pack and the escape-value promotion funnel), the relation supersession cascade with append-only decision transcripts, the observation adjudicator behind its contradiction eval gate, and a review queue with a CLI whose verdicts write reversible, auditable rows.
• Phase 1, complete — the walking skeleton works end to end as library code: a document ingests through upload → conversion → structure → chunking → embeddings → claim extraction (with a deterministic grounding gate and a full decision ledger) → entity resolution and fact normalization → the Lance search indexes, and is queryable through a minimal HTTP query API (resolve, lookup, search, hydrate) whose every answer carries a self-accounting envelope (grain, validity, freshness, typed negatives).
• Phase 4, complete — the projections: derived, rebuildable read surfaces. The knowledge graph rebuilds whole from PostgreSQL on every cycle (zero drift by construction, entity merges become no-ops), validates before publishing — a merge cycle or a row-count mismatch aborts the snapshot loudly — and ships as an immutable snapshot that read-only readers hot-swap to. Agents query it through a typed graph primitive: neighborhoods and shortest paths, filtered as-of a point in world or system time, with explicit truncation markers instead of silent top-k caps and typed answers when there is nothing to return. Community detection, PageRank, k-core, and connected components run natively on each fresh snapshot and write back to PostgreSQL. And the corpus filesystem is browsable: a generated directory tree where one index file tells an agent what every document in a directory is about, canonical per-document and per-entity paths never move across rebuilds, and the original files sit deliberately off the navigation path behind audited access.
• Phase 5, complete — retrieval complete. The full zero-LLM primitive set is implemented (fuse, rerank, transcript, delta, pages_about, enumerated aggregate, and a streaming batch scan, alongside resolve/lookup/search/graph); recipes are registry rows whose declared grain is enforced mechanically; and the self-accounting envelope surfaces contradictions, withdrawn support, mixed-grain parts, identity regime, belief horizons, freshness, and typed negatives. API, CLI, and MCP all render the same deployment registry. The consumption skill is guarded by the repeatable S58 cold-agent eval, while the retrieval spike battery records filtered Lance search at 10 million rows and graph pagination at a 100,000-edge hub. The client-first rememberstack base wheel contains the typed SDK, remote remember CLI and MCP, lineage-aware E0 ingest, and typed remote connector-management commands plus their deployment composition port, without server dependencies; [server], [connectors-watched-directory], and [k] name the heavier install surfaces.
• Phase 6, complete — Plane K now includes the deterministic control plane and crash-safe single-committer driver, exact rule routing and staleness, deterministic fact-sheet and agent-written prose bands, planner/reflection decisions with quarantine and adoption, and authored-page frontmatter, citations, watches, review flags, and debounced workflow dispatch. D73 removed the proposed K3 tier: personal or organizational principles are authored K2 content, supported by compiled scope pages and never machine-promoted or rewritten.
• Phase 3, complete — the evidence lifecycle: documents that change. Watched sources poll as recorded sync cycles (a local-directory watcher ships; revision and content no-ops, debounce, source-deletion detection); an edited document becomes a new version of its lineage and the full structure route (a deterministic heading-parsed section tree, sanity-checked by a bounded flash-class judge, with a string-anchored LLM fallback for deficient trees) re-reads it; unchanged chunks reuse their prior claims, prefixes, and vectors so cost is proportional to the edit (measured hit rate 0.79 on the spike corpus); reconciliation transitions testimony currency on an append-only ledger, recounts evidence by distinct current lineages, closes solely-supported facts when the source withdrew them (at the sync-cycle barrier, so a moved section is a support swap — never a retract flicker) and flags them for review when only the toolchain changed; deletion removes a document's contribution uniformly while keeping its claims as history; and a standing lifecycle eval suite guards the cache/ledger/count invariants with planted regression canaries. Those lifecycle events now feed the graph, corpus filesystem, and Plane-K control plane described above.
• Phase 7, complete — operational correctness now includes resumable initial-load and version-bump backfill on the same work ledger, a reproducible PostgreSQL scale battery for the designed partitions, indexes, hubs, and batching invariants, and optional per-deployment/stage/lane cost ceilings. Budget exhaustion parks healthy work durably without consuming an attempt; the local CLI reports current spend, remaining budget, tier attribution, and parked work from the same authoritative rows. Typed worker telemetry preserves complete exception chains, while bounded CLI inspection exposes pipeline/DLQ state, poison targets, P2/P3 pointers, and currency-ledger drift; an explicit one-row replay grants only additional attempts, and rebuild drills call the production P2/P3 builders. Hard-forget now purges every active library-controlled surface and replays a separately durable, content-free manifest before serving after a restore. Portability is deliberately smaller than a backup product: operators move PostgreSQL, objects, and Git with native tools; the library defines the fail-closed restore order and rebuilds P1/P2/P3. The fresh-deployment Compose profile now runs PostgreSQL, MinIO, the API, and all ten implemented continuous E/P1 routes, with explicit one-shot P2/P3 publication and machine-verifiable per-version readiness. Release engineering enforces one semantic version across PyPI, the shared API/worker GHCR image, and Compose; CI measures cold start and drills a schema downgrade followed by migrations-before-process-restart. The public v0.1.0 release proved the complete tag workflow across PyPI, GHCR, and GitHub Releases; trusted publishing, protected release tags, and the required repository-native bounded-CLA check are active. Timings remain measurements rather than hosted SLAs, and no export/import CLI, dashboard, billing policy, HA topology, backup scheduler, or control plane enters the library.

The build order

Implementation follows plan/plans/roadmap.md — nine phases, each keyed to the designs it realizes:

How these docs grow

These pages are part of the repository and ship with the code: every change that alters user-facing behavior updates its documentation in the same pull request. As each phase lands, its pages appear here — getting started, ingestion and connectors, retrieval, configuration, deployment, and the API/CLI/MCP references. Until a page exists, the design documents linked above are the truth.

Following along

• Watch the GitHub repository — work lands as reviewed pull requests referencing roadmap work packages.
• Read the decision log for the ideas; skim merged PRs for the build.