Documentation

Perdura Roadmap — Updated with Phase 0

Overview

Perdura is a persistent knowledge graph where LLMs are ephemeral workers. The roadmap now includes Phase 0 — foundational research on memoric binary, a native epistemic encoding that enables early contention detection and per-model track records in vector space.

Phase 0: Memoric Binary (NOW)

Validation status (2026-06-25): synthetic arm — experiment 1 (hidden disagreement, temporal replay) passes; exp 3 near-miss (0.87); exp 2 needs real data. First real-session arm hit consensus collapse (two frontier models agree, 1 contradicts edge in 18 turns); fixed by adversarial boarding (--adversarial-every N), validated to lift contention 0→0.13 on real models. Second real arm (contested seeds + adversarial): contention supply solved (12 contradicts edges), but adversarial contradictions are exogenous — unpredictable by scatter by construction — so exps 1/3 score at chance on them. Inversion finding: contradicting claims are lexically closer than random pairs, not farther — distance locates disagreement but can't measure it; collision_candidates() + stance-audit boarding (--audit-every N) is the repair. Exogeneity fix (2026-06-25): every Node/Edge now carries boarding_mode (organic/adversarial/audit), so exp 1 and the inversion probe (collision_probe.py) score organic+audit contradictions separately from manufactured ones. Third real arm (2026-06-25, first to combine --adversarial-every and --audit-every): experiment 1 passes on real data for the first time — AUC 0.944 (simhash, organic+audit-filtered) vs 0.370 for blake2b (worse than chance, settling open question 6 against it). Exp 3 narrows to a near-miss (0.868, up from 0.56). Exp 2 stays blocked for the third real session running (0 decisions/supersessions) — a structural protocol gap, not thin data. The inversion finding now holds on organic contention too (simhash AUC 0.395), not just adversarial (0.187). Spec still not locked (needs exp 1 and 3); heterogeneous workers (local Qwen vs frontier) remain completely untested — no local model has been reachable in any environment this has run in. Full results: docs/phase0-validation.md.

Goal: Design, implement, and validate a 96-bit epistemic encoding that captures semantic content, node type, confidence, domain tags, temporal context, and supersession lineage.

Deliverables:

Validation criteria: All three experiments pass

  1. Hidden disagreement detection: AUC > 0.7 (embedding scatter predicts contradicts edges)
  2. Per-model track records: |correlation| > 0.6 (vector-space reliability matches merge stats)
  3. Compression without loss: >90% routing agreement (memoric binary routes equivalently to edge-only metrics)

Success path: Lock the memoric binary spec, commit Phase 0 artifacts, then proceed to Phase 1.5.

Risk: If validation fails, revisit bit allocation, distance metrics, or domain encoding. Open question: the semantic-hash choice (Blake2b vs simhash — RFC open question 6) must be settled before the spec is locked.

Phase 1: Graph + Workers (COMPLETED)

Status: ✅ Built and verified

Graph memory + delta extraction with Claude, Gemini, and local Qwen.

Artifacts:

Phase 1.5: Memoric Retrieval Layer (STARTED)

Status (2026-06-17): retrieval layer shipped behind a pluggable Retriever interface (perdura_retrieval.py): --retriever graph (default, byte-identical Phase 1 baseline), hybrid (BM25 + dense + graph expansion), chroma (hybrid with a persistent ChromaDB index). Mind-map visualization shipped (perdura.py viz) and upgraded to a live local dashboard, the Station (perdura.py ui): real-time graph with 2s polling, questions ranked by contention, node inspector, conversation feed, and track records. The mind map now draws collision_candidates() as dotted lines (lexically-close, unlinked claim pairs from different authors) and is servable live over HTTP (GET /viz, operator-only, perdura_service.py), not just as a static file. Update (2026-06-25): memoric binary now reaches the actual briefing text — --memoric-briefings appends a bounded section of lexically-close, unlinked claims (the validated collision locator, collision_candidates()) to every worker's briefing, not just --audit-every turns, so a freshly-boarded or swapped-in model sees latent disagreement its retriever's neighborhood would miss within the same bounded budget — the concrete mechanism behind claim 1's "onload without losing context." Off by default; additive only, byte-identical output when unset (tests/test_briefing.py). Remaining: learned dense embeddings, retrieval A/B on real sessions (research question #1).

Goal: Integrate memoric binary into the graph storage and briefing assembly.

Scope:

Deliverables:

Timeline: 2–3 weeks post-Phase 0 validation

Phase 2: Track Records (STARTED)

Status (2026-06-11): engine shipped (perdura_track.py): per-model and per-domain reliability as a Laplace-smoothed Beta mean over claim outcomes (promoted +1, corroborated-by-another-worker +0.5, challenged -0.5, superseded -1), derived on demand and never persisted. Surfaced via perdura.py track and an operator-only MCP tool (workers never see attribution). Validated against planted synthetic ground truth. Remaining: accumulate real outcome data and calibrate the rubric weights.

Goal: Compute per-model, per-domain reliability from accumulated node outcomes.

Scope:

Deliverables:

Timeline: 3–4 weeks post-Phase 1.5

Phase 3: Epistemic Router (KERNEL SHIPPED — real A/B run, result against)

Status (2026-06-17): the routing kernel shipped (perdura_router.py, perdura.py run --route contention|periodic|random|cheap --budget N): model registry with per-boarding costs and tiers, contention-driven escalation (frontier chosen by live track-record reliability per cost), hard session budgets with local fallback, and a per-decision ledger. The decisive A/B harness shipped too (experiments/escalation_ab.py) with a synthetic positive control: at equal spend and equal flips, mean contention at escalation separates the arms (0.48 contention-routed vs 0.31 random vs 0.14 periodic) — frontier spend lands on actual disagreement.

Update (2026-06-25): the --real mode has now actually run six clean times against real Claude/Gemini calls (--local mock standing in for an unavailable local model). Full record: docs/phase3-ab-results.md. Targeting precision holds on real data and strengthens with the larger pool (contention-routing finds the hottest live disagreement every clean run; pooled, escalation-weighted mean contention at escalation 0.348 contention vs 0.154 periodic vs 0.226 random). But on the metric the thesis was supposed to win on — outcome flips at equal cost — contention-routing has not beaten periodic escalation once across all six clean real-worker runs (5 losses, 1 tie; pooled 5 vs 20 flips over 21 equal-cost escalations). A real confound was found and reconfirmed (this seed graph's contention dilutes with graph size regardless of budget, capping the contention arm's sample at ~4 escalations per run no matter how long the run goes — runs 6–8 repeat run 3's exact count), and a real open question (raw flip-counting may not credit resolving a dispute, only creating new ones) now holds modestly at the pooled level — the contention arm's mean final contention is the lowest of the three escalating arms across all 6 runs — but isn't yet separable from the dilution effect. Neither rescues the headline number; pooling six independent runs (21–24 escalations/arm pooled, not 1–4/run) makes the direction more settled, not less. The local-tier half of the thesis ("cheap-by-default is good enough most of the time") remains completely untested — no real cheap model has been available in any environment this has been built in. See docs/phase3-ab-results.md for the full data and verdict table.

Goal: Implement cost-driven routing: escalate to frontier/specialist models only where contention is high.

Scope:

Deliverables:

Routing decision logic:

contention_score = 0.5 * edge_signal + 0.5 * embedding_scatter
cost_budget_remaining = ...
model_registry.sort_by(
    (contention_score * reliability[model] / cost[model]) DESC
)
selected_model = next available within budget

Timeline: 4–6 weeks post-Phase 2

Enterprise Track (parallel to the research phases)

Full plan: docs/enterprise.md. The worker protocol is already a service boundary (delta plane / briefing plane / operator plane), so enterprise deployment is an application layer over the research engine — gated so it never displaces the research focus.

Phase 4: Multi-Model Collaborative Routing (Optional)

Goal: Enable workers to request specific collaborators for sub-questions.

Scope:

Timeline: Beyond Phase 3 (if proven valuable)

How the Phases Build on Each Other

Phase 0: Memoric Binary
    ↓
  Validation experiments run
    ↓ (if passed)
    ↓
Phase 1.5: Embed memoric binary into graph + retrieval
    ↓
  Workers use briefings with memoric signals
    ↓
Phase 2: Compute track records from node outcomes
    ↓
  Now we know which models are reliable where
    ↓
Phase 3: Route based on contention + reliability + cost
    ↓
  Perdura is a functioning epistemic router

Key Research Questions Answered by Each Phase

Phase Question Answer Method
0 Does memoric binary encoding work? Three validation experiments
1.5 Can we detect disagreement before explicit edges? Embedding scatter vs contradicts log
2 Which models are actually reliable? Per-model outcome distribution analysis
3 Does contention-driven routing save money? Cost tracking + performance comparison

Artifacts by Phase

Phase 0 (NOW)
├── docs/memoric-binary.md         (RFC)
├── perdura_memoric.py             (encoder/decoder)
└── experiments/memoric_eval.py    (validation)

Phase 1 (DONE)
├── perdura.py                     (CLI)
├── perdura_server.py              (MCP station)
└── perdura_graph.json             (state, local)

Phase 1.5 (PLANNED)
├── perdura.py (updated)           (graph + memoric integration)
├── visualization/                 (mind map web UI)
└── docs/

Phase 2 (PLANNED)
├── perdura.py (updated)           (track record computation)
├── track_records.json             (per-model scores)
└── analytics/

Phase 3 (PLANNED)
├── perdura_router.py              (routing engine)
├── model_registry.json            (model metadata)
└── cost_tracking.json

Success Metrics

By end of Phase 3, Perdura should exhibit:

  1. Inverted persistence claim: A model can be swapped mid-session; graph state survives unharmed. ✓ (testable in Phase 1)
  2. Epistemic track records claim: Per-model reliability measurable from embedding space and merge history. ✓ (testable in Phase 2)
  3. Contention-driven economics claim: Cheap models handle low-contention subgraphs; expensive models see only where disagreement clusters. Measured in Phase 3, real 6-run A/B (docs/phase3-ab-results.md): targeting precision ✓, but contention-triggered escalation never beat periodic on outcome flips at equal cost (5 losses, 1 tie) — ✗ on the gate metric. Router default is now periodic; --route contention ships as a research arm.

Next Steps

  1. This week: Run Phase 0 validation experiments on a real Perdura session
  2. If all three pass: Lock memoric binary spec (incl. semantic-hash choice), commit to repo
  3. Iterate: Adjust distance metrics or bit allocations if experiments hint at improvements
  4. Phase 1.5 kickoff: Integrate memoric binary into graph storage and briefing assembly