TerranSoul
Papers / The Evolution of Memory for AI Systems
← Main paper
2026-07
Research companion · field history with citations

The Evolution of Memory for AI Systems: twelve stages, and what each one broke.

Every stage of machine memory exists because the one before it failed in a way somebody wrote down. This is that chain, in order, with the failure named at each link — and, at the end, what TerranSoul covers for each stage with the measured number beside it.

What this article is. Field history. Stages 1–11 are about the industry, not about any one project; every external claim carries its citation and every figure is quoted as its source states it, not re-measured here. The narrative follows the evolution-timeline table in docs/brain-advanced-design.md [51], whose citations were web-researched and dated through July 2026. Stage 12 is the only section about TerranSoul, and it is a coverage table: what is implemented per stage, and the measured number with its qualifier where one exists. Where nothing was measured, the row says so. The measurement paper this accompanies is Three Falsifiable Hypotheses About External Memory for Frozen Language Models [52].
T
TerranSoul Research
Companion article, July 2026.
·
Source table: docs/brain-advanced-design.md · Main paper: Three Falsifiable Hypotheses About External Memory for Frozen Language Models.
Contents · 19 sections

OverviewOne chain, not a story of replacement

The history is usually told as a sequence of replacements: keywords gave way to embeddings, embeddings to vector databases, retrieval to graphs, graphs to agents. It is nothing of the sort. Every earlier substrate is still load-bearing inside its successor — BM25 inside hybrid retrievers, approximate-nearest-neighbour indexes inside every product that calls itself semantic, knowledge graphs inside 2026's agent-memory startups.

What moves forward is the failure mode. Each stage solved the documented limitation of the stage before it and, at production scale, exposed a new one. That is the argument of this article, and it is falsifiable: if a transition has no failure behind it, the link is not real. Two of the eleven transitions below do not have one, and both are marked as such rather than papered over.

Each stage is written the same way — what it is, pros, cons, and why the next stage. The cons are the load-bearing part.

This article names the techniques; it does not open them. A companion page, Retrieval and memory techniques, takes the twenty-nine mechanisms cited below — BM25, dense retrieval, Matryoshka, HNSW, IVF-PQ, DiskANN, RaBitQ, TurboVec, typed-edge traversal, GraphRAG, RAG, HyDE, RRF, MMR, Self-RAG, CRAG, MemGPT paging, bi-temporal graphs, sleep-time compute, write-time contradiction resolution, ColBERT, ColPali, MUVERA, Memory-R1, HippoRAG-2, RobustRAG and CaMeL — and gives each one an animated diagram of how it works, the failure it exists to fix, and what it charges for the fix.

Scope note: this article covers memory and retrieval for AI systems — how a language model recalls facts, conversations, and documents. It does not cover code-intelligence tooling (AST-level code graphs, symbol indexes, or coding-assistant token-saving techniques), which is a separate subsystem with its own design questions and is out of scope here.

Stage 1Lexical / keyword search — 1970s–2000s

What it is

An inverted index maps every term to the documents that contain it, so a query touches only the documents that share a word with it. A statistical ranking function then orders those documents by how unusual the matched terms are — TF-IDF first, then Okapi BM25 at TREC-3 in 1994 [1]. Matching is on the literal token; nothing is interpreted.

Pros

Fast, cheap, and inspectable: no training, no accelerator, and a human can read why a document ranked where it did. It is still a 2026 production default rather than a legacy tier — a benchmark of 23,088 queries over 7,318 financial documents put plain BM25 above text-embedding-3-large on most metrics, because exact identifiers reward exact-term matching [2]. The machinery also keeps resurfacing inside its successors: sparse autoencoders over frozen dense retrievers extract Zipfian vocabularies that can be scored by BM25 over an inverted index, matching the dense model's own accuracy [4].

Cons

Vocabulary mismatch. Two people pick the same term for the same concept less than 20 % of the time (Furnas et al., CACM 1987 [3]), so synonyms are missed and polysemous words are confused. Worse than the miss is the silence: a query phrased differently from the document returns nothing, and the system cannot distinguish “no match” from “no such document”.

LEXICAL · MATCH THE TOKEN car automobile no shared token: the document is never scored, and the miss is silent DENSE · MATCH THE MEANING car automobile one encoder, one space: the paraphrase lands beside its source, so the match survives the rewording
Two words for the same thing share no token, so a lexical index never scores the second document and the miss is silent; a learned encoder puts the paraphrase beside its source in one vector space, and the match survives the rewording.

Stage 2Semantic search — dense embedding retrieval — 2013–2021

What it is

A learned encoder maps queries and documents into one vector space, and relevance becomes cosine similarity between two points in it. Nothing is matched literally: a paraphrase lands near its source because the encoder was trained to put it there.

Pros

It closes the vocabulary gap the previous stage could not. Dense Passage Retrieval (Karpukhin et al., EMNLP 2020) scored 9–19 % higher top-20 accuracy than BM25 [5]. One index then serves paraphrase, cross-language, and cross-modal queries with no extra machinery, and the same recipe scales to instruction-tuned and multimodal encoders [6].

Cons

A capacity ceiling that is proven rather than empirical. For any fixed embedding dimension d there exist combinations of relevant documents that no single-vector embedder can return as a top-k result — a sign-rank argument, confirmed by the LIMIT benchmark where state-of-the-art embedders fall below 20 % recall@100 [7]. Dense retrieval also throws away exactly what Stage 1 was good at: an identifier, a version string, or a part number has no useful neighbourhood in embedding space. This is why 2026 practice is hybrid fusion of both channels rather than replacement of one by the other.

EXACT SCAN every vector compared, every query APPROXIMATE · NAVIGABLE GRAPH ENTRY TARGET a few hops; most of the index untouched
An exact scan compares the query against every vector, so its cost rises with the corpus; a navigable small-world graph reaches the same neighbourhood in a handful of hops and leaves most of the index untouched, buying sub-linear search with an answer that is near-certain rather than certain.

Stage 3Vector databases — approximate nearest neighbour at scale — 2016–2023

What it is

Approximate-nearest-neighbour indexes trade an exact answer for a sub-linear one. FAISS [10] and Hierarchical Navigable Small World graphs [11] provided the index; Milvus, Pinecone, Weaviate and pgvector wrapped it in a real database with persistence, filtering and metadata [12].

Pros

Billion-scale similarity search became ordinary infrastructure rather than a research result, and by 2025–2026 the category commoditized entirely: SQL Server 2025 ships native vector indexes and Amazon S3 Vectors ingested more than 40 billion vectors in preview [12]. RAM-residency, the early blocker, is largely gone — SSD-resident designs, 2-billion-vector indexes, a 50-billion-vector benchmark in 2026 [12].

Cons

Two practical and one structural. Disk-resident graph indexes remain immature at ≤15 % I/O utilization [13], and filtered search still degrades both recall and latency, so “semantic search, but only in this tenant’s documents” is harder than either half alone. The structural limit is the one that mattered: similarity is not structure. An index can say two texts are alike; it cannot represent that a person works at a company that was acquired by another.

THE QUESTION who owns the company she works for? NEAREST BY SIMILARITY returns things that resemble the query TYPED EDGES WORKS AT ACQUIRED BY SHE EMPLOYER ACQUIRER answer two typed hops, and the walk is the derivation
Nearest-by-similarity returns things that resemble the query, which cannot answer a question whose answer is two relations away; typed edges let the same question be walked one relation at a time, and the walk that finds the answer is also its derivation.

Stage 4Knowledge graphs — 2001–2010s

What it is

Facts stored as typed triples — entity, relation, entity — and answered by traversal instead of by scoring. RDF and SPARQL gave the standard, Freebase and DBpedia the public corpora, Neo4j the property-graph engine; Google's Knowledge Graph (2012; 500M entities, 3.5B facts) put entity-centric answers into mainstream search [15].

Pros

Entity disambiguation and multi-hop relational answers become exact rather than probabilistic, and the answer carries its own derivation. The structure argument has numbers: a 2023 data.world benchmark reports GPT-4 at 16.7 % execution accuracy over raw SQL against 54.2 % over a knowledge-graph representation of the same 43 insurance-domain questions — and 0 % against non-zero in the highest-complexity quadrant [16].

Cons

Graphs are chronically incomplete and expensive to keep alive. Knowledge Vault found 71 % of Freebase persons had no recorded place of birth [17]; Google shut Freebase down in 2015. Automatic construction from text does not remove the problem, it changes its shape — from missing edges to fragmented and inconsistent ones — and the curation cost never amortizes, because the world keeps changing under the schema.

Stage 5Retrieval-Augmented Generation — 2020–2024

What it is

Retrieve documents at query time and place them in the prompt, so a frozen model reads a fact instead of recalling it (Lewis et al., NeurIPS 2020 [19]). The model's weights stay fixed; the knowledge lives outside them.

Pros

It fixes three failures of frozen parametric knowledge at once and without retraining: hallucination, staleness, and the absence of provenance. Updating knowledge becomes a write to a store rather than a fine-tune, and every answer can cite. By 2026 the pattern had hardened into managed infrastructure with query planning, parallel subqueries and reranking — Azure AI Search's agentic retrieval reached general availability in April 2026 [20].

Cons

Most production failures are retrieval failures, not generation failures, and the pipeline emits no signal about which one happened: the output is fluent either way. A 2026 matched-control study found that “Lost in the Middle” reader errors come mainly from semantic competition among the retrieved passages rather than from context length — swapping in less-competitive passages recovered up to +6.0 exact match, so a longer window relocates the problem instead of solving it [21].

REWRITE + RETRY QUERY RETRIEVE GRADE GENERATE ABSTAIN GROUND AMBIGUOUS GROUNDED
A grader sits between retrieval and generation and reads the retrieved set: the grade routes the query to grounding, to a rewrite-and-retry loop, or to an abstention, so a retrieval failure has somewhere to go other than a confident answer built on it.

Stage 6Agentic / corrective RAG — 2023–2026

What it is

The pipeline grades its own retrieval and acts on the grade. Self-RAG trains one model to emit reflection tokens that decide when to retrieve and then critique its own output for factuality and citation quality [24]. CRAG adds a lightweight retrieval evaluator that labels retrieved documents Correct, Incorrect or Ambiguous and triggers knowledge refinement, a web-search fallback with query rewriting, or both [25].

Pros

It attacks the previous stage's documented failure at the place the failure occurs. It also reached quantified production form quickly: by June 2026 a published pipeline tuned explicit thresholds (crag_ok = 0.7 / crag_bad = 0.4; τ_claim = 0.3 / τ_abstain = 0.3) around a verifier benchmarked on HaluBench, landing on an operating point of 2 % hallucination at 0.908 faithfulness [26,27].

Cons

The loop is only as trustworthy as its own grader, and the grader is weak. That verifier's AUROC is 0.702 where 0.5 is chance, so grade-then-correct inherits a large share of the judge's error at any threshold — and the same operating point reports 0.46 coverage, which is how the low hallucination rate is actually reached: by abstaining or rerouting more than half of all queries [26,27]. Both 2024 originals also grade at document or answer level, so an answer with five correct statements and one fabrication still clears a document-relevance check — a gap the field only began closing in 2025 [28,29].

Stage 7LLM observability & evals — 2023–2025

What it is

Reference-free measurement of model output, and tracing of the pipeline that produced it: RAGAS, TruLens span-level tracing, and LangSmith [22,30]; MT-Bench's LLM-as-a-judge protocol [31]; faithfulness suites such as FaithJudge [23]. By 2025–2026 the layer standardized onto OpenTelemetry GenAI semantic conventions [32].

Pros

For the first time a team could detect hallucination and catch regressions without gold labels, at continuous-integration speed, and localize a failure to a span rather than to a pipeline. It made the previous two stages debuggable at all.

Cons

The judge is unreliable in a specific and uncomfortable way. Across 21 judges and roughly 541,000 judgments, judges with test-retest reliability above 0.95 also carried severe position bias (above 0.10) and rankings that shifted by up to 14 positions between benchmarks [33] — reliable and invalid at the same time, which is worse than noisy, because it looks trustworthy. The layer is also passive: it mines traces for humans and writes nothing back.

Stage 8Two-way agent memory — the read-write layer — 2023–2026

What it is

The agent reads memory and also writes it. MemGPT introduced operating-system-inspired virtual context management — the model pages its own working set in and out [35] — and the idea mainstreamed as Letta, Zep, Mem0 and ChatGPT Memory [36]. By March 2026 it was table stakes: Anthropic made Claude memory free for all users and added cross-assistant memory import [36].

Pros

Sessions stop being independent. Preferences, corrections and prior conclusions survive, and the agent can compress its own context deliberately instead of being truncated by a window it cannot see.

Cons

The write path is an attack surface with no defence in the base design — MINJA reports over 95 % injection success against unguarded stores [37], and an injected memory is permanent and retrievable rather than confined to one turn. Measurement is unresolved as well: the same LoCoMo benchmark was reported at 84 %, then 58.44 %, then 75.14 % in a public dispute [38], and MemoryArena shows systems that score near-perfectly on LoCoMo collapsing to 40–60 % on interdependent multi-session tasks [39].

NEW FACT SAME ORIGIN AGAIN UNVERIFIED CLAIM WRITE GATE ALLOW BLOCK QUARANTINE DURABLE STORE QUARANTINE A BLOCKED WRITE RETURNS BEFORE THE STORE IS TOUCHED
Every autonomous durable write is graded before it lands: an allowed write reaches the store, an unverified one is held in quarantine, and a blocked one turns back before the store is touched, so a poisoned write is never there to be retrieved later.

Stage 9Governed, self-consolidating memory — 2025–2026

What it is

A governed write path plus background consolidation. Letta's sleep-time compute replaces in-conversation self-editing with a separate agent that curates memory during idle time [41]; Hindsight's “Observations” process, which does the same job, took first place on BEAM at 73.9 % over 1M-token corpora [42].

Pros

Writes are reviewed rather than accepted, contradictions are resolved off the hot path where there is time to resolve them properly, and the store can get smaller and better rather than only larger. It is the first stage that treats forgetting as a feature.

Cons

GateMem finds that no current method achieves utility, access control and reliable forgetting at the same time [43] — a trilemma, not a backlog. Lifecycle attacks against the write path are separately formalized [44], so governance is now a named target rather than an assumed defence. BEAM shows roughly a 25 % accuracy drop from 1M to 10M tokens [42], and the category still has no consensus name, which is itself evidence that it has not settled.

Stage 10Memory infrastructure — Letta, Zep (Graphiti), Mem0 — undated in the source table; the products date from 2023 onward

What it is

The memory layer sold as a component. Mem0's extract-consolidate-update vector pipeline, Zep/Graphiti's bitemporal knowledge graph for facts that change over time, and Letta's operating-system-style runtime where the model pages its own memory via sleep-time curation [45]. Adjacent, same problem class in a different deployment shape: ApeRAG, an agentic GraphRAG platform built as a deeply modified fork of LightRAG [46] with five backing services, Kubernetes and Helm deployment, and optional MinerU document parsing [47].

Pros

A team gets typed memory, consolidation and temporal validity without building any of it. The bitemporal design in particular answers a question a flat store cannot express at all — what did we believe last March, as opposed to what is true now [18].

Cons

It is developer infrastructure, not a product, and the trade between shapes is measurable: Mem0's vector-first design scores 49.0 % beside Zep's 63.8 % on LongMemEval temporal reasoning [45], while Letta remains a runtime somebody still has to build a product on. Comparability is uneven too — ApeRAG publishes no quantitative benchmark suite, so it can only be compared by architecture [48]. And nothing in this layer answers Stage 9's trilemma; it packages the open problem rather than closing it.

Stage 11The learned & robust frontier — 2025–2026

What it is

Three shifts arriving together. Retrieval goes multi-vector: ColBERT-style late interaction [61] becomes deployable through PLAID and MUVERA's fixed-dimensional encodings [62] and extends to page images with ColPali [63], recovering the token-level matching a single pooled vector discards. Memory management becomes learned: Memory-R1 trains a reinforcement-learning policy over add / update / delete / noop from downstream reward [64], and HippoRAG-2 runs personalized PageRank over a passage graph for single-shot multi-hop recall [65]. RAG gains provable robustness: RobustRAG's isolate-then-aggregate [66] and CaMeL's capability and information-flow control [67] give certified resistance to injected passages, while extended RaBitQ pushes quantization past the previous accuracy/compression frontier with a per-vector error bound [68].

Pros

Each item addresses a limitation the chain had left standing — the single-vector ceiling, hand-tuned write heuristics, and the total absence of any guarantee against a poisoned passage — and the robustness half does it with a stated bound rather than a benchmark delta, which is a different and stronger kind of claim.

Cons

Every one of them is bought with a resource a sovereign, local, single-frozen-model deployment does not have. Late interaction needs a per-token embedder, and a local Ollama-style server returns one mean-pooled vector per text. Reinforcement-learned memory management needs an offline training loop, which a frozen model by definition excludes. RobustRAG's stronger aggregator needs decoding logits the local server does not expose. Full CaMeL needs a second, privileged planner model, which doubles the resident footprint. On a hosted stack these are line items; on a local one they are boundaries.

Stage 12What TerranSoul covers — mid-2026

This is the only section about TerranSoul. Each row names what is implemented for a stage and the measured number with its qualifier. Where something is implemented but never measured, the row says built, not benchmarked — that is an honest entry; a blank or an implied number is not.

INSIDE THE TOP k BELOW THE TOP k one question, two golds RANKED LEFT TO RIGHT GOLD GOLD recall_ANY@k satisfied — one gold inside the top k is enough recall_ALL@k not satisfied — every gold must be inside the top k
A question with more than one gold session splits recall in two: recall_ANY@k is satisfied the moment a single gold lands inside the top k, while recall_ALL@k requires every gold to be inside it, so one ranking can score high on the first measure and much lower on the second.

How to read the LongMemEval-S rows. One arm per process (cross-arm contamination in a shared process is proven, so arms are never mixed), LONGMEM_EMBED=1 with embeddinggemma:latest, n = 500, no abstentions. R@k here is recall_ANY@k — at least one gold session inside the top k — and it is a retrieval metric, not QA accuracy; every published LongMemEval leaderboard number is QA accuracy, and merging the two columns is a category error. Two floors govern these rows: the LongMemEval-S NDCG at 10 floor of 95.1 and the MRR floor of 95.9, both set on 2026-07-03 at commit 4eea9e45 (95.1108 and 95.8553 exactly). Current code measures 95.04 and 95.76 — below both. Under never-regress that is an obligation to investigate and regain, not a licence to republish the headline downward, so the floors are printed as floors and the measurements beside them. One structural limit applies to every NDCG@10 figure below: at top_k = 20, 5 of 948 gold sessions are never in the retrieved pool to be ranked — the same 5 questions of 500, reached from two independent directions — so NDCG@10 is capped at 99.738 by retrieval depth rather than by ranking, and no reranker can move it past that.

StageWhat TerranSoul implementsMeasured number, with its qualifier
1 — lexical SQLite FTS5 with BM25 as one of two candidate channels, fused by reciprocal-rank fusion at k = 60 [49]. Measured, inside the rrf arm (benchmark/results/head-full500-rrf/, 2026-07-26, n = 500): R@5 99.4 %, R@10 99.8 %, R@20 100.0 %; NDCG@10 95.04 against the published NDCG-at-10 floor of 95.1; MRR 95.76 against the 95.9 floor; mean 654.92 ms on an uncontended machine. Cost in the same row: those recalls are recall_ANY@k. On the same run recall_ALL@5 — every gold inside the top 5 — is 91.60 %, and recall_ALL@10 / recall_ALL@20 are 96.60 and 99.00. The gap is concentrated where the headline looks best: of 500 questions, 324 are multi-gold, and on those recall_ANY@5 reaches 100.00 while recall_ALL@5 is 87.96. Worst question types by recall_ALL@5: multi-session 85.71 and temporal-reasoning 85.71 (n = 133 each).
2 — dense EmbeddingGemma-768d with asymmetric query/document prefixes; Matryoshka two-stage search [53]; HyDE query expansion [50]. Measured, prefixes only: +4.21 pp R@10 — 0.63761905 with the documented prefixes against 0.59547619 without (benchmark/results/embedder-sweep/embeddinggemma.json and embeddinggemma__noprefix.json, both committed). Qualifiers that change what this means: the fixture is 240 observations / 20 queries, scored pure-vector with no hybrid scorer, so it is not the shipped ranking path; and it is a per-model result, not a property of prefixes — the same sweep records prefixes costing two other embedders recall. Matryoshka and HyDE are built, not benchmarked in isolation.
3 — ANN at scale Per-shard usearch HNSW, IVF-PQ, TurboVec, and a shard router that narrows the candidate shard set. Measured once, not re-verifiable today. BENCH-SCALE-1, 2026-05-14: two arms of 100 adversarial queries over a 1,000,000-chunk corpus, on mxbai-embed-large — not the shipped EmbeddingGemma. Routed against all-shards: R@10 60.5 vs 59.5, so routing gains 1.0 pp of R@10. Cost in the same row: it loses 1.59 pp of NDCG@10 (33.27 vs 34.86) and loses on every other ranking metric of the same run (R@1 11.5 vs 14.5, R@5 44.0 vs 46.0, MAP@10 24.64 vs 26.95, MRR@100 25.87 vs 28.39; R@100 ties), and it is slower at every percentile (mean 6,288 ms vs 5,141 ms; p99 178.1 s vs 67.1 s). Both artifacts were deleted in commit 217ffdc1; a re-run is owed.
4 — graph One bitemporal, typed, confidence-weighted, CRDT-syncable edge table with cascade-delete integrity, written by five producers; a neighbour boost and an edge-degree activation multiplier on the default fused path; cascade expansion, HippoRAG-2 personalized PageRank, and GraphRAG-style community detection [54] built but default-off. Built, not benchmarked — and the disclosure cuts against us. The bench harness only builds entity edges when LONGMEM_KG_EDGES=1, which defaults to false and appears in none of the four full-500 arm reports. The edge table was therefore empty for every LongMemEval number on this page: the neighbour boost found nothing, edge degree was uniformly zero, and the cross-round edge walk had nothing to walk. On the one lane where retrieval numbers are published, the graph has never been switched on. The single graph/no-graph comparison on record (BENCH-KG-2, 2026-05-13, 100 adversarial LoCoMo queries) measured 0.00 pp of R@10 at 2.04× the mean latency, with a fraction of a point in the graph arm's favour on NDCG@10, MAP@10, MRR@100 and R@5; its artifact was deleted in commit 217ffdc1 and a re-run is owed.
5 — RAG Retrieve → context pack → generate over a single substrate, with contextual retrieval at ingest, late chunking, and parent-child resolution (a matched sub-chunk resolves to its larger parent at query time). Measured as the end-to-end path the four arms exercise. The product's default mode, chat, re-run at full-500 on current code: NDCG@10 94.11, MRR@20 94.56, R@5 99.2 %, R@10 99.8 %, R@20 100.0 %, mean 661.11 ms (benchmark/results/rel010-full500-chat/, 2026-08-03). That arm reproduces an independent 2026-08-01 run bit-identically — NDCG@10 0.9411437551799973 and MRR@20 0.9455532467532467 on both, to the last decimal, across two days and many commits — which is what licenses publishing it as the mode's value rather than one sample of a noisy quantity. The earlier figure on this row was 93.78 (benchmark/results/head-full500-chat/, 2026-07-26) and it is superseded, not averaged: the 08-01 and 08-03 arms are strictly better on all six of R@5, R@10, R@20, NDCG@10, MRR and latency, and they are the arms that ran with OLLAMA_EMBED_NUM_GPU=99. The 07-26 arm left the embedder CPU-pinned, so the most likely cause of the 0.33-point gap is that CPU and GPU kernels return marginally different float embeddings and a handful of near-ties settle the other way — likely, because two arms differing in one environment variable is a correlation and this page does not print it as a mechanism until an arm isolates it. Cost in the same row: 0.93 NDCG@10 below the rrf arm on the same questions (narrower than the 1.26 previously published here, on the strength of the better arm), so the product path is still not the validated retrieval path. Parent-child, contextual retrieval and late chunking are built, not benchmarked in isolation.
6 — corrective RAG CRAG-style retrieval grading [25], hard abstain, claim-level weakest-link verification, and read-path spotlighting — every retrieved block fenced with an explicit trust boundary marking it as data to ground on, never instructions to obey. Built, not benchmarked. No arm isolates the corrective loop, and the four full-500 arms record zero abstentions, so the abstain path is untested by them. The perfect en/vi/ja score this row used to carry (p@10 = 1.00 on the 1M résumé gate) was produced by a bench-local harness with its own retrieval loop and its own judge, not by the shipped mode — see Corrections.
7 — observability & judge An LLM-judge rerank pass over the retrieved candidates, under the fail-open discipline that governs every judge on the read path: judges fail open and never veto, following the DEAD-JUDGE-1 incident in which a stale model tag silently erased all recall. The think mode used to run this pass on its retrieval path. Since 2026-08-02 it does not, and the reason is the measured number beside this row. Measured, and net-negative. think full-500 (benchmark/results/fix11b-full500-think/, 2026-07-27; valid only on post-76f2276d code): NDCG@10 93.26, MRR 94.21, R@5 98.6 %, R@10 99.6 %, R@20 99.8 %, mean 4,765.40 ms. Cost in the same row: 0.52 NDCG@10 below chat at 7.1× the latency (4,765.40 ms against 675.01 ms), with R@20 tied at 99.8. The commit's own acceptance line reads “bench owed: think ≥ 93.782”, which 93.26 does not clear. That obligation is discharged by construction rather than by a better score. The rerank came off think's retrieval path on 2026-08-02: think and chat now call the same retrieval function with the same arguments, and the rerank decision they pass is resolved by one shared production rule rather than by two matching literals, so the two cannot produce different orderings. The numbers above stay printed as what the reranking think actually scored — they are a record of a retired configuration, not a live rung.
CORRECTED 2026-08-11 — “call the same retrieval function with the same arguments” stopped being true that day. A cross-surface audit found think triggering a knowledge-graph multi-hop bridge on MCP's brain_search (via a separate ladder mechanism) while Desktop chat's and the CLI's think stayed on plain RRF — a direct violation of the standing rule that a surface may never differ in whether multi-hop runs. Desktop/CLI's think now pins the same multi-hop bridge explicitly, so it once again differs from chat in mechanism — and, once a same-day bench-harness bug in the chat arm itself was found and fixed (row 12), in measured value too: think NDCG@10 93.8 against chat's 93.9, close but not identical, for the reason row 12 gives.
8 — two-way memory A governed write gate on every autonomous durable write, plus provenance, typed edges, ACL and CRDT sync. A quarantine or block verdict returns before MemoryStore::add, so a blocked write cannot reach durable store; connector_burst_anomaly scores repeated same-origin re-ingestion, the MINJA persistence pattern. Built, not benchmarked. Three named regression tests pin the behaviour, one of them through the real production ingest call rather than a test double. There is no MINJA or PoisonedRAG evaluation: an audit found no run, no results directory and no benchmark row. A 0 % attack-success figure published here until 2026-07-28 followed structurally from blocking — an argument, not a measurement — and is withdrawn.
9 — governed consolidation Complementary-learning-systems consolidation with FSRS reconsolidation, a nightly maintenance job, LLM-arbitrated conflict resolution at the typed-edge layer [55], and a learned (heuristic-core) write policy. Built, not benchmarked. No consolidation-on against consolidation-off arm exists, so the retrieval effect of the whole stage is unmeasured here. Nothing in it addresses the industry-wide utility / access-control / forgetting trilemma [43], which stays open in this design as it does elsewhere.
10 — memory infrastructure One store behind three surfaces — the desktop app, the CLI (--mode), and an MCP server — with the same modes and the same retrieval path on each. Seven MCP read tools reach the graph; the graph UI has full edge CRUD. Built, not benchmarked as a surface. No arm measures surface parity. One asymmetry is worth stating rather than leaving to be discovered: the MCP write surface is append-only — brain_add_edge and nothing else — while close, delete, update and detach all exist for the UI, so an external agent can grow the graph and cannot correct it.
11 — learned & robust frontier Committed and tested, all default-off: MMR diversity re-rank, HippoRAG-2 personalized PageRank, a reason-then-rank reranker over the existing judge, a learned write policy, the extended RaBitQ quantizer, MUVERA fixed-dimensional encodings with a MaxSim kernel, and semantic-entropy plus information-flow-control kernels. Built, not benchmarked. Each is enabled in a configuration once its own bench clears the never-regress floor; none has cleared one, so none is on by default and none has a number. Four items of the frontier are not built and will not be, because each needs a resource this design excludes: a per-token embedder inside the local server, an offline reinforcement-learning loop, decoding-logit aggregation, and a second privileged planner model. Those are stated boundaries, not a backlog.
12 — the product surface Four thinking modes — chat, think, research, max — each one (reasoning-effort, harness) pair, identical across the app, the CLI and MCP, with an Auto router that dispatches per turn. Benchmarks drive the same modes a user gets. Measured, full-500, one arm per process. The four drawn modes are three measured behaviours: research = rrf 95.04 > chat 93.78 > think 93.26; max is absent. research is identical to plain rrf on every metric to four decimals with 499 of 500 orderings the same, at 4,928.46 ms against 654.92 ms — a measured no-op at 7.5× the cost. That full-500 research figure predates the current code: it was measured before commit 76f2276d, and it stays on this page labelled as such rather than being quietly refreshed. research HAS since been re-run on current code — NDCG@10 95.7 at n = 50, in the four-rung table below — and that run does not replace 95.04, because 50 questions and 500 questions never share a series in either direction. What this row still owes is a full-500 repeat, not a re-run. think shared 6 of 500 orderings with chat on those arms, so the reranker had stopped being a no-op and become a different, slightly worse answer; the 2026-08-02 rung fix then retired the reranker from think's retrieval path altogether, so think now runs chat's retrieval unchanged and the three-behaviours reading applies to the arms as measured, not to the code as it stands.
CORRECTED 2026-08-11 — think is no longer “chat's retrieval unchanged,” and all three re-run full-500. think now pins a knowledge-graph multi-hop bridge, matching MCP's own brain_search think rung (which had been running it, undisclosed on this page, since before the correction above was written — a genuine rules/one-path-three-surfaces.md gap, not a rhetorical one). Re-run in one session, one arm per process, OLLAMA_EMBED_NUM_GPU=99: chat NDCG@10 93.9 / R@5 99.2% / R@10 99.8% / R@20 100.0%, mean 223 ms; think NDCG@10 93.8 / R@5 99.4% / R@10 99.8% / R@20 100.0%, mean 1260 ms — close to chat but no longer identical (think is 0.1 NDCG@10 lower and 0.2 pp higher on R@5). A bench-harness bug was caught and fixed producing this row, not before: the chat arm calls the same shared retrieval entry point as think, which now resolves its rung from the app's reasoning effort; the harness's AppState is built without ever running the per-turn auto-mode router every real chat/CLI turn runs first, so its chat_mode stayed at the struct default (Auto), which resolves to the effort the router falls back to when the classifier is unavailable (Medium) — the same effort think uses, which now also selects the multi-hop bridge. The chat arm was therefore silently measuring think's retrieval (both scored NDCG@10 93.8395% identically) until the harness was fixed to pin ChatMode::Chat explicitly, mirroring an existing unit test that already did this correctly. This is a bench-fidelity gap, not a live product bug: a real user's turn always runs the router before retrieval. The numbers above are from the corrected re-run; R@5/R@10/R@20 are bit-identical to the independently-reproduced 2026-08-01/08-03 chat floor (99.2/99.8/100.0), and the 0.17-point NDCG@10 gap against that floor (94.11) reads as embedder run-to-run noise, the same class this page has already measured once at 0.33 points from a CPU/GPU kernel difference. What the corrected numbers DO confirm: multi-hop is close to a no-op on this corpus but not an exact one, consistent with LongMemEval-S building each question's memory store fresh with LONGMEM_KG_EDGES unset (row 4's own disclosure) — there are no graph edges to bridge over, so the residual difference comes from fusion/candidate-pool mechanics rather than the graph walk itself. The fix's real effect was proven separately with a hand-built edge-bridge corpus (a unit test, not a bench arm). research NDCG@10 moved to 94.3 (from 95.04 in the paragraph above) with R@5/R@10/R@20 unchanged at 99.4/99.8/100.0 — investigated before publishing per this page's own never-regress discipline: a git-worktree re-run of the exact commit preceding this session's changes, on the identical first 100 questions, measured NDCG@10 94.30 — byte-identical to the post-session number. The drift predates this session; its cause across the intervening two weeks of commits is unbisected and stays an open item, not attributed to the multihop fix. max at full-500 is UNMEASURED — but 50-question arms DO exist and are committed (benchmark/results/slice50-max-repeat/, slice50-max-newrubric/, and now p50-max-postrungfix/ on current code), so the earlier wording “no artifact exists at any size” was wrong and is corrected here. Those arms are drawn in the 50-question figure above and are never mixed into a full-500 series. Two of them were run at IDENTICAL configuration and returned bit-identical recall with NDCG@10 moving 1.0 and MRR moving 1.3, which is where this page’s ±1.3 noise band comes from.

Four rungs, one configuration, one code revision — n = 50

Every retrieval number in the table above was measured on its own day, and two of them on code that has since changed. The four thinking modes had never been run as a set: chat and think came from one campaign, research from another, max from a third, and the gaps between them were therefore differences between runs as much as differences between rungs. On 2026-08-03 all four were measured on one configuration, on the current code, one arm per process, over the same 50 questions, with the embedder resident on the GPU. This is the first four-rung ladder on this page that is a ladder rather than a collation. Artifacts, all committed: benchmark/results/p50-chat-postrungfix/, p50-think-postrungfix/, p50-research-postrungfix/ and p50-max-postrungfix/.

RungR@5R@10R@20NDCG@10MRR@20Mean latency
chat98.0 %100.0 %100.0 %93.591.4550.52 ms
think98.0 %100.0 %100.0 %93.591.4547.54 ms
research98.0 %100.0 %100.0 %95.794.45,024.74 ms
max100.0 %100.0 %100.0 %98.598.065,986.43 ms
What the columns mean. Rung — the thinking mode being measured: chat, think, research, max. R@5 / R@10 / R@20 — recall_ANY@k: the fraction of questions for which at least one gold session appears in the top k. ⚠ It is satisfied by a single hit, so on a multi-gold question it says nothing about whether the other golds were found — 324 of the 500 questions have more than one gold, and recall_ALL@k, which requires every one of them inside the top k, scores markedly lower on exactly those. NDCG@10 — normalised discounted cumulative gain over the top 10. Unlike recall it is rank-weighted: a gold at position 1 counts for more than the same gold at position 9, and the score is normalised so 1.0 is the best achievable ordering. It is the metric most sensitive to reordering, which is why the reranking experiments are all reported against it. MRR@20 — mean reciprocal rank: the average of 1/rank of the first gold found within the top 20 (0 if none). It is the closest thing here to a top-1 measure — rank 1 scores 1.00, rank 2 scores 0.50, rank 4 scores 0.25 — but it still only looks at the first gold, so it is rank-1 sensitive rather than a true precision@1. Mean latency — wall-clock per query for that rung, which is what makes an accuracy gain affordable or not.

Five caveats travel with that table. Publishing any of these numbers without them is a defect, not a shortening.

“Same retrieval, more reasoning” is a measured conclusion, not an unexamined default. The obvious objection to a rung that retrieves identically to the one below it is that nobody tried. Four interventions were built and benchmarked on the same 50-question slice against the same chat baseline, on 2026-08-03, and every one of them made retrieval worse than plain RRF:

Four attempts to make think out-retrieve chat. Same slice, same baseline, one arm per process. The best intervention still loses.
VariantNDCG@10MRR@20Latencyvs chat
chat — hybrid + RRF, no judge93.591.4550.52 ms—
think, no intervention93.591.4547.54 ms0.0
+ listwise judge, ungated85.981.24,677.03 ms−7.6
+ listwise judge, coverage-gated88.184.04,832.36 ms−5.4
+ MMR diversity90.186.9598.98 ms−3.4

The two failure mechanisms are different and they converge. The judges lose because a 12B ranks worse than RRF — and the gated run is what establishes that, because with malformed permutations made impossible the judge still lost 5.4 points, so the problem is ranking judgement rather than output format. MMR loses for the opposite reason to the one that motivated it: it was adopted expecting diversity to surface the secondary golds that multi-gold questions need, but on this corpus the golds are near-duplicates of each other — the same fact mentioned across different sessions — so a redundancy penalty demotes precisely the documents being retrieved. Diversity is the wrong objective when the correct answers are similar.

Both point at the same conclusion: RRF's ordering is already right for this workload, and a rung that reorders it pays latency to be worse. So think ships as the same retrieval as chat, at the same cost — 547.54 ms against 550.52 ms — with more reasoning effort (ReasoningEffort::Medium against Off). That difference is real and it is invisible here, because this harness scores retrieval and never reads an answer. Making it measurable needs an answer-generation and judging stage the harness does not have — that is a feature to build, not a run to schedule, and until it exists the honest claim for this rung is the narrow one.

That parity has since been confirmed at full-500, and the confirmation is stronger than the claim it confirms. Independent 500-question arms, one per process, on current code (2026-08-03, benchmark/results/rel010-full500-chat/ and rel010-full500-think/): R@5 99.2 %, R@10 99.8 %, R@20 100.0 %, NDCG@10 0.9411437551799973 and MRR@20 0.9455532467532467 — identical between the two modes on every retrieval metric, to the last decimal — with only mean latency separating them, 657.20 ms against 661.16 ms, a 0.6 % gap that is run-to-run noise. At 50 questions the two rungs merely scored the same, which is equally consistent with two similar-but-distinct pipelines landing on similar numbers. At 500 questions, agreement to one part in 1016 across five metrics is not a score: it is evidence that the same computation ran twice. The 2026-08-02 rung fix that took the reranker off think's retrieval path did exactly what it was meant to, and this row's claim is now measured at the same sample size as the rest of the full-500 series rather than inferred from a 50-question slice.

Two fixes outlived the attempt. The listwise judge had no quality gate: its permutation parser backfilled whatever the model omitted, so a reply naming three of twenty documents became a complete, well-formed permutation that was applied wholesale over RRF — and the same path is used by max, whose R@5 100.0 was therefore achieved despite an ungated judge. It now carries the same coverage floor the pointwise path always had. And MMR stopped being dark code: it was seeded off, so the diversified search returned plain RRF byte-identically and MMR had never influenced a published number on this page despite being described on it. It is now measured, and the measurement says leave it off for this workload.

What this ladder retires, and what it does not. It retires two sentences this page was carrying. The first was that research “has not been re-run on current code”: it has, and the result is above. That does not promote 95.7 over the full-500 95.04 — the older figure keeps its place, labelled as predating the current code, and what is owed for it is a full-500 repeat rather than a re-run. The second was that think sitting 0.52 NDCG@10 below chat was an open never-regress obligation. It is not an obligation any more, because it is not a gap that can exist: think cannot fall below chat at any sample size, because think's retrieval is chat's retrieval, reached through the same function. Stated plainly rather than favourably: the 0.52 claim was measured at n = 500 and the evidence replacing it is n = 50, so the two are not a like-for-like refutation. The argument here is not a statistical one. Identical inputs through an identical function cannot diverge, and no sample size changes that. A full-500 think arm would confirm the paths really are identical; it could not give the gap a way back.

How TerranSoul compares

Eight figures, one per measurement axis. They are separate charts rather than one because the numbers are not on a shared axis: this page’s rungs are LongMemEval-S retrieval, the 92 % peer cluster is a different corpus on a shared embedder, Mem0/Letta/Zep publish LoCoMo QA accuracy rather than retrieval, the newest additions publish answer accuracy, a host-plus-memory composite or a token-reduction factor, and the assistant row is a 22-prompt judged parity harness with no retrieval axis at all. Drawing those as one series would be a category error. Each caption carries its own corpus, sample size, judge and model, and writes “not disclosed” wherever the publisher did not state one.

The 2026-08-02 field audit. Four systems entered these figures from a field audit of the current open-source memory repositories — m_flow, OpenViking (ByteDance), MemOS and code-review-graph — joining the Graphify, OpenClaw, Hermes and supermemory rows already drawn here. Every figure attributed to them is their published number, reproduced in the unit they published it in and never converted into ours. None has been re-run here. Three of the four did not fit any axis that already existed, so each was given its own frame rather than a bar on somebody else’s: m_flow’s aligned LoCoMo run and MemOS’s LoCoMo number join the QA-accuracy figure as a third panel; the LongMemEval headline numbers that publishers report as answer accuracy, or as recall at a different k, go in a figure of their own, because they are not this page’s retrieval metric; OpenViking publishes a host-plus-memory composite, which is a property of a pair rather than of a memory layer; and code-review-graph publishes a token-reduction factor, which belongs on a context axis and nowhere near an accuracy axis.

Three disclosure gaps are large enough to print on the figures themselves. hindsight’s LongMemEval-S 94.6 lives inside an embedded image, with no metric definition, question count or judge stated anywhere public, so it is drawn and annotated as undefined rather than dropped or quietly promoted to something it may not be. MemOS attributes its LongMemEval 89.20 and its LoCoMo 88.83 to a single line — evaluated via OmniMemEval — with no sample size, no judge and no baselines; its eight other benchmarks are capability tests rather than retrieval, and none of them shares an axis here. And code-review-graph’s 0.71 impact-F1 is deliberately not charted at all: its own author states that the ground truth is derived from the same graph edges the predictor walks, so the number is circular by construction and is not an accuracy result. Only its token reduction is drawn. One last mismatch is worth stating in words as well as on the figure: supermemory’s LongMemEval headline is a recall at k = 15 and this page’s is a recall at k = 5, so the two never share a series.

The three TerranSoul blanks, revisited 2026-08-02. Three of these figures carried a TerranSoul blank while every peer on them carried a value, which is a bad look that is worth being precise about rather than filling in. One is now filled, one is filled in its own frame, and one stays blank — and the reason it stays blank turned out to be different from the reason this page had been giving.

Context reduction is now measured, and it is a per-cent. On the production default fused path, retrieval carries 2,748 tokens of context where pasting the whole corpus carries 32,660 — a reduction of 91.6 %, at n = 20 queries over 240 observations, no judge in the loop. It is drawn as a third panel rather than beside supermemory’s 99.4 % because the corpus differs: supermemory measures on LongMemEval and this is TerranSoul’s own 240-observation fixture, vendored from agentmemory, scored with the bench’s deterministic embedder rather than the shipped Ollama one. The same run re-based against a 200-line summary instead of the full paste gives 65.5 %, and both are printed, because a reduction is only ever a reduction against a stated baseline. Two things this figure deliberately does not do: it is not converted into a multiplier so that it can sit beside code-review-graph’s 82×, and it is not restated at n = 50 — the fixture is fixed-size, so the honest label is n = 20.

LongMemEval answer accuracy cannot be produced at all, and that is structural. The retrieval harness has no answer-generation stage anywhere in it — no generator prompt, no completion call. Its single LLM call is an evidence-support diagnostic that asks whether the retrieved sessions contain enough to answer, and it is deliberately never shown the reference answer, with a regression test that enforces the omission. So the QA blank is not a run nobody got round to; producing that number would mean building a generator-and-judge stage, which is a feature and not a benchmark. What the figure gains instead is a third panel carrying the metric we do have, at n = 50: recall@5 98.0, recall@10 100.0, NDCG@10 93.5 and MRR@20 91.4, chat mode against a frozen gemma4:12b-it-qat with the embeddinggemma embedder. It joins none of the four series above it, for the reason this page has always given: a recall is not an answer accuracy, and a recall at k = 5 is not supermemory’s recall at k = 15. Two caveats travel with it and are printed on the figure. Every question in that 50-question slice is single-session-user, so LongMemEval’s harder categories — multi-session, temporal, knowledge-update — are absent from it and it is favourable ground; and a 50-question slice never shares a series with a full-500 number.

The LoCoMo blank stays, but its stated reason was wrong. This page said we had “retrieval only, a documented gap”. That is not true: an end-to-end answer-and-judge pass over LoCoMo exists, runs, and has since before this page was written. What is missing is not the harness but a comparable target. The MTEB distribution of LoCoMo ships gold passages and no gold answers, so that pass necessarily grades a generated answer against retrieved passages, on a graded 0–10 rubric rather than as a proportion correct, judged by our own local 12B rather than by a third party. Three separate mismatches against what Letta, Mem0, Zep and MemOS report, any one of which would disqualify it from their series. It would be a fifth incomparable metric on a figure that already carries four, so it is not drawn at all, and the blank is left to say the thing that is actually true: we can run it, and we cannot yet make it mean what that axis says.

LongMemEval-S retrieval, full 500 questions: the four thinking-mode rungs, with max drawn as a blank track because no full-500 arm exists. Re-measured 2026-08-11 after wiring think's retrieval to bridge over the knowledge graph, matching MCP's own think rung. A same-day harness bug (the chat arm skipped the per-turn mode router and fell into an effort fallback that also selected multihop) was caught before publication and fixed; the chat row reflects the corrected re-run. R@5 NDCG@10 n = 500 · LongMemEval-S retrieval-only · recall_ANY@k · embeddinggemma · no judge 85 90 95 100 % TERRANSOUL RUNGS · 2026-08-11 chat 93.9 99.2 think 93.8 99.4 research 94.3 99.4 max full-500 arm not complete — ~144 s/q, ~20 h TERRANSOUL · EARLIER ARMS rrf · 2026-07-03 95.1 99.4 search · 2026-06-28 88.8 98.6 PEER SYSTEMS agentmemory 87.9 95.2 MemPalace · self-reported NDCG@10 / MRR not published ~96.6 Scale reference: 0.2 pp = 1 question of 500 — the whole R@5 spread drawn here is 21 questions. think’s retrieval now bridges over the knowledge graph (2026-08-11), matching the multi-hop rung MCP’s brain_search already ran. think is measurably below chat again here (93.8 vs 93.9, R@5 99.4 vs 99.2) — close but not a no-op, since this benchmark builds each question’s store fresh with no graph edges to bridge over. research’s NDCG@10 moved from 95.0 (2026-07) to 94.3: confirmed unrelated to the 2026-08-11 fixes via an isolated pre/post re-run; cause still open. chat’s own arm had a same-day harness bug (skipped mode router → effort fallback → accidental multihop, caught pre-publication); fixed and re-run, R@5/R@10/R@20 bit-identical to the 2026-08-01/08-03 floor.
LongMemEval-S retrieval, 50-question slice: the same rungs at a smaller sample, with the measured plus-or-minus 1.3 noise band shown as whiskers. The think row lands exactly on the chat row because think runs chat's retrieval path unchanged. Not comparable to the 500-question figure. NDCG@10 MRR hollow = the other repeat run of the identical config whisker = ±0.65 pp — where two overlap, the gap is under 1.3 pts and is NOT a difference. n = 50-QUESTION SLICE · NOT COMPARABLE TO THE n = 500 FIGURE 88 91 94 97 100 % chat · NDCG@10 93.5 chat · MRR 91.4 think · NDCG@10 · MRR 93.5 · 91.4 — identical to chat research · NDCG@10 95.7 research · MRR 94.4 max · NDCG@10 97.7 · 98.7 max · MRR 97.0 · 98.3 max · recall R@5 98.0 · R@10 100.0 · R@20 100.0 — identical in BOTH runs Recall was bit-identical across the two repeats. NDCG moved 1.0 pt and MRR 1.3 pts with nothing changed. Every gap between DIFFERENT retrieval paths clears that band: NDCG 2.2 and 2.0 pts, MRR 3.0 and 2.6 pts. think and chat share one retrieval path, so their 0.0 is not a small gap — it is no gap that can exist. A future gap narrower than 1.3 pts on this slice must not be published as a difference.
LongMemEval headline figures as four different publishers report them: supermemory recall-at-15, hindsight with an undisclosed metric, MemOS unnamed, and m_flow question-answering accuracy. TerranSoul is a blank on that panel because our harness has no answer-generation stage at all; a third panel carries our own retrieval metric at n equals 50, which joins none of their series. metric definition public metric definition NOT public PANEL A · LONGMEMEVAL, AS EACH PUBLISHER REPORTS IT · FOUR METRICS, NOT ONE SERIES 0 25 50 75 100 % supermemory · R@15 95 hindsight · metric undefined 94.6 MemOS · no n, no judge 89.20 m_flow · QA accuracy 89 TerranSoul no QA-accuracy number — no answer stage; retrieval in Panel C PANEL B · SUPERMEMORY, PER CATEGORY · RECALL@15 · ONE PUBLISHER, ONE METRIC 0 25 50 75 100 % Assistant-recall 100 Knowledge-Updates 99 User-recall 97 Multi-session 93 Temporal 91 Preference 90 PANEL C · TERRANSOUL, ITS OWN METRIC · LongMemEval-S RETRIEVAL · n = 50 · NOT QA ACCURACY 0 25 50 75 100 % recall@10 100.0 recall@5 98.0 NDCG@10 93.5 MRR@20 91.4 Panel A stacks FOUR different metrics in one frame, because that is how the field publishes them: supermemory reports recall at k = 15 and needs no judge; m_flow reports judged answer accuracy with gpt-5-mini answering and gpt-4o-mini judging; MemOS reports an unnamed score attributed only to “OmniMemEval”; hindsight’s definition is not public. The order is by published value, but these are four different metrics and the values are not comparable. m_flow also reports 93 on temporal (n = 60) and 82 on multi-session (n = 40) under that same judge. Question count: not disclosed by supermemory, hindsight or MemOS. Answering model: not disclosed by any of those three. Corpus is LongMemEval on every row, LongMemEval-S where the publisher says so. Panel C is TerranSoul’s own metric and joins NO series above. It asks whether a gold session reaches the top k, not whether an answer was right, and a recall at k = 5 never shares a series with a recall at k = 15. Chat mode, frozen gemma4:12b-it-qat, embeddinggemma, self-measured, no judge in the loop. Two material caveats on Panel C. All 50 questions in this slice are single-session-user, so LongMemEval’s harder categories are absent from it and it is favourable ground. And it is a 50-question slice, which never shares a series with a full-500 number on this page. The QA blank in Panel A is structural, not a scheduling gap. The harness has no answer-generation stage at all: its one LLM call is an evidence-support diagnostic that is deliberately never shown the reference answer, and a regression test enforces that. A QA accuracy would be a feature, not a run.
agentmemory-corpus retrieval on a shared embedder: TerranSoul against the peer cluster, with OpenClaw marked not run. R@5 R@10 R@20 n = 20 queries / 240 obs · peers share one nomic embedder · no judge 0 20 40 60 80 % TerranSoul — keyword / FTS 45.080.0 TerranSoul — hybrid + RRF ¹ 42.077.0 Claude Code + GENesis-AGI ² 43.672.3 shared-embedder cluster ³ 41.074.0 OpenJarvis (Stanford SAIL) 40.674.2 LlamaIndex 41.072.0 obsidian-wiki (agentic ripgrep) 37.372.6 HippoRAG ⁴ 34.068.0 Hermes-Agent ⁵ 14.2 · 15.8 · 15.8 Graphify ⁴ 11.2 at every k Memary ⁴ 6.0 · 7.0 · 7.0 GraphRAG (Microsoft) ⁴ 5.0 at every k OpenClaw not run — it HAS a full hybrid retrieval engine; never benched here ³ Mem0 · Letta · Cognee · Khoj · LangChain · Haystack · RAGFlow all publish an identical 41.0 / 61.0 / 74.0   — the embedder sets that recall, not the framework; this is not a seven-way tie between architectures. ⁴ returns entity subgraphs, not ranked passages — flat recall is a shape mismatch, not a like-for-like loss. ¹ TerranSoul’s vector arm here is the bench’s deterministic stand-in, not nomic — strategy, not embedders. ² different cloud model, ~53 s/query. ⁵ FTS5 implicit-AND: 4 of 20 queries returned any candidate.
LoCoMo question-answering accuracy as each system publishes it, in three panels, with TerranSoul an honest blank: an end-to-end answer-and-judge pass over LoCoMo exists and runs, but the MTEB distribution of the dataset ships no gold answers, so it grades against retrieved passages on a graded rubric with a local self-judge and cannot join a proportion-correct series. PANEL A · AS EACH SYSTEM PUBLISHES IT · END-TO-END ANSWER ACCURACY 0 25 50 75 100 % MemOS · metric unnamed ² 88.83 Letta / MemGPT 83.2 Mem0 68.5 Zep 34.53 Cognee in the same source table, but with no single figure — “(varies)” TerranSoul no comparable LoCoMo QA number — see the notes below PANEL B · GRAPHIFY’S OWN HARNESS · n = 300 · KIMI K2.6 JUDGE 0 25 50 75 100 % supermemory 49.7 Graphify (graph-expand) ~45.3 hybrid-RRF baseline ¹ 43.3 dense RAG 41.3 BM25 31.3 Mem0 27.3 PANEL C · m_flow’S ALIGNED RUN · LoCoMo-10, k = 10 · gpt-5-mini ANSWERS, gpt-4o-mini JUDGES · n NOT DISCLOSED 0 25 50 75 100 % m_flow 81.8 Cognee Cloud 79.4 Zep Cloud 73.4 Supermemory Cloud 64.4 TerranSoul not run in m_flow’s harness — and see the notes below Mem0 appears in two panels: 68.5 in A (its own paper) and 27.3 in B (Graphify’s harness, Kimi K2.6). Zep and Cognee appear in A and C with different figures — the harness, judge and top-k all differ. Same dataset name; different harness, judge and n. Neither is wrong — they are not the same measurement. No bar here is a retrieval metric, so none of them belongs beside charts (i) or (ii). ¹ a generic RRF baseline in Graphify’s own peer set — unrelated to TerranSoul’s identically-named rrf. ² MemOS attributes both its figures to one line, “evaluated via OmniMemEval” — no n, no judge, no baselines. Panel C excludes LoCoMo Category 5 (adversarial), as m_flow’s own report states. The TerranSoul blanks are NOT a missing harness, and this page said they were until 2026-08-02. An end-to-end answer-and-judge pass over LoCoMo exists and runs. What is missing is a comparable TARGET: the MTEB LoCoMo distribution ships gold PASSAGES and no gold ANSWERS, so that pass grades a generated answer against retrieved passages on a 0–10 rubric, self-judged by our own local 12B. A graded mean scored against a different target, by a self-judge, is not the proportion-correct that every bar above reports. It would be a fifth incomparable metric, so it is not drawn at all. Leaving the blank is the claim: we can run it, and we cannot yet make it mean what this axis says.
OpenViking host-plus-memory composite: OpenClaw, Hermes and Claude Code each measured alone and again with OpenViking added. Not a standalone retrieval score. host agent alone host agent + OpenViking LoCoMo · each row is one host agent measured twice · n, judge and answering model not disclosed HOST + MEMORY COMPOSITE — NOT A STANDALONE RETRIEVAL SCORE 0 25 50 75 100 % OpenClaw 24.20 82.08 Hermes 33.38 82.86 Claude Code 57.21 80.32 Each row is a PAIR: the same host agent with its own memory off, then with OpenViking on. The number describes the pair, not the memory layer, so it cannot be set beside a standalone retrieval score. The three hosts do not start level and they finish close together — that convergence is the result. TerranSoul is absent by construction: it is not a memory layer bolted onto a third-party host.
Context reduction, three unscaled panels: code-review-graph reports a token-reduction multiplier, supermemory reports a per-cent reduction, and TerranSoul reports a per-cent reduction on its own 240-observation fixture rather than on LongMemEval. PANEL A · CODE-REVIEW-GRAPH · TOKEN REDUCTION · 6 OSS REPOS, 5 QUESTIONS EACH · NO JUDGE 0 150 300 450 600 × code-review-graph 82× median PANEL B · SUPERMEMORY · CONTEXT REDUCTION, PER CENT OF TOKENS REMOVED · NO JUDGE 0 25 50 75 100 % supermemory 99.4 % PANEL C · TERRANSOUL · PER CENT OF TOKENS REMOVED · n = 20 · NO JUDGE 0 25 50 75 100 % vs full-context paste 91.6 % vs 200-line summary 65.5 % The three panels are NOT scaled to each other. Panel A prints a multiplier; Panels B and C print a per-cent reduction, and converting between them would invent a number nobody published. code-review-graph’s spread over the six repositories is 38× to 528×; the dot is its median, 82×. All three are context-cost results, not accuracy results. None says the system answers better. code-review-graph also publishes an 0.71 impact-F1. It is NOT drawn: its author states the ground truth comes from the same graph edges the predictor walks, so the number is circular by construction. Corpus: six open-source repositories in Panel A, LongMemEval in Panel B. Answering model: not disclosed. Panel C is a DIFFERENT corpus from Panel B and shares no series with it: 240 observations and 20 queries vendored from agentmemory, NOT LongMemEval. It is the production default hybrid_search_rrf, scored with the bench’s deterministic embedder rather than the shipped Ollama one. Retrieved context is 2,748 tokens against a 32,660-token full-context paste of all 240 observations. The second bar is the SAME run re-based against a 200-line summary instead. One measurement yields two figures because a reduction is only ever a reduction against a stated baseline. The fixture is fixed-size, so this row is n = 20 and cannot be restated at n = 50.
Agent head-to-head parity: judged answer quality, latency and cost across the assistant row. PANEL A · ANSWER QUALITY 0–10 · 22 PROMPTS, 7 ARCHETYPES · JUDGE gemma4:12b-it-qat 0 2 4 6 8 10 TerranSoul (brain → Ollama) 9.82 OpenJarvis (Stanford SAIL) 9.55 OpenClaw (agent --local) † 8.36 Claude Code + GENesis-AGI †‡ 8.24 Hermes-Agent (-z one-shot) † 6.90 Cost: $0 on every row except Claude Code + GENesis-AGI, which spent $5.94 of cloud inference. It still does not out-score the two free local stacks. PANEL B1 · LATENCY p50 · INFERENCE ONLY, EXCLUDES CLI COLD-START 0 1 2 3 4 s TerranSoul 1.1 s OpenJarvis 3.5 s PANEL B2 · LATENCY p50 · WALL-CLOCK, INCLUDING PER-CALL CLI COLD-START 0 10 20 30 40 s Hermes-Agent 10.9 s Claude Code + GENesis-AGI 17.5 s OpenClaw 38.1 s B1 and B2 deliberately do NOT share an axis: one excludes CLI cold-start, the other includes it. A bar in B2 is not slower than one in B1 — it is a different measurement. Quality is the comparable metric. † measured 2026-06-27 under the earlier single-call-judge protocol — re-measurement pending.   The top two rows use the 2026-07-03 deterministic protocol (judge = median of 3 repeats at temp 0.3). ‡ a different, cloud model (claude-haiku-4-5) — a frontier reference, not like-for-like with the local rows.

Design limits

Four limits of this design, stated as limits rather than as a roadmap. Three are shared with the field; one is specific to us.

There is no Stage 13 yet. A thirteenth stage would need a failure of Stage 11 that somebody has documented and that a new mechanism answers. The open items above are Stage 9's and Stage 11's own unfinished business, not a new limit, so the chain stops here rather than being extended for symmetry.

Design lineage

Credits the source table records for the mechanisms above: reciprocal-rank fusion with k = 60 follows Cormack, Clarke & Büttcher (2009) [49]; HyDE follows Gao et al. [50]; the temporal-edge supersession pattern follows Zep/Graphiti [18]; Matryoshka two-stage search follows Kusupati et al. (2022) [53]; the retrieval evaluator adopts CRAG's grading [25]; community detection follows microsoft/graphrag's hierarchy [54] with LightRAG-style community summaries [46]; conflict resolution adopts Mem0's LLM-arbitrated approach [55]. The self-improvement boundary — harness and memory evolve, weights never — is argued against SIA [56] and the A-Evolve line [57,58,59,60]. Benchmarks hold a frozen gemma4:12b-it-qat actor fixed even though the shipped product default is a 1-bit Bonsai 27B, so a measured change is attributable to memory and harness rather than to a model swap.

CorrectionsWhat this page got wrong, and what replaced it

Consolidated from the running notes that used to interrupt the argument. Each entry is a claim this page published, why it was wrong, and what it says now. Nothing is deleted from this trail.

  1. 2026-07-27 — chat was called byte-identical to the validated retrieval path. It is not: chat NDCG@10 93.8 against the rrf arm's 95.04, with 140 of 500 retrieval orderings different. The identity assertion was false in kind, not off by digits, so it was deleted rather than re-numbered and the mode's own measured value published instead.
  2. 2026-07-27 — the four modes were drawn as a ladder in which every rung improves on the one below. Measured, they are three behaviours, and research is a no-op against plain rrf at 7.5× the latency. The figure that drew the ladder is gone; the measured order is printed in the coverage table instead.
  3. 2026-07-28 — the think regression was overstated by an order of magnitude. This page published NDCG@10 87.25 and R@20 99.0 for think, calling it a recent collapse. Every digit came off a real artifact — and the artifact was six hours too old. Commit 76f2276d (“prompt-context retrieval reorders, it never prunes”) landed between it and the next run: the prompt path had been handing the reranker a drop threshold, so a pointwise judge was deleting the secondary golds of multi-gold questions instead of demoting them. The post-fix re-run restores R@20 to 99.8, and because reordering a twenty-item list cannot change recall@20, that recovery is proof the golds had been deleted rather than merely misranked. The genuine shortfall against chat is 0.52 points, not 6.53. The error was choosing the wrong artifact because its directory was named for the current branch, so artifacts are now pinned by date-against-commit and never by folder name. The obligation is not discharged, only resized.
  4. 2026-07-28 — a 0 % attack-success result against MINJA and PoisonedRAG was withdrawn. An audit that recomputed every published number from its own artifact found no run, no results directory and no benchmark row behind it. The gate is real and regression-tested, but 0 % followed structurally from blocking, which is an argument. A quantified security claim is the worst kind to publish unmeasured.
  5. 2026-07-28 — a “~1.15M CRUD/s write engine” figure was withdrawn. No artifact produces it: no results directory, no committed report, nothing in history. The sharded write engine is built; its throughput is unmeasured on the record. What survives the withdrawal, because it does have an artifact, is the ingest side: end-to-end ingest is measured in the thousands of rows per second, up to 22,745 rows/s at 1,000,000 rows (benchmark/results/jd-validate-1m-engine-on-v2/report.md), a single-machine self-run bench.
  6. 2026-07-27 — a perfect en/vi/ja score was attributed to the shipped max mode. It was produced by benchmark/scripts/jd-max-bench.mjs, a bench-local harness with its own retrieval loop and its own judge posting directly to Ollama. The verify-and-rank capability is real; the measurement belongs to the harness, and the product mode has never been run on that gate.
  7. 2026-07-28 — the shard router was described as holding quality “while cutting fan-out”. That reads as a speed result the run does not support: routed measured slower at every percentile. And 60.5 against 59.5 is true of R@10 only — the routed arm is behind on R@1, R@5, NDCG@10, MAP@10 and MRR@100 in the same run. Both halves are now in the coverage row.
  8. 2026-07-28 — “the KG cascade measured 0pp” read as “no effect”. The 0.00 pp is specific to R@10; on the same run the graph arm is fractionally ahead on R@5, NDCG@10, MAP@10 and MRR@100. The honest sentence is “no recall gain and a fraction of a point of ranking gain, for double the latency”. The verdict did not change; the sentence did.
  9. 2026-07-27 — the published floor was corrected for sighted readers and left stale for screen-reader users. Four copies of it lived only inside figure descriptions, where no check reached them. The wordings the numbers register still pins are LongMemEval NDCG@10 95.1, LongMemEval-S NDCG@10 95.1, Hybrid FTS5+dense+RRF · 95.1, and, from those descriptions, “reciprocal-rank fusion at LongMemEval NDCG-at-ten 95.1”. All four say the same thing. The figures are gone from this revision and the floor is stated once, in the coverage table.
  10. 2026-07-28 — the prefix gain was printed as a bare “+4.2pp R@10” with no bench named. It is EmbeddingGemma-only, on a 240-observation / 20-query fixture scored pure-vector, and the same sweep records prefixes costing two other embedders recall. Both endpoints and both files are now named in the coverage row so the subtraction is checkable by hand.
  11. 2026-07-03 floor provenance. The 95.1 / 95.9 floor run used a store shared across three systems in one process. Cross-arm contamination in a shared process has since been demonstrated, which is why every arm quoted above is its own process. That does not retire the floor — retiring a floor needs forensic proof it was a measurement error and an explicit sign-off — but a reader comparing the two should know the methodologies differ.
  12. 2026-07-28 — a Japanese NDCG@10 range was withdrawn rather than re-published. The withdrawn text read: “a CJK trigram mirror lifts Japanese NDCG@10 9.5 → 42.3–65.0 at 1M rows”. Every digit of it predates the cross-lingual tokenizer fix, no full-500 artifact exists for it, and the re-measurement it was owed has never been run — so it described code that no longer exists, at a scale nothing on disk covers. It is withdrawn rather than re-stated with a caveat, because a range with no reproducible artifact behind it is not a measurement. Multilingual retrieval numbers live in the main paper, against their own artifacts.
  13. 2026-07-28 — the two twins of this article published different structures. The HTML was rewritten around a twelve-stage what-it-is / pros / cons / why-next argument with a coverage table; the Markdown twin was left at the older eleven-stage narrative for a day, so the two artifacts of the same article disagreed on the number of stages, on which stage described TerranSoul, and on 44 quantitative claims — and the parity gate could not see it, because it compared number sets and never structure. The Markdown is now a full mirror of the argument and every number; figures remain HTML-only, by declared design, and the gate now enforces both halves of that split.
  14. 2026-08-02 — the LoCoMo blank was labelled “retrieval only, a documented gap”, and it is not that. The wording said we had never built an end-to-end question-answering pass over LoCoMo. We had: benchmark/scripts/locomo-mteb.mjs --qa-eval=mem0-paper generates an answer and judges it, and predates this page. The real obstacle is one level down and was never checked before the caption was written — the MTEB distribution of LoCoMo ships gold passages and no gold answers, so that pass grades against retrieved passages on a graded 0–10 rubric with a local self-judge, and none of those three properties matches the proportion-correct that Letta, Mem0, Zep and MemOS report. The blank does not move, because a number that cannot join the series is not a fill. The caption does: “we have not built it” and “we have built it and it does not yet mean what this axis says” are different admissions, and only the second one is ours. A related over-claim in the harness’s own console output — that its scores are “directly comparable to the Mem0-paper baselines” when the judge is gpt-4o-mini — is wrong for the same reason and independently of the judge, and is recorded here rather than quietly fixed.
  15. 2026-08-03 — the think never-regress obligation is discharged, and the 2026-07-28 entry above closes on the wrong note. That entry ends “the obligation is not discharged, only resized”. It was true when written and is not true now, and the correct fix is to say so here rather than to edit the trail. The obligation was to get think's NDCG@10 back above chat's. It was closed on 2026-08-02 by removing the reranker from think's retrieval path instead: think and chat now call the same retrieval function with the same arguments, and the rerank flag both pass is resolved by one shared production rule rather than by two literals that happened to agree. The gap did not shrink; it stopped being able to exist. Measured together at n = 50 on 2026-08-03 the two are identical on all five retrieval metrics. Two things this does not license: the 93.26 figure is not withdrawn — it is what the reranking configuration really scored — and a mechanism argument is not a measurement, so the four-rung table says n = 50 wherever it is quoted.
  16. 2026-08-03 — the four thinking modes had never been measured as a set, and the page presented them as if they had. The rungs came from three separate campaigns on three different days, two of them on code that has since changed, so every gap between them mixed a rung difference with a run difference. The page said so about individual rows and never about the comparison as a whole. Four arms on one configuration on the current code now exist at n = 50, in their own frame, and they are deliberately not merged into the full-500 coverage table: the fix for mixing sample sizes is a second frame, not a footnote.

ReferencesEvery citation, grouped by stage

Citation strings follow the source table in docs/brain-advanced-design.md [51] and its linked comparison records; tools and products without papers are cited by name as the source table cites them. One negative finding recorded for auditability: RAPTOR appears nowhere in the source document and is therefore deliberately absent from this list rather than attributed to it; HippoRAG likewise enters the main paper via its own related-work references, not via the source table.

Stage 1 — lexical / keyword
  1. Robertson, S., et al. Okapi at TREC-3 (Okapi BM25). TREC-3, 1994.
  2. Akarsu, K., et al. BM25 vs dense retrieval over financial documents (23,088 queries / 7,318 documents). 2026. arXiv:2604.01733.
  3. Furnas, G., Landauer, T., Gomez, L., Dumais, S. The vocabulary problem in human-system communication. CACM, 1987.
  4. Clavié, B., et al. Latent Terms. 2026. arXiv:2605.29384.
Stage 2 — semantic search
  1. Karpukhin, V., et al. Dense Passage Retrieval for Open-Domain Question Answering (DPR). EMNLP 2020.
  2. Google. Gemini Embedding 2 (tool, cited by name).
  3. Weller, O., et al. (DeepMind). Sign-rank limits of single-vector embedding retrieval + the LIMIT benchmark. 2025. arXiv:2508.21038.
  4. Khattab, O., Zaharia, M., et al. ColBERT-family late-interaction retrieval (cited by name in the source table).
  5. LEMUR: fast multi-vector retrieval via reduction to single-vector ANN. ICML 2026. arXiv:2601.21853.
Stage 3 — vector databases
  1. Johnson, J., Douze, M., Jégou, H. Billion-scale similarity search with GPUs (FAISS). 2017.
  2. Malkov, Y., Yashunin, D. Hierarchical Navigable Small World graphs (HNSW) (cited by name).
  3. Milvus; Pinecone; Weaviate; pgvector; SQL Server 2025 native vector indexes; Amazon S3 Vectors; DiskANN-style SSD indexes; VAST 50B-vector benchmark (tools/products, cited by name as the source table cites them).
  4. Disk-resident graph-index immaturity (≤15% I/O utilization). 2026. arXiv:2603.01779.
  5. GraphRAG-Bench. ICLR 2026.
Stage 4 — knowledge graphs
  1. RDF/SPARQL; Freebase; DBpedia; Neo4j; Google Knowledge Graph (2012) (standards/products, cited by name).
  2. data.world benchmark (2023), as cited by GraphRAG vs Vector RAG: GPT-4 SQL 16.7% vs SPARQL 54.2% execution accuracy on 43 insurance-domain questions.
  3. Dong, X., et al. Knowledge Vault. KDD 2014.
  4. Zep AI. Zep / Graphiti: A Temporal Knowledge-Graph Architecture for Agent Memory. 2025. arXiv:2501.13956. (DMR 94.8 vs MemGPT 93.4; +18.5% LongMemEval; ~90% lower latency.)
Stage 5 — RAG
  1. Lewis, P., et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 2020.
  2. Microsoft. Azure AI Search agentic retrieval (GA April 2026; product, cited by name).
  3. "Lost in the Middle" matched-control study: semantic competition, not context length. 2026. arXiv:2605.27294.
Stage 6 — agentic / corrective RAG
  1. Asai, A., et al. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. ICLR 2024. arXiv:2310.11511.
  2. Yan, S., et al. Corrective Retrieval-Augmented Generation (CRAG). 2024 (v1 January–v3 October). arXiv:2401.15884.
  3. FareedKhan-dev. rag-zero-hallucinations pipeline (crag_ok 0.7 / crag_bad 0.4; τ_claim 0.3 / τ_abstain 0.3; 2% hallucination at 0.908 faithfulness / 0.46 coverage; tool, cited by name with its published thresholds).
  4. Ravi, S., et al. Lynx / HaluBench (15K faithful/hallucinated triplets; verifier AUROC 0.702 as used above). 2024. arXiv:2407.08488.
  5. The Gray Zone of Faithfulness. 2025. arXiv:2510.21118.
  6. FaithLens. 2025. arXiv:2512.20182.
Stage 7 — observability & evals
  1. TruLens (span-level tracing; tool, cited by name).
  2. FaithJudge. 2025. arXiv:2505.04847.
  3. RAGAS; TruLens; LangSmith (tools, cited by name).
  4. Zheng, L., et al. MT-Bench / Judging LLM-as-a-Judge. NeurIPS 2023.
  5. OpenTelemetry GenAI semantic conventions (standard, cited by name).
  6. Reliability without Validity (21 judges, ~541K judgments). 2026. arXiv:2606.19544.
  7. EvoAgentBench. 2026. arXiv:2607.05202.
Stage 8 — two-way agent memory
  1. Packer, C., et al. MemGPT: Towards LLMs as Operating Systems (→ Letta). 2023. arXiv:2310.08560.
  2. Zep; Mem0; OpenAI ChatGPT Memory; Anthropic Claude memory (free for all users + cross-assistant import, March 2026) (products, cited by name).
  3. MINJA: memory injection attacks (>95% injection success). 2025. arXiv:2503.03704.
  4. The Mem0-vs-Zep LoCoMo measurement dispute (84% → 58.44% → 75.14%; named dispute, as recorded in the source table).
  5. MemoryArena. 2026. arXiv:2602.16313.
  6. TOKI: write-time contradiction resolution as concurrency control. 2026. arXiv:2606.06240.
Stage 9 — governed, self-consolidating memory
  1. Letta. Sleep-time compute. 2025. arXiv:2504.13171.
  2. Hindsight "Observations" — #1 on BEAM, 73.9% at 1M tokens; BEAM's ~25% drop from 1M to 10M tokens (named results, as recorded in the source table).
  3. GateMem — the utility / access-control / reliable-forgetting trilemma (named result).
  4. Memory lifecycle attacks. 2026. arXiv:2602.08563.
Stage 10 — memory infrastructure
  1. Letta; Zep (Graphiti); Mem0 (~47k stars; Mem0 49.0% vs Zep 63.8% on LongMemEval temporal reasoning; Open-LLM-VTuber mounts Letta as its brain) — as recorded in the source table.
  2. Guo, Z., Xia, L., Yu, Y., Ao, T., Huang, C. LightRAG: Simple and Fast Retrieval-Augmented Generation. 2024. arXiv:2410.05779 (MIT). ApeRAG's graph-RAG lineage.
  3. apecloud/ApeRAG (Apache-2.0) and its optional MinerU parsing service (tools, cited by name).
  4. ApeRAG architecture comparison record: benchmark/APERAG_COMPARISON.md (ApeRAG publishes no quantitative benchmark suite — architecture-only comparison).
Stage 11 — the learned & robust frontier (2025–2026)
  1. Santhanam, K., Khattab, O., Saad-Falcon, J., Potts, C., Zaharia, M. ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction (and the PLAID engine). 2022. arXiv:2112.01488. The multi-vector / late-interaction line.
  2. Dhulipala, L., Hadian, M., Jayaram, R., Lee, J., Mirrokni, V. MUVERA: Multi-Vector Retrieval via Fixed Dimensional Encodings. NeurIPS 2024. arXiv:2405.19504 (reduces MaxSim to a single vector that rides a standard ANN index).
  3. Faysse, M., et al. ColPali: Efficient Document Retrieval with Vision Language Models. 2024. arXiv:2407.01449 (late interaction over page images; ColQwen extends it).
  4. Yan, S., et al. Memory-R1: reinforcement-learned memory management (add/update/delete/noop from downstream reward); with Mem-α learned memory construction. 2025.
  5. Gutiérrez, B. J., et al. HippoRAG (arXiv:2405.14831, 2024) and From RAG to Memory: Non-Parametric Continual Learning for LLMs (HippoRAG 2, arXiv:2502.14802, 2025) — personalized-PageRank passage-graph retrieval.
  6. Xiang, C., et al. Certifiably Robust RAG against Retrieval Corruption (RobustRAG, isolate-then-aggregate). USENIX Security 2025. arXiv:2405.15556.
  7. Debenedetti, E., et al. Defeating Prompt Injections by Design (CaMeL; capability + information-flow control). 2025. arXiv:2503.18813.
  8. Gao, J., Long, C. RaBitQ / Extended RaBitQ — quantization with a provable per-vector error bound, dominating prior methods at equal compression. SIGMOD 2024/2025.
Design lineage and the self-evolution boundary
  1. Cormack, G., Clarke, C., Büttcher, S. Reciprocal Rank Fusion (RRF) — the rank-fusion method and its k = 60 constant. SIGIR 2009. (Short-form citation; the full published title is available under that venue and year.)
  2. Gao, L., et al. Precise Zero-Shot Dense Retrieval without Relevance Labels (HyDE). 2022. arXiv:2212.10496.
  3. Kusupati, A., et al. Matryoshka Representation Learning. 2022.
  4. microsoft/graphrag — hierarchical community detection and global/local query routing (tool; adopted per the source table's GRAPHRAG-1a/b/c record).
  5. Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory (LLM-arbitrated conflict resolution, adopted at the typed-edge layer). 2025. arXiv:2504.19413.
  6. Hebbar, P., et al. SIA: Self Improving AI with Harness & Weight Updates. 2026. arXiv:2605.27276. github.com/hexo-ai/sia (MIT).
  7. Lin, M., Lu, H., Shi, Z., et al. Position: Agentic Evolution is the Path to Evolving LLMs (A-Evolve). 2026. arXiv:2602.00359. Published-numbers record: benchmark/A-EVOLVE-COMPARISON.md.
  8. Harness Updating Is Not Harness Benefit. 2026. arXiv:2605.30621.
  9. Liu, Z., Shi, Z., Sang, Y., He, B., Lin, M., Wei, T., Wang, D., Dumoulin, B., Jin, W., Lu, H. Adaptive Auto-Harness: Sustained Self-Improvement for Agentic System Deployment on Open-Ended Task Streams. 2026. arXiv:2606.01770.
  10. Shi, Z., He, B., Sang, Y., Lu, H., Dumoulin, B. A-Evolve-Training: Autonomous Post-Training of a 30B Model. 2026. arXiv:2606.20657 (weight fine-tuning; recorded as the deliberate opposite of the frozen-actor boundary).
Source documents
  1. TerranSoul. Brain & Memory — Advanced Architecture Design: docs/brain-advanced-design.md (the evolution-timeline table this article narrates; per-stage citation trail in mcp-data/shared/memory-seed.sql, seed:lesson-worldwide-memory-evolution-2026-07-09).
  2. TerranSoul Research. Three Falsifiable Hypotheses About External Memory for Frozen Language Models: Behavioral Change, Single-Substrate Retrieval, and Measurement Discipline. 2026-07 (the main paper; all TerranSoul measurements cited there with source files).
The Evolution of Memory for AI Systems · TerranSoul Research · 2026