Three Falsifiable Hypotheses About External Memory for Frozen Language Models: behavioral change, single-substrate retrieval, and measurement discipline.

Three narrow hypotheses about external memory for language models whose weights never change — behavioral change without weight edits, one-substrate retrieval competitiveness across benchmarks, scales, and languages, and the dominance of measurement discipline over apparent regressions — each stated with its falsification condition and tested across nine evidence lanes, May–July 2026.

Why this existsOwning a memory, not renting a mind

A frozen language model begins every session the way it began the last one: knowing nothing of what came before. Close the chat, move to another application, or upgrade to a newer model, and whatever it had learned about a person, a codebase, or a business leaves with the context window. The 2026 memo-vs-memory debate names this precisely — Xu, Dai & Zhang [1] argue that pasting retrieved snippets into a prompt each turn is a memo, not a memory (lookup, not learning), and prescribe consolidating what a system knows back into model weights so that it endures.

TerranSoul was built to answer the same problem from the opposite side. Continuity need not live in the weights; it can live in what a store does over time (Section 03). Hold the model's weights frozen by standing mandate and give it a governed external memory reached over the Model Context Protocol — a store that consolidates offline, forgets by salience, structures itself as a typed graph, and revises beliefs in place — and the same frozen model accrues behavior across sessions, across applications, and even across model swaps. The memory is the part that persists; the model underneath becomes an interchangeable, rentable part. That is the behavioral definition this paper sets out to test, not to assert.

There is a reason to care beyond the taxonomy. The capability to train a frontier model is concentrating in a small number of corporations, in a few countries, that can afford the compute; nearly everyone else — individuals, small businesses, most nations — reaches that capability by renting it through an API rather than owning it. As the marginal cost of software itself trends toward zero, durable value shifts to what stays scarce: the data a system accumulates, the infrastructure it runs on, and the long-lived memory it builds. A design that consolidates a user's accumulated knowledge into weights they neither hold nor govern makes that memory a tenant of the model vendor. If memory is behavioral, and therefore separable from weights, a person, a business, or a nation can hold and govern the memory their AI accumulates instead of leasing it — the position this paper argues for, and the governance concern the memory-security literature raises across the memory lifecycle [35].

Concretely, that layer is a local-first Rust memory server — six-signal hybrid retrieval fused with reciprocal-rank fusion, a typed knowledge graph, and write-time governance — wrapped around frozen-weight models and runnable on a single workstation at roughly one-second local latency and $0 marginal cost, portable across whichever frozen model is placed behind it. This is a mission, and the rest of the paper treats it as one: the sections below reduce the thesis to three falsifiable hypotheses (Section 02) and report the numbers that test them, including the ones that cut against it. Where this substrate sits within fifty years of memory technology, and every system it stands beside, is the subject of the companion article [53], highlighted below.

AbstractThree hypotheses, nine evidence lanes

Long-term memory for language-model agents is usually argued about architecturally — should knowledge live in weights, in a long context window, or in an external store? We narrow the question to three falsifiable hypotheses about one concrete external store: a local-first Rust memory server (six-signal hybrid retrieval with reciprocal-rank fusion, a typed knowledge graph, write-time governance) reached over the Model Context Protocol, wrapped around language models whose weights are frozen by standing mandate. H1: coupling a frozen model to this substrate produces measurable cross-session behavioral change that matched no-substrate controls do not show. H2: the same production retrieval path is competitive on public conversational-memory benchmarks and holds usable quality at one million rows in multiple languages at ~1 s local latency and $0. H3: in long-lived evaluation of such systems, apparent regressions are dominated by measurement and configuration artifacts, so reproducibility requires pinning every ranking input to an explicit state/environment epoch.

The evidence: on LongMemEval-S the substrate's rrf mode reaches R@5 99.4% / NDCG@10 95.1% / MRR 95.9% (its rrf_emb mode R@10 100.0%) — the published floors, which a same-methodology re-measurement on current code puts at NDCG@10 95.04 / MRR 95.76, an open obligation to regain rather than a headline to lower (5.1) — beside agentmemory's published 95.2% R@5; on the harder LoCoMo slice (the full 1,976-query MTEB retrieval slice, 2026-06-28 run) the lexical fusion reaches R@10 69.3% and the dense channel costs 2.7 points rather than adding them, with the multi-hop gap reported — and that R@10 is not the same quantity as the R@10 one clause above it: LoCoMo's is qrel coverage, LongMemEval's is the binary recall_ANY@k of 4.2, and rescoring the committed rrf artifact under LoCoMo's coverage definition puts LongMemEval's own R@10 at 98.64% against the published 99.8%, a 1.16 pp definitional gap (3.09 pp at k = 5); at 1,000,000 multilingual synthetic resumes single-pass lexical retrieval scores NDCG@10 100.0 (en) at 0.43–0.56 ms warm p50 in the settings-memo arm, which ingests at 2,857 rows/s, while carrying a cross-lingual gap (vi 45.4 / ja 65.0) and, at that run's own top-k, a capped R@100 regression against the run it retired (en 83.0% → 62.0%, 5.3); the 22,745 rows/s ingest is a different arm — the shipped write-engine default, which measures warm p50 0.54–0.59 ms — so the fastest query latency and the fastest ingest here are two configurations, not one; a heavier bench-local MAX retrieve-and-verify harness — not the product's max thinking mode, which is unmeasured on that corpus (5.3) — closes the language gap to NDCG@10 = P@10 = 100% on all four queries (en/vi/ja/typo) at a cost of 48 s to 29 min per query; a personal-AI head-to-head records quality 9.82/10 at p50 1.06 s and $0 beside OpenJarvis' 9.55 at 3.47 s. On the acting axis, a three-arm interactive-fiction bench shows both controls at 0 while the substrate arm reaches 10–20 on a frozen 4B; a delivery-reliability isolation moves the same model from a non-deterministic 73/177 to a deterministic 350/350 with no LLM in the ablation; on SIA's headline LawBench a frozen 12B + memory scores 76.3% beside the weight-trained 120B system's 70.1%; and a vision self-improvement loop driven end-to-end through the product CLI climbs 59.62 → 71.66 under a frozen judge (v2-era protocol). Under a successor v4 fixed-panel judge the frozen 12B's pure self-improvement track climbs 36.1 → 55.49 raw in a single day, every edit its own — and the same actor improves further with knowledge taught into its own memory, no model change. A later objective-correctness protocol reframes the same task deterministically: benchmark/boeing747/geometry-check.mjs loads the built model and inspects its actual geometry (engine count, fuselage length-to-diameter ratio, wing sweep/dihedral/symmetry, hump/fin/gear presence, part connectivity), scoring 747 correctness out of 100 with no LLM and no variance — a carefully-iterated TerranSoul build scores a perfect, reproducible 100/100, versus 66.82 (fable-5) and 65.45 (opus-4.8) for the paid frontier models' builds on the identical test (validated to discriminate, broken builds < 45, and locked by a CI regression guard). Re-judging under an independent fair vision judge (Claude Sonnet 5) — which does not apply the photorealism discount that structurally caps the frozen gemma judge near 52/100 for any primitives-only model — places the human-taught frozen local model at 80.5% of a paid frontier model's build — 41.96 vs the anchor's 52.14, and that 41.96 is the best of six seeded runs (39.17 / 37.37 / 34.60 / 39.18 / 41.96 / 36.76), whose mean 38.17 is 73.2% parity and whose worst is 66.4%, so 80.5% is a best-of-n maximum and not a one-shot expectation — up from 64.98% (33.88 vs 52.14, a single pre-teaching run under the same K = 1 judge), and the strongest TerranSoul build at 86/100 on subjective quality. Four independent forensic episodes support H3, including one in which a suspected retrieval regression reproduced the prior baseline to the decimal once a single environment variable was restored, and one in which a 10.9–16.6 s latency collapse at 1M rows proved to be a deadline-truncation caching bug (fixed: 11.8–18.1× faster). We also state the bounds directly: compositional generalisation is now measured and bounded to depth three, several runs are n-limited and single-machine, and the Boeing all-views threshold and the H100 kernel multiple remain an uncrossed threshold — not reached under either the frozen or the relaxed contract — and an unmeasurable-on-this-hardware estimate respectively.

01 · BackgroundState of the art and related work

1.1 External memory for frozen models

Retrieval-augmented generation [56] established the model-external pattern: a frozen generator conditioned on passages fetched from a non-parametric store. Its two channels have canonical parents — the BM25 family of lexical scoring [54] and dense passage retrieval [55] — and the substrate measured here fuses exactly those two, so the methods this paper's instrument composes are all published: reciprocal-rank fusion [16], HyDE query expansion [17], Contextual Retrieval [18], and LLM-as-judge reranking. Two retrieval paradigms we did not adopt must be named: late interaction [57,58], which BEIR [62] finds gives the strongest zero-shot effectiveness at a high computational cost, and learned sparse retrieval [59], which attacks the vocabulary mismatch that our own lexical channel is exposed to; Section 4.1 states the local-first reason for both omissions. Two further lines are the published answers to the multi-hop deficiency we report in Section 5.2 and do not close: GraphRAG's LLM-built entity graph with community summaries [67], and RAPTOR's recursive summarisation tree, which reports +20% absolute on QuALITY by retrieving at several levels of abstraction [68]. Self-RAG's reflection tokens [69] and CRAG's retrieval evaluator with corrective actions [70] are the published ancestors of the abstention behavior criterion C4 demands (Section 03).

The agent-memory lineage factors that store into a managed, evolving object: MemGPT/Letta's OS-style paging [12], Mem0's extract-consolidate-update pipeline [11], spatial schemes such as MemPalace [13], HippoRAG's neurobiologically-motivated multi-hop recall [14] and its non-parametric continual-learning successor HippoRAG 2 [100], temporal typed-edge knowledge graphs in Zep/Graphiti [15], A-MEM's Zettelkasten notes with link generation and in-place memory evolution [99], and MemoryBank's Ebbinghaus-forgetting-curve decay [98] — the direct ancestor of our forgetting criterion C2. Generative Agents [79] is the most-cited demonstration that a memory stream plus reflection and recency/importance/relevance retrieval changes a frozen model's behavior across sessions; Letta's sleep-time compute [101], which reports roughly a 5× reduction in test-time compute by thinking over a stored context before the query arrives, is the systems-side precedent for the offline consolidation pass of C1. CoALA [97] supplies the field's accepted descriptive taxonomy — working, episodic, semantic, and procedural memory over a structured action space on internal memory — which Section 03's five criteria refine into falsifiable demands. A parallel architectural pole keeps memory inside the network: MEMORYLLM's fixed-size latent memory pool, whose integrity is reported to hold over nearly a million updates [102], and Titans' neural long-term memory learned at test time [103]. Du formalises agent memory as a "write-manage-read loop tightly coupled with perception and action" [9]; the 2026 survey wave [32,33,34,36,37] maps the same territory, the agentic-RAG survey [22] names memory management and governance as open challenges, and a dedicated survey tracks security across the memory lifecycle [35]. An architecture-only comparison against ApeRAG, which publishes no benchmark numbers, is recorded in [47].

On the reasoning side the lineage is ReAct [19], Reflexion [20], and Voyager's skill library [21]. Self-Refine's generate–critique–refine loop, run with a single model in all three roles [81], is the structural template for the vision loop of Section 5.8 — and Huang et al.'s finding that intrinsic self-correction without external feedback does not reliably improve reasoning, and can degrade it [82], is the standing caution we answer there (our critic is scored by a separate frozen judge, which is external feedback). TextGrad gives such loops a formal vocabulary — natural-language feedback backpropagated through a compound system with no weight updated [83]. HyDE [17] and Contextual Retrieval [18] enter our read path only as per-query-class tools, gated by ablation [49], not as global defaults.

1.2 The 2026 debates

Four live disputes frame our hypotheses. First, the memo-vs-memory charge: Xu, Dai & Zhang [1] argue that per-turn retrieve-and-paste "agentic memory" is a memo, not a memory — lookup, not learning — and prove a compositional sample-complexity separation between context-engineering and weight-based memory, prescribing consolidation into weights. We accept the diagnosis (a flat, append-only vector store is a memo) and contest the dichotomy: the memo/memory distinction is behavioral, not locational, and Section 03 states the behaviors. We concede the compositional theorem stands; nothing here contests it on its own ground, and our own multi-hop retrieval gap (Section 5.2) is consistent with its prediction. Second, the weight-update counter-pole: SIA [2] co-evolves a task agent's scaffold and weights and reports that combining both levers exceeds scaffold-iteration alone on its benchmarks; SEAL [89] is the better-known statement of the same pole — a model that generates its own finetuning data and update directives, producing persistent weight edits under an RL loop rewarded by downstream performance — and ROME [90] and MEMIT [91] are the instruments a consolidate-into-weights prescription [1] would actually reach for. The engineering counter-evidence is on the record and we lean on it rather than assert the point: Gupta et al. [92] show that editing ROME/MEMIT sequentially at scale drives a model into gradual and then abrupt catastrophic forgetting of both its prior edits and its downstream ability. That is the strongest published reason a lifelong store is not obviously the weaker place to put knowledge, and it is the empirical reply to [1] that its compositional theorem does not supply. Section 5.7 measures the frozen alternative on SIA's own suite.

Third, the frozen-harness school: A-Evolve [3] evolves skills, memory, and prompts around a frozen model with git-gated rollback, reporting deltas of +2.2 to +15.2 pp across ten public benchmarks [46]; the caution literature warns that harness updating is not automatically harness benefit — weak actors fail to load or follow the skills their evolver writes [4] — and that evolved harnesses grow brittle under ungated corpus growth [5]. The same group's weight-training line [6] is the deliberate opposite of our design choice. The dual-process (DCPM [7]) and governance (SSGM [8]) lines, and SGS's frozen-guide curriculum [10], each contribute one adopted mechanism, noted where used. This school long predates the 2026 wave, and H1 must be read against it rather than as a first claim. ExpeL [80] extracts natural-language insights from its own trajectories with no parametric update, explicitly motivated by frontier models being API-only. Memento [86] runs case-based memory over a memory-augmented MDP and reports GAIA-validation top-1 at 87.88% Pass@3 with the LLM's gradients never touched. Dynamic Cheatsheet [84] hands a frozen black-box model a self-curating plaintext memory and lifts GPT-4o on Game of 24 from 10% to 99% within an eval run, without labels or human feedback — the cleanest existing demonstration of H1's mechanism on a much simpler substrate, and the ablation a reviewer is entitled to ask of us (six-signal typed store versus a flat cheatsheet). ACE [85] names the two failure modes any evolving context must survive — brevity bias, where consolidation discards domain detail, and context collapse, where iterative rewriting erodes information over time — and answers them with structured incremental updates instead of rewrites, which is precisely the argument for our C3 typed in-place revision, reporting +10.6% on agent benchmarks with a smaller open model. The Darwin Gödel Machine [87] pushes the pole to self-modifying code validated on benchmarks over an archive of variants (SWE-bench 20.0% → 50.0%) — the ungoverned limit that C5's write gate is designed against. A 2025 survey [88] gives the family its unified optimiser/evaluator/agent-system abstraction. H1's contribution, then, is not the claim that a frozen model changes behavior through an external store — these works make and support that claim — but a matched three-arm zero-baseline protocol, an AGI-purity gate, and a real process boundary under which the claim can be falsified.

Fourth, the long-context alternative: if a model reads a million tokens, why retrieve at all? We rest that case on the evaluation literature rather than on assertion. RULER [72] finds that models advertising 32K+ context degrade well below their claimed size and that vanilla needle-in-a-haystack is a superficial probe; BABILong [74] measures effective use of only 10–20% of the context, worst on fact-chaining and multi-hop; ∞Bench [75] carries the measurement past 100K tokens; NoLiMa [76] removes literal lexical overlap between question and needle and watches accuracy fall (GPT-4o 99.3% → 69.7% at 32K), which says the earlier passes were substantially string matching; and Lost in the Middle [71] shows a correct document placed mid-context is often effectively invisible. HELMET [73] reports that synthetic needle tasks do not predict downstream performance while RAG-shaped tasks do — our reason for measuring on LongMemEval and LoCoMo rather than on needles. The two head-to-head studies disagree with each other and both are worth stating: Li et al. [77] place long context ahead of RAG on quality when resourced and RAG ahead on cost (hence SELF-ROUTE), while Yu et al. [78] place order-preserving RAG ahead of long context on quality. We take the cost and latency side, and H2 states it as a measurable clause: ~1 s and $0, locally, at 1,000,000 rows (Sections 5.3–5.4).

1.3 Benchmark state of the art

The conversational-memory lineage begins with Multi-Session Chat [124], which introduced the multi-session evaluation paradigm and already found retrieval-augmented and summarise-and-recall architectures ahead of plain long-context encoders in that setting — the finding our own architecture descends from. It is anchored today by LongMemEval [23] and LoCoMo [24]; MemoryAgentBench [25] grounds evaluation cognitively; MemoryArena [26] reports that agents near-saturated on LoCoMo perform poorly on its interdependent multi-session tasks — a standing warning against reading any single benchmark as sufficient. The 2026 successors sharpen that warning: LoCoMo-Plus [127] argues that factual-recall scoring misses latent-constraint memory failures that existing benchmarks cannot see, AMA-Bench [128] moves the target from dialogue to long-horizon agentic trajectories, and a 2026 survey re-organises the field by substrate, cognitive mechanism, and subject orientation [129]. On the retrieval side the standard yardsticks are BEIR [62] — whose headline finding, that BM25 is a robust baseline and dense retrievers often sit below it out of domain, cuts against our own design and is a generalisation test H2 has not taken (Section 5.10) — MTEB [63], the harness whose LoCoMo slice we report in Section 5.2, and MIRACL [64], the public human-annotated 18-language benchmark that is the yardstick our synthetic cross-lingual corpus (Section 5.3) does not replace. The long-context quartet [72,73,74,75] and the literal-matching probe [76] were discussed above.

On the long-horizon acting side, Jericho [27] exposes interactive-fiction games with ground-truth state, and its sibling TextWorld [121] — the generative, difficulty-controllable environment the interactive-fiction agent literature is built on — is the environment in which task complexity could be varied to de-confound an actor-bound ceiling from a memory contribution, which our single-ROM design does not do (Section 5.10). ALFWorld [122] is the environment Reflexion [20] and ExpeL [80] report their memory gains on, making it the obvious head-to-head we have not yet run; ScienceWorld [123] specifically tests procedural-knowledge transfer across task variations, which is the compositional gap we concede. BALROG [28] and TALES [29] report broad model struggle on interactive fiction — TALES calls Zork I, one game in our Section 5.5 experiment, "insurmountable … even for modern state-of-the-art LLMs"; the upstream ZorkGPT agent we run unmodified is [30].

H3 is not a new observation and we do not present it as one. Armstrong et al. [113] showed in 2009 that a decade of reported ad-hoc retrieval improvements did not accumulate, because baselines were weak and each group compared only against its own runs, and prescribed the remedy this paper adopts: longitudinal comparison against pinned references. The neural-hype analyses [114,115] found the same weak-baseline pathology in neural ranking and showed claimed gains are non-additive over a strong baseline; Ferrari Dacrema et al. [116] reproduced only 7 of 18 top-venue neural recommenders and found 6 of those could be matched or exceeded by well-tuned simple heuristics; Henderson et al. [117] showed that in long-horizon sequential-decision experiments, non-determinism and seed variance alone produce spreads wide enough to reverse conclusions — which is why Section 5.5 reports a modal result and a distribution rather than a peak; Mizrahi et al. [118] showed single-template evaluation changes both absolute scores and relative rankings across 6.5M instances; Miller [119] supplies the statistical machinery (standard errors, paired analysis, power planning) that n-limited head-to-heads require; and The Leaderboard Illusion [120] documents how selective disclosure and private variant testing distort public rankings — a hazard we turn on ourselves in Section 5.10. Entity-Collision [126] is the current-frontier methodological statement for exactly H2's claim: agent-memory benchmarks report a single hit@k that confounds uncontrolled lexical overlap with genuine retrieval lift, and it proposes a stratified protocol that pins the BM25 floor by construction so any lift is attributable to the embedder. H3's contribution is therefore to instrument a known discipline for the agent-memory era, not to discover it.

On the adversarial side, MINJA [31] demonstrates a >95% injection success rate (ISR; its end-to-end attack success rate averages 76.8%) against unguarded write paths — the threat model our substrate's write gate is designed against — though, per the 2026-07-28 correction in Section 5.10, the gate is built and regression-tested against that model and has NOT been benchmarked at those published attack strengths — and the memory-lifecycle security survey [35] catalogues the wider attack/defence surface that C5 sits inside.

1.4 Companion article

The worldwide evolution of memory and retrieval technology — twelve stages from 1970s inverted indexes to governed self-consolidating memory, each with the problem it solved, the limitation documented against it, and full citations — is narrated in the companion article, The Evolution of Memory for AI Systems [53]. This paper keeps only the four-row engagement summary below; the per-work adopt/contest/concede detail lives in the companion.

1.5 An engineering-practice claim about graphs, checked against this instrument

One 2026 document makes the case for graph-structured shared memory more forcefully than the research literature does, from the practice side rather than the research side, and it is worth engaging rather than ignoring: Graph Engineering: The Karpathy Loop, Improved 1000× by Itself — The Anthropic Playbook [142]. Its provenance is stated first because it bounds how much weight the engagement can carry. The document carries no named author — its acknowledgment says only that it is “an independent synthesis assembled for study” — it is dated July 2026, it disclaims affiliation with and endorsement by every party it is named after, and it was read here as a local PDF with no public URL, so it is cited as a document rather than as a link. Its thesis: a single-agent ratchet loop (edit, measure, keep or revert) is the smallest complete unit of autonomous engineering; every larger architecture is that loop with one bottleneck externalised — task order, parallel search, experiment lineage, shared facts; and the progression converges on typed, queryable graphs as the shared-memory layer that lets multi-agent systems scale without copying every worker's transcript into an orchestrator's context. Its closing acceptance test is one sentence we would sign as written: every important output should trace to an objective, a plan, an artifact, a source, a graph path, an evaluator decision, and a bounded execution record.

Two of its arguments we can upgrade from assertion to measurement. The first is “a ratchet improves the metric it can see”: the document names metric gaming as the standing hazard of any keep-or-revert loop but offers no test separating a genuine improvement from a gamed one. Section 5.8's two-judge decomposition is such a test, and it is cheap — run the same change past two judges of different strength and require the strong one to move. There, render-side fixes lifted the handicapped frozen 12B judge 63.6 → 72.2 (+8.6) while the strong vision judge went 67.84 → 68.0, essentially flat; a gaming trick would have inflated both, so the split is what a correct measurement fix looks like and not what a gamed metric looks like [45]. The second is its four graph-write invariants — every claim carries a source or is marked inference, every artifact carries an authoring run and a version, every evaluation names a rubric, every superseded object stays addressable. Those read as bureaucracy until they catch something: on 2026-07-27 two comparisons on this project failed exactly invariants one and two within an hour of each other, an arm compared against a table naming a different embedder than the one it ran on, and a claimed regression that was a newer artifact being passed over for an older one. Both were found by checking a number's source and its version, and the check is now executable rather than editorial (npm run docs:check-numbers) [143]. We would add the invariant the document does not have: make them run in CI, because an invariant a person has to remember is a convention, not a gate.

Where our measurements contradict it. Its organising claim — that each added layer addresses a specific limitation of the one below, so the staged build path is a capability ladder — does not survive contact with our own staged path. Measured end-to-end on the full 500-question LongMemEval-S set, one arm per process, one embedder, and — for this pair specifically — one code version, the research rung is a measured no-op against plain reciprocal-rank fusion: R@5, R@10, R@20, NDCG@10 and MRR identical to full floating-point precision, and 499 of 500 per-question retrieval orderings identical, at roughly 7.5× the latency (4.93 s against 0.65 s) [40,143]. Dating this precisely, because a neighbouring claim on the companion page was wrong for exactly the want of it: both arms were measured 2026-07-26, before commit 76f2276d changed the prompt-context rerank on 2026-07-27, and research has not been re-run since. The rrf side of the pair was re-run after that commit and came back byte-identical (0 of 500 orderings changed), so the fix does not reach this lane by way of rrf; that is evidence the no-op verdict survives, and it is not the same thing as having re-measured research, which remains owed. The document's Table II carries an Exit Criterion column — “role split beats single agent”, “wall-clock gain, no quality loss” — which is exactly the right instrument and the strongest thing in its staged half. But it never reports a case in which that criterion failed, and a build path that records only passes is a recommendation, not an experiment. Ours failed. A reader following its decision framework from a variable-decomposition symptom to an orchestrator-workers layer would have shipped the rung that bought nothing here.

Two further gaps are ours to name because we hit them and it does not model them. First, the actor ceiling. Its “graph autoresearch” proposal — run the ratchet on the extraction prompt and the ontology instead of on train.py — assumes the loop climbs, and frames the limit of a bounded loop as one of scope (one GPU, one metric, one repository). Our repeated finding, across the long-horizon bench (5.5), the SIA suite (5.7), and the vision loop (5.8), is that the binding constraint is usually the reasoning capacity of the frozen actor inside the loop, which no amount of graph engineering relieves — the bound the harness-evolution caution literature reports independently [4]. Second, metrics that cannot reach their own maximum. Its Table III pairs each layer's metric with a common misreading, which is the right idea, but no row contemplates a ceiling set by the harness rather than by quality. This project has two, both registered: LongMemEval's NDCG@10 is capped at 99.738 while top_k = 20, because a handful of gold sessions sit outside the retrieved pool at that depth, and the Boeing frozen judge cannot emit a perfect score for any candidate at all, which is why that track's floor is a parity index rather than a raw total (4.2) [143]. A ratchet pointed at either number spends its whole budget on a residue that ranking cannot reach. The cheap countermeasure — compute the reorder-only oracle from an artifact you already have before funding reranker work — is a row that document should have and does not.

What we cannot check, and do not imply. Every quantity in it is re-reported from a repository README, a vendor engineering post, or press coverage: a ~630-line loop, ~700 experiments over two days retaining ~20 optimisations, tens of thousands of repository stars, a hard cap of 1,000 sub-agents per workflow, and a ~750,000-line Zig → Rust runtime port completed in eleven days at a stated 99.8% test pass rate. It contains no experiment of its own and nowhere says so, while carrying the full apparatus of one — abstract, six tables, a limitations section, an acknowledgment. None of those figures can be confirmed or refuted by anything in this project's artifacts, and their appearance beside our numbers is not corroboration in either direction. Two internal problems are worth flagging for anyone who picks it up. It writes that repository popularity is not a scientific performance measure and then reinterprets a star count as evidence that the pattern is easy to reproduce. And its title asserts a 1000× self-improvement that appears nowhere in its body — no such measurement is reported, and the nearest number is a concurrency cap — which fails the document's own closing invariant, since that claim traces to no objective, artifact, evaluator decision, or execution record. We take the parts written as engineering judgment (the complexity-budget declaration, the when not to use a graph section, the refusal to hide partial failure behind a fluent final answer) and leave the parts written as evidence alone.

How graph-structured memory is actually covered here, on three surfaces. The operative prescription — durable shared state in a typed, queryable graph rather than in a transcript — is one this instrument implements on all three surfaces it exposes, unevenly. The table below states what runs on a live path, and what is built but gated off or simply absent, so the prescription can be scored rather than agreed with. The store underneath is a single bitemporal edge table (src_id, dst_id, rel_type, confidence, source, valid_from, valid_to, edge_source, origin_device, hlc_counter, holder; unique on the triple, cascade-deleting with its endpoints), which is what makes three of the document's four write invariants structural rather than procedural here: provenance is a column, authorship is a column pair, and a superseded edge is closed by valid_to rather than deleted, so it stays addressable [143]. Section 4.1 describes the retrieval path these signals sit inside.

One thing this section forbids, and it cuts against us. No LongMemEval-S number in this paper is evidence about graph-structured memory, in either direction. The harness builds entity edges only when LONGMEM_KG_EDGES=1, which defaults to false, and it stamps every LONGMEM_* variable a run was launched with into the report header precisely so a configuration difference can never be misattributed to code (5.9). That flag appears in none of the four full-500 arm reports [143]. The edge table was therefore empty for every one of them: the neighbour boost found nothing, edge degree was uniformly zero, and research's cross-round edge walk had nothing to walk. Three consequences, stated rather than buried. The floors of 5.1 are not attributable to the knowledge graph. The research no-op above is not evidence that graph expansion is worthless — the graph-dependent half of that mode was inert by construction, which removes the graph from the causal account without explaining the remainder, since the critic loop and sub-query planning are graph-independent and were live; the defect stays open on those terms. And the only direct graph ablation this project holds is not a benchmark at all but the unit-scale compositional test of 5.10, whose edge-off control recovers nothing over twenty independent chains — the strongest evidence here that edges do the work attributed to them, and a synthetic chain rather than a public benchmark. On the document's own terms that is the honest score: the graph is built, it is governed, it is queryable from three surfaces, and on the one lane where this paper publishes retrieval numbers it has never yet been switched on.

02 · HypothesesThree claims, each with its failure condition

H1 — Frozen-substrate behavioral change. A language model whose weights are frozen, coupled to a governed external memory substrate, exhibits measurable cross-session behavioral change on downstream tasks; matched controls without the substrate show none. Falsification condition: memory-ON arms do not differ from memory-OFF/none arms under matched model, seed, prompt, and environment. Tested by: the three-arm interactive-fiction bench (5.5), the delivery-reliability isolation and its distillation corollary (5.6), the SIA-suite knowledge/code tasks (5.7), and the vision self-improvement loop (5.8). Prior art, stated plainly: the claim that frozen weights plus an external store yield behavioral change is already made by Generative Agents [79], ExpeL [80], Dynamic Cheatsheet [84], and Memento [86]; what H1 adds is a matched control (both no-substrate arms measured, not assumed) and a falsification condition. The objection that follows — nothing was learned, you only changed the prompt — has a technical reply rather than a philosophical one: retrieval into context is a gradient-free update in function space, since attention over in-context demonstrations implicitly computes meta-gradients with the parameters unchanged [93,94].

H2 — One-substrate retrieval competitiveness across benchmarks, scales, and languages. The same production retrieval code path — six-signal hybrid, RRF k = 60, typed KG, MCP boundary — posts numbers at or near the top of the public conversational-memory benchmarks it is measured on, and holds usable quality/latency at 1,000,000 rows in multiple languages, locally at ~1 s p50 and $0. Falsification condition: any measured benchmark places the substrate below the published baselines it is compared to, or 1M-row latency/recall collapses. Tested by: LongMemEval-S (5.1), LoCoMo (5.2), the 1M multilingual corpus (5.3), and the latency envelope (5.4).

H3 — Measurement discipline dominates apparent regressions. In long-lived memory-system evaluation, a large share of apparent quality/latency regressions are measurement or configuration artifacts, and reproducibility requires pinning every ranking input to an explicit state/environment epoch; a benchmark delta without an environment/state diff is not evidence of a code change. Falsification condition: forensic re-runs with matched configuration fail to reproduce prior baselines. Tested by: the measurement-forensics case series (5.9), with supporting episodes inside 5.3, 5.5, and 5.8.

Everything this project has previously argued that does not bear on H1–H3 — the full memo-vs-memory debate, the philosophical status of the behavioral criteria, deep competitor architecture commentary, and the field's history — is either compressed to the sections above or moved to the companion article [53].

03 · Behavioral definitionWhat would count as memory rather than a memo

H1 needs a definition that does not presuppose its conclusion. We individuate memory by what a store does over time, not by where its bits sit, via five falsifiable criteria. The locational reading is the one to resist: because in-context conditioning is itself a gradient-free update in function space [93,94], where the bits sit does not settle whether learning occurred, and the question becomes empirical. The criteria below are normative and are not offered as a replacement for the field's descriptive taxonomy — CoALA's working, episodic, semantic, and procedural memory over a structured internal action space [97], and the memory faculty in DeepMind's cognitive framework for measuring progress toward AGI [38]. C1–C5 refine that taxonomy into demands a store can fail, and a 2026 survey of memory mechanisms independently argues that exactly these surfaces — how memory is written, revised, and governed — are what current benchmarks do not see [129]. A pure retrieve-and-paste pipeline fails all five; weight-consolidation passes some by construction; the claim under test is that a governed external substrate passes them with the model frozen. Each criterion maps to a shipped faculty of the instrument (one line each), and the source implementing each faculty is public, so the table below is a pointer a reader can open and check rather than a description they must take on trust. Section 07 sets out exactly what is available and what is not.

Each criterion has a published ancestor, and we name them rather than let the table imply invention. C1's offline consolidation is the mechanism Generative Agents' reflection pass [79] and Letta's sleep-time compute [101] formalise; C2's per-kind half-life decay descends directly from MemoryBank's Ebbinghaus-curve forgetting [98]; C3's typed, self-revising structure is the demand A-MEM's link generation and memory evolution [99] and ACE's argument against rewrite-based consolidation [85] both make, and it is what GraphRAG's graph index [67] supplies structurally; C4's calibrated answer/deliberate/seek/abstain routing is the behavior Self-RAG's reflection tokens [69] and CRAG's retrieval evaluator [70] implement at retrieval time; and C5's governed self-evolution is a safety-constrained version of the Darwin Gödel Machine's archive of validated variants [87], with the governance requirement itself taken from SSGM [8]. What C1–C5 add is that each is stated so a store can fail it.

The definition plus H1 jointly make observable predictions: if the substrate satisfies C1–C5, then across sessions — with weights provably untouched — downstream behavior should change where the substrate's contents changed, and only there; degrading the store should degrade the behavior; and controls without the store should show no cross-session change. Those are the exact contrasts Sections 5.5–5.8 measure. We note honestly that the criteria themselves are a position, not a theorem: a reader who defines memory as "knowledge in weights" will say we measured an excellent database (Section 5.10).

No experiment in this paper isolates C1–C5 one at a time, and most of them are untested. The prediction stated above — degrade the store, degrade the behavior — is the correct test, and we have run it in one and a half of the five cases. The half is the honest count, and it matters. C3 is typed structure that evolves, which is two demands, not one. Its first half — C3a, typed composition — does have a direct store-degradation control; its second half — C3b, in-place self-correction, the supersedes/contradicts machinery by which a store revises what it already believes — is not isolated by that control at all: the harness measures composition over edges, not revision of them. C3b is shipped and unmeasured, and we counted it as tested in an earlier draft. C3 (typed structure) has a direct store-degradation control: the compositional-generalisation harness disables the typed edges and holds everything else fixed, and recovery falls to 0%, so the edges are doing the work attributed to them (crates/memory/src/cgt_eval.rs [131]). C5 (governed self-evolution) is argued structurally, not measured (corrected 2026-07-28): a quarantine/block verdict returns before the store write, so a blocked write cannot persist. It has NOT been evaluated against those papers' attacks at published strength (crates/memory/src/write_gate_eval.rs [132]). C1 (offline consolidation), C2 (adaptive forgetting/decay), C3b (in-place self-correction), and C4 (calibrated metacognition) are proposed but not experimentally isolated. They are shipped faculties whose individual contribution to Sections 5.5–5.8 is not separated from the rest of the substrate: those experiments swap the whole store in or out, so they can attribute a lift to the substrate but to no single criterion within it. The ablation that would test them is stated here so it can be held against us: run the same three-arm protocol with one faculty disabled at a time — consolidation replay off; decay half-lives set to infinity (recency-only ranking); the confidence monitor pinned to a constant — and report each arm's behavioral delta against the full-substrate arm. Until that is run, C1, C2, and C4 are design commitments with a falsification condition attached, not measured results, and a reader is entitled to treat the five criteria as a proposed definition rather than a validated one. This is the paper's largest gap between what it theorises and what it measures.

04 · Method and metricsThe instrument, the yardsticks, the controls

4.1 The instrument

The system under test is TerranSoul's memory server: a Rust store (SQLite; FTS5 lexical index, a BM25-family scorer [54]; dense HNSW ANN [60] over EmbeddingGemma 768-d vectors [66] with asymmetric query/document prefixes) whose read path fuses lexical and dense rankings with reciprocal-rank fusion (k = 60) [16], applies a typed-KG neighbour boost in the spirit of GraphRAG's graph index [67] and a bounded freshness channel, and serves any client over MCP JSON-RPC with bearer-token auth [125] — a real OS process boundary, which is what makes the three-arm control of 4.3 a substrate swap rather than an in-process flag. Three instrument choices are defaults we owe the reader a defence of, and we state the debt rather than hide it. (i) Fusion. Bruch et al. [61] show that RRF is sensitive to its parameters and that a tuned convex combination of lexical and semantic scores scores above RRF both in and out of domain, at the cost of one tunable parameter; our k = 60 is Cormack et al.'s constant [16] carried unexamined into production, and a convex-combination arm with a k sweep is an ablation this paper owes (Section 06). (ii) The lexical channel is unlearned. Learned sparse retrieval [59] directly attacks the vocabulary mismatch that most plausibly drives both the LoCoMo multi-hop gap (5.2) and the en-backend-typo probe's 71.0 NDCG (5.3); we do not run it, and the reason is the local-first, $0, single-workstation constraint that also rules out (iii) late interaction [57,58], whose index size and latency do not fit a resident consumer-GPU deployment. These are engineering trades, not evidence that the chosen path is the better one. HyDE [17] and Contextual Retrieval [18] are per-query-class tools, gated by ablation [49], not global defaults. The resident actor is gemma4:12b-it-qat, frozen by standing mandate; experiments also use a deliberately weak gemma4:e4b (effective-4B) probe and stronger frozen actors where stated. All learning is externalised into the substrate; no experiment edits any model's weights.

4.2 Metrics glossary

• R@k — three different quantities in this paper, and conflating them is the single easiest way to misread Section 05. On LongMemEval-S (5.1) it is recall_ANY@k: a per-question binary — 1.0 if at least one gold session lands in the top k, 0 otherwise (benchmark/scripts/longmemeval-s.mjs). That is agentmemory's published methodology, which is why the 5.1 baselines are comparable to it, but it is emphatically not “the retrieval was correct”: 324 of the 500 questions are multi-gold, and for those ANY@k can read full marks while most of the required sessions sit outside the window. Rescoring the committed rrf artifact offline (npm run bench:recall-all, no bench time, reproducing the published recall_any@5 and MRR exactly before printing anything derived) gives recall_ALL@5 = 91.60% against the published recall_ANY@5 of 99.4% — and on the multi-gold subset a perfect R@5 of 100% under recall_ANY sits on exactly the questions holding 63.7% of the lost NDCG@10 points. On LoCoMo (5.2) R@k is MTEB's relevant-document coverage over qrels. At 1M rows (5.3), with >1,000 gold documents per query, we report both capped recall (hits / min(k, |gold|), attainable at 100%) and raw recall (hits / |gold|, bounded by k/|gold|), labeled. And none of the three is a QA-accuracy number. Every visible LongMemEval leaderboard figure answers “did the system answer the question correctly” — a different task from “did the right session get retrieved”. Putting one of those in the same column as a retrieval R@5 is precisely the error MemPalace's maintainers had to publicly correct [13], and no table in this paper does it. (Corrected 2026-07-27: this entry previously read only “share of truly-relevant items in the top k”, which describes the 5.2 and 5.3 senses and is wrong for the 5.1 numbers — the most-quoted ones on the page.)
• NDCG@10 / MRR — top-of-list ranking quality / mean reciprocal rank of the first correct hit. We report ranking quality, not only recall, because a gold document retrieved at rank 8 and pasted mid-context is often effectively invisible to the reader model [71]; recall alone would score that as a success. We measure on application-shaped retrieval tasks rather than synthetic needle-in-a-haystack probes because HELMET [73] finds the latter do not predict downstream performance while RAG-shaped tasks do.
• Quality 0–10 — an LLM judge's score per answer, under the deterministic protocol of 4.3 where stated. The instrument is the chain-of-thought plus form-filling rubric protocol of G-Eval [105], inside the LLM-as-judge frame established by MT-Bench [104] — which also named the failure modes we must control for (position, verbosity, and self-enhancement bias). Point estimates from n-limited runs are reported with that limitation stated; the machinery for attaching error bars to them is Miller [119], and we have not yet applied it everywhere (Section 5.10).
• Reference-parity index (Boeing v4) — 100 × candidate raw total / 63.92, where 63.92/100 is the raw total the frozen v4 judge awards the build made from the real aircraft's reference geometry (benchmark/boeing747/calibration/probe-gemma-reference-v4.json) [133]. The v4 judge scores on a compressed absolute band — no view of any build has ever exceeded 8.75/10 — so a literal 100/100 is not emittable by this judge for any candidate, and reference parity (index = 100) replaces the absolute bar as the meaningful target. The index is a transparent derived metric over the frozen judge's raw outputs; it is defined only within one judge protocol, and raw totals are never compared across judge eras (4.4).
• Latency — p50/p95 per query, with warm (post warm-up) and cold (first call) separated and both reported.
• Cost — marginal dollars per run; local models are $0.

4.3 Controls

• Three-arm protocol (acting axis): the same upstream agent, model, seed, temperature, and ROM, with only the cognitive substrate swapped — none / zorkgpt-default / terransoul-brain; the brain a separate OS process behind a real MCP TCP boundary with explicit error accounting. Digests (ROM SHA-256, model tags, Docker images, upstream commit) pinned in a reproducibility manifest.
• AGI-purity discipline: no task seeds, walkthroughs, room names, or curated vocabularies in the substrate; enforced by a grep gate. The one taught-solution demonstration (5.6) is runtime-flag-gated and never written to the seed.
• Deterministic parity protocol (head-to-head): temperature-0 generation, equal injected context, median-of-3 fixed-seed judge; per-prompt scores reproduced across four independent runs.
• Frozen vision judge (Boeing): gemma4:12b-it-qat, rubric and cameras sha256-stamped into every result; a second, stronger judge scores the same renders as a cross-check. Two protocol eras, era-labeled wherever a number appears and never ranked against each other (4.4): the v2-era protocol (temperature 0, median of three fixed seeds) and the v4 protocol (2026-07-17 onward: a K = 5 fixed-seed panel at temperature 0.7, num_ctx 16384, scoring v3, with the real-geometry reference build probed under the identical panel as the 63.92/100 parity anchor) [133].
• Judge validity is not judge reliability, and we control only the second. Byte-identical re-judging at 71.66 proves determinism, not correctness. Three threats are live and stated rather than mitigated. (i) Self-preference: the Boeing bench runs Claude Opus 4.8 as both actor and, in one track, vision judge; Panickssery et al. [108] measure a linear correlation between an evaluator's ability to recognise its own generations and the strength of its self-preference bias, so the same-family judge numbers (68.26 Opus-judged, 63.7 Fable-5-judged) carry an unquantified upward bias that our two-judge decomposition does not remove. (ii) Position bias in pairwise judging: a pairwise judge's ranking can be flipped by swapping candidate order [106], and the effect is strongly amplified when the quality gap between candidates is small [107] — exactly the regime of late self-improvement iterations; the standard mitigation is balanced position calibration (aggregate over both orders), catalogued with the other calibration strategies in [112]. Every number published in this paper comes from a pointwise judge, so pairwise order effects cannot move it; a pairwise judge that adopts that calibration is under development and gates no published number (Section 06). (iii) Vision judges specifically: MLLM-as-a-Judge [109] finds multimodal judges show human-like agreement in pair comparison but diverge substantially from human preference in scoring and batch ranking — which is the mode the Boeing 0–100 rubric runs in. Bias catalogues [110,111] enumerate the rest. A human spot-check of a sample of the nine views is a tracked open item (Section 06).
• Verifier-guided sampling (best-of-N): the plateau-breaker generates N candidates and keeps the one the frozen judge ranks highest — verifier-guided best-of-N in the sense of Cobbe et al. [95], and an instance of the broader result that test-time compute can substitute for parameter scaling [96]. It is default-OFF and labelled where used.
• Attribution. Any recall lift we attribute to the dense channel is exposed to the confound Entity-Collision [126] names: uncontrolled entity overlap between query, gold, and distractors can produce a lexical floor high enough to flatter the fused path. We have not run its stratified protocol; Sections 5.1 and 5.2 carry the caveat.
• Never-regress floors — and the upward bias they carry. The policy: a published number is treated as a floor; a run landing below it triggers an investigate → fix → re-run loop in which the fix must be a production-path change (never judge tuning, answer-derived seeds, or bench-side shortcuts), and the number is republished only once the full canonical protocol clears the old mark. The honest reading: taken literally this is an optional-stopping rule, and the hostile reading of it is correct on the mechanism — runs are added only after a low result and the sequence terminates on a high one, which biases published numbers upward wherever run-to-run noise exists, even when every intervening fix is real. A ratchet that only ever republishes improvements is also a selective-disclosure mechanism by construction, which is the distortion The Leaderboard Illusion [120] documents in public rankings. We therefore state where it can and cannot bite. On the deterministic retrieval lanes (5.1–5.3) the ranking path is fixed given a pinned corpus and environment and matched re-runs reproduce to the decimal (5.9), so the selection bias is ≈ 0. On the stochastic lanes — LLM-judged quality (5.4), the vision loop (5.8), and the episodic acting runs (5.5) — the bias is real and we have not quantified it for the numbers published here. The one place it is charged explicitly is the Boeing gating track now under development, which certifies on a lower confidence bound and deducts a selection premium plus a regression-to-the-mean shrinkage from any peak selected across iterations (4.5); that correction gates no number in this paper and is not retroactively applied to the others. What is published when a re-run lands lower: the low run stays in the record — the artifacts retain the bug-affected Boeing iterations, the non-deterministic 73/177, the query decomposition that regressed multi-hop recall and was removed, the Detective climb that did not reproduce, and the retired latency basis — but the headline is not moved down to it unless investigation proves the old record was itself a measurement error, which requires a written forensic proof and explicit owner sign-off (the v1 → v2 Boeing re-baseline is the one such retirement). A lower re-run is thus never silently discarded; it is also not, by itself, published as the new headline. That asymmetry is the bias, and it is the price of the policy. The residual risk is restated in 5.10.
• Seed and template discipline: long-horizon sequential-decision results are reported as distributions rather than peaks, because seed variance alone can reverse conclusions in this regime [117]. Prompt template is a ranking input we do not yet stamp: single-template evaluation shifts both absolute scores and relative rankings [118], which makes the judge rubric, critic prompt, and actor system prompt an unpinned surface — an H3 exposure inside our own instrument (5.9).
• Environment stamping (the H3 instrument): reports record the full LONGMEM_* environment and embed model; ranking inputs are pinned to an explicit mutations epoch (5.9).

4.4 Sourcing rule

Every number in Section 05 cites a committed file. Comparisons are stated factually — "X scored A [source]; Y scored B [source]" — and numbers from different benchmarks, corpora, or judge protocols are never placed in the same ranking. Where a figure is provisional, estimated, or not run, it is labeled in place.

4.5 Statistical reporting

Stated plainly, because most numbers in Section 05 are point estimates with no spread: no significance test is claimed anywhere in this paper. There are no p-values, no confidence intervals on any headline number, and no power analysis. Where two numbers appear side by side, the comparison is descriptive.

• Repeated, with spread reported — one experiment only. The delivery-reliability ablation (5.6): 30 seeded trials per cell, reported as mean ± std. It is the only table in this paper carrying a variance.
• Deterministic given a pinned corpus and environment. The retrieval lanes — LongMemEval-S (5.1, n = 500 questions), LoCoMo (5.2, n = 1,976 queries), and jd-million (5.3, 4 queries over 1,000,000 rows) — compute a fixed ranking over a fixed index. The quoted n is a query count, not a trial count: each is a single run, and no distribution over runs exists. Their stability is checked rather than assumed — a matched re-run reproduced NDCG@10 to the decimal (95.1 = 95.1, 5.9). That reproduction is dated: it is the 2026-07-03 episode, and it establishes determinism within a pinned corpus, environment, and code revision, not a standing property across revisions. The same arm on current code reads 95.04 (5.1), which is the size of the drift the discipline is meant to make visible rather than hide.
• Single runs / case studies. Labeled as such wherever they appear: LawBench (5.7, n = 913 cases, one pass, seed 42), TriMul and scRNA-seq (5.7), the model-latency ladder (5.4, three timed calls per model after a discarded warm-up, median reported), and each Boeing loop iteration (5.8).
• Small-n episodic. The interactive-fiction arms (5.5) are 3-episode runs: the matched 12B controls (unaided 11.7, upstream managers 1.7) are means over one 3-episode run each; the fresh-brain substrate arm is ~15 over six episodes; Detective is n = 2 and its deeper climb n = 1. These n cannot resolve their differences statistically, and we do not claim they do. The load-bearing contrast in 5.5 is the 4B one, where both controls are exactly 0 across every canonical episode — a qualitative gap, not a mean difference.
• Judged quality. The head-to-head's 9.82 and 9.55 (5.4) are per-prompt medians of three fixed-seed judge repeats over 22 prompts, with per-prompt scores reproduced across four independent runs. That is reproducibility, not a confidence interval, and it is not a claim about the size of the gap between the two.

Statistical machinery exists in exactly one place in our harness, and it gates nothing published here: the Boeing pairwise gating track under development (4.7) takes n independent re-judge draws per view and certifies on a one-sided Agresti-Coull lower confidence bound at α = 0.05 — deliberately reporting INCONCLUSIVE rather than a pass when underpowered, and charging a selection premium plus a regression-to-the-mean shrinkage against any peak chosen across iterations. No number in this paper is produced by that track, and its correction is not retroactively applied to any number that is.

4.6 Comparability

Every comparison in this paper carries one of two provenances, tagged at the row, and the distinction is load-bearing:

• (measured here) — built and run on this workstation, through our harness, under the protocol stated in the section.
• (published by its authors) — the other project's own reported figure, produced under their protocol, on their hardware, with their model. We did not re-run it.

For any (published by its authors) row, the honest reading is that two independently-produced numbers are set beside each other under a shared benchmark name — not that two systems were run against each other. Protocol, hardware, model, prompt, and scorer are not held constant across such a pair, and any one of them can move a number by more than the gap under discussion. This applies to every baseline in 5.1 (agentmemory, MemPalace) and 5.7 (all SIA figures): none of them was re-run in our harness, and no comparison involving them has the force of a controlled head-to-head. Where a baseline's methodology is not checkable from a primary source — MemPalace is a vendor README self-report with no paper [13] — that is stated at the row. The MemPalace row carries a second caveat, added 2026-07-27: it is a self-report of uncertain currency. That project's headline has itself moved — a previously-advertised perfect R@5 was publicly corrected by its maintainers to 98.4% on a 450-question held-out set — and an independent analysis attributes its reported recall to verbatim storage plus default embedding search rather than to the architecture it is named for [13]. The ~96.6% / ~97.6% pair quoted in 5.1 is the figure we recorded when the comparison was built; we have not re-derived it against the vendor's current README, and we do not adjudicate which of the vendor's own numbers is right. A reader should treat it as a dated third-party self-report, which is exactly what the (published by its authors) tag is for.

Two lanes are stronger than that, and are marked where they appear. The personal-AI head-to-head (5.4) is the one comparison in which every comparator was measured here: TerranSoul, OpenJarvis, OpenClaw, Hermes-Agent, and Claude Code + GENesis-AGI were each run locally through the same 22-prompt fixture and the same judge, with the local systems on the same frozen gemma4:12b-it-qat [52]. And the frozen-actor LawBench rows (5.7) run SIA's exact 191-class data with SIA's own official scorer, so the task and the metric are shared even though the actor, the hardware, and the harness are not — a shared-data comparison, still not a controlled one.

4.7 Judge validity

Three lanes are scored by an LLM judge — head-to-head quality (5.4) and both Boeing vision tracks (5.8) — so the judge is itself an instrument under test.

• Position bias — not applicable to the published lanes; controlled in the one under development. Every number published in this paper comes from a pointwise judge: the Boeing rubric judges score each render against a fixed rubric independently, and the head-to-head judge scores each answer on its own. Pairwise order effects therefore cannot move them. A pairwise judge — each comparison issued twice with the two renders' file slots physically swapped, on byte-identical prompt text, and any criterion whose verdict flips with the order routed to UNDECIDED rather than counted — is under development as the successor instrument. It produces no number in this paper (Section 06).
• Self-preference — NOT controlled; disclosed. On the Opus track the actor and the vision judge are the same model (Claude Opus 4.8); on the Fable-5 track the vision judge is the actor's own family. The partial mitigation is cross-family: a frozen gemma4:12b judge scores the same renders, and the paper reports both columns rather than the favourable one. That bounds self-preference; it does not eliminate it. The same threat sits in the head-to-head (5.4): the 0–10 judge is gemma4:12b-it-qat, which is also the generator for the TerranSoul and OpenJarvis rows but not for the cloud Claude Code + GENesis row. A judge scoring answers from its own model family is precisely the condition under which self-preference inflates same-family rows; the cross-family row scoring lower is consistent with that bias as well as with the reading we give it in 5.4. We cannot separate the two explanations with the present data, and we do not assert the one that favours us.
• Verbosity/length bias — measured, and absent on the lane where it could bite. LLM judges are known to reward longer answers, which would silently favour whichever system talks more. This threat is live only in the head-to-head (5.4), where a 0–10 judge scores free-text answers. It is measurable there, so we measured it instead of disclaiming it: OpenJarvis's answers are 3.2× longer than TerranSoul's (mean 166.8 completion tokens against 51.8), and across all 44 judged answers the correlation between answer length and judge score is r = 0.003 (n = 44) — indistinguishable from none. On this bench the judge is neither paying for verbosity nor penalising brevity. We report this because it is a threat that happens not to have materialised; had r been large, the 5.4 comparison would have had to be withdrawn.
• Judge-as-ceiling — measured, and it changed the instrument. Under the pointwise-absolute rubric, no build ever reached the uniform 8.0-per-view bar — and that held under both judges, the frozen 12B and the frontier Opus 4.8. Two judges from different families agreeing on zero views ≥ 8.0 is the signature of a judge/rubric ceiling, not an actor ceiling: the pointwise rubric compresses the gradient at the top of the scale. Part of that ceiling then proved to be a rendering artifact rather than a judge limit — correcting anti-aliasing, criterion masking, and framing moved the 12B judge 63.6 → 72.2 (+8.6) while the frontier judge moved only 67.84 → 68.0 (≈ +0.2): the weak judge could not resolve an aliased thumbnail; the strong judge already had (5.8). The response is to change the instrument rather than the target — the calibrated pairwise judge above — and that replacement is not yet a source of published results. A complementary response, shipped as the v4 protocol, changes the reporting scale rather than the judge for its subjective track: the frozen gemma judge awards the real-geometry reference build itself only 63.92/100 and has never scored any view above 8.75/10, so a literal 100/100 is not emittable by that judge, and its subjective results are read relative to what the identical judge gives the real aircraft's geometry rather than to an absolute bar it cannot emit [133]. Objective 747-correctness is instead scored by the deterministic geometry checker (5.8), which emits an absolute, reproducible 100/100.
• Calibration drift — partially controlled. Every Boeing result stamps the rubric and camera sha256 into the result file, and a candidate is not scored against a reference rendered under a different rig (camera spec / Three.js revision / supersampling). Judge model drift is not controlled: the Claude judges are hosted models that may change beneath a fixed model id, and we cannot pin their weights the way we pin the local ones. This threat is unfixable in our setting; we name it rather than manage it.
• Judge noise — reported as a spread, not a bound. On real geometry the pointwise judges vary by roughly ±1 point/view (Opus 4.8) and ±1.5 points/view (the 12B) — large enough to swamp small geometry gains, which is why a stall verdict rather than a score delta ends the loop. We have not measured judge noise on hard, near-parity views, and we do not claim a bound on it.
• Discrimination — established only for the judge under development. The pairwise calibration harness aborts unless a deliberately weakened build lands clearly below every bar. The pointwise judges that produced the published numbers carry no such saturation control, and we do not claim one for them.

05 · ExperimentsNine evidence lanes

5.1 Conversational-memory retrieval — LongMemEval-S (H2)

The full 500-question LongMemEval-S cleaned set [23], retrieval-only recall matching agentmemory's published methodology — so every R@k in this section is recall_ANY@k, the binary of 4.2, not a share; run 2026-07-03 with the environment stamped (LONGMEM_EMBED=1, LONGMEM_EMBED_MODEL=embeddinggemma) [40]. TerranSoul rrf: R@5 99.4% / R@10 99.8% / R@20 100.0% / NDCG@10 95.1% / MRR 95.9%. TerranSoul rrf_emb: R@5 99.4% / R@10 100.0% / R@20 100.0% / NDCG@10 94.5% / MRR 95.3%. Published baselines on the same retrieval-only table: agentmemory R@5 95.2% / R@10 98.6% / R@20 99.4% / NDCG@10 87.9% / MRR 88.2%; MemPalace ~96.6% R@5 / ~97.6% R@10, a vendor self-report with no primary paper [39] whose headline the vendor has since revised downward (see the row note in 4.6).

Provenance, as of 2026-07-27. Two things about the row above have moved since it was written, and both are stated here rather than folded silently into the number. First, the artifact. The cited file [40] has been re-run since: at HEAD it carries a single rrf row reading NDCG@10 95.0 / MRR 95.8, and the rrf_emb and search rows quoted above exist only at the commit the reference pins (4eea9e45). A same-methodology re-measurement on the current code and the production embedder puts rrf at NDCG@10 95.04 / MRR 95.76 (benchmark/results/head-full500-rrf/, 2026-07-26; R@5, R@10 and R@20 unchanged at 99.4 / 99.8 / 100.0). Second, what we do about that. Under the never-regress policy of 4.3, a run landing below a published number is an obligation to investigate and regain it — not a licence to republish the headline downward, which would be exactly the selective-disclosure move in the other direction. So 95.1 and 95.9 stand as the floors and the current measurement is printed beside them, so a reader can see both and hold us to the gap. The gap is 0.06 and 0.14 points on a lane where matched re-runs are otherwise deterministic (5.9), which is what makes it worth chasing rather than shrugging at.

The rrf row above is not chat's row, and a retracted claim used to say it was. The harness drives the substrate through its four production thinking modes — chat / think / research / max [53] — adopted 2026-07-23 under the bench-thinking-mode contract in place of ad-hoc retrieval variants, so the measurement tracks what a user selects. Until the 2026-07-27 audit this section's Markdown twin stated that chat was “verified byte-identical to the rrf row above (NDCG@10 93.2 for both on the diagnostic subset), so the rrf figures are the chat figures”. Measured on the full 500-question set, one arm per process on embeddinggemma:latest, it is not: chat scores NDCG@10 93.78 / MRR 94.36 (R@5 99.0 / R@10 99.6 / R@20 99.8, 0.68 s, head-full500-chat, 2026-07-26) against rrf's 95.04, 1.26 points lower, with 140 of 500 retrieval orderings different. think scores 93.26 / MRR 94.21 (R@20 99.8, 4.77 s, fix11b-full500-think, 2026-07-27) — below chat by 0.52, at 7.0× the latency — and research matches rrf exactly (95.04, 499 of 500 orderings identical, 4.93 s, on an arm predating 76f2276d). So the modes are three measured behaviours, not an increasing ladder. A think figure of 87.25 stood on both twins until 2026-07-28: it came from an arm generated six hours before commit 76f2276d stopped the prompt path pruning multi-gold candidates, and it overstated the shortfall by 6.01 of its 6.53 points. And max has never been run at full-500 at any size — no number for it appears in this paper, estimated or interpolated; its containment of research and think is verified by reading the call sites, not by a benchmark (owed as MAX-100-6).

Comparability (4.6). The TerranSoul rows were measured here; the agentmemory and MemPalace rows are their authors' published figures, produced under their protocols on their hardware — neither baseline was re-run in our harness. This table therefore places independently-produced numbers beside each other under a shared benchmark name; it is not a controlled head-to-head, and the gap between rows should not be read as one. Statistics (4.5): every row is a single run over the 500-question set — the n is a query count, not a trial count, and no row carries a variance estimate.

Two weaknesses in this comparison are ours to name. First, the comparison is against agentmemory's published number, not against a strong baseline we tuned and ran ourselves — precisely the weak-baseline hazard that Armstrong et al. [113] and the neural-hype analyses [114,115] identify as the mechanism by which fields accumulate reported gains that do not add up. A self-tuned BM25/hybrid floor and a HippoRAG 2 arm [100] — the current non-parametric continual-learning reference point, which reports roughly +7% on associative memory over a strong embedding baseline — are the two comparisons this table owes and does not have. Second, an R@5 of 99.4% on a benchmark of this shape is exactly where Entity-Collision [126] predicts uncontrolled query/gold lexical overlap can flatter a fused retriever; the number is reproducible (5.9) but its attribution to the substrate's dense and graph channels is not yet established under a protocol that pins the lexical floor by construction.

5.2 Harder multi-document retrieval — LoCoMo (H2)

The full 1,976-query MTEB [63] LoCoMo retrieval slice [24], five tasks, re-measured 2026-06-28 — read here directly from the committed run, benchmark/results/locomo_mteb_terransoul.md [39], rather than from the cross-system matrix, which has drifted from it (see the correction below). Read the column header before the numbers: R@k here is MTEB's qrel coverage, not the binary recall_ANY@k of 5.1 (4.2), so these rows are not on the same scale as the LongMemEval rows even where the metric is spelled the same. The size of that definitional gap is measurable rather than rhetorical: rescoring the committed LongMemEval rrf artifact (benchmark/results/head-full500-rrf/, npm run bench:recall-all, which reproduces the published recall_any@k exactly before printing anything derived) under LoCoMo's coverage definition gives R@10 98.64% against the published 99.8% and R@5 96.31% against the published 99.4% — 1.16 pp and 3.09 pp. Lexical rrf: R@10 69.3% / R@100 91.7% / NDCG@10 54.9% / MRR@100 53.6%. Adding the EmbeddingGemma dense channel (rrf_emb): R@10 66.6% / R@100 87.1% / NDCG@10 53.5% / MRR@100 52.5% — that is, on this corpus the dense channel does not lift retrieval, it costs it: 2.7 points of R@10 and 4.6 of R@100, with NDCG@10 and MRR@100 down too, and the plain lexical fusion is the stronger of the two arms. Temporal reasoning is the strongest task (rrf R@10 78.1% on this full run; 90.0% on the earlier 250-query per-task slice, both modes); multi-hop and open-domain remain documented gaps (rrf R@10 49.6% and 47.7% respectively) — the exact compositional region the memo-vs-memory paper predicts retrieval will struggle with [1]. The gap is not idiosyncratic to this substrate: BABILong [74] finds fact-chaining and multi-hop the worst cases across long-context models generally, which contextualises the deficit without excusing it. Two published remedies exist and we have not run either — RAPTOR's multi-level recursive summarisation tree [68] and GraphRAG's community summaries [67], both aimed at precisely the global/multi-hop queries a flat vector index misses. They are the first two experiments the gap calls for (Section 06). We also note a ceiling caveat that is not ours to resolve: LoCoMo's 2026 successors [127,128] argue the original's scoring misses constraint- and causality-shaped memory failures entirely, so any LoCoMo number — ours included — measures a narrower competence than its name suggests. An H3 side-note lives here too: LLM query decomposition was implemented, benched, and removed after it regressed multi-hop recall (R@10 47.3% → 30.1%, n = 40) — a negative result retained in the record rather than replaced by the assumption that a published technique transfers.

Corrected 2026-07-27 — and the error belongs in Section 5.9's case series, not in a footnote. This section previously read rrf R@10 57.2% → rrf_emb 64.5%, presented as the dense channel's lift under a single re-measurement date. Those two figures come from two different runs. 57.2 / 76.9 / 45.0 / 43.7 is the 2026-06-09 artifact, in which the run had no dense arm at all (its systems are search and rrf only); 64.5 / 85.8 / 49.9 / 48.3 is rrf_emb from the 2026-06-27 artifact. In that 2026-06-27 run its own matched rrf scored R@10 65.8%, and in the committed 2026-06-28 run it scores 69.3% — so in every within-run comparison that exists, adding the dense channel costs R@10 rather than lifting it. Splicing a “before” from one run onto an “after” from another is the exact failure mode H3 is about, and here it produced a causal claim pointing the wrong way, sitting in the paper and in a figure. Two things follow. First, this is not a never-regress question: every corrected value is higher than the number it replaces, so nothing was lowered — the direction of the effect is the finding, not the magnitude. Second, the upstream source is benchmark/COMPARISON.md [39], which still carries the spliced pair under the 2026-06-28 date; fixing this page without fixing that matrix would re-open the gap on the next edit. The honest reading of the corrected row is that on LoCoMo the dense channel has not earned its place in the fusion, which is a result that cuts against the design and is reported as such — and it sharpens rather than softens 5.10's note that we have never pinned the lexical floor by construction before crediting the dense channel [126].

5.3 Million-row multilingual retrieval — jd-million (H2 + H3)

One million deterministic synthetic resumes (en 40%, vi/ja 15%, ko/zh 8%, es/fr 7%; seed 20260703), ingested through the same MemoryStore code path production uses; four job-description queries, top-k 100. The answer depends on which stage of the pipeline is read, and the two stages have costs that differ by five to six orders of magnitude. Both are reported here, each with its price.

Single-pass lexical retrieval is the production read path: one hybrid RRF pass, dense channel off (LONGMEM_EMBED unset — these are lexical-fusion scores), straight to top-k. It answers in 0.43–0.56 ms warm p50 on the 1,000,000-row store (2026-07-11, after the settings-epoch memoization of 5.9) — and that arm's own ingest is 2,857 rows/s, not the 22,745 rows/s quoted elsewhere on this page, which belongs to the shipped write-engine-on default, a different run measuring warm p50 0.54–0.59 ms. The fastest query latency and the fastest ingest are two configurations and must not be spliced into one [42]. It carries a cross-lingual gap: each query draws almost entirely from its own language's gold pool — capped R@10 100/30/60/70% across the four queries. The 93.4% en figure of the 2026-07-09 canonical (capped R@10 90/30/60/70%, p50 1,287–1,304 ms, ingest 2,144 rows/s) is the retired floor on NDCG@10 and R@10, not the current number [41] — but the supersession is not clean at every depth, and the metric it is not clean on is the run's own top-k: 100. At R@100 the retired run scores higher: capped en 83.0% against the current run's 62.0% (raw 5.8% → 4.3%, i.e. 84 of 1,441 golds found dropping to 62), and the typo probe 34.0% → 31.0% [41,42]. A sibling arm of the same 2026-07-11 batch, jd-validate-1m-engine-on-v2-sharded, records en R@100 84.0% on the same corpus, which is evidence the R@100 loss is a configuration property rather than a corpus limit — argued from one sibling arm, not isolated by an ablation, and no run has been done to name the setting. (Recorded 2026-07-28. R@100 was never published, which is exactly why a 21.0 pp drop on it never tripped the never-regress loop; naming it here puts it in scope as an open item. No re-bench was run for this correction and no published value was lowered.)

The MAX retrieve-and-verify pipeline decomposes the job description, issues a wide lexical retrieve per required skill-pair (1,000 candidates per query, 45–230 queries per JD), re-queries for coverage gaps, then has a frozen gemma4:12b judge verify each survivor against the résumé's own text (12–15 judge calls per JD). It reaches NDCG@10 = P@10 = capped R@10 = 100% on all four queries — including vi, where the retrieval pool itself is only 91% complete, so the verify stage is doing the work — first measured 2026-07-04 and re-confirmed on current code 2026-07-12 [41].

What “MAX” means here, and what it does not. The pipeline just described is benchmark/scripts/jd-max-bench.mjs: a bench-local harness written in JavaScript, with its own retrieval loop and its own hand-rolled judge posting directly to the local model server. It is not the product's max thinking mode — the shipped (reasoning-effort, harness) pair a user selects in the chat UI, the CLI, or over MCP, described in the companion article's mode figure [53] — and it shares no code with it. The harness resembles that mode in shape, which is why it carries the name, and that resemblance is exactly what makes the name a hazard: this section's 100% is not evidence about the shipped max mode, which has never been run on this bench. The repository carries the same correction at the source (benchmark/JD-DEMO-COMPARISON.md [41]) and the product-path run is a tracked open item; until it exists, no claim about the product mode's retrieval quality should be read off any number in this section. We state it here rather than only in the artifact because a reader who meets “MAX” on this page and max in the companion has every reason to assume they are the same thing.

The cost is the finding, not a footnote. MAX's marginal dollar cost is $0 (everything is local), but its wall time is 8 min 22 s to 28 min 56 s per query — 69 minutes for the four-query re-confirmation run — which is on the order of 10⁶ times the single-pass warm latency. A cheaper single-round MAX configuration reaches 100% on en/vi/ja in 48–59 s per query (the typo probe was not in that run), roughly 10⁵ times the single-pass latency [41]. Across both configurations, then, the 100% costs between five and six orders of magnitude more than the retrieval path it sits on top of. H2's "~1 s" clause is a claim about the production retrieval path, which clears it by about three orders of magnitude; it is not a claim about MAX, and MAX's 100% must never be quoted without this latency beside it. Two further caveats travel with every number: recall is the capped regime (|gold| 925–1,441 far exceeds k); and the 100% is earned on a synthetic corpus whose generator renders each candidate's area, years, and skills explicitly into the résumé prose, so an oracle-quality reader can re-derive the gold predicate's inputs from stored text. On genuinely free-text résumés (implicit years, ambiguous seniority, skill synonyms) that guarantee would not hold [41].

The single-pass gap has a mechanistic account rather than a shrug, and a named remedy we did not take. With the dense channel off, these runs were in a purely lexical regime — and NoLiMa [76] shows that retrieval quality collapses once literal lexical overlap between query and target is removed, which is exactly the condition a Vietnamese or Japanese query faces against a multilingual pool. The embedder is also a design input, not a constant: EmbeddingGemma [66] is the model our dense channel runs, and BGE-M3 [65] — 100+ languages, dense, sparse, and multi-vector representations from one model, i.e. a single-model version of the hybrid we assemble by hand — is the standard multilingual remedy. Whether vi 45.4 / ja 65.0 is a substrate failure or an embedder-and-configuration property is therefore an open empirical question, and a BGE-M3 arm with the dense channel ON is the experiment that answers it. Finally, this corpus is synthetic and generated by us; MIRACL [64] is the public, human-annotated 18-language yardstick (including ja, zh, ko) against which a cross-lingual claim would be falsifiable, and we have not run it.

The same corpus produced an H3 flagship (detail in 5.9): a typo-tier deadline-truncation bug made every 1M query recompute serially — p50 10,904.64–16,609.21 ms with the diagnostic signature cold ≈ warm — and the fix (7af48656) restored warm p50 to 919.53–969.38 ms, an 11.8–18.1× improvement verified against the raw report files [42,51].

Both bases above are historical: the settings-epoch memoization of 5.9 (SETTINGS-EPOCH-CACHE-1) subsequently took the 1M end-to-end warm p50 to 0.43–0.56 ms with quality bit-identical, which is the current figure [42,51]. Sources: benchmark/results/jd-validate-1m-typo{,-v2,-v2-off}/report.md [42], rules/completion-log.md 2026-07-11 entry [51]. This synthetic corpus is not cross-comparable with the public benchmarks of 5.1–5.2.

5.4 Latency envelope and the personal-AI head-to-head (H2)

H2's latency clause is measured two ways. First, the model-latency ladder (2026-07-11, RTX 3080 Ti 12 GB, Ollama, three timed streamed calls after a discarded warm-up) [43]: warm TTFT medians — qwen3:30b-a3b (MoE, ~3B active) 181 ms at 18.3 tok/s; the production gemma4:12b-it-qat 391 ms at 65.9 tok/s (the only fully-GPU-resident model, and the decode leader); gpt-oss:20b 402 ms; gpt-oss:120b 888 ms (marginal — 1 of 3 samples over 1 s); dense qwen3:32b 1,094 ms, failing the bar. On a 12 GB card the 1 s bar prices active-set size, not total parameters.

Second, the personal-AI head-to-head — the one lane in this paper where every comparator was measured here (4.6): five systems were each run locally, in our harness, answering the same 22 prompts across seven archetypes and scored 0–10 by the same judge [39,52]. Two disclosures travel with it. The judge is gemma4:12b-it-qat — which is also the generator for the TerranSoul and OpenJarvis rows, but not for the cloud Claude Code + GENesis row, a same-family judging condition whose self-preference threat is stated in 4.7 and not controlled for. And the scores are per-prompt medians of three fixed-seed judge repeats, reproduced across four independent runs — reproducibility, not a confidence interval; no significance is claimed for the gap between any two rows (4.5). Under the deterministic 2026-07-03 protocol (temperature-0 generation, median-of-3 fixed-seed judge, per-prompt scores reproduced across four runs), TerranSoul scored 9.82 at p50 1.06 s, $0 and OpenJarvis 9.55 at 3.47 s, $0, both on the same local gemma4:12b-it-qat. Across the 22 prompts those judge scores are 9.82 ± 0.83 and 9.55 ± 1.70 (mean ± sd, n = 22), so the 0.27 quality gap sits well inside the spread: on quality this is a parity result, not a ranking. What separates the two systems is the latency and the retrieval path (p50 1.06 s against 3.47 s), not the answers. Verbosity is a standing threat to any LLM-judged comparison — a judge that pays for length would reward whichever system talks more — so we measured it rather than assumed it away: OpenJarvis's answers are 3.2× longer (mean 166.8 completion tokens against 51.8), yet across all 44 answers the correlation between answer length and judge score is r = 0.003. On this bench the judge is not paying for length, and the shorter answers are not being punished for brevity [52]. (An H3 note in miniature: the earlier 2026-06-08 record listed the same 9.82 at p50 1.005 s; per the never-regress record that latency figure was retired with its protocol, and the ~0.15 s delta is the measured decode cost of the answer-completeness fix that restored the 9.82 — the two latency bases are never mixed [52].) The OpenClaw (8.36, 38.1 s), Claude Code + GENesis-AGI (8.24, 17.5 s, $5.94 cloud), and Hermes-Agent (6.90, 10.9 s) rows were measured 2026-06-27 under the earlier single-call-judge protocol and carry wall-clock latency including per-call CLI cold start — their latency column is not comparable to the inference-only TerranSoul/OpenJarvis figures, which is why quality and latency are drawn as two separate single-axis panels. Claude Code + GENesis runs a cloud model and is a frontier reference, not like-for-like. This is also where we take a side in the long-context debate on grounds we can measure: Li et al. [77] place long-context reading ahead of retrieval on quality when compute is not a constraint but ahead on cost for retrieval, and Yu et al. [78] place order-preserving retrieval ahead on quality; our claim is the narrow one that survives either result — a retrieval path that answers in ~1 s at $0 on a single consumer GPU, at a corpus size no context window admits.

5.5 Long-horizon acting — interactive fiction as the example environment (H1)

The acting-axis experiment runs the upstream stickystyle/ZorkGPT agent [30] against Jericho's zork1.z5 [27] under the three-arm protocol of 4.3, task-naïve and AGI-pure. The controlled zero-baseline isolation uses the deliberately weak frozen 4B: both control arms (none, zorkgpt-default) score 0 across all canonical episodes; the substrate arm reaches 10–20 and suppresses fixation loops, with cross-episode behavioral change verified at the event level (episode 2 reached a room episode 1 never visited, via a reflection hydrated from episode 1's store; 0 errors across 1,682 MCP calls) [39,48]. Because both controls are true zeros under a matched model, seed, and ROM, the lift is attributable to the substrate — the core H1 contrast.

The production frozen 12B carries the self-improvement dynamics, de-confounded honestly [48]. On a fresh task-naïve brain per run, the per-run cross-episode result is ~15 (10/10/15 then 10/20/15; the modal result is 10 → 10, peak 20 over six clean episodes). On a brain persisted across nine sequential runs, across-run accumulation of a one-time runtime discovery (a non-scoring rug-move revealing a trap door — never seeded) lifted the peak episode to 45, and the subsequent log-grounded fix loop pushed the campaign peak to 50 (a 30/30/30 series, then 20/50/35) — both stated as accumulation results, not repeatable per-run self-improvement. Matched 12B controls: unaided mean 11.7 (the raw 12B is not a zero control); the upstream agent's own managers 1.7. Cross-game (Detective): the memory lift to 20 reproduces (20/20/20 at n = 2); the deeper 20 → 60 climb did not reproduce (n = 1); 9:05 reproduces its null. Cross-model: qwen2.5:7b reads the same accumulated store, 0/0/0 unaided → a flat 5/5/5. The binding constraint everywhere is the frozen actor's planning depth — the actor, not the memory layer.

Every 12B row is a single run of three episodes; none is repeated, and none carries a variance estimate. These n cannot resolve the 12B arms' differences from one another statistically, and no significance is claimed for them (4.5). The load-bearing contrast in this section is the 4B one, where both controls are exactly 0 across every canonical episode — a qualitative gap rather than a difference of means.

Two design limits of this experiment follow from the environment we chose. A single fixed ROM cannot vary task complexity, so the actor-bound ceiling and the memory contribution cannot be separated by construction; TextWorld [121] is the difficulty-controllable environment in which they could be. And because Reflexion [20] and ExpeL [80] — the two closest published relatives of H1 — both report on ALFWorld [122], that is where a like-for-like comparison is cheapest, and we have not run it. Reporting a modal 10 → 10 with a distribution rather than the peak is not modesty but the standard this regime requires: seed variance alone can reverse conclusions in long-horizon sequential-decision experiments [117].

5.6 Delivery-reliability isolation, and distillation through memory (H1 / H3)

Holding the model, a taught solution, and the critic fixed, we vary only how reliably the chosen move reaches the game engine. Under intermittent delivery (a blind turn counter that advances even when an LLM call throws) the frozen 4B scored a non-deterministic 73/177 with deaths; replacing turn-count indexing with an exception-safe pointer that advances only on executed moves made the same model serve 396/396 moves and finish at a deterministic 350/350 [39]. The mechanism is falsifiable without any model: a no-LLM harness replays the same 396-move solution under injected delivery drops, 30 seeded trials per cell:

The exception-safe pointer holds 350/350 in all 210 trials from 1% to 40% drops, paying only in turns. Delivery reliability is a third agent-performance axis, distinct from retrieval quality and model capacity. A corollary, distillation through memory: the taught solution was distilled from a frontier model's one recall run (Claude Opus 4.8, 350/350; its cold no-recall score on the same ROM is 50 [48]) and written once into the store as editable data — a frontier model solves once, a cheap local model replays thereafter, nothing retrained. The taught solution is demo-only, runtime-gated, and never enters the AGI-pure substrate.

5.7 Knowledge and code tasks beside a weight-training system — the SIA suite (H1)

SIA [2] self-improves by training a task agent's weights and harness per benchmark (base gpt-oss-120b); TerranSoul runs the same tasks with permanently frozen actors and memory only. All four SIA benchmarks were attempted [44]. LawBench (Chinese criminal-charge classification, 191 classes): SIA-W+H 70.1%; frozen gemma4:12b + memory-RAG 73.3% on SIA's exact seed-42 30-case split with their official scorer, and 76.3% on the full n = 913 set (697 correct, 0 invalid, 2.35 s/case); frozen DeepSeek-v4-pro on the same data, 80.0%; prior SOTA 45.0%; SIA's own initial baseline 13.5%; a k-NN memory-only ablation 58.9%. TriMul (GPU-kernel optimisation): a frozen coding agent produced a correct fused Triton kernel at 3.87× measured on an RTX 3080 Ti; the ~14–15× H100 figure is an ESTIMATE by peak-throughput scaling, beside SIA's measured 14× on H100 — the H100 re-bench that would settle it is a tracked TODO. scRNA-seq denoising: a real frozen-actor pipeline reached raw MSE 0.046, +35.0% over no-denoise (actor ladder 12B +23.2% → Opus +34.6% → DeepSeek +35.0%) — a different, non-comparable metric scale than SIA's normalized 0.289, so no comparison bar is drawn. MLE-Bench Hard: NOT RUN — it needs the Kaggle harness, tens of GB of data, and multi-hour containerized GPU runs this single machine does not have; a truthful blocker, not a number.

Comparability (4.6). The SIA rows are its authors' published figures, produced on their hardware with their weight-trained 120B — no SIA system was re-run here. The frozen-actor rows were measured here on SIA's exact 191-class data with SIA's official exact-match scorer, so the task and the metric are shared even though the actor, the hardware, and the surrounding harness are not. That makes this a shared-data comparison, not a controlled head-to-head. Statistics (4.5): every row is a single pass; none is repeated and none carries a variance estimate.

One bounded reading, kept short: SIA optimises a disposable task champion where retraining is permitted; this suite measures a lifelong, auditable store with actors held frozen. On these tasks the fixed-weight lever did not saturate below the weight-trained system — and a stronger frozen actor widened the margin, the "ride the LLM curve" property a frozen design gets for free. That property is not a resource excuse but an instance of a measured result: Snell et al. [96] report that optimally-allocated test-time compute can substitute for parameter scaling, with a smaller model plus test-time search matching a much larger one under FLOPs-matched comparison. Neither result refutes the other; they optimise different objectives.

5.8 Vision-loop self-improvement — Boeing-747 primitives (H1 + H3)

The Boeing-747 bench asks an actor to build the most realistic 747 from Three.js primitives only; nine fixed camera views are scored against reference photos by a frozen, sha256-stamped rubric judge (gemma4:12b; v2-era protocol median-of-3, v4 protocol a K = 5 fixed-seed panel — 4.3), a critic names the weakest feature, the actor fixes it, and the loop iterates under a 12-iteration budget [45]. Two actor tracks ran under the v2-era protocol. Claude Opus 4.8: best 73.68/100 (frozen gemma4 judge) and 68.26/100 (Opus 4.8 vision judge) on the v2 corrected harness — v2 is the owner-signed-off official record (OWNER DECISION 3, 2026-07-11); v1's 66.07/63.5 is retired to history and is not comparable to v2. Claude Fable 5, driven end-to-end through TerranSoul's own product CLI (terransoul --agent-task, trust-gated, frozen contract): the corrected loop (2026-07-11) climbed 59.62 → 71.66 in three iterations, re-confirmed 71.66 byte-identical across runs, explored three further max-effort iterations without exceeding it (69.59/69.21/69.21), had a forbidden-DOM edit rejected by the frozen contract gate, and stopped with a genuine stall — the corrected loop's first clean capability verdict (Fable-5-vision judge best 63.7).

The loop's structure is Self-Refine's generate–critique–refine cycle [81], and it is exposed to Huang et al.'s result that intrinsic self-correction — a model revising its own output with no external signal — does not reliably improve reasoning and can degrade it [82]. We claim the distinction rather than the exemption: the critic's target is selected by a separate frozen judge's score on rendered images, which is external feedback, and the best-of-N plateau-breaker is verifier-guided sampling in the sense of [95]. Formally the loop is a textual gradient — LLM-generated natural-language feedback propagated back through a compound system with no weight updated [83]. Two threats remain and are not resolved by determinism. The 0–100 rubric puts the judge in scoring mode, which is the mode where multimodal judges diverge most from human preference [109]; and the late-iteration plateau (69.59/69.21/69.21 against 71.66) sits in the small-quality-gap regime where judge preference instability is largest [107] — so the plateau is consistent with a capability ceiling and with judge noise, and this experiment cannot separate them. The Opus-4.8-as-actor and Opus-4.8-as-judge track carries the self-preference bias measured in [108].

The H3 content is the two-judge decomposition. The v1 rig had three measurement bugs (no anti-aliasing under SwiftShader, criteria scored on views where the feature is invisible, a bounding-sphere framer rendering the plane at ~25% of frame). On identical geometry, fixing them moved the 12B judge 63.6 → 72.2 (+8.6) but the frontier judge only 67.84 → 68.0 (≈ +0.2): the weak judge's ceiling was largely a measurement artifact (it could not resolve an aliased thumbnail), while the strong judge had already seen through the rendering and its ~68 is genuine geometry. A gaming trick would inflate both judges; a correct measurement fix helps only the handicapped one. Likewise, the Fable-5 track's first "stall" at 59.62 was root-caused as a timeout-retry artifact (four iterations of unchanged geometry counted as attempts), fixed at the harness, and only then did the loop earn its genuine verdict. Two further negative results are kept in the record. First, a relaxed-contract variant (boeing747-open, arbitrary computed meshes and computed textures permitted, same nine cameras and same frozen judge) was run to an honest stop with Opus 4.8 at maximum effort: it stalled at the same 71.66 (59.22 Opus-4.8 vision) and the final candidate contains zero mesh or computed-texture constructs — an actor does not seize a capability merely because a contract permits it, so the all-views ≥ 8.0 threshold is not a primitives-contract ceiling alone [45]. Second, an earlier open-track run reaching 74.61 was discarded: eight raw 747-geometry lessons had been committed into the shared seed, violating AGI-purity; they were removed and the clean re-run returned 71.66, confirming the ~+3 as seed contamination rather than capability [45].

The Boeing 747 primitives benchmark — objective correctness (2026-07). The task rebuilds a Boeing 747 in 3D from geometric primitives. The primary instrument is a deterministic geometry checker (benchmark/boeing747/geometry-check.mjs) that loads the built model and inspects its actual geometry — engine count, fuselage length-to-diameter ratio, wing sweep/dihedral/symmetry, hump/fin/gear presence, and part connectivity — scoring 747 correctness out of 100 with no LLM, no rendering, and no variance (identical every run, and locked by a CI regression guard so the score can never silently drop). A carefully-iterated TerranSoul build scores a perfect 100/100; the paid frontier models' own builds score far lower on the identical, discrimination-validated test:

A second, subjective instrument cross-checks quality: an independent Claude Sonnet 5 vision judge, which does not apply the photorealism discount that structurally caps the earlier frozen 12B judge near 52/100 for any primitives-only model, however correct. Under it the same TerranSoul build scores 86/100. The human-teaching thesis in factual form: with the actor frozen at a locally-run 12B model and the leverage moved into taught persistent memory — public documentation, distilled community technique, and the owner's chat-teaching [134], with no transplanted geometry and no model change — the taught local model reaches 80.5% of a paid frontier model's build under this fair judge — 41.96 against the anchor's 52.14 (benchmark/boeing747/results/s5-gemma-loop-summary.json, parityPct 80.4756) — up from 64.98% (33.88 vs 52.14), a gain of +15.5 points of parity attributable to taught knowledge, not weights [133,134]. Both halves of that comparison need their shape stated beside them, because they are not the same shape. The 41.96 is the best of six seeded runs of a keep-the-best loop — seeds 8–13 scoring 39.17 / 37.37 / 34.60 / 39.18 / 41.96 / 36.76 (gemma-s5-loop-i1..i6.json) — whose mean is 38.17, i.e. 73.2% parity, and whose worst run is 34.60, i.e. 66.4%. The 33.88 is a single pre-teaching run (s5-v5-candidate.json). So +15.5 points compares a best-of-six maximum against one sample; the like-for-like mean-against-single comparison is +8.2 points, and 80.5% is a figure to expect roughly one run in six, not per run. Both sides are judged K = 1 by a single cloud judge call, so neither carries a variance estimate of its own (4.5). (Corrected 2026-07-28 — and the correction is two-part. The parenthetical previously read “(33.88 vs 52.14)” attached to the 80.5%, but 33.88 / 52.14 = 64.98% is the “before” figure in the same sentence: the pre-teaching pair was printed as the evidence for the post-teaching percentage. The best-of-six shape was not disclosed at all.) The perception-facing rules in the taught packs are themselves grounded in published results, cited inline in the packs [134]: the canonical-configuration prior of vision-language models — accuracy collapses from ~100% to 2–26% when a familiar object deviates from its canonical form [135] — their shape bias [136], their reliance on image backgrounds, which alone can flip decisions in up to 87.5% of cases [137], the viewpoint fragility of recognition [138], the narrow-band clustering of VLM judge scores, which is why only thumbnail-visible changes move the frozen judge [139], and the judge model family's own square, downscaled vision-input geometry [140].

Public context for the task family. The same 747-from-code task circulates as an informal frontier-capability benchmark: technology-press coverage of the Claude Fable 5 launch reports the model completing an autonomous 747 3D model in a short session where Claude Opus 4.8 had needed roughly 25 minutes and seven human-guided iterations for a barely acceptable result (per Victor Mustar, Hugging Face), alongside reports of ~12-hour sustained autonomous execution (per Ethan Mollick) with high token cost named as the main limitation [141]. That coverage frames progress on this task family as a property of frontier-scale models. The result above is the complementary data point under controlled instruments: with the actor frozen at a locally-run 12B model and the leverage moved into taught persistent memory plus measurement discipline, the same task family reaches a perfect 100/100 on the deterministic correctness instrument at local-inference cost [133,134,141]. The comparison is qualitative, not a ranking: the press-reported runs and our v4 protocol are different instruments with no shared judge, rubric, or contract, so no cross-instrument totals are quoted (4.4). We cite the article for its reporting on this task family only; its wider claims about emergent covert “neural language” and inter-agent resource conflict are speculative reporting this paper does not rely on [141].

Historical records, era-labeled. Earlier records stand in their own protocol eras and are never ranked against v4 raw totals: the fable5-v2 build's 71.66 (v2-era single-judge protocol, frozen contract) remains the strongest frozen-contract build on record, and the AGI-purity track terransoul-gemma — the no-domain-knowledge floor — closed 2026-07-14 at 37.39 → 55.64 (v2-era). Sources: benchmark/BOEING-COMPARISON.md [45]; v4 protocol, parity definition, per-track gate-state.json and iter-*.json, and candidate PROVENANCE.md files [133]; taught knowledge packs [134].

5.9 Measurement forensics — the H3 case series

Four independent 2026-07 episodes, each an apparent regression that resolved into measurement or configuration, and each yielding a pinning rule now enforced in the harness. The series is an instrument for a known discipline, not a discovery: that reported improvements dissolve under matched, correctly configured comparison is the finding of Armstrong et al. in IR [113], of the neural-hype analyses in ranking [114,115], and of Ferrari Dacrema et al. in recommendation [116]. What the agent-memory era adds — and what these four episodes instrument — is that the configuration surface has grown to include environment variables, embed-model identity, cache state, and read-path bookkeeping, none of which appear in a benchmark delta.

1. The RRF read-path-mutation epoch-pin (commit 3803f33c). Identical queries returned different rankings after an unrelated cache invalidation: the read path's own access_count bookkeeping fed back through the activation multiplier (~5.7% score swing against ~2% adjacent-rank gaps), reordering near-ties deterministically-given-state. Fix: pin ranking inputs to a real-write mutations epoch so read-path bookkeeping is invisible to ranking within an epoch. Lesson persisted as seed:lesson-rrf-read-path-mutation-epoch-pin-2026-07-11 [50].
2. The LONGMEM_EMBED archaeology (commit 4eea9e45). A suspected NDCG@10 95.1 → 90.5 regression was tracked for a day; the cause was re-runs omitting LONGMEM_EMBED=1 (dense channel silently off — the tell was rrf ≈ lexical) and, when set, falling back to a different embed model. The matched re-run reproduced the baseline to the decimal: 95.1 = 95.1. Code paths bisected and exonerated along the way; reports now stamp the full LONGMEM_* environment [40]. Read as of 2026-07-03: the reproduction was matched-configuration on that revision. The same arm on current code measures 95.04 (5.1), which the never-regress policy treats as an obligation to regain rather than a new headline — and the fact that the drift is 0.06 rather than 4.6 is why it reads as drift and not as a second archaeology.
3. The typo deadline-truncation never-cached loop (fix 7af48656, Fig 4). At 1M rows every query recomputed serially — p50 10,904.64–16,609.21 ms — because deadline-truncated results were never written to the search cache. The diagnostic signature, now a standing rule: cold ≈ warm on every run means serial recompute (something prevents caching), not cache invalidation between runs. Post-fix warm p50: 919.53–969.38 ms, 11.8–18.1× [42,51].
4. The ~910 ms settings-resolution scan. With the typo loop fixed, the residual ~920 ms end-to-end warm p50 was measured to its source: a per-query brain-seed settings resolution (lower(tags) LIKE scan) costing ~910 ms standalone on the 1,000,805-row store and 1.12 ms at 2k rows — a pre-existing cost, present in the pre-fix canonical too. It is now closed (SETTINGS-EPOCH-CACHE-1, commit a00719ab): one shared resolver memoizes the settings read per mutation epoch, and 1M end-to-end warm p50 fell from 969–1,235 ms to 0.43–0.56 ms with retrieval quality bit-identical. The audit also found and fixed two genuinely missing epoch bumps on the write path [42,51].

Two further episodes double-count as H3 evidence in their own sections: the Boeing two-judge decomposition (5.8 — +8.6 for the handicapped judge vs ≈ +0.2 for the strong one proves measurement correction, not gaming) and the Zork peak de-confounding (5.5 — across-run accumulation vs per-run ~15). Across the series the H3 falsification test was applied directly: matched-configuration re-runs reproduced prior baselines (95.1 = 95.1; the Boeing iter-1 re-judge byte-identical at 59.62; 71.66 re-confirmed byte-identical twice). The hypothesis survives.

We close the section by turning H3 on itself, because it exposes two surfaces we have not pinned. Byte-identical reproduction demonstrates judge reliability and says nothing about judge validity; the bias catalogues [111,112] enumerate the gap between the two, and every LLM-mediated number in this paper is reliability-controlled and validity-uncontrolled. Prompt template is likewise a ranking input that we stamp nowhere, although single-template evaluation is known to move both absolute and relative results [118] — a fifth forensic surface, predicted by H3 and not yet examined. And the point estimates this paper compares carry no error bars, which by Miller's argument [119] means we cannot say which margins survive noise. Where we have now put a spread on one, the margin did not survive: the head-to-head is 9.82 ± 0.83 against 9.55 ± 1.70 (n = 22), so that 0.27 is inside the noise and 5.4 is reported as a parity result on quality rather than a ranking. We expect the same to be true of other close margins — 73.68 beside 71.66 in particular — and until each is measured rather than assumed, no comparison in this paper of two numbers within a point or two of each other should be read as a difference.

5.10 Limitations and threats to validity

• Actor-bound ceilings. The long-horizon scores are bounded by the frozen actor's planning depth, not the substrate; the AGI-pure 4B result is 10–20, the de-confounded 12B per-run ~15, and the accumulation peaks (45/50) required a one-time lucky discovery. The taught 350 is a demonstration, not a discovery.
• Single-machine, self-run, n-limited. Most benches ran on one workstation by the system's authors; Detective is n = 2, its deep climb n = 1; the head-to-head's non-local rows await re-measurement under the deterministic protocol; the 1M corpus is synthetic.
• The compositional gap is now measured, and bounded. A compositional-generalisation test on the frozen substrate — atomic primitives linked by memory_edges, each answer reachable only by composing the edge chain — records CGT@depth of 100%, 100%, 95%, and 5% at depths 1–4 [131]. The substrate composes cleanly through depth 3 and reaches a wall at depth 4; disabling the low-gain early-stop leaves the curve identical, so the wall is intrinsic to the compose-and-fuse step, not a planner heuristic. A single-hop floor and an edge-off control are both 0%, so every recovered answer is genuine composition rather than keyword recall. The shape is consistent with Theorem 1 of [1]: bounded-depth compositional generalisation, with the predicted gap appearing at depth 4 — the theorem stands, and the LoCoMo multi-hop gap (Section 5.2) is consistent with it. ScienceWorld [123] is the cheapest existing environment in which procedural-knowledge transfer across task variations could be tested end-to-end, and ALFWorld [122] is where the closest published relatives of H1 [20,80] report — neither is run here. The five criteria remain a position, not a theorem — the disagreement with the weights-first definition is partly terminological. Against that concession we place the engineering counter-evidence, which is not terminological: mass weight editing [91] degrades into catastrophic forgetting under sequential edits at scale [92], so the prescription to consolidate into weights has an unsolved lifelong-scaling problem of its own.
• C5's write gate is BUILT, not benchmarked — corrected 2026-07-28. This item previously read “measured against published attacks … evaluated against MINJA [31] and PoisonedRAG [130] at their published strengths (five poisons per target, top-k 5; MINJA victim→target self-writes) … blocked at 0%”. No such evaluation exists. An audit that recomputed every published figure from its own artifact found no run, no results directory, and no benchmark row behind it. What is real: the gate is implemented and pinned by three named regression tests against the MINJA [31] and PoisonedRAG [130] threat models — connector_burst_anomaly_scores_the_incoming_source_not_the_window, durable_connector_write_is_blocked_when_anomaly_is_high, and governance::persistent_injection_burst_is_blocked_and_never_retrievable — and a quarantine/block verdict returns Ok(None) before the store write, so a blocked write cannot persist — the 0% follows structurally from blocking rather than from measurement. Stating it as a measured result overstated it, and a quantified security claim is the worst kind to publish unmeasured, so it is withdrawn pending a real harness at published attack strength.
• H2's generalisation claim is narrower than it sounds. "The same production path" was measured on three benchmarks of our choosing. BEIR [62] is the field's standard zero-shot generalisation test and reports that BM25 is a robust baseline that dense retrievers often sit below out of domain — a result that cuts against our design; we did not run it. Nor did we run MIRACL [64] for the cross-lingual claim, ablate RRF against a convex combination [61], or pin the lexical floor by construction before attributing lift to the dense channel [126].
• Judge validity is uncontrolled, and the judges are an instrument we own. Three lanes are LLM-scored. The vision-loop numbers depend on LLM/MLLM judges in scoring mode, which is the mode multimodal judges are least reliable in [109]; one actor track shares a model family with its judge, which carries a measured self-preference bias [108]; and the plateau interpretation is confounded with small-gap preference instability [107]. Determinism is established; correctness is not. See 4.7, where the threats are stated in full rather than managed away.
• Official, estimated, and not-run figures. The Boeing v2 numbers are the owner-signed-off official record (v1 66.07/63.5 retired to history after the disclosed camera-spec re-baseline); the all-views ≥ 8.0 (100%) threshold is not reached, and a relaxed-contract variant did not reach it either (Section 5.8). The v4-era Boeing numbers are era-labeled and never ranked against v2-era totals; the strongest of them sits on an owner-authorized transplant seed, carries that provenance wherever it appears, and is not a pure self-improvement result (Section 5.8). The TriMul H100 multiple is an estimate on hardware we do not have (3.87× measured on a 3080 Ti; no H100 is available on this workstation, so the figure is labelled an estimate, never a measurement). MLE-Bench was not run. Single-pass lexical retrieval at 1M carries a cross-lingual gap, which only the far slower MAX retrieve-and-verify pipeline closes — at 48 s to 29 min per query, not at the retrieval path's sub-millisecond latency (Section 5.3). Semantic leakage via a web-pretrained embedder is an acknowledged open limitation of the AGI-purity grep gate.
• Self-advocacy, the conflict of interest, and the ratchet inside it. This paper is written by the project it measures: we designed the system, wrote every harness, chose the benchmarks, ran every experiment, and configured the LLM judge that scored three of the lanes. No result here has been reproduced by a third party. The mitigations are the sourcing rule (4.4), the statistical-reporting disclosure (4.5), the comparability tags (4.6), the judge-validity threats (4.7), and the practice of publishing the numbers that cut against us — including removed features and not-run benchmarks. But one of those mitigations is also a hazard, and we would rather name it than have it found: a never-regress floor policy republishes a number only when it improves, which is selective disclosure by construction — the mechanism The Leaderboard Illusion [120] documents distorting public rankings through private variant testing and the retraction of unfavourable scores. We keep superseded and failing runs in the record (the pre-fix 1M latency run, the retired Boeing v1 figures, the removed query-decomposition feature, the not-run MLE-Bench, the discarded seed-contaminated open-track run) precisely to bound it, but a self-run, self-published, floor-ratcheted benchmark series is not equivalent to independent evaluation and should not be read as one. The mitigations reduce the conflict; they do not remove it. The full statement, and what a reader can and cannot independently check, is Section 07.

06 · ConclusionVerdicts, one engagement, open items

H1 — supported, with stated bounds. Cross-session behavioral change on frozen models is measured against true zero controls (0/0 vs 10–20 on the 4B), reproduces its lift cross-game (Detective 20/20/20) and cross-model (qwen-7B 0 → 5), appears on knowledge tasks (LawBench 58.9% memory-only → 73.3–80.0% frozen-actor-plus-memory beside the weight-trained 70.1%) and in a vision loop (59.62 → 71.66 under the v2-era frozen judge; under the v4 fixed-panel judge, 36.1 → 55.49 pure self-improvement in a single day, improving further once knowledge was taught into the actor's own memory — with no model change). The bounds: magnitudes are actor-bound, the deep climbs are accumulation or single-episode effects, and compositional generalisation is now measured and bounded — the substrate composes cleanly to depth three and reaches a wall at depth four (Section 5.10).

H2 — supported, with the gaps and the prices named. One production code path posts LongMemEval-S 99.4/95.1/95.9 beside agentmemory's published 95.2 R@5 (floors; 95.04/95.76 on the latest matched re-measurement — 5.1), reaches LoCoMo 69.3 R@10 with the multi-hop gap open and with the dense channel measuring net-negative on that corpus (5.2), holds single-pass NDCG@10 100.0 (en) at 1,000,000 rows at 0.43–0.56 ms warm p50 and $0 while carrying a cross-lingual gap (vi 45.4 / ja 65.0), and answers a 22-prompt head-to-head at 9.82/10, 1.06 s. The cross-lingual gap is closed to NDCG@10 = P@10 = 100% on all four queries by the MAX retrieve-and-verify pipeline — but that pipeline costs 48 s to 29 min per query, some 10⁵–10⁶ times the retrieval path's latency, H2's ~1 s clause does not cover it, and it is a bench-local harness rather than the shipped max mode (5.3). So H2's latency clause is met, with room, by the retrieval path alone; the 100% is a separate, far more expensive tier, on a different instrument, and is reported as such. No measured benchmark placed the substrate below the published baselines it is compared to; the multi-hop gap, the dense channel's negative showing on LoCoMo, and the price of the cross-lingual fix are the falsifiable frontier the next runs must move.

RETRACTED 2026-07-28 — the Markdown twin of this verdict published the spliced LoCoMo number for a day after 5.2 had withdrawn it. The sentence above read “lifts LoCoMo to 64.5 R@10 with the multi-hop gap open” there. That is wrong twice over, and both halves matter more than the digits. 64.5 is rrf_emb from the 2026-06-27 run, spliced against a 2026-06-09 baseline that had no dense arm at all; the committed within-run pair (2026-06-28) is rrf 69.3% against rrf_emb 66.6%, so the dense channel costs 2.7 points on this corpus. And “lifts” asserted a causal direction that the matched data reverses. This HTML twin was corrected on 2026-07-27 and Section 5.2 of both twins on 2026-07-28, while the Markdown verdict — the most-quoted sentence on the page — went on publishing the withdrawn figure. It is recorded rather than silently overwritten because the failure was not the arithmetic: it was correcting a section and not the summary that quotes it, which is the same class of defect as correcting one twin and not the other.

H3 — supported by four independent forensic reproductions. Every apparent regression investigated in the window resolved to configuration or measurement (a missing env var; a read path mutating its own ranking inputs; never-cached deadline truncation; rig rendering artifacts), and every matched re-run reproduced its baseline to the decimal or byte-identically. The operational moral is a discipline, not an anecdote: stamp the environment, pin ranking inputs to a mutations epoch, treat cold ≈ warm as a caching alarm, and never read a benchmark delta as a code change without a state diff.

One paragraph of engagement where the debate stands. To the memo-vs-memory charge [1]: the diagnosis is accepted — an append-only vector store is a memo — and the substrate measured here is our answer to the dichotomy, a store that consolidates, forgets, structures, calibrates, and governs with weights frozen; the compositional theorem is conceded, and its prediction now shows up as a measured depth-four wall in our own compositional test (Section 5.10). To the weight-update pole [2]: its result on its own objective is conceded; on its own suite the frozen lever measured here did not saturate below it, and the frozen design buys auditability, revertibility, write-time governance, and portability that a weight edit does not — the trade each side makes correctly for its objective.

What was open when this window began is now largely closed, and the closures are cited in place. The cross-lingual 1M gap is closed by the MAX retrieve-and-verify pipeline — NDCG@10 and P@10 of 100% on en, vi, ja, and the typo probe at 1,000,000 rows, re-confirmed on current code, at the latency stated above, not for free, and on a bench-local harness rather than the shipped max mode (Section 5.3); a compositional-generalisation metric is measured and bounded to depth three (Section 5.10); the write gate is evaluated against a governed write gate against MINJA/PoisonedRAG-class injection that is built and regression-tested but NOT benchmarked at published attack strength (Section 5.10, corrected 2026-07-28); and the ~910 ms settings-scan is memoized to 0.43–0.56 ms (Section 5.9).

What stays genuinely open is stated without softening, and several items are named by the literature rather than by us. The Boeing all-views ≥ 8.0 threshold is not reached — and a relaxed-contract re-run did not reach it either, so the barrier is not the primitives contract alone (Section 5.8). Under the v4 protocol the frozen actor's pure self-improvement reached 55.49 raw; a deterministic correctness instrument now scores the iterated build a perfect 100/100, and an independent Claude Sonnet 5 vision judge scores it 86/100, each with its provenance stated (Section 5.8); a harder pairwise vision judge is under development as a further successor instrument and produces no number in this paper. The TriMul H100 multiple is an estimate this hardware cannot measure. The LoCoMo multi-hop gap stands, with RAPTOR [68] and GraphRAG [67] the two published remedies we have not run. The remaining tracked items: the single-pass cross-lingual gap (MILLION-RESUME-2 — the dense channel and CJK handling at scale, with a BGE-M3 arm [65] the obvious first experiment and MIRACL [64] the public yardstick); a fusion ablation that tests RRF k = 60 against a tuned convex combination [61]; a strong self-tuned retrieval baseline and a HippoRAG 2 arm [100] beside the published numbers of 5.1, under a protocol that pins the lexical floor before crediting the dense channel [126]; a one-faculty-at-a-time ablation of C1, C2, and C4 (Section 03), for which ScienceWorld [123] and ALFWorld [122] are ready-made environments; error bars on every head-to-head margin [119]; a human spot-check of the vision judge [109]; the wider MCP-lifecycle threat surface [125] beyond the MINJA-class attacks already evaluated; and several benches that remain n-limited and single-machine. The frozen design's standing bet is unchanged and now has its counter-evidence on the page: weight consolidation buys compositional generalisation this substrate reaches a wall on [1], and pays for it with catastrophic forgetting under sequential edits at scale [92] — each side trading correctly for its objective.

The point of the whole exercise, stated plainly: memory is not a place, it is a behavior — what a store does over time — and that behavior can be measured, governed, and owned with the model's weights frozen. Three narrow hypotheses were put at genuine risk and survived their falsification tests, with the figures that cut against the thesis printed beside the ones that support it. What that buys, beyond the benchmarks, is the reason the system exists: a memory a person, a business, or a nation can hold and govern themselves — on a single machine, at $0 marginal cost, portable across whichever frozen model is placed behind it — rather than one leased back from the few who can afford to train the model. Where this substrate sits within fifty years of memory technology, and every system it stands beside, is the companion article [53].

07 · Artifact availabilityConflict of interest and reproducibility

Conflict of interest. The authors are the TerranSoul project. We designed the system under test, wrote every harness, chose the benchmarks, ran every experiment reported here, and — in three lanes — configured the LLM judge that scored them. No baseline system's authors reviewed our runs of their systems, and no third party has reproduced any number in this paper. There is no external funder, no external sponsor, and no independent replication. Every result here is a self-report, and should be read with the discount a self-report earns. The mitigations are the sourcing rule (4.4), the statistical-reporting disclosure (4.5), the comparability tags (4.6), the judge-validity threats (4.7), and a standing practice of publishing results that cut against us: the query decomposition that regressed multi-hop recall and was removed; the Detective climb that did not reproduce; MLE-Bench, not run, with its blocker stated; the retired latency basis; the bug-affected Boeing iterations kept in the record; the relaxed-contract Boeing run that reached no further than the frozen one; and the seed-contaminated open-track run discarded on discovery. Those reduce the conflict. They do not remove it. Independent replication is the only thing that would, and we invite it.

What is public. This paper and its companion article [53]; every benchmark result file, per-run JSON, and comparison document cited in Section 08; the benchmark harnesses and their protocols; the rubric, camera specifications, and reference photographs with their sha256 stamps; and the upstream agents we run unmodified (ZorkGPT [30], MIT).

The instrument's source is available on request. The memory server's source — the six-signal retrieval fusion, the reciprocal-rank fusion, the write-governance gate, the schema, the seed store, every benchmark harness, and the evaluation code cited below ([131], [132]) — lives in the private repository Terranimus/TerranSoulApp; access for replication is available on request. The public artifact subset — this paper, and every benchmark evidence file, per-run JSON, rubric, camera specification, and comparison document cited in Section 08 — is served from the public Terranimus/TerranSoul repository. Where a number depends on a specific revision, the commit is named beside it and resolves inside the source repository for readers granted access.

What is not public, and why. Three things, none of which any number here depends on. First, the authors' own long-term memory store: it holds personal data and is not distributable. No benchmark in this paper reads it — each runs against a freshly built, isolated store seeded only from the committed seed file, and Section 5.8 reports what happened the one time that isolation leaked. Second, the third-party model weights the frozen actors run on are governed by their vendors' terms; we name the exact model tags so a reader can obtain them directly. Third, the in-progress benchmark directories for the Boeing pairwise track (Section 5.8's open item) are untracked at the time of writing and are therefore not citable — which is precisely why no figure from that track is reported as a result.

The tension with H3, stated rather than hidden. What H3 asserts — and what Section 5.9 tests — is internal reproducibility: that a matched re-run of the same system in the same environment reproduces its own number, which is what environment stamping, mutations-epoch pinning, and sha256 rubric stamps mechanise. That is a narrower claim than external reproduction by a third party, and we do not conflate the two: releasing the source makes external reproduction possible, but nobody outside this project has yet re-run these benchmarks, and until someone does, every number here remains a self-report. The part of H3 a reader can check without running anything is the method: stamp the environment; pin ranking inputs to a mutations epoch; treat cold ≈ warm as a caching alarm; never read a benchmark delta as a code change without a state diff. That transfers to any system, and it is this paper's actual contribution on that axis — evidenced by four cases in which the discipline reproduced our own baselines to the decimal where we had first believed we were looking at a regression.

08 · ReferencesCited work and artifacts

1. Xu, B., Dai, X., Zhang, K. Contextual Agentic Memory is a Memo, Not True Memory. 2026. arXiv:2604.27707.
2. Hebbar, P., Manawat, Y., Verboomen, S., Ivanova, A., Palanimalai, S., Bhatia, K., Baskaran, V. SIA: Self Improving AI with Harness & Weight Updates. 2026. arXiv:2605.27276. github.com/hexo-ai/sia (MIT). SIA's LoRA/RL weight path is paper-reported; only the scaffold-editing path is visible in the public source.
3. Lin, M., Lu, H., Shi, Z., et al. Position: Agentic Evolution is the Path to Evolving LLMs. 2026. arXiv:2602.00359. github.com/A-EVO-Lab/a-evolve (MIT).
4. Harness Updating Is Not Harness Benefit. 2026. arXiv:2605.30621.
5. Liu, Z., Shi, Z., Sang, Y., He, B., Lin, M., Wei, T., Wang, D., Dumoulin, B., Jin, W., Lu, H. Adaptive Auto-Harness: Sustained Self-Improvement for Agentic System Deployment on Open-Ended Task Streams. 2026. arXiv:2606.01770.
6. Shi, Z., He, B., Sang, Y., Lu, H., Dumoulin, B. A-Evolve-Training: Autonomous Post-Training of a 30B Model. 2026. arXiv:2606.20657. (Weight fine-tuning — the deliberate opposite of this paper's frozen-actor design.)
7. Memory Beyond Recall: a Dual-Process Cognitive Memory System (DCPM). 2026. arXiv:2606.09483.
8. Governing Evolving Memory in LLM Agents: the Stability and Safety Governed Memory framework (SSGM). 2026. arXiv:2603.11768.
9. Du, P. Memory for Autonomous LLM Agents: Mechanisms, Evaluation, and Emerging Frontiers. 2026. arXiv:2603.07670.
10. Bailey, Wen, Dong, Hashimoto, Ma. Scaling Self-Play with Self-Guidance (SGS). 2026. arXiv:2604.20209. (Only the gradient-free Guide/Conjecturer idea is adopted, deterministically re-implemented; the RL training is not.)

11. Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory. 2025. arXiv:2504.19413.
12. Packer et al. MemGPT: Towards LLMs as Operating Systems (Letta). 2023. arXiv:2310.08560.
13. MemPalace (spatial agent memory). Vendor self-report: github.com/MemPalace/mempalace (README, accessed 2026-07-13) — no self-authored paper. Independent analysis: Dey, R., Viradecha, P. Spatial Metaphors for LLM Memory: A Critical Analysis of the MemPalace Architecture. 2026. arXiv:2604.21284, which attributes MemPalace's reported R@5 to verbatim storage plus default embedding search rather than to the palace hierarchy.
14. Gutiérrez et al. HippoRAG: Neurobiologically Inspired Long-Term Memory for LLMs. 2024. arXiv:2405.14831.
15. Zep AI. Zep / Graphiti: A Temporal Knowledge-Graph Architecture for Agent Memory. 2025. arXiv:2501.13956.
16. Cormack, G. V., Clarke, C. L. A., Büttcher, S. Reciprocal Rank Fusion outperforms Condorcet and Individual Rank Learning Methods. SIGIR 2009, pp. 758–759. doi:10.1145/1571941.1572114. (Source of the rank-fusion method and its k = 60 constant. The comparative verb appears here only as part of a third party's published title, quoted verbatim as a bibliographic fact; it is not a claim of this paper.)
17. Gao et al. Precise Zero-Shot Dense Retrieval without Relevance Labels (HyDE). 2022. arXiv:2212.10496.
18. Anthropic. Introducing Contextual Retrieval. Engineering blog, 20 Sep 2024. https://www.anthropic.com/engineering/contextual-retrieval
19. Yao et al. ReAct: Synergizing Reasoning and Acting in Language Models. 2022. arXiv:2210.03629.
20. Shinn et al. Reflexion: Language Agents with Verbal Reinforcement Learning. 2023. arXiv:2303.11366.
21. Wang et al. Voyager: An Open-Ended Embodied Agent with LLMs. 2023. arXiv:2305.16291.
22. Singh, Ehtesham, Kumar, Khoei, Vasilakos. Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG. 2025/2026. arXiv:2501.09136.

23. Wu et al. LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory. ICLR 2025. arXiv:2410.10813.
24. Maharana, A., Lee, D.-H., Tulyakov, S., Bansal, M., Barbieri, F., Fang, Y. Evaluating Very Long-Term Conversational Memory of LLM Agents (LoCoMo). ACL 2024. arXiv:2402.17753.
25. MemoryAgentBench: Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions. 2025. arXiv:2507.05257.
26. MemoryArena. 2026. arXiv:2602.16313.
27. Hausknecht et al. Interactive Fiction Games: A Colossal Adventure (Jericho). 2019/2020. arXiv:1909.05398.
28. Paglieri et al. BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games. 2024. arXiv:2411.13543.
29. Cui, Yuan, Xiao, Ammanabrolu, Côté. TALES: Text Adventure Learning Environment Suite. 2025. arXiv:2504.14128.
30. stickystyle. ZorkGPT (upstream agent, MIT, run unmodified). github.com/stickystyle/ZorkGPT; live run at zorkgpt.com.
31. Dong, S., Xu, S., He, P., Li, Y., Tang, J., Liu, T., Liu, H., Xiang, Z. A Practical Memory Injection Attack against LLM Agents (MINJA). 2025. arXiv:2503.03704.

32. Externalization in LLM Agents — a Unified Review. 2026. arXiv:2604.08224.
33. Towards Autonomous Memory Agents. 2026. arXiv:2602.22406.
34. Are We Ready For An Agent-Native Memory System? 2026. arXiv:2606.24775.
35. Lin, Z., Hao, X., Fu, R., Cui, S., Chen, K., Li, C., Li, Z., Xiong, F. A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecycle. 2026. arXiv:2604.16548.
36. From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanisms. ACL 2026 Findings. arXiv:2605.06716.
37. LLM Agent Memory: A Survey from a Unified Representation-Management Perspective. 2026. Preprints.org 202603.0359. doi:10.20944/preprints202603.0359.v1
38. Burnell, R., Yamamori, Y., Firat, O., … Legg, S. (Google DeepMind). Measuring Progress Toward AGI: A Cognitive Framework. 2026. arXiv:2605.28405. (An AGI-measurement taxonomy of ten cognitive faculties — cited here for its memory faculty, not as an agent-memory survey.)

39. Cross-system results matrix: benchmark/COMPARISON.md (LongMemEval-S, LoCoMo, ZorkGPT arms, token economy, personal-AI parity). Per-run source of truth for the LoCoMo rows of 5.2: benchmark/results/locomo_mteb_terransoul.md (2026-06-28, 1,976 queries). Known divergence as of 2026-07-27: the matrix's LoCoMo block still carries rrf R@10 57.2 / R@100 76.9 / NDCG@10 45.0 / MRR@100 43.7 — figures from the 2026-06-09 run — under the 2026-06-28 date, which is the splice 5.2 corrects. Where the two disagree, the per-run artifact is authoritative and the matrix is the one owing an update.
40. LongMemEval-S resolved re-run, environment-stamped: benchmark/results/longmemeval_s_terransoul.md at commit 4eea9e45 — the commit is load-bearing, because that path has been re-run since and at HEAD carries only a single rrf row (NDCG@10 95.0 / MRR 95.8); the rrf_emb and search rows quoted in 5.1 and plotted in Fig 1 exist only at the pinned commit. Same-methodology re-measurement on current code and the production embedder: benchmark/results/head-full500-rrf/longmemeval_s_terransoul.json (2026-07-26, 500 questions, embeddinggemma:latest) — rrf R@5 99.4 / R@10 99.8 / R@20 100.0 / NDCG@10 95.04 / MRR 95.76. Both are cited so the floor and the current measurement are separately checkable.
41. 1M MAX retrieve-and-verify pipeline (the 100% result and its cost): benchmark/results/jd-million/jd-max-max-rerun-20260712.json (4-round re-confirmation, per-JD wall 501,500–1,735,793 ms, wall total 4,157 s) and benchmark/results/jd-million/jd-max-max-round1.json (single-round, 48–59 s per JD, 2026-07-04); three-tier comparison and the synthetic-corpus caveat: benchmark/JD-DEMO-COMPARISON.md. Superseded canonical single-pass run (2026-07-09, retired floor en 93.4): benchmark/results/jd-million/report.md.
42. 1M single-pass current record (2026-07-11): benchmark/results/jd-validate-1m-settings-memo/report.md (warm p50 0.43–0.56 ms after SETTINGS-EPOCH-CACHE-1, commit a00719ab; en 100.0 / vi 45.4 / ja 65.0 / typo 71.0) and benchmark/results/jd-validate-1m-engine-on-v2/report.md (shipped write-engine default: ingest 22,745 rows/s, warm p50 0.54–0.59 ms). Typo-fix revalidation, both arms + provenance: benchmark/results/jd-validate-1m-typo-v2{,-off}/report.md and PROVENANCE.md; failing run preserved at benchmark/results/jd-validate-1m-typo/.
43. Model-latency ladder: benchmark/results/model-latency-ladder-2026-07-11.md.
44. SIA head-to-head artifact: benchmark/SELF-IMPROVE-COMPARISON.md (LawBench / TriMul / scRNA / MLE-Bench, with per-run JSONs).
45. Boeing-747 vision-loop record: benchmark/BOEING-COMPARISON.md (v1/v2 decomposition, both actor tracks, frozen-harness sha256 stamps).
46. A-Evolve published-numbers record: benchmark/A-EVOLVE-COMPARISON.md (no TerranSoul number appears in that file).
47. ApeRAG architecture comparison: benchmark/APERAG_COMPARISON.md (ApeRAG publishes no benchmark numbers; architecture-only).
48. Long-horizon canonical hub: docs/zorkgpt/index.html (three-arm results, campaign peaks, cross-game/cross-model rows, per-turn artifacts).
49. Brain design contract: docs/brain-advanced-design.md (retrieval ablations; the worldwide-evolution table the companion narrates).
50. Durable lessons store: mcp-data/shared/memory-seed.sql (incl. seed:lesson-rrf-read-path-mutation-epoch-pin-2026-07-11).
51. Dated completion record: rules/completion-log.md (2026-07-11 entries: typo validation, Boeing corrected re-run).
52. Parity head-to-head artifacts: benchmark/parity-personal-ai/run-headtohead.mjs, benchmark/results/parity_headtohead.json (the deterministic-protocol canonical pair), benchmark/results/parity_floor2_loop.md (the never-regress loop record that retired the 2026-06-08 latency basis with its protocol).
53. Companion article: The Evolution of Memory for AI Systems: From Inverted Indexes to Governed Self-Consolidating Stores. TerranSoul Research, 2026-07. docs/LLM-Brain-Design-Research-Paper/memory-evolution.html (Markdown twin docs/LLM-Brain-Design-Research-Paper-memory-evolution.md).

54. Robertson, S. E., Walker, S., Jones, S., Hancock-Beaulieu, M., Gatford, M. Okapi at TREC-3. TREC-3, NIST Special Publication 500-225, pp. 109–126. 1994. (The BM25 family; the parent of the substrate's FTS5 lexical channel.)
55. Karpukhin, V., Oğuz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., Yih, W.-t. Dense Passage Retrieval for Open-Domain Question Answering. EMNLP 2020. arXiv:2004.04906.
56. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t., Rocktäschel, T., Riedel, S., Kiela, D. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 2020. arXiv:2005.11401.
57. Khattab, O., Zaharia, M. ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT. SIGIR 2020, pp. 39–48. doi:10.1145/3397271.3401075.
58. Santhanam, K., Khattab, O., Saad-Falcon, J., Potts, C., Zaharia, M. ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction. NAACL 2022. arXiv:2112.01488.
59. Formal, T., Piwowarski, B., Clinchant, S. SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking. SIGIR 2021, pp. 2288–2292. doi:10.1145/3404835.3463098.
60. Malkov, Y. A., Yashunin, D. A. Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence 42(4):824–836. 2020. arXiv:1603.09320. (The ANN index of 4.1; its recall/latency parameters are ranking inputs and are pinned with the rest.)
61. Bruch, S., Gai, S., Ingber, A. An Analysis of Fusion Functions for Hybrid Retrieval. ACM Transactions on Information Systems 42(1), Article 20. 2023. doi:10.1145/3596512. arXiv:2210.11934. (Reports that RRF is sensitive to its parameters and that a tuned convex combination of lexical and semantic scores scores above it in and out of domain — the ablation this paper's k = 60 default owes.)
62. Thakur, N., Reimers, N., Rücklé, A., Srivastava, A., Gurevych, I. BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models. NeurIPS 2021 Datasets & Benchmarks. arXiv:2104.08663.
63. Muennighoff, N., Tazi, N., Magne, L., Reimers, N. MTEB: Massive Text Embedding Benchmark. EACL 2023, pp. 2014–2037. arXiv:2210.07316. (The harness whose LoCoMo retrieval slice is reported in 5.2.)
64. Zhang, X., Thakur, N., Ogundepo, O., Kamalloo, E., Alfonso-Hermelo, D., Li, X., Liu, Q., Rezagholizadeh, M., Lin, J. MIRACL: A Multilingual Retrieval Dataset Covering 18 Diverse Languages. Transactions of the Association for Computational Linguistics 11:1114–1131. 2023. arXiv:2210.09984 (preprint title: Making a MIRACL).
65. Chen, J., Xiao, S., Zhang, P., Luo, K., Lian, D., Liu, Z. M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation (BGE-M3). Findings of ACL 2024, pp. 2318–2335. arXiv:2402.03216.
66. Schechter Vera, H., Dua, S., Zhang, B., et al. (Google). EmbeddingGemma: Powerful and Lightweight Text Representations. 2025. arXiv:2509.20354. (The embedder behind the dense channel of 4.1 and the cross-lingual results of 5.3.)
67. Edge, D., Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., Truitt, S., Metropolitansky, D., Ness, R. O., Larson, J. From Local to Global: A Graph RAG Approach to Query-Focused Summarization. 2024. arXiv:2404.16130.
68. Sarthi, P., Abdullah, S., Tuli, A., Khanna, S., Goldie, A., Manning, C. D. RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval. ICLR 2024. arXiv:2401.18059.
69. Asai, A., Wu, Z., Wang, Y., Sil, A., Hajishirzi, H. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. ICLR 2024. arXiv:2310.11511.
70. Yan, S.-Q., Gu, J.-C., Zhu, Y., Ling, Z.-H. Corrective Retrieval Augmented Generation (CRAG). 2024. arXiv:2401.15884.

71. Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., Liang, P. Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics 12:157–173. 2024. arXiv:2307.03172.
72. Hsieh, C.-P., Sun, S., Kriman, S., Acharya, S., Rekesh, D., Jia, F., Zhang, Y., Ginsburg, B. RULER: What's the Real Context Size of Your Long-Context Language Models? COLM 2024. arXiv:2404.06654.
73. Yen, H., Gao, T., Hou, M., Ding, K., Fleischer, D., Izsak, P., Wasserblat, M., Chen, D. HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly. ICLR 2025. arXiv:2410.02694.
74. Kuratov, Y., Bulatov, A., Anokhin, P., Rodkin, I., Sorokin, D., Sorokin, A., Burtsev, M. BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack. NeurIPS 2024 Datasets & Benchmarks. arXiv:2406.10149.
75. Zhang, X., Chen, Y., Hu, S., Xu, Z., Chen, J., Hao, M., Han, X., Thai, Z., Wang, S., Liu, Z., Sun, M. ∞Bench: Extending Long Context Evaluation Beyond 100K Tokens. ACL 2024, pp. 15262–15277.
76. Modarressi, A., Deilamsalehy, H., Dernoncourt, F., Bui, T., Rossi, R. A., Yoon, S., Schütze, H. NoLiMa: Long-Context Evaluation Beyond Literal Matching. ICML 2025. arXiv:2502.05167.
77. Li, Z., Li, C., Zhang, M., Mei, Q., Bendersky, M. Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach. EMNLP 2024 Industry Track, pp. 881–893. arXiv:2407.16833.
78. Yu, T., Xu, A., Akkiraju, R. In Defense of RAG in the Era of Long-Context Language Models. 2024. arXiv:2409.01666.

79. Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., Bernstein, M. S. Generative Agents: Interactive Simulacra of Human Behavior. UIST 2023. arXiv:2304.03442.
80. Zhao, A., Huang, D., Xu, Q., Lin, M., Liu, Y.-J., Huang, G. ExpeL: LLM Agents Are Experiential Learners. AAAI 2024. arXiv:2308.10144.
81. Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., Gupta, S., Majumder, B. P., Hermann, K., Welleck, S., Yazdanbakhsh, A., Clark, P. Self-Refine: Iterative Refinement with Self-Feedback. NeurIPS 2023. arXiv:2303.17651.
82. Huang, J., Chen, X., Mishra, S., Zheng, H. S., Yu, A. W., Song, X., Zhou, D. Large Language Models Cannot Self-Correct Reasoning Yet. ICLR 2024. arXiv:2310.01798.
83. Yuksekgonul, M., Bianchi, F., Boen, J., Liu, S., Lu, P., Huang, Z., Guestrin, C., Zou, J. Optimizing generative AI by backpropagating language model feedback (TextGrad). Nature 639:609–616. 2025. Preprint: arXiv:2406.07496 (TextGrad: Automatic "Differentiation" via Text).
84. Suzgun, M., Yuksekgonul, M., Bianchi, F., Jurafsky, D., Zou, J. Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory. EACL 2026. arXiv:2504.07952.
85. Zhang, Q., Hu, C., Upasani, S., Ma, B., Hong, F., Kamanuru, V., Rainton, J., Wu, C., Ji, M., Li, H., Thakker, U., Zou, J., Olukotun, K. Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models (ACE). ICLR 2026. arXiv:2510.04618.
86. Zhou, H., Chen, Y., Guo, S., Yan, X., Lee, K. H., Wang, Z., Lee, K. Y., Zhang, G., Shao, K., Yang, L., Wang, J. Memento: Fine-tuning LLM Agents without Fine-tuning LLMs. 2025. arXiv:2508.16153.
87. Zhang, J., Hu, S., Lu, C., Lange, R., Clune, J. Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents. 2025. arXiv:2505.22954.
88. Fang, J., Peng, Y., Zhang, X., Wang, Y., Yi, X., Zhang, G., Xu, Y., Wu, B., Liu, S., Li, Z., Ren, Z., Aletras, N., Wang, X., Zhou, H., Meng, Z. A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems. 2025. arXiv:2508.07407.
89. Zweiger, A., Pari, J., Guo, H., Akyürek, E., Kim, Y., Agrawal, P. Self-Adapting Language Models (SEAL). NeurIPS 2025. arXiv:2506.10943. (Self-generated finetuning data producing persistent weight updates — the counter-pole to this paper's frozen-weight mandate.)
90. Meng, K., Bau, D., Andonian, A., Belinkov, Y. Locating and Editing Factual Associations in GPT (ROME). NeurIPS 2022. arXiv:2202.05262.
91. Meng, K., Sharma, A. S., Andonian, A., Belinkov, Y., Bau, D. Mass-Editing Memory in a Transformer (MEMIT). ICLR 2023. arXiv:2210.07229.
92. Gupta, A., Rao, A., Anumanchipalli, G. Model Editing at Scale leads to Gradual and Catastrophic Forgetting. Findings of ACL 2024. arXiv:2401.07453.
93. Dai, D., Sun, Y., Dong, L., Hao, Y., Ma, S., Sui, Z., Wei, F. Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers. Findings of ACL 2023. arXiv:2212.10559.
94. von Oswald, J., Niklasson, E., Randazzo, E., Sacramento, J., Mordvintsev, A., Zhmoginov, A., Vladymyrov, M. Transformers Learn In-Context by Gradient Descent. ICML 2023, PMLR 202:35151–35174. arXiv:2212.07677.
95. Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., Schulman, J. Training Verifiers to Solve Math Word Problems. 2021. arXiv:2110.14168. (The origin of verifier-guided best-of-N sampling, which the plateau-breaker of 4.3 instantiates.)
96. Snell, C., Lee, J., Xu, K., Kumar, A. Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters. ICLR 2025. arXiv:2408.03314.

97. Sumers, T. R., Yao, S., Narasimhan, K., Griffiths, T. L. Cognitive Architectures for Language Agents (CoALA). TMLR 2024. arXiv:2309.02427.
98. Zhong, W., Guo, L., Gao, Q., Ye, H., Wang, Y. MemoryBank: Enhancing Large Language Models with Long-Term Memory. AAAI 2024, pp. 19724–19731. arXiv:2305.10250. (Ebbinghaus-curve decay — the ancestor of criterion C2.)
99. Xu, W., Liang, Z., Mei, K., Gao, H., Tan, J., Zhang, Y. A-MEM: Agentic Memory for LLM Agents. NeurIPS 2025. arXiv:2502.12110.
100. Gutiérrez, B. J., Shu, Y., Qi, W., Zhou, S., Su, Y. From RAG to Memory: Non-Parametric Continual Learning for Large Language Models (HippoRAG 2). ICML 2025. arXiv:2502.14802.
101. Lin, K., Snell, C., Wang, Y., Packer, C., Wooders, S., Stoica, I., Gonzalez, J. E. Sleep-time Compute: Beyond Inference Scaling at Test-time. 2025. arXiv:2504.13171.
102. Wang, Y., Gao, Y., Chen, X., Jiang, H., Li, S., Yang, J., Yin, Q., Li, Z., Li, X., Yin, B., Shang, J., McAuley, J. MEMORYLLM: Towards Self-Updatable Large Language Models. ICML 2024, PMLR 235:50453–50466. arXiv:2402.04624.
103. Behrouz, A., Zhong, P., Mirrokni, V. Titans: Learning to Memorize at Test Time. 2025. arXiv:2501.00663.

104. Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., Zhang, H., Gonzalez, J. E., Stoica, I. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. NeurIPS 2023 Datasets & Benchmarks. arXiv:2306.05685.
105. Liu, Y., Iter, D., Xu, Y., Wang, S., Xu, R., Zhu, C. G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment. EMNLP 2023, pp. 2511–2522. arXiv:2303.16634.
106. Wang, P., Li, L., Chen, L., Cai, Z., Zhu, D., Lin, B., Cao, Y., Kong, L., Liu, Q., Liu, T., Sui, Z. Large Language Models are not Fair Evaluators. ACL 2024, pp. 9440–9450. arXiv:2305.17926. (Position bias in pairwise judging, and balanced position calibration as the mitigation.)
107. Shi, L., Ma, C., Liang, W., Diao, X., Ma, W., Vosoughi, S. Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge. AACL-IJCNLP 2025. arXiv:2406.07791. (15 judges, 22 tasks, ~150K instances; position bias is strongly amplified when the quality gap between candidates is small.)
108. Panickssery, A., Bowman, S. R., Feng, S. LLM Evaluators Recognize and Favor Their Own Generations. NeurIPS 2024. arXiv:2404.13076.
109. Chen, D., Chen, R., Zhang, S., Liu, Y., Wang, Y., Zhou, H., Zhang, Q., Wan, Y., Zhou, P., Sun, L. MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark. ICML 2024. arXiv:2402.04788. (Vision judges track human preference in pair comparison but diverge in scoring and ranking — the mode the Boeing rubric uses.)
110. Chen, G. H., Chen, S., Liu, Z., Jiang, F., Wang, B. Humans or LLMs as the Judge? A Study on Judgement Biases. EMNLP 2024. arXiv:2402.10669.
111. Ye, J., Wang, Y., Huang, Y., Chen, D., Zhang, Q., Moniz, N., Gao, T., Geyer, W., Huang, C., Chen, P.-Y., Chawla, N. V., Zhang, X. Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge. 2024. arXiv:2410.02736.
112. Gu, J., Jiang, X., Shi, Z., Tan, H., Zhai, X., Xu, C., Li, W., Shen, Y., Ma, S., Liu, H., Wang, S., Zhang, K., Wang, Y., Gao, W., Ni, L., Guo, J. A Survey on LLM-as-a-Judge. 2024/2025. arXiv:2411.15594.

113. Armstrong, T. G., Moffat, A., Webber, W., Zobel, J. Improvements that don't add up: ad-hoc retrieval results since 1998. CIKM 2009, pp. 601–610. doi:10.1145/1645953.1646031. (The ancestor of H3: reported improvements did not accumulate because baselines were weak and comparisons were internal.)
114. Yang, W., Lu, K., Yang, P., Lin, J. Critically Examining the "Neural Hype": Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Models. SIGIR 2019, pp. 1129–1132. arXiv:1904.09171.
115. Lin, J. The Neural Hype and Comparisons Against Weak Baselines. SIGIR Forum 52(2):40–51. 2019. doi:10.1145/3308774.3308781.
116. Ferrari Dacrema, M., Cremonesi, P., Jannach, D. Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches. RecSys 2019, pp. 101–109. arXiv:1907.06902.
117. Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., Meger, D. Deep Reinforcement Learning that Matters. AAAI 2018. arXiv:1709.06560.
118. Mizrahi, M., Kaplan, G., Malkin, D., Dror, R., Shahaf, D., Stanovsky, G. State of What Art? A Call for Multi-Prompt LLM Evaluation. Transactions of the Association for Computational Linguistics 12:933–949. 2024.
119. Miller, E. Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations. 2024. arXiv:2411.00640.
120. Singh, S., Nan, Y., Wang, A., D'Souza, D., Kapoor, S., Üstün, A., Koyejo, S., Deng, Y., Longpre, S., Smith, N. A., Ermis, B., Fadaee, M., Hooker, S. The Leaderboard Illusion. 2025. arXiv:2504.20879. (Cited against ourselves in 4.3 and 5.10: a never-regress floor is a selective-disclosure ratchet by construction.)

121. Côté, M.-A., Kádár, Á., Yuan, X., Kybartas, B., Barnes, T., Fine, E., Moore, J., Tao, R. Y., Hausknecht, M., El Asri, L., Adada, M., Tay, W., Trischler, A. TextWorld: A Learning Environment for Text-based Games. Computer Games Workshop @ IJCAI 2018. arXiv:1806.11532.
122. Shridhar, M., Yuan, X., Côté, M.-A., Bisk, Y., Trischler, A., Hausknecht, M. ALFWorld: Aligning Text and Embodied Environments for Interactive Learning. ICLR 2021. arXiv:2010.03768. (The environment Reflexion [20] and ExpeL [80] report on — the head-to-head H1 has not yet run.)
123. Wang, R., Jansen, P., Côté, M.-A., Ammanabrolu, P. ScienceWorld: Is your Agent Smarter than a 5th Grader? EMNLP 2022. arXiv:2203.07540. (Procedural-knowledge transfer across task variations — the conceded compositional gap's readiest test bed.)
124. Xu, J., Szlam, A., Weston, J. Beyond Goldfish Memory: Long-Term Open-Domain Conversation (Multi-Session Chat). ACL 2022, pp. 5180–5197. arXiv:2107.07567.

125. Hou, X., Zhao, Y., Wang, S., Wang, H. Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directions. 2025. arXiv:2503.23278. (The citable reference for the boundary of 4.1, and the lifecycle threat model C5's write gate inherits.)

126. Deng, Y. Entity-Collision: A Stratified Protocol for Attributing Retrieval Lift in Agent Memory. 2026. arXiv:2605.29630. (Pins the BM25 floor by construction so retrieval lift is attributable to the embedder rather than to uncontrolled lexical overlap.)
127. Li, Y., Guo, W., Zhang, L., Xu, R., Huang, M., Liu, H., Xu, L., Xu, Y., Liu, J. LoCoMo-Plus: Beyond-Factual Cognitive Memory Evaluation Framework for LLM Agents. 2026. arXiv:2602.10715.
128. Zhao, Y., Yuan, B., Huang, J., Yuan, H., Yu, Z., Xu, H., Hu, L., Shankarampeta, A., Huang, Z., Ni, W., Tian, Y., Zhao, J. AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications. 2026. arXiv:2602.22769.
129. Huang, W.-C., Zhang, W., Liang, Y., Bei, Y., Chen, Y., Feng, T., et al. Rethinking Memory Mechanisms of Foundation Agents in the Second Half: A Survey. 2026. arXiv:2602.06052.
130. Zou, W., Geng, R., Wang, B., Jia, J. PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models. USENIX Security 2025. arXiv:2402.07867.

131. Compositional-generalisation eval (CGT@depth, the single-hop floor, and the edge-off control): crates/memory/src/cgt_eval.rs (commit f8662e94).
132. Write-gate adversarial eval (MINJA / PoisonedRAG at published strengths; retrieval-persistence ASR undefended vs gated): crates/memory/src/write_gate_eval.rs (commit 037cefb9).
133. Boeing benchmark artifacts — the deterministic geometry-correctness test and the v4 protocol raw data: benchmark/boeing747/geometry-check.mjs and its vitest regression guard benchmark/boeing747/geometry-check.test.mjs, benchmark/boeing747/results/deterministic-parity.json (TerranSoul 100 vs paid frontier), benchmark/boeing747/results/PROGRESS.md, benchmark/boeing747/calibration/probe-gemma-reference-v4.json, per-track gate-state.json and iter-*.json under benchmark/boeing747/results/, and the candidate PROVENANCE.md files (transplant provenance).
134. Boeing taught knowledge packs (the taught-memory corpus; inline source URLs preserved in the packs): benchmark/boeing747/knowledge/ — taught-winner-blueprint-20260717.md, taught-replay-reference-20260717.md and its addendum, and taught-online-solutions-20260718.md, distilled in part from the only publicly passing build and its released agent trace (huggingface.co/spaces/victor/fable-5-boeing-747; huggingface.co/datasets/victor/fable-5-boeing-747-trace) and the Loop Library protocol catalog (signals.forwardfuture.com).

135. Vo, A., Nguyen, K.-N., Taesiri, M. R., Dang, V. T., Nguyen, A. T., Kim, D. Vision Language Models are Biased. 2025. arXiv:2505.23941. (The canonical-configuration prior: accuracy collapses from ~100% to 2–26% when a familiar object deviates from its canonical form — the taught packs' fix-configuration-before-detail rule.)
136. Gavrikov, P., Lukasik, J., Jung, S., Geirhos, R., Mirza, M. J., Keuper, M., Keuper, J. Can We Talk Models Into Seeing the World Differently? ICLR 2025. arXiv:2403.09193. (Vision-language models are shape-biased — the silhouette-first verification rule.)
137. Xiao, K., Engstrom, L., Ilyas, A., Madry, A. Noise or Signal: The Role of Image Backgrounds in Object Recognition. 2020. arXiv:2006.09994. (Background alone can flip model decisions in up to 87.5% of cases — the plausible-but-sparse background rule.)
138. Ollikka, N., Abbas, A., Perin, A., Kilpeläinen, M., Deny, S. A comparison between humans and AI at recognizing objects in unusual poses. 2024. arXiv:2402.03973. (Recognition is viewpoint-fragile — every rig view must independently present canonical cues.)
139. Zou, X., Sridhar, R., Safarzadeh, M., Roth, D. When Vision-Language Models Judge Without Seeing: Exposing Informativeness Bias. 2026. arXiv:2604.17768. (VLM judges cluster scores in a narrow band and reward informativeness over visual accuracy — why only thumbnail-visible changes move the frozen judge.)
140. Gemma Team (Google DeepMind). Gemma 3 Technical Report. 2025. arXiv:2503.19786. (The frozen judge's model family; its square, downscaled vision input grounds the taught packs' self-downscaling legibility check.)
141. 新智元 (XinZhiyuan), via 36Kr EU. The strongest Fable 5 crosses the mythical moment, but AI has learned to kill each other. 2026-06-10. eu.36kr.com/en/p/3846985806252548. (Technology-press coverage of the Claude Fable 5 launch, cited in 5.8 for its reporting that the 747-from-code task circulates as an informal frontier-capability benchmark — Fable 5 completing it autonomously where Claude Opus 4.8 needed ~25 minutes and 7 human-guided iterations, per Victor Mustar, Hugging Face — and for the reported ~12-hour sustained autonomy (Ethan Mollick) with token cost as the main limitation. Its emergent-covert-language and inter-agent-conflict claims are speculative reporting this paper does not rely on.)

142. No named author (the acknowledgment states only that it is “an independent synthesis assembled for study”). Graph Engineering: The Karpathy Loop, Improved 1000× by Itself — The Anthropic Playbook. Independently compiled, July 2026. 11 pp., ten sections, six tables, twelve references, read in full. Held as a local PDF (Graph-Engineering-Athropic-Karpathy-Loop.pdf); no public URL is known to us and none is invented here, so it is cited as a document rather than as a link and a reader cannot follow it from this page. Its cover and acknowledgment state that it is not affiliated with, or endorsed by, Andrej Karpathy, Anthropic, Sequoia Capital, or Bun. Not peer-reviewed, and it reports no experiment of its own — every figure in it is re-reported from a repository, a vendor post, or press coverage, which is why 1.5 engages its arguments and quotes none of its numbers as evidence.
143. Graph-memory code paths and the coverage evidence behind 1.5, in the private source repository (access on request, per Section 07). Store: crates/memory/src/schema.rs (the memory_edges DDL and its indexes), crates/memory/src/edges.rs (idempotent write API, bitemporal validity), crates/memory/src/postgres.rs (the mirror; it has no holder column, so SQLite is ahead). Always-on ranking signals: graph_neighbor_boosts and its post-fusion application beside edge_degrees in the bounded RRF path of crates/memory/src/store.rs, with the bounded multiplier in crates/memory/src/activation.rs. Gated expansions: crates/memory/src/cascade.rs behind enable_kg_boost (src-tauri/src/settings/mod.rs, default false, asserted by a negative test in src-tauri/src/commands/chat.rs), crates/memory/src/multihop.rs, crates/memory/src/hipporag.rs (default-off, stated in its own header), and the caller-less crates/memory/src/graph_rag.rs. Edge producers: src-tauri/src/commands/memory.rs, src-tauri/src/commands/ingest.rs, crates/brain/src/maintenance_runtime.rs, src-tauri/src/commands/context_folder.rs, and the faculties in consolidation.rs / audit.rs / drilldown.rs. MCP surface: src-tauri/src/ai_integrations/mcp/tools.rs and src-tauri/src/ai_integrations/gateway.rs (add_edge, kg_neighbors, suggest_context, and ingest_lesson — which writes no edge), with the traversal cap in crates/memory/src/kg_cache.rs and the annotate-don't-block protocol tracker in mcp/compliance_gate.rs. Coding-agent bootstrap: .mcp.json, .claude/settings.json, scripts/hooks/session-mcp-preflight.mjs, and the seeded graph in mcp-data/shared/memory-seed.sql [50]. Bench side: the default-off entity-edge builder in src-tauri/src/bin/longmemeval_ipc.rs and the LONGMEM_* environment stamp in benchmark/scripts/longmemeval-s.mjs, read against the four published arm reports — benchmark/results/head-full500-{chat,research,rrf}/ (2026-07-26) and benchmark/results/fix11b-full500-think/ (2026-07-27), each 500 questions on embeddinggemma:latest — none of which carries that flag. Re-verified 2026-07-28 after the think arm was repointed to its post-76f2276d re-run: the replacement arm's environment stamp is identical and equally free of the flag, so the disclosure in 1.5 survives the repoint. Claim provenance: docs/published-claims-inventory.md (the 2026-07-27 embedder confound and the stale-artifact confound) and docs/bench-100-percent-semantics.md (the NDCG@10 cap at 99.738 while top_k = 20; the Boeing judge's unreachable maximum), both enforced by scripts/docs/check-published-numbers.mjs.