Inside the brain: how an LLM gets memory, recall & skill
A language model is brilliant in the moment and forgets everything between turns. TerranSoul wraps that model in the regions of a real brain — focus, memory, recall, association, feeling and skill — so it can remember, learn from outcomes, and grow more useful over time. This is the what and the why, in plain language.
How the brain works together
Brain Architecture Design
Many specialised parts, one coherent loop.
An LLM core, wrapped in brain regions
Picture the model as the brain's thinking surface — the part that reasons, in the moment, about whatever is in front of it. On its own it has no memory of its own and no past. TerranSoul grafts on the other regions of a brain, each a small module that gives the core something it lacks.
Not every message deserves deep thought. Focus sizes each turn and routes it — trivial ones skip the heavy machinery, hard ones get the full treatment. It keeps the brain fast and cheap when it can be.
Forms new memories from what happens and files them for later. Memories flow through tiers and fade unless they're used again — much like human reinforcement and forgetting.
Stores facts as a web of links, not a flat list. By chaining links the brain can reach an answer that no single memory ever spelled out — connecting people, places and ideas across many small facts.
Surfaces the memories worth seeing for the question at hand. Several different search methods each weigh in, and their verdicts are merged into one trustworthy short list.
Reads the emotional tone of the brain's own reply, so a companion avatar can show the matching expression. Feeling is part of how it presents, not just what it says.
Learned how-to procedures that build confidence as they succeed and get benched when they keep failing — the brain getting better at the things it does often.
How a fleeting moment becomes a lasting memory
Human memory isn't one bucket — a phone number you hold for ten seconds is different from a skill you keep for life. The brain uses three tiers, and memories graduate upward only when they earn it.
Holds the immediate flow of conversation. It's there so the brain stays coherent turn to turn, and it's allowed to disappear — most of what's said doesn't need to outlive the chat.
The session's active picture — the facts, threads and goals that matter right now. A staging area where things that keep coming up start to look worth keeping.
Durable memory that survives across sessions. Crucially it isn't frozen: unused memories slowly decay, so the brain forgets the trivial and keeps the meaningful — like a person would.
Four kinds of knowing — because not all memory is the same
A tier says how long a memory lives. But memories also differ in what kind of knowing they are — and the right one to surface depends on the question. The brain recognises four cognitive axes, drawn straight from how cognitive science describes human memory.
Episodic
What happened, and when. The story of past events and conversations — "last time we tried this," "what we decided yesterday."
Semantic
Facts and meanings that hold regardless of when you learned them — definitions, properties, "what is this thing."
Procedural
How to do something — the steps of a task. "How do I deploy this," recalled as a procedure rather than a fact.
Judgment
Durable rules, heuristics and lessons — the hard-won "do it this way" wisdom that should outrank a one-off anecdote.
The reason this matters is recall quality. A "how do I…" question should reach for procedures first; "what did we decide…" wants the episode; "what is…" wants the fact; and a settled rule should win over a passing remark. By understanding which kind of knowing a question is really asking for, the brain surfaces answers that feel right rather than merely related.
One answer, many points of view
No single way of searching memory is trustworthy on its own. Keyword search nails exact terms but misses meaning; meaning-based search catches paraphrases but fumbles precise identifiers; following links between facts reaches things neither would find alone. So instead of betting on one, the brain asks several — then fuses their answers.
The signals it fuses
- Lexical match — exact words and identifiers, the things meaning-search tends to blur.
- Meaning (vector) similarity — paraphrases and "this is about the same thing" matches.
- Knowledge-graph links — facts reached by following relationships between memories.
- Freshness — how recent and current a memory is.
- Activation — how connected and how often-used a memory is, a stand-in for "how alive it is in the brain right now."
Their separate rankings are blended into one, then a final pass reorders the survivors for genuine relevance. The result reads as a single, confident answer — but it's a consensus of several independent opinions, which is exactly why it's hard to fool.
There is one more pass after the rerank: a set-level bias audit (the BCF-15 faculty) inspects the whole short list for statistical skew — one source over-represented, everything clustered in a single recent window, near-duplicate dominance, or members that contradict each other — and gently down-weights a skewed set before it ever reaches the model. Hybrid recall checks not just which memories surfaced, but whether the set as a whole is balanced.
For research and agent turns, this recall also feeds an optional deep-research answer-mode that loops — critiquing its own coverage and following the knowledge graph — to assemble thoroughly-cited answers. It widens answer coverage, not retrieval: at production cutoffs its candidate set is the same one this single pass already returns.
Same brain, your choice of mind
The LLM at the core is pluggable. You can run a private model on your own machine, or reach for a frontier model in the cloud — and everything around it stays the same. Memory, recall, the graph, the skills: identical surface, swappable core.
A model running entirely on your own device. Private by default, works offline, nothing leaves the machine — the brain's memory and skills wrapped around a mind that's wholly yours.
A frontier model reached over the network for maximum capability when you want it. Because the surface is identical, the brain can lean on the cloud and still fall back to local — so it keeps working even if the cloud blinks.
The right gear for every turn
Choosing the mind is one axis; choosing how hard it should think is another. A greeting should answer instantly; a multi-step build deserves an agent loop; an open-ended question may need a full evidence sweep. TerranSoul has seven real thinking modes, and on every turn the brain reads the message and shifts into the one that fits — automatically, unless you pin a mode yourself.
Each mode sets two dials at once — how much the model deliberates, and which behavioural harness wraps the reply:
- Chat — fast, grounded answers straight from the knowledge graph (one hybrid + graph-edge pass, no chain-of-thought), for greetings and quick questions.
- Think — extended step-by-step reasoning in a single pass, for analysis and hard questions that don’t need to go out and search.
- Research — an agentic multi-step search: memory + knowledge-graph walk → self-critique for coverage gaps → re-query → cited answer, for fact-checking and open-ended questions.
- Max — the most thorough gear: retrieve a wide candidate pool, verify every item against the request from its own source text, and loop until the result is complete and fully ranked.
Exhaustive, fully-cited answers
Most turns are well served by a single recall pass and a short, confident answer. For research and agent work, the research and deep modes can instead engage an optional deep-research answer-mode: it loops — retrieve, ask a completeness critic what the answer still hasn't covered, follow knowledge-graph edges, re-query — then spend its deepest thinking on a thoroughly-cited synthesis. The point is coverage and answer quality, not more recall.
- Retrieve per sub-question. Each critic-generated frontier query runs the same multi-signal recall with a graph-edge bridge — the loop widens the set of questions asked, it does not re-rank or deepen any single query's retrieval.
- Critique coverage. A completeness critic asks the brain "what relevant memory did I not surface?" over everything seen so far, and re-queues the still-missing aspects.
- Expand along the graph. The top results seed a walk over knowledge-graph edges, pulling in linked-but-unsurfaced memories and chaining across rounds.
- Stop only when dry. The loop ends when coverage is saturated, gains flatten, the frontier empties, or a generous round cap is hit — every bound a brain-seeded setting, none hardcoded.
- Synthesise with citations. The final answer is forced through the deepest thinking tier with full source context and rendered with
[n]markers and a sources list — and the findings flow back into memory so the next deep pass starts further ahead.
A brain that learns from its own outcomes
The point of giving a model memory isn't just to look things up — it's to get better. TerranSoul closes the loop: what the brain does, how it turns out, and what it remembers all feed back into what it recalls next time. The cycle is simply write → manage → read.
As the brain works, it records lessons, facts and the results of what it tried — turning experience into memory rather than letting it evaporate.
New memories are tidied, linked to related ones, and consolidated so the store stays coherent. Skills that keep succeeding gain standing; ones that keep failing are set aside.
On future turns the brain recalls what's relevant — and because outcomes fed back in, what surfaces is shaped by what actually worked before.
One brain, reached and fed through one connection
Every region connects to the world through a single nerve: the Model Context Protocol. That means any client can query the same brain — and the brain, in turn, keeps itself current by pulling from its own sources. Knowledge written by one client is legible to the next.
Coding assistants like Claude Code and Codex, desktop and mobile apps, and even another TerranSoul — all reach the same brain over one protocol. Whatever one client learns, the next can read.
The flow runs both ways. The brain also pulls in fresh material from connectors, from per-project repo brains, and from peer instances — so its memory stays current without you feeding it by hand.
Twenty-one named faculties, each in a real brain region
The six regions are the coarse map. Inside them the brain is growing a layer of finer cognitive faculties — BCF-01 through BCF-21 — drawn from cognitive neuroscience and each anchored to the part of a human brain that does the same job. The roster was completed against a four-angle research sweep (cognitive-neuroscience taxonomy, the cognitive-architecture canon, LLM-agent architectures, and deep dives on insight, creativity, metacognition and curiosity), which added three faculties the literature treats as core — insight, creativity, and cognitive control. All twenty-one are now implemented (v1) — the complete roster, each anchored to its matching region on the interactive brain map.
Four faculties, up close
A second-order monitor that estimates a calibrated confidence in the brain's own answer before it is spoken, and routes the turn to one of four honest choices — answer, deliberate, seek more evidence, or abstain. It reads only structural signals (top score, margin, retrieval quality, agreement across samples, supporting links) and learns its own calibration curve from logged outcomes, so a small local model can decline to bluff rather than hallucinate confidently.
Endogenous "what should I do now" for an always-on resident. A set of homeostatic drives — coverage (curiosity), competence (mastery), integrity (coherence), upkeep (tidiness) and bond (connection) — each measured from real store metrics. Deviation from a setpoint becomes a need signal, and closing a gap or tidying the store is intrinsic reward. The highest-need drive surfaces as an intent for the goal-stack to act on; drive weights are learned from whether pursuing them tends to succeed.
The agenda that turns a fleeting drive into sustained pursuit. A prioritised stack of goals that persists across sessions, with arbitration (priority × drive-weight × due-ness, plus a commitment bonus), a suspend / resume / done / abandon lifecycle, progress tracking that auto-closes finished goals, and commitments with optional deadlines. Stale goals decay so neglected ones sink. This is the difference between a reactive chatbot and an agent that follows through.
A fairness check on the whole retrieved set, run after rerank but before the model sees it. Four structural metrics — source concentration, recency skew, single-cluster dominance and contradiction density, each normalised to a 0–1 score with no domain word-lists — flag a skewed set and down-weight it. It judges the set, where the per-document quality check judges each row, and its thresholds self-tune from their own flag-rate history.
The full faculty roster & its brain-region map
Each faculty sits beside the anatomical region that performs the matching function in a human brain — consistent with the placements on the interactive brain map. Every faculty below is implemented (v1) today — the full twenty-one-faculty roster, complete.
Knows when it doesn't know — calibrated self-confidence that gates answer vs. abstain vs. seek-more.
Setpoint need-variables plus intrinsic reward from learning-progress — the source of self-directed activity.
A durable, arbitrated goal stack with suspend/resume and commitment tracking that persists across sessions.
Appraises events against goals into a stored mood that decays — and modulates what becomes salient.
A winner-take-all bottleneck arbitrating memories, drives, intentions, affect and surprise for the one context window.
An explicit model of what the user knows, wants and feels, held apart from the agent's own.
A "do Y when X happens / at time T" store that fires an intention when its cue is met — follow-through beyond the turn.
Decides how much deliberation to spend, escalating only when confidence is low — System-2 on demand without retraining.
Acts on detected gaps with curiosity-driven questions and self-directed search, rewarded for closing a gap.
Causal "if I do X then Y" transitions for forward simulation and counterfactuals — the source of prediction-error.
Idle replay that distils episodes into durable generalisations and revises stale beliefs in place.
Self-verified, precondition-keyed runnable skills with an automatic curriculum — learning to act, not just recall.
A continuous self-narrative that binds sessions into one persistent "I" — the heart of resident embodiment.
Progressive summarisation / virtual-context paging that keeps long sessions coherent without abrupt truncation.
Audits the retrieved set for statistical skew and down-weights it before injection.
Aligns relational structure across domains to transfer a solution schema — human-like generalisation.
A perception front-end mapping image / screen / spatial input into the same memory schema with spatial graph edges.
When analytic search stalls, it re-encodes the stuck problem and reaches a remote association — the Aha! leap, measured as a real solution jump, not a token.
Generates many varied candidate ideas then filters them against constraints — balancing novelty against appropriateness on the four classic divergent-thinking axes.
Inhibits a prepotent or unsafe answer, monitors conflict between candidate responses, and catches its own errors to trigger self-correction.
A write-time safety gate over the brain's own autonomous writes — guarding against belief drift, poisoned or anomalous sources, and unbounded growth, and quarantining peer-brain imports until corroborated.
One clean schema, sharded for the write storm
The memory store is being rebuilt around a single, self-contained schema instead of a long chain of in-place upgrades — and split into purpose-built shards sized for the order of a million memories, so the high-frequency writes of an always-on resident never block recall. This section is a design contract: the spec a follow-up change implements, not yet live in the store. Because TerranSoul is pre-release there is no production data to migrate, so the schema can be recreated clean.
Why redesign — three structural debts
- A 33-step migration chain no real database needs. The old store carried one canonical schema plus ~20 self-heal helpers that dragged a hypothetical old table forward column-by-column — dead weight that bloated startup and made the true schema hard to read.
- One monolithic database file. A single SQLite file is single-writer: ingest, access-counter bumps, embedding writes, decay and append-only logging all serialised behind one lock, and every checkpoint replayed all of it.
- Unbounded append-only logs. Telemetry (~86k rows/day), traces, sync and calibration logs had no retention policy — inflating the write-ahead log and slowing every scan, even though their write-once, sequential access pattern is the opposite of the random-access knowledge table.
One definition, five table groups
The canonical schema plus its version-helper chain collapse into a single schema applied once (version reset to 1). Every column exists in its CREATE TABLE from the start — no backfills, no "create indexes last" ordering hazard. The schema reads as five clearly fenced groups:
| Group | Purpose | Holds |
|---|---|---|
G1 core memory | the knowledge base + provenance | memories, versions, conflicts, sources, offload payloads |
G2 embeddings & index | vector + keyword retrieval surfaces | embeddings, pending-embeddings, full-text index + its sync triggers |
G3 knowledge graph | typed edges + pre-aggregated stats | edges, graph clusters |
G4 append-only logs | write-heavy, time-ordered, retention-bounded | traces, telemetry, sync log, calibration log, safety decisions |
G5 cognitive-faculty tables | learning / metacognition state on the hot path | reinforcements, trigger patterns, gaps, tool-skills, action-trust ledger, decomposition cache, paired devices |
To keep the hot row narrow, the rarely-read procedural ledger (success / failure counts, activation tier, deprecation flag) moves out of the wide memories table into a keyed G5 side table — so the 90% retrieval path reads a compact row and the promoter pays a cheap join only when it needs the counters.
Sharded for high-throughput writes
The store opens one primary connection and attaches siblings — a single-process scale-out where joins still work but each file gets its own write-ahead log, checkpoint cadence, cache and lock. Routing is keyed on the two axes the memory model already encodes — tier (working vs. long) and access pattern (random-access knowledge vs. sequential logs):
The canonical read-heavy RAG database — core memory, embeddings and the knowledge graph. Tuned with the largest cache for fast fused recall.
Working-memory churn that promotes and evicts every turn, aggressively checkpointed so it never dirties the durable index.
Append-only telemetry and traces on a separate checkpoint, decoupling high-frequency log writes from retrieval latency.
On top of the shards, several mechanisms absorb the write rate of a resident that records something on nearly every turn:
- In-memory write buffer for the hottest counter. The per-search access-count bump is the worst write-amplifier; instead of one disk write per query it is accumulated in memory and flushed asynchronously, cutting write-ahead-log frames by roughly two orders of magnitude and removing the read/write lock fight on the hot path.
- Group-commit batching. A chat turn's many reads run in one transaction, and ingest lands in ~1000-row micro-transactions rather than row-at-a-time, so the durable index commits in batches instead of thrashing the log.
- Per-shard write-ahead tuning. Each attached file carries pragmas matched to its role — a large cache and relaxed checkpoint for the read-heavy durable shard, tight checkpoints for the churny working and log shards — with a one-writer / many-reader connection pool.
- Time-partitioned logs with retention. Each append-only table rotates into monthly partitions; a background task moves cold partitions to an archive and hard-deletes past a generous per-table window, so the logs never grow without bound.
- Hot/cold split inside the durable shard. Partial indexes let recall scan only the live ~10% of the corpus for far better cache residency, while decay and garbage-collection touch the cold rows without stalling reads.
Best ideas from rival agents, re-grounded as our own
TerranSoul studies competing agent systems and folds their strongest generic patterns into the brain — reimplemented on existing modules as domain-agnostic primitives. No upstream source, prompts, branded identity or schema is copied; each adoption enhances a part of the brain rather than bolting on a parallel one. Influences are credited in the project's CREDITS file.
A study of GENesis-AGI contributed several generic mechanisms, each reimplemented in Rust: an activation signal as an extra recall term (how alive a memory is right now), an echo-collapse diversity pass that stops near-duplicates dominating a result set, auto-linking on write so new memories join the graph immediately, consolidation safety gates (reversible rollback, faithfulness review, a shrink guard), and outcome-classified write-back that routes a five-class session outcome to reinforce, quarantine or extract.
Reverse-engineered from the Hermes agent (generically, nothing vendored): a closed learning loop — observe → author → validate → register → reuse → refine — that detects a recurring or workaround-successful trajectory and creates a fresh reusable skill, the one capability the brain lacked. New skills are governed by the existing confidence ledger, surfaced by description-first retrieval (names and one-liners in the prompt, bodies hydrated on match), and curated through an Active → Stale → Archived lifecycle. Design contract; not yet shipped.
A blueprint unifies a parameterised skill and a human-friendly schedule into one typed object — no raw cron is ever written. When the brain notices a routine worth automating it adds a consent-first suggestion to a queue: it proposes, you accept or dismiss, and only an explicit accept ever schedules anything. The proposal bar is learned from how often suggestions are taken up, so it grows more or less forward over time.
A continuously-running companion should notice the follow-ups a conversation leaves dangling — "I'll let you know when the build finishes," "check in after their interview" — without being asked. Inferred open-loops are stored as future intentions, and a gentle heartbeat surfaces the due ones through the same attention gate as everything else, held back by quiet-hours, busy-skip and don't-repeat guards so it never nags. How proactive it is, is itself learned.
The patterns it borrows — and where they land
- A proof-before-trust ledger. A synthesized skill is born "cold" and advisory-only: retrievable for audit but never auto-injected until it earns promotion through the confidence ledger and clears an offline replay check that confirms it doesn't lower answer quality. Unproven know-how can't change live behaviour.
- Snapshot-style checkpoints. Every self-authored write is snapshotted before it mutates anything and archived rather than deleted, so a bad consolidation can be rolled back — and a frozen-snapshot write persists to disk now but only folds into the prompt next session, keeping the model's prefix cache stable mid-conversation.
- Self-repairing skills. Re-authoring an existing skill is an idempotent patch keyed on source + name, so an outdated procedure is updated in place instead of spawning a duplicate — repair, not proliferation.
- A pre-write safety scan. Before any self-authored artefact is stored, a generic content scan (prompt-injection, credential-exfiltration, hidden-character checks) gates it behind a trust tier and quarantines anything below the floor — complementing the faithfulness review that already guards fidelity.
- An import seam for external tool-skills. The same skill catalog can ingest tool-skills from sibling agents such as OpenClaw and Hermes, tagged by source, so imported, synthesized and built-in skills share one search, install and uninstall surface.
The whole idea
One frozen LLM, one nerve (MCP), and the 21 BCF components above sitting between them — each one reads/writes its own tables, each one is reachable by the model on every turn. Nothing here retrains the model; every capability lives in the memory layer, which is why it's auditable and swappable independent of the model.