On this page
I spent an afternoon reading the source of eight open-source AI memory tools, hunting for one field. Not one of them has it. When any of these tools hands back an answer, nothing in that answer says which version of your notes it was built from.
That sounds like a small omission. It is the difference between a tool you can check and a tool you have to trust.
The copy problem
Almost every memory tool has the same anatomy. Your actual notes live somewhere durable: markdown files, a database, a Git repository. Sitting on top is a search index, which exists because the durable thing is the wrong shape for answering questions quickly.
That index is a photocopy. Photocopies do not update themselves.
So there is always a gap, usually small and occasionally not, between what you have written and what the search box knows about. Add a note, ask about it a second later, and a perfectly healthy system will tell you the note does not exist. Nothing errors. No log line goes red. You simply get a wrong answer wearing the same confident formatting as a right one.
Everything in my fridge admits how old it is. My search index would not.
Three questions I asked every tool
I wanted to know whether a tool could be caught out, so I scored each one on three things that build on each other.
Does a result carry its version? When you get an answer back, is the revision it was derived from sitting right there in the payload, next to the content?
Is there a way to check the copy against the original? Can you ask the system whether its index and its source actually agree, and get a straight answer naming the laggard?
And if they disagree, does it stop? Or does it shrug and answer anyway, from the copy it already knows is behind?
A tool that does all three can be checked by whatever is calling it. A tool that does none has to be taken at its word.
What eight tools answered
I read the read path in every one of these. Star counts are from the day I checked.
| Tool | Stars | Carries a version | Can check the copy | Stops when they disagree |
|---|---|---|---|---|
| mem0 | 66,303 | No | No | No |
| Hindsight | 42,133 | No | No | No |
| cognee | 31,210 | No | No | No |
| DiffMem | 904 | No | No | No |
| Remnic | 211 | No | Partly | No |
| Inspeximus | 7 | No | No | No |
| Talamus | 3 | No | Partly | No |
| GitOfThoughts | 1 | Only one you passed in | No | No |
I also want to be careful about what the table means. It does not say these are bad tools. Most of them are working on recall quality, which is a harder problem and a more useful one to most people than the question I turned up asking. Several are very good at it. What the table says is that a property I care about is absent from all eight, including the ones built on Git, where the version is sitting right there for free.
I read the function that answers a search in each of the eight and listed the fields it returns. For Hindsight I read its generated API client models, which are the contract its callers actually receive. If I have missed a version field somewhere, I would genuinely like to hear about it.
So close, and yet
The near-misses are where this got interesting, because three of these tools clearly thought hard about exactly this kind of ambiguity and then thought about a different axis of it.
mem0 gives every memory a hash. A content hash, of that one memory. So it can tell you whether a particular note has changed since you last saw it, which is genuinely useful. It cannot tell you whether the store you just searched was current. It dates the page, not the edition.
cognee has the richest provenance of anything I looked at. Its evidence references carry chunk_id, chunk_index, assertion_id, source_node_id, document_name, source_ref_key. It will tell you, in forensic detail, precisely which fragments of which documents fed the answer you are holding. What it will not tell you is what state the dataset was in when it did that. This is the finding that convinced me the gap is real and specific: this is not a field that forgot about provenance. It is a field that solved provenance beautifully and never asked “as of when”.
Hindsight sweats the small ambiguities. Its search results return a snippet, and the documentation for that one field explains that a page with no body comes back saying “No content yet.” in words, rather than as an empty string, so that a caller can tell an unwritten page from a page that failed to render. Someone sat down and thought carefully about a caller being silently confused. They did it for one field. Currency was not the field.
The two traditions that almost get there
Step back and the eight stop in two distinct places, approaching from opposite directions.
The first tradition is cache invalidation. Remnic and Talamus both keep a version number and use it to throw derived data away when it goes bad. They are not the same number: Remnic’s tracks content and bumps whenever the corpus changes, while Talamus’s tracks the cache format and trips only when the layout on disk stops matching what the code expects. Remnic’s is the closer of the two to what I wanted, and there is a comment in its source warning that keying the cache on the wrong signal would let a second process serve a stale map missing newly written memories. Exactly the right instinct, and it goes further than I first credited: the counter is backed by a sentinel file on disk, so peer processes see the same value rather than each keeping a private tally. What it does not do is leave the building. The search result you get has a title, a score, a path, and no version at all.
The second tradition is the tamper-evident log. Inspeximus is the most serious piece of verification engineering in the whole survey, from a project with seven stars. Merkle trees, signed checkpoints, external witnesses, consistency proofs: the machinery certificate transparency runs on. You can prove nobody rewrote its history. What you cannot do is look at a recall result and see which state of that log produced it, because the log machinery and the read path never meet each other.
One tradition keeps a copy honest behind the scenes, for speed. The other proves the past was not edited, after the fact. Neither carries the version across to the reader at the moment the reader is about to act on it.
Once I saw it in that shape the gap stopped looking like an oversight and started looking like unclaimed ground between two mature neighbourhoods.
The four days of silence
I care about this because it has bitten me, and never once as an error message.
My own vault has three things reading from it: a wiki that renders the pages, a local search index, and a backup mirror on another machine. I watched that mirror report success and exit cleanly while pushing precisely nothing, because the filesystem layer underneath was serving a cached view from before the commit. Every indicator was green. The mirror was a day behind and perfectly happy about it. The fix was not a better mirror. It was making every reader state the revision it read, and refusing to call the system current unless all of them said the same thing. I wrote that health model up in proving your docs are actually up to date, and the agent side of it in making your agents read the docs first.
The same shape turns up everywhere once you have a name for it. My download client once reported a correctly synced port while moving zero bytes for four days. The part that did the configuring reported on the configuring. Nothing checked the outcome.
A memory tool that cannot report its revision is that trap with better typography. It will answer from a week-old index just as smoothly as from a current one, in the same font, with the same confidence.
Check any tool yourself in ten minutes
You do not need to install anything. This is reading, and it is the same three steps in any language.
Find the function that answers a search or recall call. Follow it to whatever structure it returns and read the field names out loud. You are looking for anything that names a version: a commit hash, a revision id, a snapshot number, a sequence number, an ETag. If every field is content or scoring, you have your answer.
Then search the whole project for that same vocabulary. If the only hits are in tests, database migrations or the benchmark harness, it is not in the serving path. That distinction caught me repeatedly: three projects turned out to be full of careful staleness language that lived entirely in their documentation tooling and benchmark harness, and never touched a real query.
What you are hoping to find, and what I did not find anywhere, is the version sitting in the same structure as the answer.
If it matters to you, there is a small and answerable question to put to a project: would you take a patch that returns the corpus version alongside search results? For a tool already computing one internally, that is a modest change, and it turns a private detail into something a caller can act on. The spec writes the check out in full, including the two harder tests this section skips: whether a system can compare its copies against the source at all, and what it does when they disagree.
What I actually want
The property has a plain description, and none of it is exotic. Every read reports the revision it came from. Every copy has to report the same revision as the original. When they disagree, the system says so instead of answering. I have written that up properly as a conformance specification, with the requirements in RFC 2119 terms and five conformance levels, because “I would like this” is easier to argue with than “here is the property, here is how to test it”.
That last clause is the one with a price on it, because it means occasionally refusing to be helpful. A tool that declines to answer while its index rebuilds feels worse for about five seconds and is better to rely on forever. For anything I am going to act on, I will take that trade every time.
None of this needs the Git-backed design I happen to run. My vault is plain markdown in a Git repository with a local embeddings index over the top, and Git hands me a revision identifier for nothing. A sequence number from a database would do the same work. The identifier is not the clever part. Carrying it to the reader is.
Related posts:
- Prove Your Homelab Docs Are Actually Up to Date — the health model this post measures other tools against, with the checks written out.
- Make Your AI Agents Read the Docs First, Every Time — how an agent verifies the revision before trusting anything it loads.
- Share Your Vault Across Machines Without Split-Brain — many readers, one authoritative copy, and a reader that labels what it serves.
- Never Lose a Homelab Note Again: A Git + Markdown Vault — the storage layer underneath all of this, and where the revision identifier comes from.
- Semantic Search for Your Notes, Running Locally — the derived index that has to agree with the source, and what happens when it drifts.
- Why Auto-Syncing Your Notes Repo Will Corrupt It — the write side, where single-writer discipline is what keeps a revision meaningful.
Comments
Comments are powered by GitHub Discussions — sign in with a GitHub account to join the conversation.