Why your AI assistant answers from an outdated document

An employee asked the company assistant a routine question: what is the current process for changing my address? The assistant answered confidently and cited a document. The document was two versions out of date. The process it described had been replaced months earlier, and nobody noticed until the employee followed it and the request bounced back.

This is one of the most common failures in enterprise AI right now, and it is almost never a model problem. The model did what it was asked to do with what it was handed. The failure happened before the model was involved.

Retrieval ranks by similarity, not by currency

Most assistants answer from retrieval: embed the question, find the nearest chunks of text in an index, hand them to a model, ask for prose. That pipeline optimizes for exactly one thing — semantic closeness between the question and some text. Nothing in it asks whether the text is still true.

Worse, the outdated document is often the better match. Superseded versions tend to be longer, more keyword-dense, and written in the same vocabulary as the question, because the replacement was rewritten to be shorter and clearer. Retrieval rewards the old document for being wordier about the thing you asked.

Similarity is not currency. An index that ranks by resemblance will confidently prefer a document that is well written and wrong.

The index does not know the document is dead

Lifecycle state lives in the system that manages the document — a status column in SharePoint, a row in a CMS, a folder convention, a filename ending in _FINAL_v3. It does not live in the sentences. When a document is chunked and embedded, that state is left behind. What lands in the vector store is a paragraph of prose with no memory of having been retired.

So the retrieval layer has nothing to filter on. It cannot exclude the superseded version because, as far as the index is concerned, there is no such thing as a superseded version. There is only text that resembles your question to varying degrees. Add the usual duplication — the same policy living in a team site, a personal drive, an email attachment, and an archive folder — and the current version is competing against four copies of its own past.

The usual fixes, and why they don't hold

• Prompting. "Only use current documents" asks the model to verify something it has no way to check. It will comply in tone and not in fact.
• A better model. A stronger model reasons better over what it receives. It does not know what it wasn't given, and it cannot detect that the passage in front of it was retired.
• Better embeddings. These improve relevance. Currency is not a kind of relevance.
• Deleting the old files. Usually not permitted. Retention obligations are frequently the reason the superseded version still exists.
• Filtering by modified date. File timestamps move for meaningless reasons — migrations, permission changes, metadata edits. A stale document touched last week outranks a current one untouched for a year.
• Adding a disclaimer. This relocates liability. It does not change the answer.

What actually fixes it

Lifecycle has to become a fact the system can act on, and it has to be enforced before anything is ranked. In practice that means every governed artifact carries, as structured data rather than prose:

• An effective window — the date it took effect, and the date it stopped applying.
• A supersedence chain — what this replaced, and what replaced it.
• An owner and a review date, so staleness is visible before it is discovered by an employee.
• A content hash, so you can prove which exact bytes were served in a given answer.

With that in place, choosing a document stops being a similarity contest and becomes a decision you can inspect. Eligible artifacts are selected on the record; ineligible ones are excluded with a stated reason — retired, superseded by a named successor, outside its effective window, wrong jurisdiction, wrong audience. The model, if one is used at all, receives only what survived that filter. It is no longer the component deciding what is true.

This is the order we build in, and the order matters: model the work, make the rules mechanical, then add AI to whatever is left. Handing a language model your document sprawl and hoping it sorts out which version is authoritative is the opposite of that order.

A test you can run this week

1. Pick a document your organization superseded in the last twelve months — a policy, a procedure, a form with a new revision.
2. Ask your assistant the question the old version answered.
3. Check three things: which version it cited, whether it named the current one, and whether it can tell you why that one and not the other.

Most systems fail the third check, and the third check is the one that matters. If your assistant cannot explain why it chose a document, you cannot audit the answer, which means you cannot trust it for anything consequential — and HR, benefits, payroll, and compliance are all consequential.

Common questions

Why does my AI assistant cite an outdated document?

Because retrieval selects text by similarity to your question, and lifecycle state is not part of the text. The superseded version is often the closer match, since it is usually longer and uses more of the question's vocabulary than its replacement does.

Can I fix it by telling the assistant to only use current documents?

No. The instruction asks the model to verify something it cannot see. Nothing in the retrieved passage records whether the document is current, so the model has no basis on which to comply.

Does upgrading to a better model solve this?

No. A stronger model reasons better over the passages it receives, but it has no way to know which passages it was not given, or that the one in front of it was retired. The problem is in selection, which happens before the model is called.

If you want to see what governed selection looks like when every decision is visible, the Context Engine demo runs the whole path in your browser — mapping, routing, exclusions, and a receipt for the answer — over synthetic data.