Knowledge & actions · Explore this field ↗ · Operations · 3 min read
Knowledge freshness and document lifecycle: make retrieval expire on purpose
A governance loop for source ownership, effective dates, replacement, deletion, access changes, and answer verification.
Freshness is more than a timestamp
A retrieved page can be recently crawled and still be wrong for the user. Give every source a content owner, authoritative location, jurisdiction or audience, effective date where available, review interval, lifecycle state, and a stable source identifier. Distinguish published, draft, superseded, withdrawn, archived, and unknown. A document that lacks an effective date is not automatically stale, but it should not silently acquire one in a generated answer. For policy, price, eligibility, medical, legal, or operational material, have the answer layer expose the source and its relevant date when that context changes the decision. If two approved sources conflict, route the conflict to the owner instead of asking the model to choose a winner.
Model the source and its derived artifacts
A document lifecycle includes more than the original file. Extraction output, chunks, embeddings, summaries, citations, cache entries, enriched fields and evaluation cases may all survive a source update. Maintain a lineage record that connects each derived artifact to the source version and ingestion run. Then a withdrawal can identify what must be removed or made unavailable. This is also how a team answers a practical incident question: did the service retrieve the old policy because it was still in the source system, because deletion was not propagated, because a cache was stale, or because the retrieval filter ignored the lifecycle state?
Update and deletion need separate tests
Microsoft’s Azure AI Search documentation is a useful concrete warning, even for teams using another stack: incremental indexing detects new and changed content through source-specific mechanisms, while deletion handling must be designed rather than assumed. Its documentation notes that reset and rerun do not themselves remove orphaned documents. Treat that as a general design lesson. Test a changed source, an access-label change, a moved canonical URL, a soft deletion and a hard deletion. Confirm that an answer which should change does change, and that a formerly retrievable passage is no longer returned to the agent or cited. Do this from the user’s authorization context, not only from an administrator console.
Put a gate before retrieval
At query time, filter by tenant, entitlement, lifecycle state, audience, locale and effective period before ranking relevance. Do not retrieve broadly and ask the model to ignore material it should not see. Access decisions made only at ingestion can become outdated when a user’s role or a document’s classification changes. A response policy should also decide when to decline a precise answer: source state unknown, review overdue, conflict unresolved, or no authorized current source. That is an honest service condition, not a prompt failure.
Rehearse the replacement path
Choose a small set of sentinel questions for each high-consequence collection. When a new policy version enters, run those questions before promotion and record expected citations and constraints. When it replaces an old version, test negative retrieval for the old wording. Keep the withdrawn source in a restricted provenance register if it is needed for audit, but do not leave it in the general answer index. Assign an operator to review failed freshness checks and a time limit for repairing them. NIST’s Generative AI Profile is useful here as a risk-management framing: lifecycle controls, documentation and evaluation belong to the system, not just the model.
Do not confuse index currency with truth
An index can be synchronized perfectly to a source that is itself incomplete, misleading, or legally inapplicable to the current user. Retrieval quality, source authority and policy interpretation remain separate review questions. Vendor documentation describes a particular product’s behavior; it is not a universal lifecycle standard. Keep the contract specific to your sources and revisit it when data ownership, connectors or access rules change.
Primary reading
Sources and limits
These links support the architecture, policy, or product behavior discussed above. Vendor documentation describes vendor features; it is not independent proof of performance. Current details should be rechecked before a production decision.