An Embedding Index Is Not Your Database

A team embeds every row of a customer table and starts answering questions from the vector store. The phone number changes in the database. The embedded text still has the old one. A query for "this customer’s phone" returns a fluent sentence with the old number, because the nearest passage is stale and the model is not looking at the row.

Lewis, Perez, Piktus, and colleagues introduced retrieval-augmented generation as a way to condition a generator on passages fetched from a corpus. That is a search-then-read design. It is not a storage design. The corpus has a home. The index points at it. Constraints, transactions, and "which value wins" live in the home, which is the system of record. AI Features Need an Owner for the Record names that owner. Production RAG data freshness covers how the index falls behind. This article is the prior decision: do not appoint the index as the home.

What an embedding is good at

• Finding passages that are about the same thing when the words do not match exactly.
• Narrowing a large corpus to a handful of candidates a person or a model can read.
• Ranking unstructured notes, tickets, or documents.
• Not deciding that two customers are the same entity. That is a key, or a reviewed match.
• Not storing the only copy of a balance, a permission, or a price.
• Not applying a tenant filter unless you filter explicitly. Nearness will cross tenants if you let it.

Illustrative scenario

This scenario is illustrative. Support notes are embedded so an agent can find similar cases. The ticket row, including status and customer id, stays in the relational database. The search returns note ids. The application loads those tickets with the caller’s tenant and permission, then shows them. If the index returns an id the caller cannot read, the application drops it. The model, if one is used, sees only the tickets that survived that load. When a note is deleted, the row deletion and the index deletion are both required. The index is a derived copy. Losing it means you rebuild it. Losing the table means you lost the tickets.

Checklist

• Name the system of record for each fact the assistant might say.
• Use the index to propose ids, then load those ids through the normal permission checks.
• Do not answer a numeric or legal fact from passage text if the record has a field for it.
• Update or remove index entries when the source changes. A missing deletion is a stale answer.
• Include the tenant in the filter, not only in the embedded sentence.
• Be able to rebuild the index from the source. If you cannot, the index has become the source by accident.

Questions that usually come next

Should we embed structured columns at all?

Embed the text you would otherwise search by wording. Filter and join the columns in the database. Embedding a price so you can "search" for it is how you get a nearby price instead of the price.

Is a vector database a replacement for Postgres?

It replaces neither transactions nor constraints. Some systems can store both vectors and rows. The role split remains: rows for facts you write, vectors for candidates you retrieve. Database Design for AI Agents is about agent state, which is also a row, not a vector.