An AI assistant needs to know the difference between ‘for now’ and ‘from now on’

Imagine a sales rep tells an assistant: “Use blue shirts for today’s order.” A few minutes later, someone says: “This customer now prefers blue shirts.” The words are nearly the same. The system should not treat them the same way.

The first sentence is a one-time instruction. It should help with this order and then get out of the way. The second might be a real customer preference that should help the next sales rep too. If an assistant cannot tell the difference, it either forgets useful context or collects a pile of stale instructions and calls it memory.

The problem was not recall. It was authority.

While building the sales assistant in SpotBook, I kept running into one uncomfortable truth: “memory” is a lazy word. It makes several very different jobs sound like one feature.

• The assistant needs recent messages to understand what “the first one” or “make it 40 instead” means.
• The business needs a reliable record of what happened in a conversation, including failed attempts.
• A customer preference may need to remain useful weeks later.
• Stock, price, customer status, and permission still have to come from the live system right now.

Putting all of that into one giant chat history is tempting. It is also how an assistant starts confusing a past conversation with a current fact.

We gave four kinds of information four different jobs

1. The recent conversation

This is a small moving window of recent messages. In SpotBook, it is ten messages. Its job is continuity, not permanent truth. It lets the model understand a follow-up without making the entire lifetime of a chat part of every request.

2. The permanent record

Every user and assistant message is stored separately as an audit transcript. That helps with debugging, replaying an incident, and understanding whether a turn completed or failed. It is deliberately not dumped back into the model every time someone asks a question.

3. Durable context

This is where a reusable preference can live, such as a customer’s packaging preference, payment terms, or usual carrier. It has a scope: it may belong to one sales user, one customer, or the whole organization. A customer preference should not disappear because one sales rep leaves. It also should not become visible to someone from another tenant just because it sounds relevant.

4. The live business system

This decides what is true now. A remembered note can say that a customer usually buys a product. It cannot prove that the product is in stock, that the price is still valid, or that the caller is allowed to book it.

flowchart TB
  A[Recent conversation] --> E[Assistant response]
  B[Audit history] --> E
  C[Durable memory] --> V[Check scope and access]
  V --> E
  D[Live business system] --> E

Different stores answer different questions. Only the live business system decides today’s stock, price, and permissions.

Useful memory can make an answer more relevant. It cannot make an outdated fact true.

“For today” must not silently become “from now on”

This is the line that makes agent memory hard. A current instruction may temporarily override a remembered default without replacing it. A genuine correction should replace the old durable record. Two pieces of information can look similar in a chat while having completely different lifetimes.

For example, “Send today’s order without the usual packaging” changes this one task. “This customer no longer wants the usual packaging” is a durable correction. The architecture needs both a temporary override and a way to mark the older durable memory as replaced.

We call that replacement supersession. The old memory is not magically erased from every derived system. It becomes inactive and points to its replacement. That matters because search results can lag behind a correction.

Search can suggest a memory. It should not approve one.

We use semantic search, often called vector search, to find stored notes that appear related to a request. Think of it as a librarian who can point to a shelf based on meaning rather than exact wording. That is useful, but a librarian should not also be the person who approves a sale.

Search returns candidate memory IDs. SpotBook then reads the actual relational records and accepts only the ones that are still active, belong to the authenticated tenant, match the allowed scope, and refer to an authorized customer. A missing, inactive, or foreign-tenant record has zero authority, even if search thought it was a perfect match.

Search proposes. The database validates. The live backend decides.

There is an important consequence. If semantic search is unavailable, the assistant can continue with no long-term memory and say less. It must not quietly trust old search data or invent a substitute. Losing optional recall should reduce helpfulness, not weaken the rules around truth and access.

The conversation ID is not a permission slip

A conversation ID helps find the current discussion. It does not prove who is allowed to see a customer preference. Durable memory retrieval is tied to authenticated server-side identity: tenant, user, and role. The chat can tell the system what someone is following up on. It cannot decide what they are authorized to retrieve.

One subtle case made this real. A request could identify one customer clearly and leave another name ambiguous. The assistant may keep the partial result to ask a better clarification question, but it should not use customer-specific durable memories while the request is still unresolved. Partial understanding should grant partial authority, not full authority by accident.

Why we do not save every sentence

A full transcript is evidence, not automatically durable knowledge. Saving every sentence would preserve one-off instructions, contradictions, noise, and possibly sensitive context. Instead, a background process can propose a memory operation, while application rules check its tenant, scope, subject, and whether it replaces something older before changing durable memory.

The technical machinery includes queues, retries, and duplicate protection so a repeated job does not create two copies of the same durable fact. The important idea is simpler: a model may notice that something looks worth remembering, but it does not get to make that decision alone.

The hard part was giving memory less power

The interesting question was never “Which vector database should we use?” It was “What is each piece of remembered information allowed to mean?” Once recent context, audit history, durable knowledge, authorization, and live business facts were separated, the assistant became easier to reason about and safer to extend.

An AI assistant should remember enough to be useful. It should forget enough to stay honest. And when it does remember, it should know whether that memory belongs to this moment, this customer, this organization, or nobody at all.