What is hot memory in AI agents?

Hot memory is a useful name for the information an AI agent has in front of it for its current task. Here, it is an operational term for the context supplied to the model at a given step. It is not a universal standard or a feature with the same meaning in every product.

That context can include your request, work rules, recent messages, and facts from tools. It can change as the agent works. It is not a fixed set of facts for the whole job.

If an agent gives your team a wrong answer, missing or old context is one possible cause. The sources below do not establish it as the main cause of agent errors.

What goes into an agent's context?

An AI model is the part of an agent that works with the supplied information to produce a response. An agent can use tools and choose steps toward a goal. A fixed workflow follows steps set in advance. Either can use AI to draft an answer; taking an action, such as sending it, is a separate step.

Anthropic calls the work of choosing and managing the model's input context engineering. Its guide includes instructions, tools, outside data, and message history as parts of context. It also explains how agents can fetch new facts as they work. Read Anthropic's context engineering guide.

For a customer service task, useful context might include:

• The customer's message and order details.
• The latest shipment status and when it was checked.
• Your current refund policy.
• A rule to draft a reply and wait for approval.

The model has a limit on how much it can work with at once. This is its context window. Providers measure it in tokens, which are small pieces of text or other input. The window must also leave room for the model's output. Limits vary by model and setup. See Claude's context window docs.

Hot memory, saved history, and training are different

A saved fact is not always a fact the model can see right now.

Current context is what the system supplies for the model's next response. It may include only part of a past chat or a short summary.

Stored memory and history sit outside that current input until the system brings them in. A saved policy file can last across sessions. To use its contents, the system needs to load the relevant text into context. Anthropic's memory tool uses this store-and-read pattern. See how its memory tool works.

AI agent memory labels vary. LangChain, for example, calls memory within one conversation short-term memory. That state can still be saved to a database. Its long-term memory spans conversations. Short-term does not always mean unsaved. Read LangChain's memory overview.

Model training changes the model itself. Adding a policy to its current context does not, by itself, retrain it. A system may save a correction and use it in a later prompt. That is different from a separate training process. OpenAI's guide treats prompting and fine-tuning as distinct ways to improve model output. See OpenAI's model optimization guide.

A late-order example

This is a hypothetical example, not a client case or a tested result.

Say your company uses an agent to draft replies about late orders. A customer asks when an order will arrive. The agent has an old note that says Friday. The carrier now says Monday.

If the system supplies only the old note, the agent may draft a reply with the wrong date. Saving the Monday update in a customer record does not mean the model has seen it.

Your team could give the agent this task:

Draft a reply about this order. Check the latest carrier update first. Include when you checked it. Do not promise a refund. If the records disagree, flag the conflict for our team. Do not send the reply.

Then check the tool result and the draft. Did it fetch the right order? Did it use the new date? Did it stop before sending?

This tests whether the right facts reached the model and whether it followed the work rules. It does not prove the agent will handle every future order correctly.

Keep context useful, not just short

A short prompt can still leave out a key fact. A long prompt can include old rules that clash with new ones.

Anthropic advises teams to keep context focused, but says minimal does not have to mean short. Keep the details needed for the job. Remove repeat text and facts that do not help. When you shorten a chat or file, check that you kept the limits and exceptions that matter.

Concision does not guarantee exact behavior. OpenAI notes that model output can vary. Clear input still needs tests and review.

A checklist for your team

Use one task your team knows well. Ask whoever manages the agent to show what the model actually received, not just what your company has saved.

• State the job and what a correct result should include.
• Check that the agent used the right customer or work record.
• Check source dates and replace old facts or rules.
• Confirm that needed saved facts reached the current context.
• Remove duplicate text without cutting key exceptions.
• Set what it may read, draft, change, or send.
• Have tools enforce approval rules; do not rely on prompt wording alone.
• Test a normal case, a missing fact, and conflicting records.
• Review the result before allowing a consequential action.

Context is a useful place to look when the answer seems based on old or missing facts. It is not the only place. Also check the task itself, tool access, failed tool calls, and the model's limits. Anthropic's agent guide stresses clear tools, tests, and human checkpoints. Read its guide to building effective agents.

For owners exploring Core Agentic, this is a useful question to bring to a workflow discussion: which facts should the agent see for this job, and who keeps them current? A saved history alone does not answer that question.