Why Fresh Conversations Beat Long Ones

TL;DR;

  • Context windows are limited working memory - Every message adds to the AI’s context window, a finite space that fills up faster than you think, causing performance degradation.
  • Bigger isn’t always better - Even models with millions of tokens suffer from “lost in the middle” syndrome, where they lose track of information buried in long conversations.
  • Context rot makes AI less effective over time - As sessions grow longer, the AI gets overwhelmed by information volume, not less intelligent, leading to degraded responses.
  • Three simple habits keep AI sharp - Place key instructions at the end of prompts, start fresh sessions frequently, and ask the AI to summarize context before beginning new chats.

Every time you send a message to an AI assistant, you’re adding to something called the context window. Think of it as the AI’s working memory—a limited space where everything from your conversation gets stored. As that space fills up problems start to emerge and this can happen surprisingly quickly.

Context windows are measured in tokens, roughly equivalent to word fragments. Different models have different limits: some handle a few thousand tokens, others can stretch into the millions. But bigger isn’t always better. As conversations grow longer, models struggle to maintain focus. Research has identified a “lost in the middle” phenomenon: models perform best with information at the beginning and end of the context, while content in the middle gets overlooked. Different models handle long contexts differently—some are optimized for longer contexts while others work best with shorter, focused sessions. Your careful instructions from twenty messages ago? The model may have effectively lost sight of them.

This degradation—sometimes called “context rot”—means the AI gradually becomes less helpful during extended sessions. It’s not getting dumber; it’s getting overwhelmed by the sheer volume of information it’s trying to track. As context fills up, the risk of hallucinations increases because the model’s pattern recognition weakens.

The practical solutions are straightforward:

  • Remind the LLM of key points at the end of your prompt, where the model pays closest attention
  • Start fresh sessions frequently rather than pushing one conversation to its limits
  • Ask the AI to summarise key points and decisions into a prompt you can use to seed a new chat

Combine these context management habits with clear, structured prompts for maximum reliability. These simple habits keep your context clean and your AI sharp.

Things to try

Try these techniques in your next AI conversation to see the difference fresh context makes.

  1. Test the Reminder Technique: Open a long conversation with your AI assistant. Scroll up and copy an important instruction from early in the chat. Paste it at the end of your next prompt and notice how much better the AI follows it.

  2. Start a Fresh Session: If you’ve been working on something complex for more than 10-15 messages, ask the AI to summarize the key decisions and context. Copy that summary, start a new chat, paste it in, and continue your work. You’ll likely notice sharper, more focused responses.

  3. Compare Models with a Long Document: Paste the same lengthy document—meeting notes, a report, or a code file—into Claude, ChatGPT, and Gemini. Ask each one to find a specific detail buried in the middle. You’ll quickly see which models handle “lost in the middle” better than others.

Want to know more?

Context windows sound huge—hundreds of thousands of tokens—but context rot kicks in well before you hit those limits. So how much can you actually use?

“Roughly what percentage of a context window can I use before an AI starts losing track of earlier information?”

The answer might surprise you—and explains why starting fresh sessions often beats pushing a long conversation further.