Context Windows: Why AI Forgets the Beginning of Long Conversations

You're working through a complex medication question with an AI assistant. You started by explaining your mother's kidney disease, her recent medication change, and her allergies. Fifteen minutes later, you ask a follow-up question, and the AI seems to have forgotten she has kidney disease and starts suggesting things that aren't safe for her condition.

This isn't the AI being dumb. It's hitting a limitation called the context window—think of it as the AI's short-term memory limit for any single conversation.

What's Actually Happening

Every AI model can only "see" a certain amount of text at once. This limit is measured in tokens (roughly equivalent to words or small chunks of text). Claude can remember up to 200,000 tokens in one conversation. ChatGPT's free version? About 4,000 tokens. Google Gemini sits somewhere in the middle.

Here's the practical translation: if you've already written 3,000 words of background information before asking your question, and your AI tool only has a 4,000-token window, you're using 75% of its available memory just on context. Any nuance gets dropped.

For caregivers, this is particularly frustrating because your situations require context. You can't safely ask about pain management without explaining the patient's kidney function, current medications, and previous reactions to opioids.

Why This Matters

In caregiving, losing context is dangerous. An AI might give you technically accurate information that's wrong for your specific person. The solution isn't to find an AI with infinite memory (they don't exist yet). It's to structure your conversations strategically.

The best approach: create a context document—a single, comprehensive summary of the person's medical situation that you paste at the start of every conversation. Instead of repeating details across multiple questions, you say, "Here's the full context (paste document), and here's my specific question about today."

Common Misconception

Many people think larger context windows always mean better care coordination. Actually, how you structure information matters more than how much the AI can technically hold. A well-organized 2,000-token context document beats a scattered 20,000-token rambling.

Try this: Create a one-page medical snapshot document for each person you care for. Include: primary conditions, medication list, recent changes, allergies/reactions, and kidney/liver function status. Before your next AI conversation about care decisions, paste this document first, then ask your specific question. Notice how much more relevant the answers become.