Context Windows: Why Long Health Questions Get Worse Answers

Imagine trying to explain a complicated medical situation to a doctor, but there's a strict rule: you get exactly 500 words to tell your entire story, including your symptoms, medical history, medications, and what you need help with. That's roughly what a context window is—the amount of information an AI can consider at one time.

Every AI tool has a limit to how much text it can hold in its "working memory" at once. Think of it like the capacity of a notepad. ChatGPT can hold roughly 4,000 words in a conversation (the "4k context window"). Claude can hold roughly 75,000 words (the "100k context window"). Older or lighter AI models might only hold 2,000 words.

Here's why this matters for healthcare: medical situations require context. If you're asking AI "should I be worried about this symptom?" the quality of the answer depends on whether AI knows about your diabetes, your medications, your allergies, previous similar episodes, and what you're already concerned about. If you try to cram all of that into a context window that's too small, AI has to pick and choose what to keep—and it might drop important details.

A concrete example: You paste your entire medical history (medications, conditions, surgeries, family history) plus your three current health concerns plus your latest test results—maybe 2,000 words total. You ask AI to help you think through whether you need to see a doctor urgently. Now you've used up most of a smaller context window, and when AI generates its response, it has to be brief because there isn't much room left for it to "think" through your question.

Result: shorter, less detailed answers than you'd get with a larger context window where AI has space to reason through complex scenarios.

The practical implication: be strategic about what you include. Don't paste your entire medical record for one simple question. But if you're tackling something complex (multiple symptoms, multiple conditions, medication interactions), use an AI tool with a larger context window like Claude, which has room to really think through your situation.

Also, know that context windows reset between conversations. If you've had a long conversation with lots of medical back-and-forth, starting a new conversation means you lose all that context and need to reintroduce it.

Try this: Ask the same complex health question to both ChatGPT (smaller context window) and Claude (larger context window) with the same detailed medical background included. Notice whether Claude's response tends to be more thorough or considers more nuances.