Context window

What it is

Think of the context window as the model’s desk. Everything it can use for this answer has to fit on the desk at once. Anything not on the desk, the model can’t see.

How it works

Each request bundles instructions, attached files, prior messages and the space reserved for the reply. The total is counted in tokens. If the bundle exceeds the window, something has to be cut, summarized or retrieved selectively (see RAG). Chat apps re-send the conversation on every turn, which is why long chats eventually lose their early details.

Sizes (checked 2026-10-17):

• Claude API: 1M tokens on current models.
• OpenAI API: its flagship lists about 1.05M tokens.
• Claude paid apps: 500K to 1M tokens, depending on the model.
• ChatGPT: the Instant model’s window is 27K tokens on Free, 54K on Go and Plus, and 128K on Pro.

Benefits

• Whole-document work. Contracts, transcripts and reports can go in complete, so nothing is silently left out.
• Fewer moving parts. Andrej Karpathy has said he’d rather have all the material in the context window than trust a retrieval layer to choose for him.
• Simpler checking. If everything was given, you know what the model saw.

Limits and risks

• Cost scales with input. Every token in the window is billed on every request.
• Big isn’t the same as well-read. Research such as Lost in the Middle (Liu et al.) found that models used information at the start and end of long inputs better than information in the middle (see context rot).
• Windows differ by model and plan. App plans can allow far less than the API maximum.