Context Window
The maximum number of tokens (words and punctuation) a language model can process at once in a single prompt plus response.
What Is Context Window?
The context window is the "working memory" of a language model — the maximum number of tokens it can attend to at one time. Everything the model knows about the current conversation or task must fit inside this window: the system prompt, the conversation history, retrieved documents, the user's question, and the model's response. Tokens that fall outside the window are effectively invisible to the model.
Context windows have grown dramatically: early GPT-3 had 4k tokens; GPT-4 Turbo extended this to 128k; Gemini 1.5 Pro to 1 million tokens; and Claude 3.5 supports 200k tokens. A 200k context window can hold roughly 150,000 words — about the length of a full novel.
Why It Matters
Context window size determines what kinds of tasks an AI can perform in a single call. A small context means you must summarise or truncate long documents before processing them, losing information. A large context enables processing entire codebases, legal contracts, or research papers in one shot. However, longer contexts also increase inference cost and can degrade model performance at very long lengths ("lost in the middle" phenomenon).
Real-World Examples
Codebase Analysis
A 200k-token model can read an entire medium-sized codebase in one call and answer questions about any part of it.
Long Document Summarisation
A 128k-token model can process a full annual report (typically 50–80 pages) without truncation.