The race to expand context windows was supposed to democratise deep analysis - feed an entire codebase to a model, drop in a year of meeting transcripts, ask it to synthesise everything at once. Google’s Gemini models have supported context windows of one million tokens or more since 2024. Anthropic’s Claude has offered comparable capacity. The capability exists. What hasn’t arrived is any coherent product thinking about what people should actually do with it.
Most AI tools today still behave as though they’re optimising for short interactions. The default UI for nearly every major AI assistant is a chat box. You type a question. You get an answer. Maybe you paste a document. The metaphor is conversation, not analysis - and conversation doesn’t scale to a million tokens in any natural way. Nobody talks to a corpus.
There’s a specific gap here that doesn’t get discussed enough: retrieval-augmented generation has become the dominant architectural pattern for enterprise AI precisely because long-context models turned out to be harder to use than to build. RAG systems chunk documents, embed them, and fetch the relevant pieces at query time. It’s a workaround for a product design problem that was supposed to be solved by raw context capacity. The fact that RAG is still the default in 2026 is telling.
The Attention Problem Is Real
Part of this is technical. Research has repeatedly shown that transformer-based models tend to weight information at the beginning and end of a long context more heavily than material in the middle - sometimes called the “lost in the middle” problem. Feeding a model a 500-page document doesn’t guarantee it processes all 500 pages with equal fidelity. So long context isn’t a free lunch, and product teams building on top of these models know it, which is why they don’t lean on it.
What Actually Gets Built
The applications where long context is genuinely being used well tend to be narrow: legal contract review, pharmaceutical research, specific code review workflows. These are high-stakes domains where the investment in custom tooling makes sense. For everyone else, the million-token window is mostly a marketing figure.
What’s missing isn’t the model capability - it’s the interaction paradigm. Nobody has built the equivalent of a spreadsheet for long-context reasoning: something that makes the structure of a large input visible, navigable, and queryable without requiring the user to just trust that the model read it all. Until that interface exists, context windows will keep growing on spec sheets while most people prompt chatbots about their emails.