Most people using large language models today are still writing prompts the way they did in 2022 - a sentence or two, maybe a short paragraph, then waiting to see what happens. The models they’re talking to have changed dramatically. The habits haven’t.
Gemini 1.5 Pro shipped with a one-million token context window. Gemini 1.5 Ultra pushed further. Anthropic’s Claude models have supported 200,000 tokens for a while now. These are not theoretical limits buried in a spec sheet - they’re live, usable, billed capacity. You can paste an entire codebase, a year of meeting transcripts, or a 700-page PDF into a single prompt and the model will work with all of it at once.
And yet the dominant use pattern remains: short prompt, short response, repeat.
Why the Gap Exists
Part of this is friction. The chat interfaces most people use - ChatGPT, Claude.ai, Gemini - are still visually designed around conversation threads. There’s no obvious affordance that says paste your entire project here and ask a holistic question. The text box is small. It expands, technically, but nothing about the UI suggests you should dump 80,000 words into it.
Part of it is also a holdover from when context length was actually a real constraint. The original GPT-3 had a 4,096 token limit. People learned to work around that - breaking tasks into chunks, summarising earlier context manually, structuring prompts to fit. Those habits calcified.
The third factor is cost opacity. Token pricing still confuses most users, and the fear of accidentally spending a lot on a single massive prompt is real, even if the actual numbers are often modest. This is a reasonable thing to be cautious about, but it’s keeping people from using capacity they’re already paying for in subscription tiers.
What’s Actually Possible Now
Long-context models are genuinely better at tasks where the full picture matters: code review across multiple interdependent files, legal document analysis where clause relationships span hundreds of pages, competitive research where you want synthesis across dozens of sources simultaneously rather than sequential summaries.
The limitation isn’t the model anymore. It’s that nobody trained the average user to think differently when the ceiling went from 4K to 1M. The demos exist. The technical write-ups exist. The default behavior didn’t shift.
That’s less a criticism of users than an indictment of how these tools have been presented. A million-token context window, marketed primarily as a benchmark number, is still mostly a benchmark number.