LLM API Context Window Comparison

The token count on the spec sheet is the number vendors want you to remember. It’s also the least useful number when you’re deciding which provider to trust with a task that genuinely needs to hold a lot of information at once. A 2-million-token window that degrades to noise past 200K is worse than a smaller window that stays sharp all the way to its limit. This comparison focuses on how models actually use context — not just how much they advertise.

What context window sizes look like across providers today

The raw numbers, as of mid-2026:

• Google Gemini 3.1 Pro — 1M tokens standard, 2M on enterprise Vertex AI tiers
• Anthropic Claude 5 / 4.x — 1M tokens across the full lineup (Opus 5, Sonnet 5, Fable 5, and the 4.x series), with Haiku 4.5 at 200K
• OpenAI GPT-5.5 / GPT-5.6 — 1M tokens on GPT-5.5 and GPT-5.4; GPT-5.6 advertises 1.05M but charges a surcharge past 272K
• DeepSeek / Qwen / Llama — DeepSeek V4.1 and Qwen 3.7 hit 1M; Llama 4 Scout claims 10M

For a current, maintained breakdown of exact input and output limits per model, the community-run llm-context-limits repo and the Morph context window comparison are more reliable than any single vendor’s marketing page. The context window growth timeline shows how quickly these numbers have inflated — from 4K in early GPT-3.5 to 10M (Llama 4 Scout) in under four years.

But sizes are the easy part. The interesting question is what happens at the top of that range.

Does a bigger context window actually mean better performance?

No — and this is the single most important thing to internalize before you pick a provider based on token count.

The failure mode has a name now: “context rot,” a term that emerged on Hacker News and was later formalized in a widely cited Chroma study in July 2025. The core observation is that models get less effective as their context fills up. As one HN commenter put it: “the more context it has, the less it weights individual items.” You can hand a model 500K tokens, but its ability to reason over any one of them drops as the window saturates.

The related and better-known problem is “Lost in the Middle” — information placed in the center of a long context gets retrieved far less reliably than information at the start or end. Benchmarks have measured accuracy drops of over 30% when the relevant fact sits in the middle of a large prompt. For RAG pipelines this is brutal: you can build a retrieval system that correctly surfaces the right chunk, place it at position #3 of 5, and still watch the model answer from chunks #1 and #5 with something plausible and wrong.

So the practical framing is: advertised windows are close to fiction at scale. A model supporting 1M tokens does not mean a model that reasons well across 1M tokens.

Which provider handles long-context retrieval most reliably?

If your task is straightforward retrieval — needle-in-a-haystack factoid lookup across a huge document — Gemini is the clear leader. Google’s own research on Gemini 2.5 Flash found it can answer needle-in-a-haystack questions “with great accuracy regardless of document position,” effectively neutralizing the Lost in the Middle effect for simple factual Q&A. When dropping an entire codebase, a long spec, or a set of transcripts into one call and asking targeted questions, Gemini is the one worth trusting to actually find the thing.

The caveat is the word simple. Positional robustness on factoid retrieval does not automatically translate to positional robustness on multi-hop reasoning across a saturated window. Gemini’s edge is retrieval reliability at length, not immunity to quality degradation as the window fills.

Where Claude wins on context quality

Now that every major provider ships a 1M-token window, the differentiator has shifted from size to what the model does with what you put in it. Claude Opus 5 leads SWE-bench Verified at 96%, with Gemini 3.1 Pro at 80.6% — a gap that widened compared to the previous generation. The takeaway: the model that uses its context best for your specific task matters more than the model with the biggest context.

This aligns with what the creator of aider posted on Hacker News — that context handling is “perhaps the #1 problem users have,” and that very large context “seems to lure users into a problematic regime” for coding. Coding is dense, high-value context where every token matters. Stuffing a 1M-token window with your whole repo is exactly the wrong instinct; you get worse output and pay for every one of those tokens.

The cost trap nobody puts on the spec sheet

Every token you put in the window is a token you pay for on input, on every single call. Redis engineers framed it well: treating the window “as a hard limit to fill misses the point,” because longer contexts both cost more and hurt reasoning quality.

Run the math on a genuinely long-context workflow. A 500K-token input at typical GPT-5.5-class pricing can cost several dollars per request, and if you’re calling it in a loop or a multi-turn agent, that compounds fast. This is where OpenAI’s conversation state guidance matters: managing state deliberately (truncating, summarizing, offloading to retrieval) is a first-class engineering concern, not an afterthought. The window isn’t a bucket you fill; it’s a budget you spend.

The factor that actually tips the decision

When choosing a provider for a task that needs a large window, the token count is rarely the deciding factor. What tips it is retrieval reliability at depth combined with cost-per-effective-token — how much usable attention you get per dollar, not how many tokens the model will nominally accept.

Put concretely, my decision heuristic:

• Massive-document factoid retrieval, position-agnostic → Gemini. Its long-context robustness and 2M enterprise tier are the real differentiators.
• Coding and dense reasoning over curated context → Claude. Same 1M window as the competition, but it uses what you give it far better — and you shouldn’t be filling 1M tokens for code anyway.
• Cost-sensitive high-volume workloads → the answer depends on how aggressively you can trim context; a 128K model you keep 30% full often beats a 1M model you keep 70% full on both quality and price.

I’ve gone deeper on provider trade-offs beyond context in our breakdown of the best LLM APIs for developers, which covers latency, tooling, and pricing tiers alongside these capabilities.

Context engineering beats context stuffing

The consensus that has taken hold since Shopify’s Tobi Lütke coined the term context engineering in mid-2025 is that the winning move is assembling exactly the right context per task, not brute-forcing the maximum. This is why RAG isn’t dead. In an era of million-token windows, retrieval becomes more valuable, not less, because it acts as the curator deciding what deserves a spot in an expensive, degradation-prone window. The future isn’t RAG versus long context; it’s RAG feeding a deliberately managed context.

The verdict

Stop comparing context windows by their advertised ceilings — that number is marketing. The real comparison is: does the model retrieve reliably across depth (Gemini wins), does it reason well over dense curated context (Claude wins for code), and what does effective context cost you per call (always run the math)?

Choose the provider whose failure mode you can engineer around. Then keep the window as empty as your task allows. A smaller, sharper context beats a bigger, rotting one every time — and your accuracy numbers and your bill will both reflect it.