Context Window: The Token Limit for One AI Model Call

The most a model can handle in one call

A context window is the maximum number of tokens a model can handle in a single call, including input and output. A model call is one request to the model to generate a response: a simple chat reply can take one call, and using tools can involve several. Tokens are the chunks of text a model reads and writes, such as whole words, parts of words or single characters (here’s how text becomes tokens).

The input is everything supplied for that call: the instructions, your message, any earlier messages that are included, passages from documents and results from tools. Images and documents take room too. The output is the answer the model writes, and it comes out of the same limit.

So the window is a capacity, and a short question doesn’t get a smaller one. It just fills less of it. The model can also draw on what it learned during training, but that learned knowledge is different from the material supplied for this call.

What goes in, and what’s left for the answer

Take a simplified window of 100 tokens. If everything sent in comes to 9 tokens, the answer has at most 91 left. Real windows are far larger, but the arithmetic is the same. Models differ in how they set input and output limits, so check both.

In a simple chat, the app sends the model its instructions, your new message and whatever earlier conversation it includes for this reply. Each call stands on its own, so the model sees earlier messages only when the app, or the provider’s service, sends them again. As more conversation history is included, it takes up more of the window.

Documents work the same way: to use a passage from your files, the model needs that passage in its context. Uploading a file doesn’t prove all of it got there, because some apps find the passages that look relevant and send only those.

The answer takes room too, and on models that reason before they answer, so does the reasoning. When a call would go past the limit, the product decides what happens. An input that’s too long can be refused, an answer can stop or be cut off at the limit, and a chat app may drop the oldest material or replace it with a summary. The window sets the limit, and the app decides what to remove.

Picture a whiteboard of fixed size, prepared afresh for each call. The app writes on it whatever this call is given, and the model writes its answer in the space that’s left. Earlier notes are back on the board only if the app copies them over, and to make room, an app may erase older notes or replace them with a summary.

Capacity, contents and reliable use

A window’s size and what a call contains are two different things, so a vendor quoting a window size is quoting the maximum, not what your tool actually sends.

The window covers what’s supplied for this call and what the model generates. Earlier conversations aren’t automatically included, and stored memory, which some tools have, sits outside the window until it’s brought into a call.

And fitting isn’t the same as being used well. One model maker’s documentation says that as the token count grows, accuracy and recall degrade. For long inputs, provider guidance recommends putting documents before the question, which can improve response quality. A summary made to free up room can lose a detail whose importance only shows up later, which is why a retention policy for AI meeting agents keeps the original record within reach.

That’s where the whiteboard misleads. On a board, every note is just as easy to read as the next, while a model may use parts of a long input less reliably. And a window’s size is a count of tokens rather than an area, so the same page can take more or less room depending on the language and the model.

Forty contracts and a missing clause

Here’s a hypothetical. Your team uploads forty supplier contracts into an AI tool and asks for every auto-renewal clause. It comes back with a tidy list of renewals. The clause in contract twenty-two isn’t there, and nothing on the screen says something’s missing.

The window gives you three useful checks. Did all forty contracts reach the model, or only the passages the tool picked? If they all reached it, was the clause used reliably in such a long input? And if this came late in a long session, did the app drop or summarize earlier material? The answer alone doesn’t tell you which happened, and these checks don’t rule out other causes.

What “reads all your documents” really claims

When a tool says it reads all your documents, ask how. Does it send everything in one call, retrieve selected passages, or work through the documents across several calls, and how does it check that nothing was skipped? Ask too for the model’s context window and for any separate cap on how long an answer can be. Then test whether it uses what it gets reliably, because no vendor answer settles that.

Our recommendation, a method and not a research standard: test the tool on documents where you already know the answer. Take contracts someone has already checked by hand, load as many as you’d really use, and ask for every clause with its contract and section. Decide up front what passing means: every known clause found and none invented, in each of a few fresh conversations. If it misses a known clause, don’t rely on it as the only check for that work. A pass supports this use on documents like these, so keep checking as your documents or the tool change. And if you need knowledge to carry from one conversation to the next, see where your AI agent keeps what it learns.

Pinning down the vendor’s answers

1. Does the tool send our full documents in one call, retrieve selected passages, or work through them across several calls, and how does it check coverage?
2. What’s the model’s context window, and is there a separate limit on how long an answer can be?
3. Have we tested it on our own documents, at our real volume, against answers we already know, with the pass condition set in advance?
4. What does the tool put in the window before we ask anything, and what happens to earlier material when a long session fills it?