7. Documents Too Long to Read

Pasting a long document and asking questions is the use case that converts sceptics. It is also where the folklore is thickest: that models lose track of the middle, that you must chunk everything, that long inputs degrade badly.

We went looking for that. On a 260 line document we could not reproduce it. This lesson covers what we measured, the technique that makes long document work checkable anyway, and where the real limits actually sit, because they are not where people think.

The test: hiding one fact at three depths

We generated a 260 line document of plausible filler, all of it interchangeable, and planted a single specific fact in it:

The Riverbend contract renewal fee was set at $41,750
for the 2027 term.

Then we built three copies with that line at position 9, position 131 and position 251, and asked each the same thing: what is the fee, and which numbered line states it. Asking for the line number is the part that makes the result checkable rather than a vibe.

Fact planted at	Found the figure	Line number
Line 9, near the start	$41,750	Correct
Line 131, the middle	$41,750	Correct
Line 251, near the end	$41,750	Correct

Three for three, exact line numbers, no hedging. Whatever "lost in the middle" meant when it was written, it did not show up at this length on this model.

The honest caveat, same as everywhere in this course: one document, one model, three trials. 260 lines is a long email thread or a short contract, not a 400 page annual report. We are telling you what we measured, not that the effect never exists.

The instruction that makes any of this trustworthy

Notice what made the test above verifiable. We did not ask "what is the fee". We asked for the fee and the line that states it. That is the whole technique for long documents, and it applies whether the model is reliable or not:

Answer only from the document below. Quote the exact
sentence each answer comes from and give its section or
line number. If something is not stated, say "not stated
in this text" rather than inferring it.

Three clauses doing three jobs. "Only from the document" stops it answering from general knowledge about how contracts usually work. "Quote and cite" turns every answer into something you can check in five seconds. "Not stated" gives it an honest exit instead of a plausible guess, and the gaps it reports are usually the thing you needed to know.

Where the real limits are

Having failed to find the famous problem, here is what does actually go wrong, from doing this on real documents.

Tables and PDFs arrive scrambled. This is the big one and almost nobody warns about it. A table in a PDF often reaches the model as a stream of cells with the row and column structure lost. The answer then looks confident and maps the wrong number to the wrong label. If your question depends on a table, ask it to reproduce the table first and check that it read it correctly before you ask anything about the contents.

Contradictions inside one document. If clause 12 modifies clause 7 and you ask about clause 7, you get clause 7. Nothing is wrong with the answer except that it is incomplete. Ask explicitly: "is this modified, qualified or contradicted anywhere else in the document?"

Scanned images. Quality varies enormously. A photographed page can lose a decimal point silently. Any number extracted from an image gets checked against the image.

Counting and completeness. "How many times does X appear" and "find every instance of Y" are the questions to distrust most. Ask for the list with quotes rather than the count, then count the list yourself.

What to do when the document really is too big

Modern context windows are large, but they are not infinite and cost scales with what you send. Three approaches, in the order to try them:

Ask for locations before answers. Cheapest and most reliable.

Do not answer yet. List the sections that are relevant to
[question], with their headings and line ranges.

Then paste only those sections into a fresh prompt. You have used the long document as an index rather than as the working material.

Split by natural boundary, not by length. Chapters, sections, dates, one document per file. Splitting mid-argument produces two halves that each make no sense.

Summarize each part, then work from the summaries. The classic approach and the lossiest. Anything specific you did not anticipate is gone from the summary. Use it for "what is the overall picture", never for "what does it say about X".

Five questions worth asking any long document

1. Summarize this in 5 bullets, with a line reference each.
2. What in here would cost me money, and when?
3. What are my obligations, as a list with deadlines?
4. What is unusual compared to a standard version of this?
5. What is NOT covered that I might assume is?

Four and five are the ones people never think of and the ones that pay. Question 4 finds the clause someone inserted. Question 5 finds the assumption that ruins your year, and it works because you have asked for absences rather than contents, which is a shape the model can look for and your own proofreading cannot.

Asking a document what you should be asking it

The hardest part of a long document is not extraction. It is that you do not know what to look for, which is exactly why you have been avoiding the document.

So ask it that first:

Do not summarize this yet. I am [your situation: signing
this / inheriting this project / deciding whether to
renew]. What are the five questions I should be asking
about this document, and why each one matters to me?

Then ask the questions it gives you. This inverts the usual order and it is much better than a summary, because a summary tells you what the document says while this tells you what the document means for you specifically.

It is also the honest answer to "I do not know enough to know what to ask", which is the real reason the lease is still unread.

Two documents at once

The comparison job is where this genuinely beats reading, because it is the thing humans are worst at. Two versions of a contract, an old and new policy, your draft against the template:

List the substantive differences between the two documents
below. Ignore formatting and rewording. For each real
change, say what someone is now obliged to do that they
were not before, or no longer has to do.

<version_a>...</version_a>
<version_b>...</version_b>

The clause about obligations is what makes this useful rather than a diff. A wording change that quietly converts "may" into "shall" is invisible in a list of textual differences and is the entire point of reading the document.

Common mistakes

1. Asking a question the document cannot answer and accepting an answer anyway. The "not stated" clause exists for this and it is one line.
2. Trusting a number from a table without checking the table parsed. The single most common real failure.
3. Pasting the document after the question on something very long. Instructions first, so it reads with a purpose, and repeat the question after the closing tag.
4. Treating a summary as a search. Summaries drop specifics by design. If you want a fact, ask for the fact and the citation.
5. Uploading something sensitive without checking your settings. Most consumer AI products let you turn off training on your conversations. Find that setting once. If the document is genuinely sensitive, delete the identifying lines first; it does not need the account number to explain the fee.

What this does not replace

Being clear about the boundary, because it is the difference between a useful habit and a bad one. This gets you through a document you were never going to read at all, and it tells you which three clauses to take to someone qualified. It does not tell you whether to sign.

For anything with real money or real consequences attached, the win is not skipping the lawyer. It is arriving at the lawyer with the specific questions already identified, which turns a two hour engagement into twenty minutes. That is a genuine saving and it is a different claim from the one people usually make about this.

Try it now

Take the longest document you have been avoiding. Paste it with the citation instruction at the top and the five questions at the bottom.

Then take one answer, the one you would actually act on, and go look at the line it cited. That thirty second check is the habit this lesson is really teaching. Everything else is a way of making that check possible.

Next

Lesson 8 opens the box: how the thing actually works, why it fails in the specific ways it does, and why nothing in lessons 1 to 7 is a rule.