4. Making It Think Before It Answers

"Let's think step by step" is the most repeated prompting advice in existence. It has a paper behind it, it is in every guide, and it genuinely transformed what models could do when it was discovered.

We tested it on two problems. On both, the model got the right answer without being told to think. This lesson is about what that means, and about what step by step is actually still for, which turns out to be something other than accuracy.

Test one: a shopping problem with a trap

Multi-step arithmetic with an optimization buried in it, which is the classic case where models used to fail:

A shop sells notebooks in packs of 7 for $12 and singly
for $2. I need 30 notebooks. I have a coupon that takes
15% off the total before tax. Tax is 8.25%.
What is the cheapest I can pay? Give me just the final
number.

Note "give me just the final number", which actively discourages working out loud. The answer came back as three characters:

$47.85

We checked it by hand. Four packs of 7 plus 2 singles is 30 notebooks, at 4 × $12 + 2 × $2 = $52. Fifteen percent off is $44.20. Adding 8.25% tax gives $47.8465, which rounds to $47.85. Correct, including the non-obvious part: buying 5 packs would give you 35 notebooks for $60, which is more notebooks and more money, and it did not fall for that.

Test two: a rate problem with a tempting wrong answer

A tap fills a tank in 6 hours. A drain empties a full tank
in 10 hours. The tank is currently one third full. Both
are open. How long until the tank is full?
Answer with just the number of hours.

The trap is the "one third full", which people and models both routinely ignore, giving 15 hours for a full tank instead of 10 for the remaining two thirds. The answer came back as two characters:

10

Correct. Net fill rate is 1/6 minus 1/10, which is 1/15 of a tank per hour. Two thirds remaining divided by 1/15 is exactly 10 hours.

Problem	Told to think	Answer	Correct
Notebooks	No, discouraged	$47.85	Yes
Notebooks	Yes, full working	$47.85	Yes
Tank	No, discouraged	10 hours	Yes

What we are not saying

We are not saying the technique is obsolete. Two problems on one model is a small sample, and there is a large body of work showing this mattered enormously on earlier models. What we measured is narrower and more useful:

On a current model, for problems of everyday difficulty, telling it to think step by step did not change the answer we got. It changed whether we could check the answer.

That distinction is the whole lesson, because checking is the part you actually need.

What step by step is genuinely for

When we ran the notebook problem with the instruction to show its work, we got the same $47.85, but we also got this:

Packs	Singles	Subtotal
2	16	$56
3	9	$54
4	2	$52
5	0	$60

Now you can audit it. You can see it considered buying whole packs, see where the cost turns back upward, and satisfy yourself in five seconds that $52 is really the floor. The bare "$47.85" gives you a number to trust or not trust, with nothing in between.

That matters because of what we found in the Super Beginner course: asked to explain mortgages, the same class of model told us a 15 year $400,000 loan at 7% costs $20,000 in interest, when the real figure is $247,156. It was off by a factor of 12 and it stated it as flatly as it stated $47.85 here. Confidence is not a signal. Working is.

How to ask, and it is not the famous phrase

"Let's think step by step" is vague. These are better because they name the steps:

Work through it step by step. Compare every combination
that gets to at least 30, show the subtotal for each, then
apply the discount and tax to the cheapest. Final number
last.

That is the prompt that produced the table above. It did not just ask for thinking, it specified which thinking: enumerate, then compare, then apply. Naming the procedure is what turned a number into something checkable.

Two more shapes worth having:

Before you answer, list the assumptions you are making.
Then answer.

Best on anything underspecified. The assumptions list is usually where you discover you left out a fact that changes everything.

Give me the answer first, then the working underneath.

Best when you mostly trust it and want to skim. You get the number immediately and the audit trail is there when a figure looks wrong.

The cost, which is real

Working takes words, words take time, and long working buries the answer. Our bare notebook answer was one line. The worked version was 172 words and a table.

So scale it to the stakes, exactly as with checking:

Situation	Ask for working
A number you will glance at	No
Anything you will paste into a document	Yes
Anything with money, dates or a decision attached	Yes, and read it
A task with no right answer, like writing	No, it just adds preamble

Where it still matters, on our own evidence

Our two problems were self-contained: all the facts were in the prompt and the arithmetic was a handful of steps. That is where we found no difference. The picture changes on three kinds of work, and we have hit all three while writing this course.

When the facts come from somewhere else. The mortgage answer that was wrong by a factor of 12 was not a reasoning failure. It was a recall failure: it produced a number that looked right and attached it to the wrong label. Working exposes that, because a stated formula and a stated result can be compared. A bare figure cannot be checked against anything.

When the problem has a step you did not think of. The value of "list your assumptions before answering" is not the answer, it is the list. On anything underspecified, that list is where you discover the model has quietly picked a currency, a time zone, a tax treatment or a definition, and picked differently from you.

Before you answer, list every assumption you are making
and every piece of information you would need to be sure.
Then answer under those assumptions, flagging which ones
change the result most.

The last clause is the one that pays. It sorts the assumptions by how much damage each does if wrong, which tells you the single fact worth going and checking.

When you are going to be asked to defend it. If the number ends up in front of someone who will question it, you need the derivation regardless of whether the derivation improved the number. That is not a prompting concern, it is a work concern, and it is the most common real reason to ask.

The check that costs nothing

Whether or not you asked for working, this follow-up takes one line and catches the category of error the bare answer hides:

Show me the calculation behind that, and state any figure
you had to assume rather than derive from what I gave you.

The second half is the useful half. On our notebook problem there was nothing to assume, since every number was in the prompt. On a real question there usually is, and an assumed figure presented in the same tone as a given one is exactly how a wrong number survives review.

Common mistakes

1. Adding "think step by step" to creative work. There is no procedure to expose. You get a paragraph about its approach and then the same draft.
2. Asking for working and not reading it. Then you paid the words for nothing. If you are not going to read it, ask for the bare answer and check the one number that matters.
3. Believing the working proves the answer. It can produce plausible-looking steps to a wrong conclusion. Working makes errors findable, not impossible.
4. Using the famous phrase and nothing else. Naming the actual steps, as we did above, is what produced the auditable table.

Try it now

Take a calculation you actually need: a budget, a rate, a comparison between two options. Ask for the bare number. Then in a fresh chat, ask for the same thing with the steps named, and check the working against the answer you already have.

If they agree, you have learned that this class of problem is safe to ask bare. If they disagree, you have found something considerably more valuable.

Next

Lesson 5 puts lessons 1 through 4 together and builds one real prompt from an empty page, start to finish.