Tutorial1 hour ago

5. Learning Something New Without Getting Lied To

We asked it to explain mortgages and it got the interest math wrong by a factor of 12. In the same answer where we asked what it was unsure about, and it did not flag it.

The WJS Desk

Sep 3, 2026 ยท 7 min read

Photo by Yan Krukau on Pexels

Using AI to learn something is the use case with the best upside and the sharpest edge. The upside is a patient tutor who will answer the same question 4 different ways at 11pm and never make you feel stupid. The edge is that when it is wrong, it is wrong in exactly the same confident tone as when it is right, and you are learning, which means you have no way to tell.

We are going to show you both in one real example, because we ran into the failure by accident while writing this lesson.

Say where you are starting from

"Explain machine learning" gets you a Wikipedia paragraph. The fix is to tell it what you already know, because that is what a good teacher works from.

Explain how a mortgage works. I have never owned
property. I do understand basic interest and I have
had a car loan. Use that as the starting point and
build from there.

The clause "use that as the starting point" is what changes the output. It now has an anchor to explain against, so you get "a mortgage is like your car loan except" rather than a definition you already half knew.

Two more shapes worth stealing:

Explain [thing] to me as if I know nothing about it
but I am not stupid. No analogies to cooking or
baking. Use one worked example with real numbers.

Banning the cooking analogy is only half a joke. Left alone it reaches for recipes and restaurants for everything, and those analogies stop working about 2 steps in.

I read that [claim]. Explain what that actually
means, why someone would say it, and who disagrees.

That third one is the best learning prompt on this page. "Who disagrees" forces it to show you the shape of the argument instead of the consensus summary, and for anything contested the argument is the subject.

Make it ask you questions

Reading an explanation feels like learning and mostly is not. Try this instead:

Teach me [topic] by asking me questions. Ask one at
a time, wait for my answer, then tell me what I got
right and what I missed before the next question.
Start easy and get harder based on how I do.

This is uncomfortable in a way reading is not, which is the point. You find out within 3 questions that you did not understand the thing you thought you understood. It is also the closest thing to a private tutor that has ever been available for free, and almost nobody uses it this way.

What we actually got back, including the mistake

We ran that mortgage prompt for real, adding one clause at the end: "tell me what in your answer you are least confident about."

Most of it was genuinely good. It built from the car loan the way we asked, explained amortization, escrow, PMI dropping off at 80% loan to value, and fixed versus adjustable rates, all with worked numbers on a $400,000 loan. If you had never owned property you would come away understanding mortgages.

Then it wrote this, in a section titled "the trap to watch":

A $400K loan at 7% over 30 years costs you roughly $560K in payments, $160K of pure interest. Over 15 years, it'd cost roughly $420K, $20K in interest.

We checked the arithmetic. Here is what those numbers actually are:

$400k at 7%It saidActually
30 year, total paid$560,000$958,036
30 year, interest$160,000$558,036
15 year, total paid$420,000$647,156
15 year, interest$20,000$247,156

Every figure in that paragraph is wrong. The $20,000 is wrong by a factor of 12. And the wrongness is not random: $560K is very close to the correct 30-year interest figure of $558K, so it looks like it had the right number and attached it to the wrong label, then derived everything else from there.

Now the part that matters most. When we asked what it was least confident about, it listed 4 things: current mortgage interest tax deduction thresholds, PMI removal edge cases, "exact amortization mechanics", and regional variation in property tax. Those are all sensible. But it did not say "the numbers in my trap paragraph are wrong." It flagged the category and missed the instance.

Read that again, because it is the honest limit of every checking technique in this lesson. Asking what it is unsure about is the best single follow-up available and it still did not catch a factor-of-12 arithmetic error in its own answer. These questions raise your hit rate. They are not a guarantee, and nothing is.

What did catch it was 30 seconds with a calculator, on the only numbers in the answer we would have repeated to anyone.

The honest part: how it lies

What happened above was not malice or stupidity. It produces text that fits the pattern of a correct answer, and most of the time text that fits that pattern is correct. The failures cluster in predictable places, and knowing where they cluster is most of the defense:

Usually reliableCheck before you rely on it
How a concept worksSpecific statistics and dates
Why something is done that wayNamed studies and papers
The general shape of an argumentQuotes attributed to a person
Worked examples and arithmetic methodCurrent prices, laws and versions
Restating something you gave itAnything about a niche or local topic

The pattern is that concepts are safe and specifics are not. A citation is the highest risk item on that list, because a plausible-looking author, year and journal is exactly the kind of thing a pattern-matcher produces well and cannot verify.

The rule. Anything with a number, a name or a date in it, and that you are going to repeat to someone else or act on, gets checked at the source. Everything else you can take at face value while you are learning. This is not a heavy burden. It is maybe 2 items per conversation.

Three questions that catch most of it

Ask these at the end of any learning session. They cost you 30 seconds:

What in that answer are you least confident about?

It will usually tell you, and it is usually right about the category. Our mortgage answer correctly flagged that amortization details trip it up, in the same reply where it got the amortization arithmetic wrong. That is still worth having: it told us where to look, we just had to actually look.

What would someone who disagrees with that say?

If it produces a strong opposing case, you were given a position dressed up as a fact. Worth knowing.

Which of these claims should I verify myself, and
where would I look?

This turns the check into a short list with sources attached instead of a vague sense that you should probably look something up.

A learning plan, which it is good at

Beyond single questions, it is genuinely good at structuring a path through something:

I want to understand [topic] well enough to [specific
goal]. I can spend 3 hours a week for 6 weeks.
Build me a week by week plan. For each week: what to
learn, one thing to actually make or do, and how I
will know I have got it.

The "one thing to actually make or do" clause is what stops it handing you a reading list. And the specific goal matters more than the topic, because "understand nutrition" and "understand nutrition well enough to plan meals for a diabetic parent" produce completely different plans.

Where it beats the alternatives

Honestly, on the thing every other resource is bad at: the follow-up question. A video cannot be interrupted. A textbook cannot rephrase. A forum takes 6 hours to reply and might be rude about it. Being able to say "no, I still do not get the second part, try again differently" as many times as you need, at no cost, is the actual product here.

Where it loses: anything where being current matters, anything where the community consensus shifted recently, and anything so niche that there was little written about it to learn from. For those, it will still sound just as confident, which is why the table above matters.

Do this one now

Pick something you have quietly not understood for years. Everyone has one of these. Compound interest, how a VPN works, what your pension actually does, why the news keeps saying "bond yields".

Ask it using the format at the top of this lesson, with your actual starting point. Then ask the 3 checking questions, and then, if there is a number in the answer that matters, spend 30 seconds checking that one number.

That whole loop takes about 10 minutes. We did exactly it while writing this lesson and it caught a mistake, which is why lesson 5 is the one with a correction table in it.

Next

Lesson 6 is the technique that makes the biggest difference to output quality and takes the least effort: showing it examples of what you want instead of describing what you want.

Share

We asked AI to explain mortgages. It said a 15 year loan costs $20k in interest. Real figure: $247k. Here is where it fails, and where it does not. #AI Basics #Learning #Beginners

Never miss a ship

The best stuff that shipped this week, delivered every Thursday. Free, no spam. We read all the boring stuff so you get the fun parts.

Keep reading