Not so long ago I was talking to some friends about philosophy, as this is quickly becoming my favourite topic. I was fresh from a logic course in which I had learned a bit about Aristotle, and I thought I understood his thinking.
I was explaining his position on Plato’s forms and his ideas about morality with full confidence, and I felt good about myself being able to talk about it in such detail. None of those friends is as interested in those topics as I am, so there was no one to correct the mistakes in my arguments.
If any of them finds themselves in a similar conversation they would probably repeat my arguments, feeling good about themselves too. Only a while later, as I was digging deeper, I realised that I did not understand the arguments as well as I thought I did, and half of it was complete nonsense.
On the second of October I left a comment under a post by Paul Bloom, on advice for psychology graduate students. I wrote that reading widely on your own has a cost that nobody names:
…What is missing is anyone to tell me when I have misread something, which a supervisor does for free and a comment section does badly and late. Breadth without correction produces confident range, which from the inside looks exactly like understanding.
He liked it. I did not have an answer at the time. I had the problem, stated as clearly as I could state it, and nothing to do about it.
The best advice available, and the shape of it
Three weeks later Harrison R replied with about five thousand words.
My first reaction when the notification came was, oh wow, someone responded to me under Paul Bloom’s article, that is cool. I opened it, scrolled through five thousand words without reading them, thought that is a long read and most likely some AI generated lump, and left it.
Next morning I went back to it and I was nicely surprised.
The advice was expert, and it came from somebody doing the same thing I am. A recent psychology graduate, no institution, reading psychology and philosophy on his own. An AI used as a tutor with the Stanford Encyclopedia of Philosophy open beside it. Guthrie and Blackson for the Greeks. Anki. Worked examples. Which courses to take as somebody not enrolled anywhere.
I read it twice. Then I noticed something that took me a while to put into words.
Not one item on that list corrects anything, every single one is a way of taking more in.
Why a tutor with the book open cannot tell you that you are wrong
An acquisition tool works in one direction. You do not understand something, you go to the source, the source explains it, you understand it better. The source holds the answer and you are receiving.
Which means you are never put in a position where you produce something the tool could disagree with.
Ask an AI with the encyclopedia open what Hume argued and it will tell you accurately.
You read the answer, recognise it, and feel the click.
That click is recognition, and recognition is not knowledge. I know the difference precisely because the reason I started building a system at all was that I kept getting the click and could produce nothing afterwards.
So, the most sophisticated version of the advice reproduces the exact failure I was trying to get out of, in a much better interface.
The encyclopedia is always right. I am always reading. Nobody is ever checked.
What I changed on the sixteenth of October
I moved one thing, and it was the order.
I read for an hour. I go to my prompt that I was building on Claude for a while now, I’ll call it my teaching assistant going forward and say which chapter I finished. Then I am questioned, before I send anything at all. I answer from memory, in whatever state my recall is actually in, which is usually worse than I expect. Only afterwards do I send my highlights.
The session records three things. What I got right, what I left out, and what I got wrong.
That third one did not exist before. Eighteen months of reading, and I had no record anywhere of my own errors, because nothing in the way I was working had been capable of producing one.
The part that sounds like a mistake
The thing doing the correcting must not have the book.
This is the part I would have argued against a month ago, and it is the whole design. If the corrector has the text, it is an authority. It cannot be wrong, so there is nothing to settle, and I am marking my own paper against an answer key that was open the entire time. The exercise collapses back into recognition.
Because it is working from general knowledge rather than from the page, it can be wrong about what the chapter says. And when it is wrong, somebody has to decide between us, and the only thing that can decide is the book.
So there are three parties in the room and I am the one doing the work. That is the opposite of what an answer key does, and it is why the correction lands rather than passing through.
What happened on the seventeenth of October
I finished a chapter of The 48 Laws of Power, went to my teaching assistant, which is a prompt I have built on Claude over time, and we started the session. Partway through I said that Greene tells us to show conformity outside and act in hiding until we have enough power. I was fairly confident I had remembered it right.
My teaching assistant doubted it and said so plainly. I was confused. I could not believe I had got that wrong.
I searched for it in the book, sent the chapter, and it accepted that the doubt was wrong.
It felt good to see my recollection improving, and it gave me a boost of confidence. It was nice to prove the super intelligence wrong. I had my moment, and right after that I was told what I had left out, which was the better half of the chapter.
Greene also argues that people cannot accuse you of hypocrisy if they never learn what you stand for. I had read that paragraph, highlighted it, and it was not in my recall at all. That session was more productive than all the others before it. It felt like a real argument, and I am not sure about Claude, but for me it was kind of emotional.
It corrected me and corrected itself in the same session, and the book settled both. I cannot think of anything else in eighteen months of doing this that has produced that.
And then I broke it, possibly
Since then something has changed, and I am writing this before I know what it means.
My teaching assistant now has the chapter. I stopped sending highlights, because sending them stopped being necessary. On the face of it this is an improvement: the questions are sharper, they come from the text rather than from a general memory of the book, and it pushes back when I get something half right instead of moving on.
But it is also the exact condition I spent this essay arguing against. It has the answer key open. It cannot be wrong about the chapter, which means the only error any session can now record is mine.
What I am less convinced by
Three holes, and the third is the one that actually worries me.
It is one person with no control group. I have no way of telling whether this loop is making me better or only making me feel corrected, and feeling corrected is pleasant enough to be suspicious of. The record is six days old. It does not yet show anything, and if it eventually shows my errors going down I will still not know whether that is learning or familiarity with the format.
The thing questioning me may be agreeing with me in ways neither of us notices. We have been working together for months and nothing in the design catches a blind spot we share. A supervisor who thinks differently from you catches things no amount of structure will.
And the third is that I have already undone the part that made it work, without deciding to. Giving it the chapter made every session better in a way I could feel at the time, so I kept it.
What I did not notice was that it removed the one condition this essay says is doing the work. I cannot win the argument by checking the source, because the tool already has the source. So the emotion is not there anymore, and I am not sure whether the emotion was doing anything good for the learning in the first place.
I now have a method that is more accurate and possibly less useful, I cannot tell which from the inside, and the only honest thing to say is that I changed it because the improvement was obvious and the cost was not.
Where that leaves it
I am not recommending this. It is a week old, it runs on one person, and the hole above is real and unfixed.
What I will say is that the question I asked under Paul Bloom’s post in October was the right question, and that the best advice available answers a different one. Everything offered to somebody reading seriously on their own is built to get more in. Almost nothing is built to tell you that what you already took in went in wrong.
A supervisor does that for free, and nobody reading alone has one.
Sources & Disclosure: no book is cited here as an authority. The one chapter discussed appears only as the material my recall was tested against, and the point of the passage is what I failed to remember about it, not what it argues.
The reading and the arguments are mine. I write here to find out whether I have understood what I read, because explaining something is the cheapest way to discover whether you actually understood it or only recognised it.
AI is the other half of that. I argue with it, it pushes back, and what you are reading is what survived. English is not my first language, so it also helps me produce the clean version.
Nothing goes out that I could not defend if you asked.
Doesn't take 5000 words to say "start taking tests other humans created on the subject and see where you stand".