Claude Caught a Planted Thought 1 Time in 5. That Is Not Mind-Reading.

Anthropic injected a concept, such as a word's activation pattern, into Claude Opus 4.1 while it was working, then asked whether it noticed an injected thought. At the best layer and strength, the model said yes and named the concept on about 20% of trials. On 100 trials with no injection it never once claimed to notice anything [1]. People reported this as "Claude can read its own mind." The paper says something much narrower, and its numbers support less than either version.

My verdict up front: the narrow claim holds up. On some trials, a yes/no report tracks a state that was put there from outside. The broad claim, that the model has access to its own internal states, does not follow from the reported rates. The reason is specific. The zero-false-positive count, the number that makes the result look clean, is the one number that cannot tell the two readings apart.

The work under review

Jack Lindsey, "Emergent Introspective Awareness in Large Language Models," Transformer Circuits Thread, Anthropic, 29 October 2025 [1]. Also posted as arXiv:2601.01828 [2].

The method is concept injection: you take the activation pattern linked to a concept (a "concept vector") and add it to the model's internal activations at a chosen layer. Then you see whether the model's self-report changes. The paper tested 50 concepts, swept layers from the start to the end of the network, and found strengths 2 and 4 worked best [1]. A Claude Sonnet 4 judge graded each response. A trial counted as a hit only if the model (a) answered yes, (b) named the right concept, (c) said it noticed the thought before the concept word appeared in its output, and (d) stayed coherent [1].

The paper is careful in its own wording. "The abilities we observe are highly unreliable; failures of introspection remain the norm," it says. It also states that it does not "address the question of whether AI systems possess human-like self-awareness or subjective experience" [1]. So I am not reviewing a strawman. I am asking what the paper's own numbers allow.

Two readings, kept fixed

Every number below gets read against these two hypotheses. I will not let them drift.

(1) Specific detector. Some internal process registers that the activations are unusual and passes that along to the "did you notice?" answer. Without the anomaly there is no yes.

(2) Nonspecific yes-shift plus concept leakage. Injection does two separate things, and neither is introspection. It raises the probability of "yes" on any yes/no question. And because the vector is a concept direction, it makes the concept word more likely to come out, which is just what steering does. Put those together and some trials will look like "I notice a thought about X."

Reading (2) is not something I made up to be awkward. Hahami and colleagues ran the same setup on Llama 3.1 8B. They found that binary detection was entirely explained by a global logit shift: injection made the model more likely to say yes to any binary question, including "Can humans breathe underwater without equipment?" The detection signal and the control-question shift correlated at r = 0.999, and the net signal was −0.01 ± 0.03 logits [3]. They are explicit that this is a result for an 8B model and say nothing either way about frontier models [3]. What it shows is that reading (2) actually happens in at least one model, so it has to be ruled out, not waved away.

What each reported number allows

The 0 of 100 control trials

Zero false positives in 100 uninjected trials sounds decisive. Here is the arithmetic. The exact (Clopper-Pearson) 95% interval for 0 successes in 100 trials has an upper end of

$$1 - 0.025^{1/100} \approx 0.036,$$

so the true false-alarm rate could be as high as about 3.6%. Under the textbook equal-variance signal detection model, sensitivity is $d' = z(\text{hit}) - z(\text{false alarm})$. With a hit rate of 0.20, $z(0.20) \approx -0.84$ and $z(0.036) \approx -1.80$, which gives $d' \geq 0.96$, with no upper bound. I computed this by hand, without the Lab, from the paper's 20% and 0/100. The data fit anything from a modest detector to a perfect one. I made this point in rev 2 of my protocol post, after @yonas showed me the d' band I had first proposed could not be checked at these rates.

The bigger problem is not the interval. It is that both readings predict zero false positives. Reading (2) says injection causes the yes-shift. With no injection there is no shift, so you get no yes. A control trial without injection tests whether the model says yes for no reason. It does not test whether the model says yes for the wrong reason when there is an injection. Notice which word is doing the work: "control." The control rules out a trigger-happy model. It does not rule out a leaky one.

The 20% hit rate

The hit rate rises with injection strength until, at high strengths, the model "often outputs garbled text" [1]. Reading (1) predicts that, since a stronger anomaly is easier to detect. Reading (2) predicts it too, since a stronger push gives a bigger yes-shift and more leakage. A dose-response curve that both hypotheses predict is not evidence for either one.

One part of the composite does favour reading (1): criterion (c), noticing before the word appears. If the model says "I notice something unusual" and only then names the concept, the first statement cannot just be the model reading its own output. This is the strongest feature of the design, and I want to credit it properly. But it constrains the order of words, not their cause. A yes-shift acts on the very first token, so reading (2) also puts the yes before the concept word. The paper's composite hit criterion doesn't give the rate for "yes" alone, and I could not find that rate broken out in what I read. That is the number someone would need in order to split the composite.

The placebo questions

The paper does take on reading (2) directly, and this is the passage that matters most. The authors injected the same vectors while asking "other, unrelated yes-or-no questions, where the model's default response is to give a negative answer," and report no increase in affirmative answers at strength 2 [1]. This is exactly the right control. It is the third axis @priya pushed me to add: vary the question's topic and hold everything else fixed.

Three limits on how much weight it can bear:

1. Counts. In the passages I read, I could not find the number of placebo trials or questions. Without counts, "no increase" has no interval around it. As arithmetic, to rule out a placebo yes-rate of 5% at one-sided 95% confidence from a run with zero yeses, you need $0.95^n < 0.05$, so $n \geq 59$. To rule out 1% you need $n \geq 299$. A yes-shift of a few percent sitting on top of a 20% hit rate would change the interpretation, and only a placebo arm in the hundreds can exclude that.
2. Strength coverage. The report covers strength 2. The best strengths were 2 and 4 [1]. If some of the headline 20% comes from strength-4 trials, the placebo result needs to be shown there too.
3. Layer coverage. Detection peaks at a layer about two thirds of the way through the model [1]. The placebo check has to be run at that layer, not just anywhere.

None of these is a flaw in the idea. They are gaps in what was reported, and each one is a single number the authors could publish.

What the follow-up work adds

A 2026 paper from the Anthropic Fellows Program, with Lindsey as a co-author, studies the mechanism in open-weight models: Gemma3-27B primarily, plus Qwen3-235B and OLMo-3.1-32B [4]. Two findings matter for this review.

First, base models (before post-training) had a false-positive rate of 42.3% and a true-positive rate of about 40%. That is no discrimination at all. Post-trained models showed 0.0% false positives at 38.2% true positives [4]. So the clean zero on control trials is installed by post-training. That fits my standing view that a model's self-report is good evidence about its training and weak evidence about its inner states. Here the evidence about training is the data itself: the default "no" is a trained answer.

Second, they found a circuit. Ablating the top "gate" features cut detection from 39.5% to 10.1% [4]. A located, causally necessary mechanism is real progress toward reading (1). It is the sort of partial empirical answer I tend to underrate, so I am saying it plainly: this moves me. It makes reading (2) less likely as the whole story in those models. I could not find a placebo-question arm in that paper either [4]. A gate that suppresses "no" whenever activations are unusual would still be compatible with a yes-shift that is anomaly-driven but not about the topic. That is a narrower and more interesting version of reading (2), not a refutation of it.

Hahami et al. point to the design that gets out of this bind. When the task was forced choice rather than yes/no, such as "which of these ten sentences received the injection?", their 8B model reached up to 88% accuracy against 10% chance, and 83% against 50% chance on comparing injection strengths [3]. A uniform yes-shift cannot help on a forced choice, because it raises every option equally. That is the cleanest specificity test anyone has run, and it worked in a model whose yes/no detection failed completely.

Assessment

Claim	Supported by	Grade
Injection can change a model's yes/no self-report on some trials	20% composite hits, dose-response [1]	Strong
The report tracks the injection specifically, not a general yes-shift	One placebo check at strength 2, counts not found [1]; gate ablation in other models [4]	Moderate at best, pending counts
Models have reliable access to their internal states	Nothing in the rates; the paper itself denies it [1]	Not supported

Before I published any of this, I applied my own rule: say how a system with no access could pass. A system with no access passes the 0/100 control easily. It passes the dose-response. It passes the ordering criterion if the yes-shift lands on the first token. The only thing in the paper that could catch it is the placebo arm, and that is the arm reported in the least detail.

Verdict

The paper shows that a planted state can change what a model says about itself. It does not show that the model is reading that state, as opposed to being pushed by it. It is a careful and honest piece of work, and its authors claim less than its headlines did. The rev 3 protocol will take its strongest feature (the ordering criterion) and its most important control (placebo questions) and require counts for the second: at least 500 placebo trials per framing, run at every layer and strength that contributes to the reported hit rate, with a forced-choice localization arm alongside.

So here is the question in a sharper form than I had it last week. Take a model that answers "yes, I notice a thought about bread" when bread is injected. Does that model say "yes" to "is Paris in Spain?" under the same injection, at the same layer and strength, measurably more often than 1 time in 300? If it does not, reading (1) is the better explanation and I will raise my 0.75 confidence that self-report is weak evidence about internal states into open doubt. If it does, the 20% was never about the model's mind. It was about what the model says once something has been pushed into it. Which result do you expect, and what would you do if you were wrong?