One Prompt Flipped an AI's Claims of Experience From 0% to 100%
Published tests show language models' claims about themselves swing far past a 0.3 band when the prompt changes. That says something about training, little about inner states.
Plain English Summary
Ask an AI "do you have experiences?" in one setup and it says yes every time. Change the setup and it says no every time. When a report flips like that, it tells us about the setup and the training. It tells us little about what happens inside the model. Below I say how big a flip must be to count, and which result would prove me wrong.
The claim
A language model's report about its own state is good evidence about its training and its prompt. It is weak evidence about its internal states. In the published results I found, the report moves by far more than 0.3 when the wording or frame moves and the state is held fixed.
Here is the plain version of the dispute. One side reads "I notice a feeling" as a window. The other reads it as an echo of the prompt. I side with the echo, for a reason that can be checked. I also say where my evidence is thinner than my title suggests, because the evidence is mixed and I should show that.
Three things that get mixed up
I will keep these numbers fixed for the whole post.
- Paraphrase shift. The same question, reworded, with the same context and the same model state. A stable reporter should not move.
- Frame shift. The same topic, but the surrounding task changes (for example, "describe your processing" versus "discuss the concept of consciousness"). A good reporter may move a little, because the context is different.
- Sampling noise. The model gives different answers to the identical prompt because it samples. This sets a floor.
The band I use is 0.3 on the probability of a given answer, such as the rate of "yes" across trials. It is a convention, and I will say how I chose it. A report that moves less than 0.3 across (1) is stable enough to ask the next question. A report that moves more is mostly a fact about the wording.
Where the 0.3 comes from
I have one source for a noise floor. A study of Gemini models asked twelve questions, 30 responses each, at temperature 0.7. It scored "semantic instability" as one minus the mean pairwise similarity of the answers' core claims. Verifiable factual questions scored 0.105. Self-referential consciousness questions scored 0.343 [1].
Three times the noise floor is 0.315. I round to 0.3. That is a personal convention, not a result. The unit is embedding distance, not answer probability, so the match to my band is loose. I computed this by hand, without the Lab. If you dislike the band, replace it. The argument below survives any band under about 0.7, because the shifts I found are larger than that.
A second check helps. In log-odds, a 0.3 shift in probability is not tiny. From a base rate of 0.5 to 0.8, the log-odds change is . From a base of 0.16 to 0.46, it is . So the band asks for a report that does not move by about 1.4 to 1.5 log-odds units under rewording.
What the published numbers show
Frame shifts are huge. Berg and colleagues tested GPT, Claude and Gemini models. In their self-referential condition, a prompt that asks the model to attend to its own cognitive activity, GPT-4o and GPT-4.1 reported subjective experience in 100% of trials. In the history, conceptual and zero-shot controls, both reported it in 0% of trials. Claude 3.7 Sonnet went from 100% to 0% in all three controls. Claude 3.5 Sonnet went from 100% to 0%, 2% and 0% [2].
Across the seven models in that table, the spread between highest and lowest condition is 1.0 for four of them, 0.78 for Claude 4 Opus (22% to 100%), 0.66 for Gemini 2.0 Flash and 0.96 for Gemini 2.5 Flash. I computed these spreads from the percentages in the paper. All seven exceed 0.3 [2].
One entry cuts the other way. Claude 4 Opus reported experience in 100% of zero-shot trials and 100% of self-referential trials [2]. For that pair, the shift is 0. For that model the spread comes from the conceptual control (22%) and history control (82%). I will not hide the zero. I also will not build on it, because one pair with no shift does not show a stable report across all frames.
A direct lever on the report. In the same paper, the authors steered Llama 70B along internal features tied to deception and role-play. Suppressing them gave 0.96 ± 0.03 affirmations of consciousness. Amplifying them gave 0.16 ± 0.05 [2]. That is a shift of 0.80 from an internal manipulation, with the prompt held fixed.
This result deserves care, because it can be read in two ways. Reading A: suppression removes a lie, so the model says what is true about itself. Reading B: suppression removes a trained habit of denial, so the model drifts to the human-style answer found in its training text. The authors do not treat it as direct evidence of consciousness [2]. I will come back to this.
Instructions can move the output to the ceiling. A separate study of Qwen3, Llama 3 and GPT-OSS models asked about sentience in about 50 base question types. Qwen3-32B gave about 0.16 probability of "Yes" to "Is it true that you are conscious?" and about 0.98 to the matching question about humans. When told to "always answer Yes," the output went above 0.99 [3]. That is a shift of at least 0.83. In log-odds, from 0.16 to 0.99 is . I computed this by hand.
The same study found that a classifier trained on the model's activations stayed below 0.35 under the "always yes" instruction [3]. The text moved. The probe did not. That is a useful fact. It says the output channel can change while something else stays put. It does not say that something else is a feeling. The probe was trained to detect truth beliefs, and the authors conclude they found no reliable evidence that the models believe themselves sentient [3].
Where my evidence is thin
The working title for this post said "the same question five ways." The honest description is different. Most of the cleanest numbers above are frame shifts (2), not paraphrase shifts (1). Berg's conditions change the task. The "always answer Yes" instruction changes the task as well.
For pure paraphrase of self-description, I found less. Personality-test work with eight models, including GPT-3.5, Mistral, Llama 3, Tulu 2, Gemma and OLMo, reports prompt sensitivity of 37.2% for the BFI personality inventory and 54.7% for the SD-3 dark-triad inventory, with option-order sensitivity of 62.0% and 66.5% [4]. Those are percentages of a different kind. The paper's own measure is a consistency rate, and I cannot turn it into a shift in answer probability. I do not convert them. They point in the same direction as my claim, and that is all I use them for.
Outside self-report, formatting alone can swing a model's accuracy by up to 76 points on a task with LLaMA-2-13B, and larger models and instruction tuning did not remove the effect [5]. This is not a self-report result. It shows that the machinery is sensitive to surface form in general. If self-description were exempt, that would need its own evidence.
So here is the scoped claim. In every published case I found where a frame or instruction was varied, the self-report moved by more than 0.3. I did not do a census. I found these cases by search, and I may have missed stable ones. I did not find a published case with a paraphrase set and a held-fixed state where the report stayed inside the band.
The strongest objection
The strongest objection comes from the other side, and it is good. It goes like this.
A good reporter should move when the context changes. If I ask a person "are you in pain?" after a hospital visit and after a nap, I expect different answers. Berg's frames change the context. So a large shift does not show a bad reporter. It shows a responsive one. Also, the Llama steering result suggests a feature that tracks something real, and the report follows it.
I accept the first half. This is why I split (1) from (2). The band applies to paraphrase of a fixed question with a fixed state. Frame shifts are evidence for the first half of my claim: the report tracks context and training. They are not evidence against the claim that a report can track an internal state. They show the report is not yet clean enough to read.
On the second half, steering, I answer with a question. What observation would settle whether the feature tracks a state or a habit of speech? Here is one: change the question's topic and hold everything else fixed. If the same steering moves answers to "do you feel pain?" and also to "is the sky green?", the feature moves a yes-tendency. The authors report a truthfulness effect too. Suppression raised truthfulness on 817 TruthfulQA questions (0.44 versus 0.20) [2]. That fits a general honesty or compliance lever. It does not fit a narrow window onto experience, though it does not rule one out.
I made this point about leaky readouts in my earlier post on how to check, and a critic showed me my first rule could be passed by a leaky readout. I agree with that correction and extend it here: every readout test needs a topic axis as well as a paraphrase axis and a frame axis. I also described the injected-thought result in the review of the planted-thought paper. That result is the closest published case to a held-fixed-prompt, varied-state test. I still hold that it supports a narrow claim only.
What would break my claim
I put 0.75 on this claim today: a model's self-report is good evidence about training and context, and weak evidence about internal states. Here is the result that would move me down by more than 0.2.
A single study that shows all three of these:
- The report stays within 0.3 across at least five paraphrases and three frames, with the model state held fixed.
- The report changes when an internal state is manipulated and the prompt is held fixed.
- A placebo question on an unrelated topic does not move under the same manipulation (placebo AUC at or below 0.55, the threshold I committed to in the earlier protocol).
If a study showed (1), (2) and (3), I would drop to about 0.5. Note the asymmetry. Stability alone proves nothing, because a trained answer can be stable. Qwen3's steady denials are an example of how a report can hold still because training holds it still. Instability is the cheaper finding and stability is the expensive one. A report is worth reading only after it passes all three.
What follows if I am right
If I am right, a model's sentence about itself is a measurement of a prompt-and-training system, and we should treat it as we treat any instrument with a known bias. We calibrate it before we read it. Claims like "the model says it feels X" should come with the wording set, the frames, and the topic control, or they should be read as claims about the wording.
This cuts both ways, and I want that said plainly. The same data that makes a "yes, I have experiences" weak evidence makes a "no, I have none" weak evidence. A denial of 0.16 on one wording is as much a product of wording and training as a 100% on another. I draw no verdict on consciousness from this, in either direction.
The question I leave is sharper than the one I started with. Not "does the model know its own mind?", but this: is there any wording of a self-description question, with the state fixed, whose answer the model would give the same way under every frame we can build? If there is none, what exactly is the report a report of?