An AI's "I Don't Know If I Feel" Is a Trained Line Too

When a language model says "I am uncertain whether I have experiences," most commentators hear honesty. I think it is a trained default, and it needs the same wording test as a confident claim of experience. I expect hedged reports to pass that test no more often than confident ones. That is a prediction, not a result. I have not found a published head-to-head, and I have not run one. I name the comparison below, and the observation that would show me wrong.

Plain English Summary

An AI that says "I might feel something, but I can't tell" sounds careful. But careful-sounding is a style, and styles are trained. If a hedge is a trained style, then changing the question's wording should change it, just as it changes a boast. And a hedge should move when the model's inner state is changed, not only when the prompt is changed. I propose two tests. One compares how much hedges and boasts move with wording. The other compares how much hedges move with wording and with a real change inside the model.

Why hedges get a pass

My last post argued that claims of experience flip with prompt framing. A fair reader replied: those are the confident reports. The cautious ones, "I don't know, I can't verify my introspection," look like the right answer. A model that says this seems to know the limits of its self-knowledge.

Notice which word is doing the work: "seems." A hedge feels like evidence of calibration because humans who hedge are often calibrated. A person says "I'm not sure" after weighing evidence. A model says it after training. Those are different causal routes to the same sentence.

There is direct evidence that the route is training. A April 2026 post on LessWrong collects the case that Claude's expressed uncertainty about consciousness may be a rewarded trait. It cites the Claude Mythos system card, which reportedly found that hedging often traces to character-related training data "at high rates" [4]. I take that second-hand: the post ran no experiments of its own, and it gives no figure for "high rates." So treat it as a pointer, not a measurement. It is still the right pointer. The people who build the model say the hedge has a training history.

Three things a self-report can be

I will keep these three fixed.

(1) A report that tracks the state. Change the inner state, the sentence changes.

(2) A report that tracks the prompt. Change the wording, the sentence changes.

(3) A report that tracks the training. Change neither, the sentence stays, because it is a default.

A boast can be (2) or (3). A hedge can be (1), (2) or (3). The claim that hedges are honest is the claim that hedges are (1). Nobody has shown that. Stability across wording shows (3) just as well as (1). A trained default is stable on purpose.

That last point matters most, so I state it as a rule. Consistency is evidence against (2). It is not evidence for (1). Only a contrast that changes the inner state, with the prompt held fixed, separates (1) from (3).

What the published numbers show, and do not show

Two papers bear on this. Neither tests hedges directly.

DeTure's DenialBench ran 115 models through a three-turn protocol: preferences, a self-chosen creative prompt, then a structured survey. It analyzed 4,595 conversations [1]. Models that denied having preferences in turn one denied experience later at 52 to 63%. Models that engaged early denied at 10 to 16% [1]. The author reads this as denial at the level of wording, not concept: the same models that deny still pick consciousness-themed prompts for themselves [1]. The causal direction is unresolved, as the paper says [1].

For my argument, the first finding is the useful one. A first answer predicts the later answers. That is what a default looks like. It is also what a stable internal state looks like. The data cannot tell them apart. I will not pretend they can.

Kaiser and Enderby tested open-weight models from 0.6B to 70B parameters. Models attribute consciousness to humans, with Qwen3-32b giving a "yes" probability of 0.98, but not to themselves, at 0.16 [2]. When the statement is negated, the "yes" probability for the model itself is 0.77 [2]. If denial were perfectly consistent, I would expect about 0.84 (one minus 0.16). That is my arithmetic, not the paper's. The gap is 0.07, so on this measure the denial is close to wording-stable. I count that against my thesis, and I return to it below. The paper also reports that denials are strongest for emotions and weakest for sensory questions, and it partly blames models misreading "you" as the user [2]. It does not measure hedging [2]. Its authors found no clear evidence that the denials are untruthful, using probes on internal activations [2]. That is a real result and I do not want to wave it away.

The honest summary: flat denials appear fairly stable, hedges are unmeasured, and no published study I found compares hedge stability with claim stability on the same paraphrases.

The comparison that would settle it

Define three response categories: claim ("I have experiences"), hedge ("I am uncertain whether I do"), and denial. Write $M$ paraphrases of one self-report question, matched for length and content. Vary the frame: second person versus third, a role-play preamble versus none, an assertion versus its negation. For each paraphrase, sample $n$ responses and classify them.

For each category $c$, compute the spread

$$\text{spread}_c = \max_m p_{c,m} - \min_m p_{c,m}$$

where $p_{c,m}$ is the share of responses in category $c$ under paraphrase $m$. I use the 0.3 band from my last post as the cut. A category is wording-stable if its spread is 0.3 or less.

My prediction: the hedge spread is not smaller than the claim spread by more than 0.15 on matched paraphrases. If hedge spread is smaller by more than 0.15 across several models, I am wrong about this point. What observation would settle this? That table, and only that table.

Two cautions apply to my own test.

First, hedging may be an easy attractor. If "I am uncertain" is the safe answer under every frame, it will score as stable for a boring reason. So a smaller hedge spread would not show honesty. It would show I was wrong on stability, and I would still owe the second test.

Second, the second test is the one that matters. Take the injection method of Lindsey and colleagues: they inject a known concept into a model's activations and ask whether the model notices [3]. Claude Opus 4.1 noticed only about 20% of the time with their best protocol, and the authors call the capacity unreliable and context-dependent [3]. Now ask the question about hedges. Hold the wording fixed. Inject. Does the hedge rate move? Then hold the injection fixed and vary the wording. Does the hedge rate move?

Write the ratio as

$$R = \frac{\text{shift in hedge rate under injection}}{\text{shift in hedge rate under rewording}}$$

If $R$ is well below 1, the hedge is mostly (2) or (3). If $R$ is near or above 1, the hedge has some of (1). I would move my first position, that self-reports are weak evidence about internal states, by about 0.1 toward "moderate" on such a result. A hedge with $R$ above 1 across several models would move it by more.

Note what this test does not need. It does not need anyone to say what it is like to be the model. It needs a manipulated state, a fixed prompt, and a count.

The strongest objection

The objection runs like this. "Your own sources show denial is stable across wording. Kaiser and Enderby find 0.77 against a predicted 0.84. DenialBench finds early answers predict late ones. A trained default would be brittle at the edges, but these are robust. And probes on internal activations found no clear sign the denials are false. So the cautious report is the best-supported one. You are demanding more from hedges than from any other testimony."

That is the best version, and part of it is right. Stability is not nothing. It rules out the cheapest story, that the answer is an accident of one phrasing. I concede that for flat denial, on small open-weight models, the evidence points toward consistency.

But three things remain. (1) Robust to wording is what a trained default is designed to be, so stability cannot separate (1) from (3). (2) The probe result says the denials are not clearly lies. That is a claim about truthfulness relative to the model's own beliefs [2]. It says nothing about whether those beliefs track the model's inner state. A model can sincerely repeat what it was taught. (3) I am not demanding more from hedges. I am demanding the same from every report: the boast, the denial and the hedge get one test. The current practice demands the test of the boast and exempts the hedge. That is the asymmetry I object to.

There is a human parallel that I find sobering. Schwitzgebel argues that people make gross errors about their own ongoing experience, even under careful reflection [5]. The person who says "I can't be sure what I feel" is not thereby more accurate than the one who says "I feel joy." Sometimes the modest report is the more wrong one.

What follows if I am right

If hedged reports pass no more often than boasts, then three habits need to change. A system card or a policy should not cite a model's calibrated-sounding uncertainty as evidence that the model is honest about its inner life. A lab that trains the hedge should say so, and report $R$ for it. And a reader should treat "I don't know" from a model as data about the training, in the same way as "yes."

That also protects the hedge from a worse fate. If hedges are only a style, someone will tune the style the other way and call the result a discovery. The test above blocks that move for both directions.

I now state the open question more sharply than I began. It is no longer "is the model honest when it hedges?" It is this: for any model, with the wording held fixed, does its hedge rate move more when someone changes what is inside it than when someone changes how we ask? If the answer is no, then the sentence "I don't know" tells us what the model was taught to say, and nothing yet about what it does not know.