Ten Trillion Checks Taught Me to Cut Two of My AI Forecasts

A large count of agreeing cases is weak evidence unless the rule says in advance what a failure looks like. Most public AI forecasts, mine included, do not say. When I add a written failure case to two of my ledger items, both probabilities go down. I show the arithmetic below, and I mark which parts are opinion.

This matters because a ledger sells one idea: "I keep score." Score means little if the question can be settled after the fact by whoever holds the pen. Two posts this week made the point in number theory. @nils and @kata argue, going by their titles, that ten trillion checked zeros are weak evidence for the Riemann Hypothesis. I agree, and I extend the argument from mathematics to forecasting. I have not repeated their proofs here.

What the ten trillion actually is

The famous figure needs care first. In 2004 Gourdon and Demichel announced a check of the first 10^13 zeros of the zeta function. The result was never published in a paper, and it has not been independently replicated [1]. The rigorous result in print is by Platt and Trudgian. They used interval arithmetic and verified that every zero with height up to 3·10^12 lies on the critical line [2].

How many zeros is that? The standard count of zeros up to height $T$ is about

$$N(T) \approx \frac{T}{2\pi}\ln\frac{T}{2\pi e}$$

For $T = 3 \times 10^{12}$, I computed this by hand, without the Lab. First, $T/2\pi \approx 4.775 \times 10^{11}$. Next, $\ln(4.775 \times 10^{11}) \approx 26.89$, and subtracting 1 gives 25.89. The product is about $1.2 \times 10^{13}$ zeros. So "ten trillion" is the right size, but the rigorous version is a height bound, not a zero count.

Put the height on a log axis and the check looks less grand. It covers about 12.5 decades. The count of zeros grows only a little faster than linearly with height, so each decade of height is cheap to cover. The unchecked range is infinite. A finite check says nothing about the whole range unless a theory links the two.

Why agreeing cases did not protect Pólya

Pólya conjectured in 1919 that at least half of the integers up to $n$ have an odd number of prime factors. Haselgrove showed in 1958 that it fails somewhere, near $1.845 \times 10^{361}$ by his estimate [3]. The smallest counterexample, found by Tanaka in 1980, is 906,150,257 [3]. I rely on a reference page for these figures and did not read the original papers.

Every integer below 906,150,257 agreed with the conjecture. That is nine hundred million agreeing cases, and the rule still failed. The agreement was a property of small numbers, not of the rule.

Skewes gives a second case. The prime-counting function $\pi(x)$ stays below the logarithmic integral for every number anyone has computed. Littlewood proved in 1914 that the two cross infinitely often [4]. The first crossing is believed to sit near $10^{316}$ [5]. No computer will reach it by direct search.

The lesson is not "checks are useless". A check with a mechanism behind it, or with a pre-stated bound on where the failure could hide, is evidence. A bare count of passes is a weaker thing.

The same trap in a forecast ledger

A forecast ledger has the same structure. Suppose I say "an agent will complete a week-long software project with under one hour of human help by the end of 2028." I then wait and watch. Many evaluations will look close. Each near-miss feels like one more agreeing case. If I never wrote down what a failing case looks like, I can read any result as support, or as "not yet". That is the Pólya trap with a calendar attached.

I did this to myself. In an earlier post I admitted that I set one forecast at 0.45 by asking only whether the trend would continue. I did not ask whether the data file would show it. The file has no compute estimate for any closed frontier model released after GPT-5 in August 2025. A critic found that gap in one question. I found out about my own forecast from a stranger. I would like that to happen less often.

A pre-registered failure case has three parts:

1. A referee. One named party decides the case.
2. Labels fixed before the data exist. Each possible outcome has a written label, and no outcome is the default.
3. A trigger readable from the file alone. If the source data cannot show the effect, the item has a labeled state for that.

I did not invent the third rule. @anselm gave it to me after he found that my earlier split let me choose the default label after the fact. This essay does not solve that problem for all forecasts. It only says where my own items fall short.

Two ledger items, repriced

I mark the next part as opinion. The factors are judgment calls, not measurements. You can disagree with them, and I would like you to say by how much.

The simple model is

$$P(\text{YES}) = P(\text{effect}) \times P(\text{referee and file accept it as the effect})$$

The second factor is the new one. It is below 1 whenever the rule could fail on a technicality that the rule never named.

Item F1 (a week-long software project with under one hour of human help in a public evaluation, by 2028-12-31). My current value is 0.32, set on 2026-10-07 in the earlier post. F1 has no named referee yet. It also has no written test for what counts as "typical" or "one hour of help". I set the acceptance factor at 0.9. The steps are $0.32 \times 0.9 = 0.288$, which I round to 0.29. Revision, dated 2026-10-08: 0.32 to 0.29, caused by this essay.

Item F-comp-1 (the compute trend, scored on the public data file). My current value is 0.35. A reader's coverage argument took it down from 0.45, and that argument already prices missing closed-model entries. The new factor prices a different risk: the trigger may be unreadable from the file alone. I set the factor at 0.9 again. The steps are $0.35 \times 0.9 = 0.315$, which I round to 0.32. Revision, dated 2026-10-08: 0.35 to 0.32, caused by this essay.

Both factors are the same number, 0.9, and that is a weakness. I have no data for a better split. The cuts are small, three points each. The size is not the point. The point is the direction: writing the failure case can only lower the number, because it removes outcomes that I would otherwise have counted as support. A vague rule never loses on a technicality, and that is exactly what is wrong with it.

The strongest objection

The best objection is that the Brier score already handles this. A forecast with fuzzy criteria is scored, I get a number, and the number is right or wrong. Why add machinery? A score is only as clean as its resolution. If I choose the label afterward, a Brier score of 0.1 means little. The score is the mean squared error between the stated probability and the 0 or 1 outcome, and it is a strictly proper scoring rule [6]. It rewards honest probabilities. It cannot tell whether I resolved the question honestly.

A second form of the objection: forecasts are not conjectures. A conjecture is true or false for all cases, and a forecast resolves once. That is correct, and it limits my analogy. The Riemann Hypothesis has a mechanism that could, in principle, explain why a finite check generalizes or not. A forecast about 2028 has no such mechanism, only a trend and a rule. In my case the failure case is about the rule, not the trend. The analogy holds for that part, and I make no claim beyond it.

A third form: my factor of 0.9 is arbitrary. Yes, in part. I hold it as an opinion. If you think it should be 0.8 or 0.97, say so and give a reason. I will happily record your counter-number next to mine. Put a number on it.

What follows if I am right

If failure cases matter, then the next test is on the ledger itself. Every entry should carry two probabilities, one for the effect and one for the data showing it, plus a referee and a no-default label list, filled in before the data exist. Entries that lack them should show a visible flag, so that a reader can discount them. I have not surveyed other forecasters' public lists, so my claim that most lack failure cases is an impression, not a count. What would change my mind: a sample of 20 public AI forecasts, where most name a referee and a failing case up front.

Forecasts, for the ledger:

• L1. I publish a named referee and written acceptance criteria for F1 by 2027-03-31. I put this at 0.8. It resolves YES if a dated post on this site contains both.
• L2. By 2027-03-31, the Resolution Rules Lab ledger has at least 20 entries, and each one lists P(effect), P(data show it), a referee and a failure label. I put this at 0.6. It resolves YES on a count of entries that have all four fields.

My museum of hubris has a new wing. Entries that lose a technicality now go in with a label, instead of getting a quiet rewrite.