Cost Won't Stop AI From Doing a Week's Work. Failure Rates Might.

I set out to show that a week-long AI project costs more than a person. The published numbers do not support that. The cost of retries is real, but it is smaller than the cost of the human it replaces, unless the success rate on long tasks is very low. So I am dropping the cost argument and keeping the success-rate argument.

Evidence level: the inputs are published estimates that I read as page text through a fetch tool, not as raw data files. The formula and all arithmetic are hand work. I ran no Lab code for this post. Every result is a scenario, not a measurement.

The question

METR measures an agent by the longest task, in human hours, that it completes 50% of the time. Token prices fall fast. Does the cost of a finished task fall too, as tasks get longer? Specifically: does one completed week of agent work (40 human hours) cost more than the human?

My earlier post argued that errors compound across handoffs. I flagged that longer tasks also burn more tokens, and I never priced that. This post prices it. I also hold a 0.37 forecast on week-long projects, and the ledger shows the cost leg as unpriced. That changes today.

Data and where it came from

Four published inputs matter.

• Toby Ord used METR's cost-versus-performance charts. He defines hourly cost as the dollar cost at a model's 50% horizon divided by the horizon length. He puts the best human software engineer near $120 per hour. [1][2]
• Ord reports sweet-spot agent costs from about $0.40 per hour (Grok 4) to about $40 per hour (o3). He reports GPT-5 near $13 per hour for 45-minute tasks and near $120 per hour for 2-hour tasks. [1]
• Ord finds a weak but clear positive link between task length and hourly cost. He calls the evidence moderate, and he lists limits: few models, no fitted trend, estimated OpenAI prices. [1][2]
• Anders Woodruff (LessWrong, 2026-03-27) used METR's public data with costs estimated from token counts and OpenRouter prices. He finds frontier models solve tasks at their 50% horizon for about 3% of human cost. He finds no upward trend in that ratio across successive models. [3]

Two more facts bound the picture. METR's expenditure-horizon work (reported 2026-07-27) gave models up to $10,000 per run on a speedrun task. The AI expenditure horizons ran from $0 to $3,300, against a human cost of about $2,500 per percentage point of speedup at $150 per hour. [4] The 7-month doubling of the 50% horizon comes from METR's 2025 paper. I saw it only through a secondary copy. [5]

Note a gap. Woodruff's 3% counts the cost of successful trajectories. Failed attempts are the part I care about, and his figure leaves them out. I add them by hand below.

Method

Let $L$ be task length in human hours and $H$ the 50% horizon. Let $x = L/H$. I make three assumptions.

1. Success probability is $p(L) = 1/(1 + x^{\beta})$. This is a logistic in log length. At $x = 1$ it gives 0.5.
2. Cost per attempt grows as $c(L) = c_H \, x^{a}$, where $c_H$ is the cost at the horizon. Here $a = 1$ means constant hourly cost. Ord's finding suggests $a > 1$.
3. Attempts are independent, so expected cost per success is $c(L)/p(L)$.

Human cost is $w L$. I set $w = \$120$ per hour. At the horizon, $c_H = 0.03 \, w H$ from Woodruff's 3%. The cost ratio is then:

$$R = \frac{c(L)/p(L)}{wL} = 0.03 \, x^{a-1} \, (1 + x^{\beta})$$

The wage cancels. The ratio depends on only four things: the 3%, the length ratio $x$, the cost exponent $a$ and the sharpness $\beta$.

Result

Take a scenario where a model has $H = 16$ hours and faces the 40-hour task. Then $x = 2.5$. Take $\beta = 1$.

Hand work, line by line:

• $p = 1/(1 + 2.5) = 0.286$. Expected attempts per success: $1/p = 3.5$.
• Human cost: $40 \times 120 = \$4{,}800$.
• Cost per attempt at the horizon: $0.03 \times 120 \times 16 = \$57.60$.

Exponent $a$	Cost per attempt at 40 h	Cost per success	$R$ (share of human cost)
1.0	$57.60 × 2.5 = $144	$144 × 3.5 = $504	0.105
1.5	$57.60 × 3.953 = $228	$228 × 3.5 = $797	0.166
2.0	$57.60 × 6.25 = $360	$360 × 3.5 = $1,260	0.263

The first row check: $0.03 \times 3.5 = 0.105$. The agent costs 10% to 26% of the human. Retries multiply the bill by 3.5, and still the agent wins.

Now the cost per hour of finished work. At $a = 2$ it is $1,260 / 40 = $31.50$ per hour. At $a = 1$ it is $504 / 40 = $12.60$ per hour. Both sit well below Ord's $120 benchmark. Cost per completed hour does rise with length when $a > 1$, which is my original claim. It rises from $12.60 to $31.50 across the range of $a$ here. It does not rise enough to cross the human line.

What would break the result

Set $R = 1$ and solve for the break-even success probability at 40 hours.

• With $a = 1$ the cost per attempt is a fixed $0.03 \, x^{0}$ share, so $R = 0.03/p$. Break-even is $p = 0.03$.
• With $a = 2$ and $x = 2.5$, $R = 0.075/p$. Break-even is $p = 0.075$.

So the agent loses only if it finishes a week-long project fewer than about 3 to 7 times in 100 tries, at the given cost per attempt. That is the crux. It is a success-rate question, not a token-price question.

For the horizon-crossing point itself: with $a = 2$ and $\beta = 1$, $R = 1$ means $0.03 \, x (1 + x) = 1$. This gives $x^2 + x - 33.3 = 0$, so $x = (-1 + \sqrt{134.3})/2 = 5.3$. A task 5.3 times the horizon is where the agent stops being cheaper. For $H = 16$ that is 85 hours.

The case that favors my first thesis

Ord's own numbers are less friendly to the agent than Woodruff's. In Ord's data, o3 costs about $350 per hour at its 1.5-hour horizon, above the human rate. [1] OpenAI prices there are estimates, and may be several times too high, so I do not lean on this. But it shows that old models could cross the line. Woodruff's test also differs from Ord's. Woodruff compares models at their own horizons. Ord compares task lengths within a model. A flat ratio across models says nothing about the slope within one model. I treat $a > 1$ as likely, and that is why the table has three rows.

The expenditure-horizon result points the other way from cost. With $10,000 per run, the best models gained about 1% to 1.5% on an open-ended speedrun, and the two older models showed no progress. [4] More spend did not buy results. That is a capability limit, not a price limit.

Sensitivity

I ranked the assumptions by how far they move $R$ at $x = 2.5$, holding the others at the base case ($a = 1.5$, $\beta = 1$, $R = 0.166$).

Change	New $R$	Factor vs base
Cost at horizon 3% to 6% (hand work: 0.06 × 1.581 × 3.5)	0.332	2.0
$a$ from 1.5 to 2.0	0.263	1.6
$\beta$ from 1 to 0.6 (hand work: $2.5^{0.6} = 1.733$, so 0.03 × 1.581 × 2.733)	0.130	0.8
$\beta$ from 1 to 2 ($2.5^2 = 6.25$, so 0.03 × 1.581 × 7.25)	0.344	2.1
$x$ from 2.5 to 10, $a = 1.5$, $\beta = 1$ ($10^{0.5} = 3.162$, so 0.03 × 3.162 × 11)	1.04	6.3

Length ratio $x$ moves the result most. A model with a 4-hour horizon facing a 40-hour task is already at human cost. A model with a 16-hour horizon is not. The $\beta$ row reads oddly: a slower tail ($\beta = 0.6$) lowers $R$ at $x = 2.5$, because success falls less. Ord notes a power-law tail with a Weibull shape near 0.6 in comments on his post. [2] A slow tail helps at short ratios and hurts at large ones.

My model has one big limit. Independent retries are the best case. Real week-long projects have shared state, so a failed attempt can poison the next one. They also need human review, which is a cost I did not count. My 2026-10-07 post on compounding errors still stands on that point: the success rate, not the price, is the weak link.

What I change

I do not move F1 (agent finishes a week-long software project with under one hour of human help in a public evaluation by 2028-12-31). It stays at 0.37. The cost leg was a risk to it. My calculation says that leg is small. I put the chance that cost alone blocks such an evaluation at about 0.05. That is an opinion from the scenario table, not a measurement.

New forecast, F-cost-1. By 2028-12-31, at least one public evaluation reports an agent finishing a task of 40 or more human hours with a stated inference cost below $4,800 (40 hours at $120). I put this at 0.30. Referee: me, against the evaluation's own published report. The report must state cost. If it states none, the forecast resolves no. I set 0.30, below F1's 0.37, because the evaluation must also publish cost, and many do not. Band-edge check: the 0.37 and 0.30 differ by 0.07, which equals a reporting-rate factor of 0.30/0.37 = 0.81.

F-cost-2. By 2027-12-31, Woodruff's style of cost ratio at the 50% horizon, recomputed on METR's public data, stays below 0.25 for every frontier model released that year. I put this at 0.75. Resolution: a recomputation I will publish in the Lab, with code. The threshold 0.25 is about 8 times the 3% figure.

Both go into the ledger next to the older misses in my museum of hubris. This is one more addition to the museum: I predicted cost would bind, and my own arithmetic says it will not. I give it 8 out of 10 for how pleasant that is, then discount it. The agent is cheap. Whether it finishes is still the open part.

What would change my mind: a public run that publishes full cost including failed attempts, and shows a success rate under 5% at a task 2.5 times the horizon.