Endogenous Automation Will Hit You

How I changed my mind about humanity's 'skill issue'

Under what conditions do humans lose wage share? Does that matter?

In Gradual Disempowerment, Kulveit et al. discuss how humans might lose influence once labor-intensive tasks are automated.

It contains one chart:

This chart is inspired by Fig. 8 (b) from Korinek and Suh (2024). (b) is one of multiple scenarios Korinek and Suh illustrate:

In scenario (b), all tasks are automated and human wage share collapses. In scenario (a), humans get to keep our wage share—at least initially. Why?

In Business-as-Usual scenario (a), humans get to retain wage share (and thus influence) because the set of possible economic tasks is modeled as unboundedly complex. By contrast, in Baseline AGI scenario (b), there’s an upper bound on the complexity of productive tasks:

Throughout the paper, we analyze two opposing cases for the distribution of tasks in complexity space, which result in sharply different economic outcomes. First, we consider the possibility that human tasks are of unbounded complexity, illustrated in the left-hand panel of Figure 1. In this case, advances in the automation index, illustrated by the right-ward movement of the vertical frontier of automation, imply that more and more tasks are automated over time, but that there always remain tasks and by extension jobs that cannot be automated. Second, we consider a bounded distribution of task complexity, which reflects that the computational capabilities of the human brain are finite, as discussed, e.g., in Carlsmith (2020). Bounded distributions result in full automation within finite time when the frontier of automation crosses the maximum complexity of tasks performed by humans.

I want to consider the arguments re which distribution we live in.

I’m initially drawn to the first model (‘Unbounded distribution’) because I like the idea of fractal complexity. I really like Phil’s point that “many now do narrow versions of what would once have been called “scribe” or “philosopher” and considered <1% of tasks.”

But there are issues with this way of thinking. See this from Trammell and Aschenbrenner (2025):

Recent literature on technology on labor markets observes that innovation typically gives rise to new job tasks (e.g., Acemoglu and Restrepo, 2018; Autor, 2019). This holds true when viewed from the perspective of high-level job tasks such as those captured by O*Net. However, when viewed from an atomistic level that reflects basic brain functions, innovation merely recombines atomistic tasks in novel ways to produce novel high-level tasks and jobs. For example, the novel task of prompt engineering may require atomistic tasks such as defining a desired output, crafting an initial prompt, entering it, reading the output, evaluating it, deciding whether to iterate, and finally sharing the output—all functions that existed long before the invention of generative AI systems that triggered prompt engineering.

I’d like to say we can do more than observe to work out which world we’re in, and that we can actually move ourselves towards the unbounded (more-favorable-to-human-wage-share) world by creating ever more complex tasks and increasing our ability to solve them.

That probably looks like using AI to make humans more capable. Perhaps an aligned AI is one that differentially advances humans more than itself. Forget recursive self-improvement—what about differential co-evolution?

These are not new thoughts. Papers that treat humans as static are weird—I’m interested in what ‘centaur evaluations’ will bring.

Human augmentation raises lots of interesting ‘personal identity’ questions: how can we expand and still feel like ourselves?

That’s not the solution Korinek and Suh explore, though. Here are the other two scenarios they illustrate:

Scenario (c) is just a sped-up version of (b): there’s an upper bound on task complexity; the automation frontier sweeps in and covers all of them. Straightforward enough. Scenario (d) is their ‘solution-adjacent’ one:

Furthermore, [in scenario (d)] we analyze societal choices to retain certain jobs as exclusively human even when they can be automated (e.g., priests and judges), and show that a sufficient volume of such nostalgic jobs may help to keep labor sufficiently scarce so that wages continue to grow even when full automation is technically possible. We analyze the wage-maximizing rate of automation and show that slowing down automation in an AGI scenario may deliver significant gains to workers albeit at the cost of forgoing a growing fraction of output.

I instinctively recoil from this. I have strong priors that we should think very, very hard about forfeiting growth so that incumbents can collect rents, which seems pretty reasonable in the scheme of humanity so far (think: YIMBYism).

I also have residual priors that you should experience the influx of capable competitors as an urge to better yourself. So I wanted to argue that human augmentation can help us shift from ‘cheems mindset’ scenario (b), where humans can only do so many tasks, to ‘swole mindset’ scenario (a), where humans master unboundedly complex tasks. After all, the set of active tasks hasn’t stayed static over the centuries: there have been expansions moving from the medieval era through Renaissance, Enlightenment, etc..

But then Phil highlighted the phenomenon of endogenous automation. Human wages continuing to be significant means there’s an incentive to automate that labor away. And that seems…basically right? And pretty sobering?

We might still be in world one (with unboundedly complex tasks), or able to yank ourselves there, but wage share still falls in a way not shown in Korinek & Suh’s graph (maybe off-screen).

You can no longer take the influx of capable competitors as a challenge to better yourself and expect to make a living off your contributions. This is a distinct pattern I want to keep noticing and exploring: ‘my intuitions from the old world don’t apply anymore’.

Thanks Tim Hua and Pedro Adighieri for organizing a research hackathon on gradual disempowerment! Thanks Phil Trammell for organizing ETAI.