Will MacAskill’s latest post helps me identify that my focus when establishing ‘vitae per person’ was on making ‘primary goods’ — health, material abundance — accessible to everyone. That requires me to justify why I think this isn’t priced-in / won’t automatically happen by default.
I think the most likely reason it doesn’t happen is we can’t find a way to build AI that does all three:
(1) advances R&D / manufacturing at the necessary rates
(2) is objectively safe — doesn’t hurt lots of people (vs. counterfactual world without it)
(3) satisfies the naysayers (Yudkowsky & Soares, CAIS, equivalently pessimistic policy efforts)
(1) & (2) should satisfy (3).
Interestingly, public access to frontier AI systems is not required for (1). This suggests I should shift my focus towards advancing AI systems that are safe when wielded by only good-faith actors, rather than ensuring frontier systems are safe when accessible by everyone with every intention.
Good-faith actors might take many (overlapping) forms:
Trustworthy government agencies
‘AI for science/society’
Anthropic / OpenAI / …
‘Aligned’ models we’re using for oversight
I’ll coin this:
The Good-Faith Actor Principle: Frontier technologies should be safe when wielded by people who are setting out to cause good, not harm. They don’t have to be safe when wielded by people setting out to cause harm.
We already apply the Good-Faith Actor Principle to nuclear and biological technologies. Given the stakes associated with increasing primary goods provision, I think we should apply it to AI systems.
There are three reasons you might think it’s important to make frontier AI systems safe when used by every possible actor, not just good-faith ones.
Security. You might think we can’t secure a frontier system: that ‘bad-faith’ actors might get their hands on it despite our best efforts. I’m cautiously optimistic about securely limiting access.
We need all hands on deck. You might think that the cause of advancing science / society is a collaborative effort, and that limited actors (frontier labs, government agencies) can’t do it alone: you need all of society to use their creativity and ingenuity to apply frontier systems to their particular problems.
To discuss this, let me introduce three levels of technology access: Universal, Restricted, and Controlled.
Right now, Opus-4.5 and GPT-5.2 are ‘Universal’ technologies. Proprietary base models have often been ‘Restricted’ technologies. Staged releases often pass through the three levels, from Controlled to Restricted to Universal, to allow for early user testing and feedback.
I’m optimistic about us being able to reap the benefits of society-wide creativity while keeping frontier AI systems a ‘Restricted’ technology.
Inseparability: Aside from the sociotechnical problem of vetting ‘good-faith’ actors to be granted access, you might think that the safety of AI systems when wielded by (A) bad-faith actors and (B) good-faith actors are correlated, such that improving (A) improves (B).
This is not obvious. Some of the techniques you have to apply to regularize / reduce variance in AI behavior have tradeoffs — muting beneficial variance, and, importantly, introducing brittleness.
An AI system optimized for Universal access might look quite different to an AI system optimized for Restricted access.
I’d be concerned if we forfeit lots of benefits from frontier AI systems by prioritizing Universal access.
Looking at the design/spec of a system, I evaluate it according to the Good-Faith Actor Principle: checking good-faith actors can reliably use it to good ends. That’s because I think the following:
Expanding primary good provision is really important.
Powerful technologies can help us with this.
It’s easier to design helpful, reliable ‘Restricted’ technologies than helpful, reliable ‘Universal’ technologies.
This implies that when the development of Restricted and Universal technologies trade off, we should prioritize Restricted.
I'm pessimistic that
> (1) advances R&D / manufacturing at the necessary rates
will naturally flow through to primary good abundance.
I think the most likely scenario is that human workers are gradually replaced with AI. COGS goes down but I don't foresee commensurate drops in prices. I think my view here is too complicated for me to try to explain in a comment, and I should just write a post if I'm going to go into it more. I'm not sure if anyone would read it though.
Basically I think a lot of people are going to get laid off and lose their purchasing power while simultaneously the corporations adopting AI and doing the layoffs will not have profit incentives to drop their prices enough to allow disenfranchised labor to maintain their standard of living, let alone abundance.
So primary good abundance depends on political/government action. I think governance failures are the most likely reason why primary good abundance doesn't happen.
Democracy and collective self-interest will not necessarily work here. A lot of people still fundamentally have an adversarial attitude towards other humans, and especially welfare recipients. Corporate media incentives feed these tendencies. AI cannibalizing the memetic landscape won't help either.
My prediction is that after AI productivity gains, eventually governments will adopt mass redistribution on an unprecedented scale, but the transitional decades will be very painful with a dramatic bifurcation between those that manage to stay employed (or have assets) and those that don't. The political fights over redistribution will remain equally acrimonious, and households that lost out during the transitional decades will never really catch up.