A tiny amount of force applied in just the right place

& unified theories thereof

edit: rediscovered the post through which i must’ve internalized this frame:

A relatively small amount of force applied at just the right place

Dynamical systems as a frame for impact

I’ve previously written about backdoors as a Lipschitz clamping problem: the idea that various inputs can induce/elicit downstream behavioral distribution shifts, and we say ‘backdoor’ only when the input is particularly tiny/well-formatted and the distribution shift is particularly nasty/undesired.

The funny thing is, this mirrors the problem we face as researchers, or people, trying to have a positive impact in the world. We are trying to find the sensitive points where the application of just a little pressure can lead to overwhelmingly positive downstream effects. It’s a symmetric problem.¹

Dynamical systems:

Infinitesimal perturbations causing macroscopic qualitative changes are literally the definition of bifurcations. “Large downstream effects” = change of attractor, basin switch, chaos onset. This is not metaphorical.

So in which senses are we on the rim between basins?

---

the unbearable customizability of new shiny tools

I already knew how to say basically anything with statistics or philosophy. Now I feel able to say basically anything with AI/ML systems. I feel as though I could make a model organism that says anything — challenge me with something I can’t make a model organism for?²

When the world feels too customizable, things get confusing. You can make the models do anything. So of course it finally gets important to figure out what you actually want them to do.

You need to figure out the intersection of ‘what you actually want them to do’ and ‘what they’re not going to do by default — or get be made to do by default’. You can get into an endless game of chicken with other scientists.

In ‘Vitae per Person’, I suggested that “my top priority is moving us towards a world with at least 8 billion vitae (one for each person alive today).” Recently, I’ve been thinking vitae per person might be ‘priced in’. Conditioned on keeping control of advanced AI systems in good hands, we might just get ‘abundance by default’.

That leads to some sort of crisis / loss of meaning. If I understand correctly, ‘Deep Utopia’ is about ‘life and meaning in a solved world’. Maybe I ought to give it a spin. We have Adrià’s ‘alignment by default’ and now Dalrymple singing a similar tune. What’s left to do?

Constantly caught between “oh my goodness, there’s so much left to do…there’s so much that it would be nice to know” and “what is really important to know? What questions would I seriously pay for the answers to?”

an attempt at answering that

I would seriously pay for cybersecurity assurances. I’m unconvinced we get ‘cybersecurity by default’, even if history might lull us into a false sense of security.

---

There are still lots of things I want to understand out of curiosity, and within that set I can order them by ‘how much the world needs to stare at this’. And that’s probably what I’ll continue doing for now. But I notice I don’t have a strong theory of “what’s not priced in?”

I hope to flesh one out.

1. It’s also a problem I faced a lot when mentoring & investing resources into younger people under time/resource constraints, like being a counselor at rationality camps: the most cynical possible instantiation of this. Who yields huge ROI — a small nudge and they’re off to the races, for themselves and others — and who could you sink dozens of hours into talking to without getting anywhere?

yes these tradeoffs were real. yes this frame is problematic and versions of it at varying levels of implicit/explicit also served us really well sometimes.

2. i believe there are probably many such things! i think it would be interesting to try and work out what they are, and the reasons we can’t make them (i.e. in each case is it just an ephemeral model capability issue, or something broader/deeper)