Part One: 'The cost of mastery’ || Part Two: ‘The cost of specialization’
1. Competence
a. Intrinsic Cost
While working on research into increasing LM consistency, I ran into an unpleasant observation:
If consistency is actually a dangerous capability, then I should be cautious about pushing it.
Right now, I get to be curious because I’m small. My experiments are micro-scale, within the ambient research distribution. Nothing I do is out-of-distribution enough to bear real moral load.
But if that changed—if I started doing more robust or decorrelated things—I’d inherit obligations I currently avoid. And I can feel the incentive to not let those conditions change, because I like the freedom I have.
That’s the pull of amateurism.²
b. Opportunity Cost
Competence tightens the constraint set. Public mastery implies a heavier burden of evaluation: what I do, why I do it, which trajectory I’m nudging. That trades directly against curiosity.
2. Recognition
I currently open-source almost everything—drafts, experiments, stray thoughts. I don’t worry about being scooped; if someone copies me, they know who originated it; that’s enough. The rewards of sharing freely / openly are so high.³ Besides, I have more ideas than I know what to do with, and it’s a blessing to see someone else did ‘my thing’—that means I can move onto more exciting things! I really enjoy inhabiting this fearless mindset, and want to continue doing so.
But if I became more original or more visible, the calculus flips. A larger audience amplifies impact, which increases the cost of mistakes, which pressures the mind to regularize what it produces.
a. Moderating what you post
If your reach grows, every post inherits latent second-order effects: what you normalize, what you inadvertently prototype for others, the degree to which you shape the field’s priors. This creates a subtle incentive to cap audience size—stay amateur, protect freedom.
b. Moderating what you allow yourself to think
The deeper danger: thought-space collapse. Public thinking creates an internal critic that optimizes for legibility. Gavin once warned me: “public intellectuals can become hostages to their audience.” It’s easy to end up with tamer thoughts, prematurely rounded edges, a mind optimizing for scrutiny instead of exploration.
Conclusion
Competence and visibility both erode intellectual freedom because the opportunity cost landscape sharpens. Once your moves matter, the tradeoffs stop being hypothetical.
This piece tries to expose that structure and commit, in public, to choosing competence + responsibility anyway.
¹ subjective-lifespan effects; more in part two.
² very AI-safety-specific; sometimes i think about how to avoid this.
³ transparency fights insularity harder than secrecy fights “being scooped.”
come to think of it this is not so much the cost of getting good as it is the cost of getting famous/looked up to for opinions in the field.
IMagine 2 scientists, both are equally cracked and in the exact same field. Except one has a substack with 10k followers and is twitter famous and the other one maintains no online presence whatshowever.
It's clear that the latter is way more free from audience capture and all the other things mentioned here, you're using being cracked as a proxy for fame/recognition (ie: percieved competence)
That image is so fucking true. great post