I fine-tuned Llama-3.1-70B on my blog posts

soon: a tool for you!

TL;DR

• Used Tinker to fine-tune Llama-3.1-70B on my blog posts. Liking the outputs so far. I’ll make it easy for you to do the same.

Coming Soon

Is it actually good?

I think so.

Here’s an annotated sample output:

captures my actual views + lydia-coded stylistic elements

You can view more on this document.¹

What’s more…

sufficiently well-fed fine-tuned models are indistinguishable from human writers. inkhaven coaches, watch out! ;)

Plans:

High-Priority:

1. Create a LydClone wrapper that generalizes

   1. It will look like a prettier version of this:

i wish ethan / mindthemanifold wrote more blogposts, but three was a good number for prototyping today :’D

2. Get the ‘Substack URL → neatly-wrapped fine-tuned model’ pipeline working properly!

   1. Later, extend to non-Substack blog formats.

Medium-Priority:

1. Test out other models:

   1. Qwen3-235b-a22b-instruct-2507 (Instruction, MoE, large)

   2. Deepseek-v3.1 (hybrid, MoE, large)

   3. OpenAI gpt-oss-120b (reasoning, MoE, medium)

2. Convert my reactions / comments, annotations, & feedback on the outputs of LydClone into actually-useful training signals

   1. ChatGPT gives some pointers here

      1. Pairwise preference DPO

      2. Edit-based supervision

3. Stress-test / benchmark the model

   1. Can the model reveal my true priorities?

   2. Or accurately predict my takes on things I haven’t encountered yet?

   3. Can it tell what I’ve encountered & what not?

4. Cost estimates

Scaffolding

OK, so now there’s the question of how I put this fine-tuned version of myself to good work.

Research

The cool thing is this model can read as well as write. So I can get it to funnel insights into my research agenda.

1. LydClone stays abreast of current ML research

   1. Reads weekly ML research

   2. Feeds back how it relates to research agenda / priorities

   3. Flags highest-priority papers for me to read

   4. AI Scientist integration

Feed

I want to simulate my first-pass interaction with almost everything. I want to improve my Twitter feed / the posts that make it to me

• This is most appropriate for cases where false negatives are ok—i.e. feeds I wouldn’t possibly have time to monitor + scrutinize by myself, but would benefit from seeing hidden gems from

  • Like SciRate or Science Talk

• Probably LydClone just needs an RSS feed in general, and now RSS will actually be manageable, because LydClone will filter out all the fluff.

  1. Filtering by writer doesn’t work for me—I have to filter by writer and topic, i.e. I’m most interested in a) technical topics & b) actual good meta life advice / applied rationality / frame-sharing stuff²

Group Cognition

1. Rather than reading auxiliary papers myself, I’ll probably send LydClone off to read 3-4 each morning and report back

2. I can also set LydClone off to read the most relevant new releases³

Bene Gesserit stuff

1. Model-merging

   1. cf. K-Values of Speculative Kinship

The Existential

I recently asked a non-blogger friend “if you die tomorrow, what will be left of you?”. I felt kind of emotional seeing there is now something left of me where this time last year there was nothing.

Updated Priorities

1. I was already planning to continue writing daily post-Inkhaven. The early LydClone results reinforce this conviction, cast-iron.

2. It also updates me towards thinking metacognition posts are extremely worthwhile. I think these have been key to LydClone producing outputs I recognize as coming from a ghost, whisper, shadow of myself.

========================================================================
SAMPLING FROM YOUR REAL FINETUNED LLAMA 70B
========================================================================

Checkpoint: tinker://5e055c1d-a64d-5886-bb21-d59f26ce83b2:train:0/sampler_weights/ephemeral_175

Loading sampling client from checkpoint...
✓ Loaded!

Getting tokenizer...
✓ Tokenizer ready!

========================================================================
GENERATING SAMPLES
========================================================================

1. These seem like promising outputs to me. I’d endorse reading more like them.
It looks like LydClone can give ~my thoughts on topics I haven’t considered before. Seems like a powerful tool for combining ideas / topics in ways I’d endorse.

2. Zvi recommended some posts today; I like stuff like these & the IFS guide

3. Jenn mentions her rationality meetup group split ‘Situational Awareness’ and each read a section, then reported back, meaning they could have a high-quality discussion the week it came out without each person having to read 150+ pages!