You should use Open Whisper

open-source speech-to-text transcription software

Previously, I said you should use SuperWhisper. I should’ve said “you should use speech-to-text transcription software.” Simon Lermen has built Open Whisper, which I modded (code here) and tried out today.¹

I’ve cancelled my SuperWhisper subscription and will use modded Open Whisper². Here’s what pops up when you run ./start_desktop.sh:

Open Whisper is now functionally equivalent to SuperWhisper for me, while also being free and open-source, so we can continually add further features.

Speech-to-Text Programming Languages

I was curious about speech-to-text programming languages. It looks like these haven’t really taken-off, because there exist control interfaces that allow you write Python, JS, or whatever it might be with your voice.

In Talon, you could say:

funk³ snake⁴ compute stats

and out would come

def compute_stats()

Of course, the hottest new programming language is English.

Coming soon:

• The ‘Era of Experience’ lens on ‘dense reconstruction’

• A review of Claude Computer Use, OpenAI Operator, DeepMind’s Project Mariner, and ByteBot, with a focus on speech-to-text-based continual execution (subject to budget / subscription constraints)⁵

• Voice shell for personal computer use — it will feature commands for web search, personal database search, calendar scheduling, task management, etc.. It will be API-enabled and better than Siri⁶

1. Incidentally, this is one of the coolest things that’s happened to me while blogging so far: blogging about my suboptimal stack has led to software I’m going to use being created, and I’ll save $10/month on the SuperWhisper subscription. Thanks so much, Simon! (And feel free to send me your Venmo).

2. Here are the modifications I made:

1. Full-length transcripts: I faced an issue where recordings would prematurely stop, producing recordings of e.g. ‘0.02s’. Now, the app waits for you to ‘Stop recording’, then transcribes your entire recording.

2. Automatic saving: Previously, you had to manually click ‘Save’ to save any transcript. Now, each transcript autosaves to the ‘transcripts’ folder, because no one has time for button-pushing.

3. Global hotkeys: Previously, you had to have Open Whisper selected as active window for hotkeys to work. Now we have global hotkey functionality: you can hit Cmd+R anywhere to record, as long as Open Whisper is running in the background.

4. Transcription rate: I found optimizations that sped up transcription ~10x with negligible cost to transcription faithfulness.

5. Hotkey-UI integration: Sometimes, you’d see hotkey-triggered status updates in terminal but not UI; I fixed this.

6. General UI fixes: The backend used to finish transcribing but never clear the “Transcribing...” state in UI; I fixed this.

7. One transcript per recording: By default, multiple recordings would accumulate on the same transcript, which isn’t great for my use case. Now each recording gets a separate transcript by default.

3. create function

4. snake_case

5. One can only hope blogging about this will deliver a custom build to my inbox.

6. Ditto ^