WARNING: This Rust code was written by hand.

back to the old days

I recently had a bit of a self-identity crisis; of course, it was tied to spending way too much time on social media. I do try to avoid the worst of it, got rid of Twitter (X) this year, and mostly stick to Substack, LinkedIn, YouTube, occasional Reddit, and Hacker News. Probably still too much.

Anyways, I’ve always been one to keep as close to the bleeding edge as possible throughout my career, at least from my view, without wandering over the edge. I’ve always seen it as a form of self-preservation, probably due to growing up the youngest of five in the middle of the prairie next to the river, in the only county without a working stoplight.

I try to be pragmatic and never dogmatic, except when someone says “Production” and “Notebook” in the same sentence.

Oh, back to my identity crisis. I probably just need to turn off social media, but I’m constantly torn about my future as a data and code person. In my heart of hearts, I believe things will be fine for those who care for their craft, as applied to the business of software, but that could just be wishful thinking.

Again, I digress; what I’m trying to say is that I miss writing code. Myself. Without a machine looking over my shoulder. Rust, I miss that nasty little borrow checker and compiler. I miss struggling.

If life writing code for two decades has taught me anything, it’s that struggle and exercising one’s mind are critical to success, personal and professional, and I’ve felt it sorta creeping up on me, slowly and sneakily.

• Don’t get me wrong: spending time on systems design, architecture, business requirements, project planning, the big picture, nitpicking Claude solutions, and providing value through software solutions … it’s important work.

But I can only accomplish those tasks well insofar as my current mind is continually exercised and acquainted with the intricacies of software used to solve data problems and provide value. Building systems that run well, are cost-effective, and are generally a pleasure to work with.

---

Customer-facing analytics that ship in days, not quarters

Thanks to Cube for sponsoring today’s post! Without companies like Cube, this content wouldn’t be possible. The best way to support this Substack is to click the links below and check out Cube!

Customer-facing analytics are slow, the warehouse bill keeps climbing, and every new chart turns into an engineering project. Cube is the agentic analytics platform built for embedding: drop governed dashboards and an AI analyst into your product in days, with built-in caching that keeps queries fast and warehouse costs down. Pelago, Constant Contact, GoFundMe, and 400+ other SaaS companies ship customer-facing analytics on it.

Get started for free.

Check out Cube today!

---

I, for one, find it surprising how quickly those finer details seem to fog over in the Cold Claude War we find ourselves in. Sure, it’s hard to erase 20+ years of code and experience, but it can get a little opaque.

So today, I’m here to do a simple thing with you. Bust out the terminal, vim, Rust, cargo … and solve a problem I deal with daily and find annoying. The most important part is that my own two fingers, with a few Google searches, are going to do this work.

WARNING! No LLMs were used in generating the following code.

---

The problem.

So, what is this itching problem I deal with way too often each week? Well, I’m sure, or I hope you are, that, along with the written word I produce for you, more than I should, includes a lot of Video/Podcasting content. Substack Podcasts, YouTube, etc.

The problem is simple: I need the transcripts from my video and podcast recordings. For all sorts of reasons. I need the transcript to summarize and find …

• What I/we talked about, aka topics covered

• figure out a good title and description

• etc

There are a lot of AI tools I can pay for to do this for me, but I want my own, My Precious. So, we are going to whip out Rust and bang our heads against a wall until we figure this out.

cargo init transcription-rs
cargo add unbundle

I’m thinking we just want to accomplish a few simple things …

• command-line CLI to run this tool

• reduce dependencies

• rip out audio media from a video file

• … try to transcribe the audio file

I’m excited to write some Rust code without any LLMs, to think through the problems and struggle on my own, like the good old days.

• I found a Rust crate that uses the powerful and apparently popular FFmpeg, which has been used since the beginning of time.

---

Re-learning Rust without LLMs.

Let’s start our journey by using this crate to do something simple, knowing it probably won’t be our final solution. But can we open an MP4 video file, a clip from one of my podcasts, and have an audio file ripped from it?

use unbundle::{self, MediaFile, AudioFormat};
use std::env;

fn main() {
    let args: Vec<String> = env::args().collect();
    let input_file = &args[1];
    println!("Starting to work on {input_file}");
    let video_data = unbundle::MediaFile::open(&input_file).expect("Failed to open video file");
    process_video(video_data);
}

fn process_video(mut video_data: MediaFile) {
    video_data.audio().save("output.wav", AudioFormat::Wav).unwrap();
}

We will probably need to use clap for proper CLI features at some point, but raw arg input parsing will work for now. So far, it’s pretty straightforward, at least in proving out our idea.

• It’s a good reminder of Rust's core principles.

When it comes to mutable and immutable, we can think of our data and variables in such terms. This is far from the land of Python, where everything can be anything and do whatever. Also, another Rust beauty is borrowing: can we borrow a copy of a pointer to some data or thing? For example, our argument and variable for input_file should be immutable and borrowable. No need for it to change.

This is in contrast to our video_data, which holds the data from our MP4 file; it is mutable because we need to extract the audio data from it.

---

< Always thinking about data and how it’s used. >

• Immutable data …

  • A file_name would/should have no need to ever change.

  • We should probably always just borrow the file_name if we need it

• Mutable data …

  • If you scroll down a little, you will see we eventually declare video_data as mutable, because we need to rip the audio off it.

If you work with data, you should train yourself to think of data points like Rust does: Is this piece of data changeable? Should it ever change? Does it become something else if I change or consume it? Where is the data used and how is it used?

It forces a deeper understanding of the data problem space you’re working on.

---

One thing I noticed was that this solution depends on the machine having a proper installation of FFmpeg. I would prefer a pure Rust implementation, but maybe we can deal with that later. Let’s run our little bitty pieces of Rust and see what happens.

danielbeach@Daniels-MacBook-Pro-2 transcription-rs % cargo run -- test.mp4
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.06s
     Running `target/debug/transcription-rs test.mp4`
Starting to work on test.mp4

• Well, our hacky little Rust script works; here is our output.wav audio file ripped from a video MP4.

Next, transcription: hackey is fine, but we need to pull text from the audio file.

---

Transcription with whisper-rs.

One thing I enjoy about hand-rolling code is not handing off the thinking part to our benevolent master Cladius Maximus. Working on scratching out a version of what I want requires me to do my own research into what others have found useful for solving problems.

The problem with The Vibes is that we don’t question the LLM’s choices enough. We tab and move on. What causes the invisible gremlins, the weights trained in some dark evil frontier lab, to pick one tool over another for solving a problem?

• Those reasons won’t be the same for a human.

OpenAI has a model, Whisper, that is a general-purpose speech recognition. Look, maybe we will turn back to the models, but not for our code.

Looks like we have some pretty small Whisper models available, which is great; we can probably use anything from tiny to base.

Of course, we could find the Rust wrapper for this tool, basically the Rust bindings to whisper.cpp

Ok, so let’s see if we can add some more hacky Rust code to get whisper-rs to extract a transcript from our audio file. Don’t worry, we will revisit the code at some point to do some refactoring, but I’m mostly just thinking about getting the end-to-end working at the moment.

• Doing things “right” can come later.

Ok, let’s download one of the Whisper models onto our machine.

mkdir -p models
curl -L -o models/ggml-small.en.bin \
  https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-small.en.bin

Easy enough, now for the Rust bindings.

use unbundle::{self, MediaFile, AudioFormat, AudioHandle};
use std::env;
use whisper_rs::{WhisperContext, WhisperContextParameters, FullParams, SamplingStrategy};


fn main() {
    let args: Vec<String> = env::args().collect();
    let input_file = &args[1];
    println!("Starting to work on {input_file}");
    let mut video_data = unbundle::MediaFile::open(&input_file).expect("Failed to open video file");
    let audio_data = process_video(&mut video_data);
    
    let path_to_model = "models/ggml-small.en.bin";
    let audio_samples = pull_audio_samples(audio_data);
    pull_transcript(path_to_model, &audio_samples);
}


fn process_video(mut video_data: &mut MediaFile) -> AudioHandle<'_> {
    return video_data.audio();
}

fn pull_audio_samples(audio_data: AudioHandle<'_>) -> Vec<f32> {
    let samples: Vec<f32> = audio_data
      .sample_iter().expect("Failed sampling")
      .flat_map(|chunk| chunk.unwrap().samples)
      .collect();
    return samples;
}


fn pull_transcript(path_to_model: &str, audio_sample: &Vec<f32>) {
    let ctx = WhisperContext::new_with_params(
        path_to_model,
        WhisperContextParameters::default()
    ).expect("failed to load model");

    let params = FullParams::new(SamplingStrategy::BeamSearch {
        beam_size: 5,
        patience: -1.0,
    });

    let mut state = ctx.create_state().expect("failed to create state");
    state
        .full(params, &audio_sample[..])
        .expect("failed to run model");

    let mut transcript = String::new();
    for segment in state.as_iter() {
        transcript.push_str(&segment.to_string());
    }
    println!("{transcript}");
}

You can see that not much has changed, just the addition of whisper. When we process_video we return the raw audio, then we chunk that raw audio data out into a Vector, this allows us to pull_transcript with whisper.

Running the command.

PATH="/opt/homebrew/bin:$PATH" \
  PKG_CONFIG_PATH="/opt/homebrew/opt/ffmpeg/lib/pkgconfig" \
  cargo run -- test.mp4

This is hilarious.

whisper_full_with_state: decoder  4: score = -5.57022, result_len =   4, avg_logprobs = -5.57022, entropy =  1.38629
whisper_full_with_state: best decoder = 0
single timestamp ending - skip entire chunk
seek = 36985, seek_delta = 2141
 I live in the capital of Twerba and Trutar. [growling] I am a woman. I am... my wife. 
[sigh] [growling] [growling] [growling] [growling] [growling] [growling]
 [growling] [growling] [growling] [growling] [growling] [growling]
 [growling] [growling] [growling] [growling] [growling] [growling] 
[growling] [growling] [growling] [growling] [growling] [growling]
 [growling] [growling] [growling] [growling] [growling] [growling] 
[growling] [growling] [growling] [growling]

So, apparently, we have something running end-to-end, although I can tell you this is not how the conversation went. Maybe with the exception of “my wife. [sigh].”

Clearly, we have a frigging in the rigging somewhere, but I see this as a positive: the fact we have ~50 lines of Rust code and can take a video file and get something that is supposed to be a transcript out … well, that’s fun!

I’m no audio or video expert, but one can imagine this has something to do with the audio or video format, probably the audio being pulled from the video. From what I can tell, the Whisper model requires “16 kHz sample rate, 16-bit samples for audio.”

• Great, now I, who know nothing, need to figure out how to resample audio data.

Of course, a quick Google search shows rubato, a small Rust package we can use to resample. The rubato docs show a simple example of how to do the resampling, so hopefully this shouldn’t be too bad.

One thing we should also do is add a check to get the current sample rate, then convert it to the 16Hz required for Whisper to work. I download video files from all sorts of sources: Riverside.fm, Substack, etc. Ok, let’s update our pull_audio_samples method to determine the sample rate of the chunks we are working with.

fn pull_audio_samples(audio_data: AudioHandle<'_>) -> (Vec<f32>, u32) {
     let mut sample_rate = 0u32;
     let mut samples = Vec::new();
     for chunk in audio_data.sample_iter().expect("Failed sampling") {
         let chunk = chunk.unwrap();
         sample_rate = chunk.sample_rate; // same on every chunk
         samples.extend(chunk.samples);
     }
     (samples, sample_rate)
 }

Then, the nasty old complicated resampling; it’s ugly to look at, too verbose for my liking, but it works.

fn resample(input: &[f32], from: u32, to: u32) -> Vec<f32> {
    let mut resampler = Fft::<f32>::new(
        from as usize, // input rate
        to as usize,      // output rate (16000)
        1024, // chunk size in frames
        1,                  
        1,   // channels = mono
        FixedSync::Both,
    )
    .expect("resampler construction failed");

    let in_len = input.len();
    let out_len = resampler.process_all_needed_output_len(in_len);

    let input_buf = InterleavedOwned::new_from(input.to_vec(), 1, in_len)
        .expect("input buffer construction failed");
    let mut output_buf = InterleavedOwned::new(0.0f32, 1, out_len);

    let (_nbr_in, nbr_out) = resampler
        .process_all_into_buffer(&input_buf, &mut output_buf, in_len, None)
        .expect("resample failed");

    let mut out = output_buf.take_data(); // mono => frames == samples
    out.truncate(nbr_out);
    out
}

Can you imagine some Rust wizard sitting in a dark basement somewhere who thought it was fun to write a Rust crate for resampling audio? Good Lord.

Anyways, with these two update/new methods, let’s re-run our script and see if we can get a real transcript back, and not “I live in the capital of Twerba and Trutar. [growling].”

single timestamp ending - skip entire chunk
seek = 12328, seek_delta = 852

Yeah, so I predict that companies are gonna continue to struggle to get return. They’re gonna pay a lot of money to Enthropic and to OpenAI and wonder where are the results. I mean, I think there are results, but there’s some results that are positive, but there’s also a lot of waste and slop and consequences of the slop. And yeah, and again, right, I think the foundation labs, like as long as they sell tokens, like as long as people are still paying for the tokens, like they’re happy, but I think as companies put like harder constraints on token budgets and things like that, it’s gonna be really interesting to see how things evolve. But maybe there’ll be some radical technological innovation that makes like producing tokens a lot cheaper, such that what I really want is I got my hands on GLM 5.2, kind of like one of the latest open weights Chinese models and running on physical infrastructure, and it worked great. And I’m just like, how long will it be till I can just have like a server in my closet that runs one of these, one trillion parameter models, open weights models, and I don’t have to, I can just pay for the power for that server, and I don’t have to, it’d be nice for that server to like not cost 50 or $100,000. I think that’s how much a server that can run these models costs right now, but maybe in the fullness of time, like the open weights models would get good enough, and the hardware will become affordable enough that there won’t be a need to send all of our money to Dario or Sam Altman, and just plug those in, put the machine into the wall and get coding. I think I’m looking forward to that.

Well, look at that! We got our own decent video-to-text transcription tool! Of course, we could have used a slower, more accurate Whisper model, but this looks good enough for what I need! All I need to do is maybe add another command-line argument to write the transcript to a text file.

use std::fs;

let ouput_file = &args[2];
...
let transcript = pull_transcript(path_to_model, &samples);
fs::write(ouput_file, transcript).expect("Should be able to write to {output_file}");

---

Thoughts and more.

Well, that was fun, wasn’t it? A little wandering down memory lane to the good old days of fighting the borrow checker and compiler. Trust me, there was plenty of that going on that I didn’t show you.

If you want, as always, you can get this codebase on GitHub.

To be honest, it was good to have those old feelings of not being able to figure something out right away. It took me two days, off and on, during my free time to hack this code together. Remember to exercise my brain about data, mutable or not, borrow it or not, data types … you know … just programming stuff.

Sure, I bet Claude could write this Rust code way better than I, much prettier, catch all the rough edges, who knows what path it would have chosen. That’s ok. I enjoyed the process of researching and thinking myself. Not worried about anyone else but me.

It was a nice reminder of minimalist code: not everything needs a ton of boilerplate, “to be done right,” or anything fancy. Seems like Rust is like riding a bike; it comes back awful quick. Maybe I will make it a habit.