I think including the word “OpenAI” in the post name is somewhat a misnomer that implies an encrapification not really happening to the FFMPEG project.
Yes, it is true OpenAI originally developed the Whisper model, and I hate OpenAI; however:
- Whisper is actually open source, unlike most OpenAI crap.
- FFmpeg isn’t even directly using the OpenAI version, written in Python - they’re using a port to C++ called Whisper.cpp
- We’ve been able to use speech recognition for decades, so unlike other AI models, I don’t think a speech recognition model that does it better is problem.
- You don’t even necessarily have to compile FFmpeg with Whisper support.
I get the dislike of AI, but the idea of association with OpenAI is overblown and not really reflective of reality. Now I can get not wanting to use open source projects whose developers don’t reflect your principles; however, I think this ethical issue is more indirect than may initially appear and is not a strong reason to quit using what is still the most effective media conversion tool.
Hopefully the speech recognition is better than whatever the fuck most online video platforms use for automatic subtitles at the moment.
I’ve built an app with Whisper, the level of ‘hit or miss’ entirely depends on the size of the model and language. Even audio quality is a lesser factor in my experience. So, it depends…
I was messing around with HomeAssistant the other day, which uses the same speech recognition engine, and I found it to be decent.
has anyone compared vulkan av1 to nvenc or vaapi? too new still?
ugh so what’s the alternative package to ffmpeg?
No need to panic in this case. While I hate OpenAI, there’s two things to note here:
- Whisper is an open source library for speech recognition rather than generative AI, run entirely locally. It’s just using ML to do something we could already do with computers (speech recognition), but better.
- They aren’t even directly using the OpenAI version - they’re using whisper.cpp, a port of the model.
Good luck with that… ffmpeg is the de facto standard.
This is one of the actually decent uses of this model. I have used Whisper to transcribe to phone calls, and just the other week I had to export the audio from a video I was working on to run whisper to get subtitles for the video. It’s still not a set it and forget it solution, but correcting it’s small mistakes here and there is so much faster than manually transcribing the audio.
Given how modular ffmpeg is with the way the switches work a user never has to interact with that portion of the application. I can technically use ffmpeg to trsnscode an mp3 without ever using the video components.
