Tired of foreign subtitles, I built a video voice translation tool for myself

One evening last year, I was watching an insightful tech video on my computer. It was in English with Chinese subtitles. As I watched, a thought struck me: I wish I could just listen to the Chinese version. No need to stare at subtitles all the time. I could listen during my commute, before bed, wherever.

Eventually, I actually built it.

Watch this film clip translated from Chinese to English 👇

But from the moment the idea struck me to creating a product everyone could use, it took far longer than I expected.

After the idea came, I tried a few video translation tools. Chinese translation was there, but something always felt off when I listened.

The person in the video was still there, but it was no longer their voice. The emotion in how they spoke each sentence, also gone.

Imagine a face you know well, suddenly speaking in a different tone, a different voice. You can follow the meaning, but that connection breaks. It's like watching a familiar person while hearing a stranger speak for them.

What I really wanted was simple. Translate video language, but keep the original voice intact. Keep the emotion, the nuance in how they say each word. These small details in a voice matter. The same sentence said softly versus forcefully hits you completely differently.

I searched around and found a model approach. At the time, options were limited, but this approach's voice preservation quality gave me hope.

Having a model is one thing. Getting a finished video is another. The inference was painfully slow, and I couldn't just throw an entire video at it and wait for perfectly translated output with audio synced with the video, all while preserving background music. I had to wire everything together myself.

After I shared some samples, quite a few people asked if I could turn it into a tool everyone could use.

Watch this clip translated from Chinese to English 👇

It’s quite interesting in itself to hear a familiar character speaking another language. What I care more about, though, is whether I can still recognize the original vibe of that character after the language switch.

Watch this clip translated from Japanese to Chinese 👇

Emotional expression in anime is relatively distinct. You can pay attention to the characters' tones when they speak.

Back to the original requirement, translate the English video into Chinese.

Watch this clip translated from English to Chinese 👇

Back to what we were just saying. After I shared some samples, quite a few people asked if I could turn it into a tool everyone could use. I thought so too. Honestly, with AI-assisted coding, I figured it would be fast, right? The core translation pipeline was working. Everything else? Just building a website for video uploads.

Turns out, it took a year.

Turns out building something you can demo and building something people can reliably use are very different things. Using it myself, all I needed was proper video processing. But for others, I needed registration, login, data persistence. One user's data can't leak to another's. Multiple concurrent uploads can't crash the system.

What if the system goes down? What if users multiply and I need to scale? Would users instinctively know how to use it? Databases, servers, object storage, domains, DNS, one by one, you have to get them right. Each piece seems fine alone, but how do you wire them together so people can smoothly use your service? There's always something else, and something else after that.

AI can help. But whether it's right, what to do when something breaks, that's still on you.

So why keep going? Because I promised. I promised the people who wanted this, promised myself. That thought kept me going.

Now, finally, I can deliver.

It's called SoulDub, dubbing with soul. It supports 30+ languages and some dialects.

Using it is straightforward:

Web way: Go to souldub.ai, upload a video, pick a language, hit start. That's it.

Agent way: Search for github-souldub-skills in your browser and install it via your preferred Agent (ChatGPT, Claude, Gemini, etc.) and have it install the skill.

After that, your Agent can translate your local videos. The translated video and subtitles automatically download to the original video's folder.

Videos from both methods show up in your SoulDub account.

One more critical piece: manual review and editing.

If a subtitle isn't quite right, if a dubbed voice needs adjustment, you edit it directly on the site. The AI does the first pass, then you refine the parts that matter to you. Quality keeps going up.

Speaking of this, I'd like to chat with you about whether using SoulDub actually costs anything.

To be frank: you can use it free every month, but not unlimited free.

The computing power is rented from cloud servers, and so is the storage. Every month when the cloud provider's bill arrives, we have to pay it so the servers keep running.

Many people think it'd be best to translate videos without spending anything at all. I genuinely understand that feeling. I used to think the same way when looking for tools myself.

But when you're the one maintaining a service, you're the one picking up the bill. If you can't handle it, the thing simply can't last.

If you've used SoulDub, please share your feedback with me anytime. Let's make it better together.

Thank you to the readers who gave me feedback a year ago, when I was only beginning to have an idea.

And thank you to myself, for the persistence over this past year.

Now, let me share some reflections on AI that I've gathered this year.

AI has turned some things that could only be imagined before into reality.

Like video voice translation, it actually got built. Just this alone is enough to excite me.

But excitement aside, there's something I've become more and more certain of this year.

AI, no matter how powerful, is just a tool.

We can keep it close at all times, but we can't keep it in front of us at all times, doing everything for us.

It's like using a hammer to drive a nail. Your eyes need to watch the nail and the wood, and your hands need to feel every vibration the hammer sends back.

If your eyes are only on the hammer and your hands don't touch the wood anymore, eventually you forget what the nail looks like, forget the feel of the wood, forget what vibration means.

But these are what life should feel like.

So when building SoulDub, I insisted on putting human review and human editing functions right into the core. No matter how strong AI gets, it can't do without human involvement. Without human involvement, there's no personality. Without personality, there's no soul.

The world is built up bit by bit in the interactions between people, between people and things, between people and circumstances.

A flower is called a flower because we call it a flower. A tree is called a tree because we call it a tree.

That's why animals have no world, only humans do. It sounds like a stretch, but think about it. It's true.

My world can't be constructed by me dealing with AI, and then letting AI deal with people, things, and circumstances on my behalf. That would be AI's world, not mine.

My world has to be built by me engaging directly with people, with things, with circumstances. I have to build it with my own hands. And only then, for certain questions where I still have judgment, should I let AI step in.

That's my world, not AI's.

What does it matter to me if a flower blooms and falls in the mountains? It matters the moment I go see it myself.

If I let AI see it for me, my heart and that flower remain equally silent. The fact that I won't even move to see it means that flower was never really in my heart to begin with.

If I live this way for too long, wouldn't my heart become empty?

A year has passed. Finally, I can write this essay.

Thank you for reading this far.