Last verified
Convert YouTube or Bilibili videos into dubbed, subtitled versions using Whisper, VoxCPM2 voice cloning, and Demucs, running entirely on your own hardware.
Creators who publish to both YouTube and Bilibili can run YouDub-webui on a local machine or small server to automate dubbing across new uploads and back-catalog videos. The author's 20,000+ video Bilibili channel runs entirely on this pipeline.
If you process fewer than a handful of videos per month, a one-off cloud service may be simpler than the multi-step local setup and the GPU infrastructure overhead.
The FastAPI backend and modular pipeline design make YouDub-webui a reasonable starting point for a production localization service. Each stage (ASR, translation, TTS, mixing) is clearly separated, and the OpenAI-compatible translation hook makes swapping providers straightforward.
If you need managed infrastructure, SLA guarantees, or a multi-tenant architecture out of the box, a commercial API is a faster starting point than building on YouDub-webui.
Teachers and learners can use YouDub-webui to create dubbed versions of YouTube or Bilibili content for study use, preserving the original background audio while replacing the speech track in the target language.
If your source material is not on YouTube or Bilibili and you do not want to manage local file uploads, the two-entry-point design limits your workflow options.
Localizing a video into another language requires chaining six distinct processes: downloading the source video, separating speech from background audio, transcribing speech with accurate word-level timestamps, translating the transcript sentence by sentence, generating dubbed speech that matches the original speaker's voice, and mixing the result with background audio before burning subtitles. Assembling these tools into a reliable pipeline is significant engineering work.
Commercial platforms like Rask AI, HeyGen, Maestra, and Dubverse handle this as managed cloud services, but charge per-minute or per-seat fees that scale with video volume. For a content creator publishing dozens of localized videos per month, a cloud service's free tier caps quickly and paid plans add up. Data residency is a secondary concern: uploading video content to a third-party cloud means your footage, speaker voice, and transcript all leave your infrastructure.
From YouTube or Bilibili URL to dubbed, subtitle-burned mp4 with no manual steps between. The pipeline runs download, source separation, ASR, translation, TTS, mixing, and subtitle burn as a single task. The web UI shows per-stage progress, timing, and logs; failed tasks resume from the failed stage without rerunning earlier steps.
Transcribes speech using openai-whisper, defaulting to the large-v3-turbo model. Output includes word-level timestamps, which are used to cut the original vocal track into per-sentence reference segments for the voice cloning stage. Runs on CUDA or CPU; MPS devices fall back to CPU for the float64-dependent timestamp alignment step.
Generates dubbed speech in the target language using VoxCPM2, cloning the original speaker's voice from reference audio segments cut from the source track. Dubbed segments are aligned to fit the original sentence durations, then mixed with the preserved background audio and burned into the final video file.
Separates the original video's vocal track from background music and sound effects using a Demucs source submodule. The background audio is preserved through the pipeline and remixed with the dubbed vocals in the final output, so background music and ambient sound carry through to the localized version.
Failed tasks restart from the stage that failed, reusing outputs from completed stages cached in the local work folder. The web UI displays per-task stage status, stage duration, and full running logs. Tasks can be deleted individually, including their log records and work folder contents, or rerun from scratch.
All intermediate files, the SQLite task database, YouTube cookies, and final output videos stay on your machine. Translation uses any OpenAI-compatible Chat Completions endpoint you configure, including locally run models. The backend enforces Argon2id password authentication and per-session CSRF protection on all API endpoints.
Rask AI is a managed cloud service for AI video localization, supporting dozens of languages through a web-based upload workflow. YouDub-webui addresses the same core need: take a video with speech in one language and produce a dubbed version in another. The key differences are deployment model, data residency, and cost structure.
| Feature | YouDub-webui | Rask AI |
|---|---|---|
| License | Apache-2.0 (open source) | Proprietary |
| Deployment | Self-hosted, local | Cloud-only |
| Pricing | Compute costs only | Per-minute subscription |
| Data residency | Local machine | Rask AI servers |
| GPU required | Yes (recommended) | No |
YouDub-webui is the stronger choice when you are processing a large volume of videos regularly and want to avoid per-minute fees. If you publish dozens of videos per month, the subscription cost of a managed service grows linearly while your compute cost stays roughly fixed. The trade-off is setup effort: YouDub-webui requires a machine with a capable GPU and a multi-step installation.
Rask AI is worth considering if you need a no-infrastructure option, support for many language pairs with no local model setup, or a managed product with customer support.
HeyGen provides video translation and dubbing through its Video Translate feature, alongside its core AI avatar product. Like Rask AI, it is a cloud-only proprietary service with credit or subscription billing. The core difference from YouDub-webui is that HeyGen processes your video content on its own servers.
| Feature | YouDub-webui | HeyGen |
|---|---|---|
| License | Apache-2.0 | Proprietary |
| Deployment | Self-hosted | Cloud-only |
| Data residency | Local machine | HeyGen's servers |
| Input sources | YouTube URL, Bilibili URL, local upload | Upload only |
| Pricing | Compute costs only | Credit-based |
YouDub-webui is the better fit for creators who need to keep video content on their own infrastructure, or who process a volume that makes per-credit billing prohibitive. HeyGen is the better choice for teams who want a polished managed product with no local GPU infrastructure to maintain.
Self-hosting requires cloning the repository with its Demucs submodule, then installing Python and Node.js dependencies before starting the backend and frontend services.
```bash
git clone https://github.com/liuzhao1225/YouDub-webui.git
cd YouDub-webui
git submodule update --init --recursive
python3.12 -m venv .venv
.venv/bin/pip install -r requirements.txt
npm --prefix apps/web install
cp env.txt.example .env
```The README documents two primary language routes: YouTube English-to-Chinese and Bilibili Chinese-to-English. Translation uses any OpenAI-compatible Chat Completions endpoint, so other language pairs work if your chosen model supports them. Whisper handles many languages for ASR. VoxCPM2 TTS quality for languages beyond the primary two has not been validated in the documented production case, so additional testing is advised before relying on other language pairs at scale.
A CUDA-capable NVIDIA GPU is strongly recommended. The README states that running Whisper transcription, Demucs source separation, and VoxCPM2 TTS on CPU is very slow for real video workflows. Setting DEVICE=cpu is supported for partial testing, but a complete video localization pipeline on CPU takes significantly longer per video. Apple Silicon (MPS) is partially supported: VoxCPM2 uses MPS, but Whisper runs on CPU on MPS devices due to float64 constraints in the word-level timestamp alignment step.
Apache-2.0 permits commercial use, modification, and distribution without a copyleft requirement. You can run YouDub-webui in a commercial content production operation or build a service on top of it without releasing your own changes as open source, as long as you retain the original Apache-2.0 license notice in any distribution.
Yes. YouDub-webui accepts local video uploads up to 4 GiB by default (configurable via LOCAL_UPLOAD_MAX_BYTES). You can also upload a pre-translated SRT subtitle file alongside a local video to skip the Whisper transcription and OpenAI translation stages, running only the voice cloning and subtitle burning steps. Uploading an SRT alongside a URL-based task is not supported in the current version.
Failed tasks can be resumed from the stage that failed without re-running earlier stages. The web UI displays per-stage status, duration, and full logs. Intermediate outputs from completed stages are cached in the local work folder and reused on resume. You can also delete a task's records and output files from the UI if you want to start fresh.
Alternative to
MacWhisper
Local speech-to-text with multi-format output and GPU acceleration
Alternative to
Rask AI
Open source video translation with AI dubbing and voice cloning
Alternative to
HeyGen
Local AI workspace for video, image, voice, and translation
Alternative toHeyGen LiveAvatar
Talking-avatar SDK for voice AI agents, MIT-licensed, Python and Node.
Alternative to
Descript
Edit videos by chatting with Claude Code, not clicking menus
Alternative to
InVideo AI
AI-powered short video generation from topic to publish
Featured
Alternative to
Superhuman
AI email assistant that drafts replies and clears your inbox
Alternative to
Grok Bot
AI teammates you own: your keys, your model, your machine.
Alternative toClaude Managed Agents
Open-source agent deployment API. Self-host or use Omnara Cloud.
The projects we add and every new guide. Free, unsubscribe any time.
Claude Code to 
OpenCode
Jira to 
Windshift
Notion to 
AppFlowy
Slack to 
Mattermost
Linear to 
GitLab
Wispr Flow to 
Handy
CapCut to 
Concat
Grok Bot to 
rakazo
Superhuman to 
Inbox Zero
Airtable to 
AITable
Muse to 
OpenClaw
GitHub Copilot to 
Cline
Datadog to 
Netdata
Microsoft Teams to 
Rocket.Chat
Pinecone to 
Qdrant
Zapier to 
n8n
Devin AI to 
OpenHands
Firebase to 
Supabase
1Password to 
KeePassXCConfluence to 
Docmost
Contentful to 
Strapi
Adobe Premiere Pro to 
Shotcut
ChatGPT to 
AnythingLLM
LangSmith to 
Langfuse
