VibeVoice: Microsoft's Frontier Open-Source Voice AI

Clone any voice in 10 seconds • Generate 90-minute multi-speaker podcasts • Real-time streaming • All local, free, and MIT-licensed

What is VibeVoice?

Microsoft just open-sourced a powerful voice AI framework that was briefly pulled from GitHub due to misuse concerns (deepfakes and disinformation). After adding watermarks, safety controls, and disclaimers, they re-released it for the world.

VibeVoice is a family of frontier models for both Text-to-Speech (TTS) and Automatic Speech Recognition (ASR). It excels at long-form conversational audio like podcasts — up to 90 minutes with up to 4 distinct speakers, natural turn-taking, emotional nuance, and real-time streaming.

No subscriptions. No ElevenLabs pricing. Runs entirely on your machine.

Key Features

🎙️ Voice Cloning

Clone any voice from just 10 seconds of audio. Perfect for custom narrators or podcast hosts.

📼 90-Minute Long-Form TTS

Generate full multi-speaker conversations (up to 4 voices) with natural pauses, turn-taking, and emotion — all in one pass.

⚡ Real-Time Streaming

First audio chunk in ~200-300ms. Supports streaming text input for live applications.

🔍 Long-Form ASR + Diarization

Transcribe 60 minutes of audio in a single pass. Identifies speakers, adds timestamps, and supports custom hotwords + 50+ languages.

🌍 Multi-Lingual

English, Chinese, and experimental support for DE, FR, IT, JP, KR, NL, PL, PT, ES + multiple English styles.

🛡️ Built-in Safety

Watermarks, audio disclaimers, and responsible-use guidelines from Microsoft Research.

History & Why It Matters

Originally released in 2025 as VibeVoice-TTS, the repo was taken down after misuse reports. Microsoft re-released it (and added VibeVoice-ASR and Realtime models) under MIT license with safety guardrails. 23K+ GitHub stars show the demand for truly open, local voice AI that rivals paid services like ElevenLabs.

This is Microsoft Research giving developers frontier capabilities — for free and offline.

Quick Start Tutorials & Guides

Everything you need to run VibeVoice locally — from one-click Colab to full local setup.

1. Official Project Page

Full demos, audio samples, and transcripts (podcasts, singing, cross-lingual conversations).

Visit Project Page →

2. GitHub Repository

Core library, docs, finetuning code, and Realtime Colab notebook.

GitHub Repo →

3. Realtime Colab Demo (Easiest Start)

Zero-setup streaming TTS in your browser. Runs VibeVoice-Realtime-0.5B instantly.

Open Colab →

4. VibeVoice-ASR Playground

Try 60-minute transcription + speaker diarization live.

Launch Playground →

5. YouTube Hands-On Tutorial

Full walkthrough: install, generate 90-minute podcasts, voice cloning.

Watch Tutorial →

6. Hugging Face Models

Download VibeVoice-1.5B (TTS), VibeVoice-ASR-7B, and Realtime-0.5B.

Hugging Face →

Installation (Local Setup)

From the official docs:

Clone the repo: git clone https://github.com/microsoft/VibeVoice
Install: pip install -e .
Use the provided Gradio interface or Python scripts in /demo
For Realtime: follow the dedicated docs here

Requires a decent GPU (8GB+ VRAM recommended for 1.5B model). Full instructions in the README.

Resources & Community

Technical Paper (TTS): arxiv.org/abs/2508.19205
ASR Technique Report: arxiv.org/pdf/2601.18184
Community Forks & UI Apps: Search GitHub for VibeVoiceFusion, ComfyUI-VibeVoice, VibeVoice Studio
License: MIT — fully open for research, personal, and commercial experimentation

Inspired by the viral thread: @heynavtoor on X
Built for MikesBlogDesign — exploring the best open-source AI tools.