YouTube tests new AI Lip Sync tool to help creators reach global audiences

YouTube is officially taking another big step in AI-powered video tools. This follows the launch of AI dubbing in December and the expansion of Multi-Language Audio tracks last month.

The platform is now testing a new AI lip sync feature for videos across different languages to make them feel more natural and immersive for global audiences.

YouTube revealed the new feature with a short, 15-second demo video demonstrating that an Italian clip was automatically translated into English, with the speaker’ s lip movements complementing the dubbed voice.

The caption read, “With upcoming lip sync technology, translated videos will now visually match the speaker’s lips with the newly dubbed language, making your content more accessible and engaging for a global audience.”

YouTube’s Head of Product for Dubbing, Buddhika Kottahachchi, said that the feature uses a custom technology toolkit “modifies the pixels on the screen to match the translated speech.”

It will show that YouTube's AI systems will modify the visuals of the speaker’s mouth to align naturally with the dubbed video and ultimately create a smoother experience for viewers.

The pilot program will support five languages: English, German, Spanish, Portuguese and French.

YouTube will further expand to include all the languages supported by its auto-dubbing feature.

This move specifically comes as demand for AI driven lip sync technology continues to evolve and modern AI tools can deliver high-fidelity with translated voices while meticulously retaining details like facial, hair details, teeth movement, and high-resolution visuals.

This innovation marks a crucial development for creators to reach a large number of audiences. Viewers can now enjoy videos that sound more natural regardless of the language they speak.