Ex-Apple researchers want to make AI voicebots feel human

Talk to most AI voicebots for more than a minute and you'll notice the lag, the flat reactions, and the sense that nobody's really listening. Seattle startup Nuance thinks it has found the fix, and investors just backed that bet with $50 million.

Cofounder and CEO Fangchang Ma spent his research career at Apple alongside his two cofounders before deciding existing AI avatars had a structural problem, not just a polish problem.

Most voicebots, he explained to Business Insider, are stitched together from separate voice-to-text, language model, and text-to-voice systems. That patchwork approach is what creates the delay and the blank stares when an AI is supposed to be listening.

The solution from Nuance is an all-in-one system that accepts audio and video input and gives back audio and video without any handovers from one model to another. Ma described this as the more difficult technical solution, but one he felt his team had no choice but to pursue in order to bridge the difference in behaviour between people and the AI.

Having trained the model in such a way means there is no place for the typical practice of getting texts from the web. The training set at Nuance comes from their own recorded conversations or from those of their partners where two people are captured on video and have natural audio with no artificiality of any kind.

According to Ma, the best sample for training is just two people talking naturally.

The company is still pre-product, although Ma has identified a range of potential applications for his AI system that include helping individuals with interview prep, presentation skills, and language acquisition, as well as interviewing, selling, and customer service AI for businesses. The plan is for a public research demo before the end of the year.

The $50 million Series A funding, which comes from Lightspeed Venture Partners with Accel, Nvidia’s NVentures, South Park Commons, and Define Ventures all participating, comes after a $10 million seed fund raised in July.

For now, the eight-member team will be using the funds to hire researchers and will work on go-to-market plans and pricing later once the product is ready for sale.