Google announces dual AI chips to challenge Nvidia lead

Morgan Stanley estimated in December 2024 that 500,000 TPU chips sold could add roughly $13 billion to Google's revenue by 2027. On Wednesday, Google gave that projection a serious tailwind.

Google announced two new Tensor Processing Units that, for the first time in the product's history, are designed for distinct jobs. The TPU 8t handles the training of the largest frontier AI models. 

The TPU 8i is built specifically for inference, the process of running deployed models at scale. Both chips are expected to be available later this year.

Google Cloud CEO Thomas Kurian called the split a natural evolution, citing power efficiency as a core design constraint. "We felt that power efficiency would become a constraint as people continue to scale both training and inference," he told reporters. 

The company says both chips represent a meaningful leap beyond last year's seventh-generation Ironwood TPU.

 For the past several years, AI investment was concentrated at the training stage, the expensive, compute-intensive process of building models from scratch. That emphasis is shifting. As leading labs have narrowed the quality gap between their models, the economic focus is moving toward agents and applications that sit on top of those models and run continuously at inference.

 The TPU 8i's headline technical advance directly targets this transition. Google says the chip makes a significant jump in high-bandwidth memory capacity, addressing what engineers call the 'memory wall', the bottleneck between how fast a processor computes and how fast it can retrieve the data it needs. 

For AI agents that must reason and act rather than simply respond, that bottleneck has become a meaningful constraint.

 "AI is evolving from answering questions to reasoning and taking action," Google infrastructure chiefs Amin Vahdat and Mark Lohmeyer wrote in the chip announcement.

Nvidia continues to be the king of silicon for AI, but Google is not totally shunning them either; the company continues to provide access to Nvidia GPUs via Google Cloud services and has announced it would soon provide its customers with the newly designed Nvidia GPUs known as the Vera Rubin series. 

However, it seems that the difference between working with and competing against Nvidia is becoming increasingly narrow.

For over ten years, Google had been working on its own silicon chips, but in recent times it has accelerated the process of development. This includes allowing TPU compatibility with Nvidia-friendly tooling, thus lowering the entry barrier for those companies using Nvidia's solutions. 

Anthropic is built on TPUs. Apple has used the TPU chips to train AI models. There are now many users of the products. 

Nvidia has also been doing its thing. It recently signed a $20 billion licence agreement with a competitor in the area of inference chips known as Groq and released its own inference chip last month.