ByteDance trains 10 trillion-parameter AI model to rival Anthropic’s Mythos

ByteDance is set for its next artificial intelligence move as the Chinese tech company is training its mega AI model with as many as 10 trillion parameters, the size that could rival Anthropic’s cutting-edge Mythos system.

According to the sources familiar with the matter and reported by Financial Times, the training of a model is at an early stage, but given the amount of parameters, the model would be more than three times the size of Moonshot’s Kimi K3 model, trained on 2.8 trillion parameters.

Right now, ByteDance’s AI model is being pre-trained and this stage requires 3-6 months. Prior to releasing to the public, the model is fine-tuned. If all goes well without any technical and security issues, then it is made public. However, as per the sources, the exact model size would be determined at a later stage.

In general terms, AI mode’s parameters refers to the numerical settings a mode learns from data. The learning involves recognizing patterns, performing tasks and generating the answers. These parameters do not necessarily represent the model's capability.

Mythos 5 is considered Anthropic’s powerful model which has been available to particular approved organisations after a temporary suspension in June over security concerns.

The announcement comes as the US and China are locked in tech rivalry marked by a competitive artificial intelligence landscape. The new push also shows the efforts of Chinese tech giants to not only catch up but outperform their US peers in the most advanced level of AI.

For instance, the Chinese AI models from Alibaba and Moonshot have shown impressive performance, lagging behind Fable 5 in certain areas.

Besides the race to develop highly advanced models, Washington and Beijing accused each other of AI distillation. Last month, the White House accused Moonshot of stealing Anthropic’s Fable 5 technology to build Kimi K3. China, however, denied such allegations. In turn, Chinese authorities not only blamed the US for training AI models on Chinese data but also threatened countermeasures.

On Thursday, ByteDance founder, Zhang Yiming, told employees to avoid using AI distillation to train models just for the sake of short-term gains. ByteDance has not yet responded to this report.