Recent community discussions have led to Chinese tech giants Baidu It may have developed important theoretical foundations for large-scale artificial intelligence (AI) models even before America’s OpenAI.
The large model, or “basic model”, is AI developmentwhose rapid iteration drives cutting-edge applications. While the United States is generally seen as leading the way in innovation in advanced AI models, some argue that China may be starting to explore these concepts sooner.
At the heart of large-scale model development is the “law of scaling.” This is a principle that states that the larger the training data and model parameters, the greater the model’s intelligence capabilities. This idea, popularized in OpenAI’s 2020 paper “Scaling laws for neurolinguistic models,” has been a cornerstone of AI research ever since.
of OpenAI This paper shows that increasing model parameters, training data, and computing resources improves performance according to a power law relationship. This insight led to the subsequent development of large-scale AI models.
But Dario Amodei, co-author of the OpenAI paper and former vice president of research at the company, said on a podcast in November that he had observed a similar phenomenon in 2014 while at Baidu.
“When I was working at Baidu with (former Baidu chief scientist) Andrew Ng in late 2014, the first thing we did was voice recognition It’s a system,” Amodei said. “We realized that the more data we gave it, the bigger we made it, and the longer we trained it, the better the model became.”
