A few years ago, AI mostly showed up as clunky chatbots that struggled with the simplest questions. Today, the same family of models writes code and powers search engines. What changed? Under the hood is a surprisingly simple idea: if you keep feeding these models more data and more computation, they keep getting better in ways you can measure and predict.
Yang Yang, a machine learning expert and distributed systems architect, has worked on both sides of that story. As co-author of “Deep learning scaling is predictable, empirically,” he helped show that model performance improves in consistent, mathematically describable ways as you scale up data and compute. Today he leads teams that turn those ideas into production systems used by millions of people, and he stays close to the research community as an IEEE manuscript reviewer and member of the editorial board of ESP. In this article, he traces how AI reached its current level, why “scale” is so important, and what hardware it takes to run these systems.
Scaling Laws 101
Scaling laws in deep learning describe how three ingredients relate to each other: how much data you train on, how large your model is, and how much compute you use. They link those inputs to performance, usually tracked by how often the model makes mistakes on fresh test data. In simple terms, you might ask, “If I double the data, how much does the error drop?”
Yang’s work showed that, when you grow both the training set and the model in a balanced way, the errors tend to follow smooth power-law curves. On a chart built for this kind of comparison, the relationship behaves almost like a straight line. The steepness of that line acts as a rough ‘learning speed’ for the task.
For the budding AI industry, this confirmed that once you know the learning speed for your model and where you stand today, you can estimate how much extra data and compute you need to reach a specific performance target, and also see where the gains start to level off.
Yang’s paper highlighted a budgeting insight. The optimal model size grows sublinearly with data size. Ten times more data does not require ten times more parameters. For the many startups comparing cloud contracts or wondering whether to buy more hardware for AI, that rule of thumb points toward growing data and model capacity together and attaching numbers to those choices.
How ‘Scale’ Became AI’s North Star
Once those relationships were clear, scale became a planning tool. Engineers didn’t have to debate in the abstract whether a model should be slightly larger, or whether another dataset would probably help. They could start from a performance target and work backward, asking: Given our learning curve, what will it take to get there, and is that step worth the money and time?
Yang’s work also offered a shared language for very different AI problems. A team building a recommendation system and another working on speech recognition could both talk about progress in terms of similar curves. The data and users differ, but the basic picture of how performance improves with more resources stays the same.
Knowing how larger datasets tend to improve results was only the first part of the story. Building systems that deliver those gains to people using phones and business tools is another challenge entirely.
Solving the Infrastructure for Scale
Analysts at Goldman Sachs estimate that AI will account for roughly a third of the entire data center market within just two years. That growth comes with a need for more graphics processing units (GPUs) and far higher power budgets than past generations of web services required. A few years ago, a server with fewer than a dozen GPUs might have counted as a serious AI system. Soon, leading systems may pack more than 500 GPUs into a single rack. Turning guidance from scaling laws into infrastructure that runs efficiently and reliably required another wave of engineering work.
After his research, Yang went on to lead the architectural design of a first-of-its-kind neural retrieval engine used for personalization and content discovery. The project began when those eight GPUs in a single server still felt ambitious. Today, the platform spans a fleet of more than a million servers and handles billions of requests per day.
Each request to that system has to pass through several demanding stages in a few dozen milliseconds. The software narrows a massive candidate pool down to a smaller set, computes neural representations of both users and items, and then ranks the strongest matches. People experience that as the right content showing up on demand, but under the hood it is a tightly timed choreography of caches and network calls built on the same scaling principles Yang once described on paper.
Yang and his colleagues worked closely with hardware specialists to use advanced accelerators, including processors in the class of NVIDIA’s Grace Hopper Superchip. By pushing for model-hardware co-design, they could fit larger and more capable models into practical serving systems. That serving layer turns abstract scaling rules into something people can use every day. Without it, even the most efficient learning curve would stay stuck in theory.
Where Scale is Pushing AI Next
Scaling laws and the infrastructure that grew around them have changed how AI is planned and delivered. The research offered a map of how performance improves as you feed in more data and compute, and the engineering work built the server farms and accelerators that make those gains available to users.
That combination now shapes choices far beyond any one company. McKinsey’s research projects that by the end of the decade, data centers will require $6.7 trillion in investment as operators race to support larger models and more intensive inference workloads. Each time a recommendation feels timely or a voice assistant responds smoothly, it reflects both sides of this story: the simple rules that govern how models scale, and the large, carefully designed systems built to put those rules to work.
