AI · Training Compute

AI Training Compute

The biggest training run ever done, doubling every five and a half months.

• 4.6× growth per year since 2013
• 191,000,000× from 2013 to Grok 4
• The record holders, not the famous names
• 1 PF-day = 8.64e19 FLOP

The compute frontier: the largest training run anyone had done at each moment, a running maximum over Epoch AI’s database of notable models. Where several records fell within a few months of each other, the chart shows the one that stood, since at this scale they would land on the same pixel. One petaflop-day is 8.64e19 floating-point operations, roughly what a machine doing a quadrillion operations per second gets through in a day. Between 2013 and 2025 the frontier grew 4.6-fold per year, a doubling every 5.4 months, finishing 191 million times higher than it started. A few curves on this site double faster, none over anything like twelve and a half years. The record has stood since Grok 4 in July 2025 because Epoch publishes no compute estimate for the frontier models that followed, which is not evidence that scaling stopped. AlexNet, the Transformer and BERT are missing for another reason: none of them was ever the biggest training run. The original Transformer paper used about 0.086 petaflop-days in 2017, when the record already stood at 77, roughly 890 times higher; it is famous for an idea, not for its scale. When AlexNet won ImageNet in 2012 the all-time compute record still belonged to a Google language model from 2007, and it went unbeaten until DistBelief in 2013, where this curve begins. Pick models by fame rather than by size and the chart lies: the Transformer would appear as a step upward while sitting three orders of magnitude below the frontier of its own year.

Source: Epoch AI, Notable AI Models

Download the data: ai_compute.csv · ai_compute.json

All curves · The chart is drawn in your browser and needs JavaScript. Every number it plots is in the table above.