"If AI inference remains as desirable as Kimball expects, the evolution is likely to follow the same trajectory as the CPU. The CPU didn’t improve along a single axis but instead across simultaneously. Once transistor scaling slowed, chip and system architecture innovations of all kinds proliferated. The list of individual innovations that led to today’s ubiquitous, powerful personal compute could fill dozens of books. A few decades from now, the history of AI inference innovation will show similar depth."
Of the areas mentioned in the article, which are the most likely to have the most prominent innovative impact, and what will they entail?
I like how the author uses the analogy of scrabble word creation to describe LLM training but unfortunately the analogy didn't continue to inference and I got lost trying to keep up.
Excellent article. I believe the majority of benchmark performance gains moving forward will come from this side of the stack enabling faster iteration/recursion.
"If AI inference remains as desirable as Kimball expects, the evolution is likely to follow the same trajectory as the CPU. The CPU didn’t improve along a single axis but instead across simultaneously. Once transistor scaling slowed, chip and system architecture innovations of all kinds proliferated. The list of individual innovations that led to today’s ubiquitous, powerful personal compute could fill dozens of books. A few decades from now, the history of AI inference innovation will show similar depth."
Of the areas mentioned in the article, which are the most likely to have the most prominent innovative impact, and what will they entail?
> The list of individual innovations that led to today’s ubiquitous, powerful personal compute could fill dozens of books.
https://a.co/d/0dMk8urP
which is the Hennesey and Patterson computer architecture book would serve the role of the "dozens of books" hyperbole rather well.
Great read.
I like how the author uses the analogy of scrabble word creation to describe LLM training but unfortunately the analogy didn't continue to inference and I got lost trying to keep up.
Excellent article. I believe the majority of benchmark performance gains moving forward will come from this side of the stack enabling faster iteration/recursion.
> And Anthropic is paying LLM competitor SpaceXAI over a billion dollars per month to lease spare compute
I knew of this but not the $ amount. Wow
Big money from taking Tesla's place in line for an Nvidia shipment during a huge shortage without compensation.