Buy the Future
About 2 and a half years ago, a leaked internal Google memo said that they "had no moat" and that open source and smaller models would be their competition in AI. They figured that OpenAI was in the same position.
I believe that this memo put too much emphasis on open source, and not enough on Moore's Law. They seemed to think Meta was going to be the winner, so it hasn't aged that well.
Running AI models is expensive because they're computationally difficult.
Training a model takes a long time, not only to download and store a copy of all the content you're stealing, but also to iterate through that stolen content and turn it into the numbers you're ultimately going to store in your resulting model. From the initial training pass, to then training it how to answer questions, to training it on common patterns, to training it how to be sycophantic and not insane - all of this takes time and computing power.
Then there's the inference - actually running the model. You take the user prompt and turn it into another sequence of numbers, then throw it to your digit soup and generate a response. Again, this is computationally expensive.
So if I'm one of these AI companies, and I've implemented this transformer idea, I'm going to be looking at what I have and thinking that it certainly has its applications. It seems to do well with coding tasks. It does a reasonable job of translation, or writing text - certainly in a way that can act as an assistant to those who are less confident. Unfortunately it also uses a lot of computing power to do those things.
That means, as a business, it's difficult to see how you make it super profitable. Sure, there's a lot of people out there who will pay you to help fix their grammar mistakes, and probably many more people who you can write code for - but the bill for the computing environment is going to be huge.
So you either:
- Believe you can make the models better and reach AGI, cure cancer, solve world hunger, and give everyone free puppies (or kittens, depending on preference).
- Tell everyone you can, whether you believe it or not.
And you use that to raise a bunch of money.
(There's the possible option 3 here, that Anthropic are maybe following, which is that you don't necessarily need all the money in the world, and that there's a good business in coding tools if you can concentrate on optimising everything to run as cheaply as possible and eventually Moore's Law will help).
This is where the bubble begins. You need to pay for all that compute somehow, while you lose money on every query. So some circular deals, shoddy accounting and stock inflation later - we are where we are now.
But eventually, what is computationally expensive today, is not going to be computationally expensive tomorrow. And there's nothing particularly special about AI training or inference that's going to change that fact. I can already run advanced models on my home Mac Studio with 96GB RAM, and my inability to run frontier models is partly limited by the fact they're not available. The only way I get access to them is via a $20 a month subscription.
There also seems to be a limit to the model size, where you can make them bigger, but they don't necessarily get any better. We're reaching the effective size that they need to be, so while good training data becomes the commodity that everyone is going to keep wanting to gather, the computational power to train and provide inference is unlikely to increase much more. Especially as they turn to optimisation.
So project that out - how many years before what I can get what OpenAI or Google can provide now to just run on a home computer? How long until it's just available offline on your phone? 2 years? 5 years? 10?
Or how long before Google's memo is right, and the model comes from open-source? Or government? And takes the bigger players out of the picture altogether?
Apple are often criticised for their lacklustre AI showing, especially when compared to the companies we've been talking about. But what they have is a world-class chip making department, who I can guarantee right now are looking at ways to design hardware to do exactly the computations that an AI model needs. The latest iPad chip marketing already claims 2-5 times faster performance than a year ago.
Why spend hundreds of billions of dollars on data centres that you hope to pay back with subscription fees, when you can sell hundreds of billions of dollars of devices to everyone and put the AI in their pocket?
Maybe OpenAI or Google or somebody running the transformer playbook will reach that mythical cancer curing moment. OpenAI's survival is predicated on making that kind of leap before Moore's Law comes knocking.
Like Marty McFly, they hope that buying the future will help them become rich. But eventually the future becomes the present, and at that point they're going to look pretty silly with a data centre filled with 10 year old graphics cards, a few dozen lawsuits and a collapsed economy behind them.