The Economics of Foundation Models - Are VCs Subsidizing A Bad Business Model?
Is this Uber all over again?
Hi Everyone and welcome to Investing In AI. I’m Rob May, CEO of Nova, and a very active angel investor in the AI space. I also host the AI Innovators Podcast. This newsletter is 100% written by me. No LLMs involved.
I’ve been writing about foundation models and the possible ways the future could play out for them, and now I want to look at the pro scenario and the con scenario for these businesses.
Foundation models have been credited with supercharging AI recently. And they are creating a paradigm shift in AI. One good mental model for thinking about foundation models comes from Percy Liang:
The term ‘foundation models’, he explains, is meant to evoke the central importance of these systems within whole ecosystems of software applications. “A foundation of a house is not a house, but it’s a very important part of the house,” says Liang. “It’s a part of the house that allows you to build all sorts of other things on top of it. And, likewise, I think a foundation model is attempting to serve a similar role.”
The question I want to address today is - would you want to be in the foundation model business? Before I give you my thoughts on situations where you would not, and situations where you would, I do want to clarify that there is a lot of nuance in these decisions. Some areas of foundation models will inevitably more competitive than others, and thus possibly less attractive, and some markets for certain foundation models will be smaller than others.
Let’s start by making the “no” case. In this case, the foundation models get widely adopted and that growth makes investors excited, but the underlying economics remain poor and VCs pour billions into subsidizing a business model that never emerges. This is what some people claim happened with companies like Uber and Doordash, which continue to lose money as publicly traded companies.
There are several ways we could end up in this situation. First of all, the marginal cost to gather more data to train bigger models could rise, while the marginal value of a better model falls. This is the reverse of what you want if you run a foundation model company. As your economics get worse, and new types of computer chips dramatically lower the cost to train these models, the defensibility of the whole business model goes down. On top of it, the scale of customer usage bogs down the company with more traditional engineering challenges that steal resources from AI innovation that could be much more valuable. In this scenario these companies may run losses indefinitely and you don’t want to be in this business.
Now let’s consider the “yes” scenario. In this situation, as adoption climbs the fixed costs of training and serving a model get spread out over more users and lead to eventual profitability. Competition stays tight with just a few players dominating the landscape. Advances that come from better chips or other widely available academic innovations are adopted fasted by the main players and they keep lowering their costs enough that rolling your own foundation models doesn’t seem worth the hassle.
What I think is happening is that the market, while realizing the possibility of the “no” scenario but desiring to get to the “yes” scenario, is starting to differentiate in different ways. Consider OpenAI and Cohere as examples. I know people at both companies but don’t have any inside knowledge of their long term strategies, but as an outside observer, here is what it appears they are each doing with respect to LLMs.
It appears the market may play out like the cloud computing market, with trained models and the tools needed to run and implement them sort of like various levels of compute abstraction - you can buy a bare metal server or a virtual machine, with levels in between. I think part of the OpenAI-Microsoft deal was for OpenAI to get access to other training data to push models forward. It’s possible that Microsoft will slide some term into Office licenses that allows it to use data for training purposes, or will find a way to make it fit their existing terms. This would continue to push OpenAI forward as the leader of horizontal AI use cases for absolute best of breed models, and they will probably quickly discount slightly less performant versions for other use cases. In this case, OpenAI becomes profitable because advances in model capabilities continue to unlock more TAM via more use cases.
Cohere, on the other hand, seems to be building lots of enterprise functionality that makes those tasks easier that are one step above being a pure foundation model. They can be, probably 98% as good as OpenAI or other models and win customers all day long with this approach, because if you are building common NLP tasks into your app, Cohere may end up being the enterprise choice. In this scenario the company becomes profitable because you are paying primarily for that extra layer above the foundation model.
The other issue that can come into play here is that maybe none of this is where foundation model companies actually make money. Perhaps by staying at the front of the pack, the distance between the market leaders and the capabilities of everyone else become bigger and bigger, and the true business model emerges from use cases that aren’t coming for another couple of years.
I think what will happen is similar to what happened in the cloud computing world. Compute costs for both training and inference on a per-unit basis will drop dramatically in the next few years, the way storage and server costs dropped for the cloud. Value added service layers on top of the basics will create demand and stickiness, the way they did for the cloud. Competition will shrink over time as the clear market winners emerge. And these business will ultimately be a good place to have bet.
I also suspect the market will be large enough to support several niche foundation models which can still have hundreds of millions in revenue. If you have ideas for those and are early in your fundraising, I’d love to hear from you.
Thanks for reading.
