Is AI Too Expensive To Justify The ROI?

Some excerpts from a debate on AI and productivity

Happy Sunday and welcome to Investing in AI. I write a lot about all the opportunities in AI but it’s important to keep a balanced perspective so today I want to talk about some of the challenges in AI.

The biggest challenge, by far, is around the ROI of adoption. From what I’ve seen at my own company and those I’m invested in, lots of AI projects are stuck in the perpetual pilot phase as people figure out whether the technology is good enough and the problems it solves valuable enough to generate real ROI.

Goldman Sachs recently published a newsletter that is the most skeptical assessment of AI ROI that I’ve read. Below is the most interesting excerpt:

The cost to develop and run AI technology means that AI applications must solve extremely complex and important problems for enterprises to earn an appropriate return on investment (ROI). We estimate that the AI infrastructure buildout will cost over $1tn in the next several years alone, which includes spending on data centers, utilities, and applications. So, the crucial question is: What $1tn problem will AI solve?

Replacing low wage jobs with tremendously costly technology is basically the polar opposite of the prior technology transitions I’ve witnessed in my thirty years of closely following the tech industry. Many people attempt to compare AI today to the early days of the internet. But even in its infancy, the internet was a low-cost technology solution that enabled e-commerce to replace costly incumbent solutions. Amazon could sell books at a lower cost than Barnes & Noble because it didn’t have to maintain costly brick-and-mortar locations.

The bolded statement above from the report rings true with my experiences in the market. Historically, tech lowers the cost to do things. With many AI applications, that isn’t the case, because AI is so expensive to run.

As an example, for BrandGuard to do an equivalent job checking a marketing asset for brand compliance as what a human would do, requires 22 different AI models. For all of those models to run in less than 60 seconds, which is a SLA similar to what a human would provide, takes several GPUs, which are expensive. And to do it quickly requires us to keep them up and ready to go even when not in immediate use. It’s not cheap to score a marketing asset with AI. But it is more scalable. Is the value of scalability worth the cost? The market hasn’t decided yet.

The question then, for investors, is how to place smart bets on things where AI can show strong ROI and ignore things where it can’t. But this is a moving target so we have to do this while factoring in that the models are constantly improving (thus providing the ability to automate more tasks) and the costs to run a model at a given performance level are decreasing (thus increasing the areas of available ROI), all while paying attention to the difficulties of dealing with AI’s jagged frontier. It’s a very challenging problem.

One reasonably safe bet though, is to look at technologies that lower the cost of AI training and inference, because those technologies will directly impact the ROI calculations of everything sitting on top of that part of the tech stack.

I’ve been a big proponent for years of a coming AI explosion, but my views have become more tempered as I’ve watched this festering ROI problem continue to grow. I still believe AI is the most important technology to ever be invented, but rolling it out through the economy is going to take much longer than I anticipated.

Thanks for reading.