Jagged Frontier: The Best New Idea In AI

It's hard to understand where to apply AI

Happy Sunday and welcome to Investing in AI! I’m Rob May, CEO at BrandGuard, host of the AI Innovators Podcast, and active AI angel investor. I don’t charge anything for this email and, if you like it, you can help support future writing by sending me an intro to Nike - it’s the one large company we haven’t been able to connect with at BrandGuard that we think would be awesome.

This week I want to write about the Jagged Frontier. Every once in a while an idea comes around and really makes you think about AI in a new way, and the Jagged Frontier fits that description perfectly. The paper that coined the term has a bunch of good stuff in it. It highlights the fact that yes, AI will have a huge impact on the way we work. I encourage you to go read the whole article about the paper.

Here is the key section that I believe is so powerful (bold is mine, not original):

AI is weird. No one actually knows the full range of capabilities of the most advanced Large Language Models, like GPT-4. No one really knows the best ways to use them, or the conditions under which they fail. There is no instruction manual. On some tasks AI is immensely powerful, and on others it fails completely or subtly. And, unless you use AI a lot, you won’t know which is which.

The result is what we call the “Jagged Frontier” of AI. Imagine a fortress wall, with some towers and battlements jutting out into the countryside, while others fold back towards the center of the castle. That wall is the capability of AI, and the further from the center, the harder the task. Everything inside the wall can be done by the AI, everything outside is hard for the AI to do. The problem is that the wall is invisible, so some tasks that might logically seem to be the same distance away from the center, and therefore equally difficult – say, writing a sonnet and an exactly 50 word poem – are actually on different sides of the wall.

The implication here is powerful. It means that the best way to understand the capabilities of AI are to explore the jagged frontier. There isn’t an analytical way to understand it, so you have to use an experimental way. Some companies, many companies in fact, aren’t built that way. Companies want to analyze things and make decisions about the path to pursue, but, according to this new research paper, that won’t work with AI. You have to try things, experiment, and explore.

So if you don’t have an AI beta program going somewhere in your company, you are falling behind, and unlikely to understand where the jagged frontier lies.

Thanks for reading.