The Economics of Edge AI Will Make It Bigger Than The Cloud
Inference at the edge is the trend to bet on.
Happy Thanksgiving Sunday and welcome to Technically Sentient. I’m Rob May, a partner at PJC. As an angel and a VC I’m invested in over 75 AI companies, and my goal with this newsletter is to discuss some of the things I’ve learned, and am learning, about how AI gets rolled out in the real world. This week I’ve written some commentary on Edge AI, which is a big investment thesis for me in 2021.
— Interesting Links —
Robots help with real estate sales. NYTimes.
Drift patents a new converational AI. Drift Blog.
GRANT is a collection of techniques for model explainability. Wagtail Labs.
Has China caught up to the US in AI? Forbes.
AI has a replication crisis. MIT Tech Review.
The forces driving sex robots. (safe for work) Dianaverse.
Facebook’s new AI misinformation detector. OneZero.
— Research Papers —
Adversarial Generation of Continuous Images. Link.
European Strategy on AI. Link.
Zero Shot Visual Slot Filling as Question Answering. Link.
— Commentary —
AI models use compute in two different forms - training, and inference. The vast majority of compute workloads in the long term will be inference - e.g. I've trained a model to detect cats vs dogs, and now I just run it on the animals I see in the world to tell me if it's a cat or dog. Training is very processor intensive and will most certainly be done in the cloud for now, but, AI edge inference will not because the economics don't make sense.
If I train a model for a security camera to detect faces, or people with guns, or something like that, it doesn't make sense for me to constantly stream data back to the cloud, then run inference to see if the current scenes contain a face, then send the answer back to the camera. It's much cheaper to buy a cheap microcontroller, deploy an edge AI model, and let it run continuously on the camera. This hasn't been possible because running inference on the edge took too much compute, but several technologies are changing this.
There are companies working on this. At the chip level, I’m an investor in Mythic, an analog AI chip for lower power edge inference. I’m also an investor in Reality.ai, which makes it easy to deploy edge models for inference on data from physical sensors. But this market is expected to grow, and edge computing workloads will outstrip cloud workloads in a few years.
This is one of the biggest trends investors, and many entrepreneurs, are missing. Cloud has been the dominate compute paradigm for 15 years. It’s about to change, and while cloud will remain important, edge AI will provide massive opportunities for innovation and scale. It’s something to pay attention to, and if you are working on an edge AI company, I’d love to talk to you.
Thanks for reading.
@robmay