The Coming Dark Patterns of Botomation In AI
Everything will be a bot so that it can collect more data
Happy Sunday and welcome to Investing in AI. I’m Rob May, CEO at BrandGuard, and co-founder of the AI Innovator’s Community. I’m also an active early stage investor (most recently Graphlan and Salessync).
Today I want to write about the dark patterns of the internet and how those are likely to get worse with AI. Dark patterns seem to be increasingly common to me. A simple example is how Amazon defaults to buying certain things via subscription, rather than one-time purchases, and you have to modify that choice before you buy. Subscriptions benefit Amazon. That’s why they are the default. I’m sure internally someone has crafted some arguments that defaulting to subscriptions benefit the user, but they are mostly like specious and self-serving.
Notifications are another dark pattern. I’ve written in other places about my distaste for the Doordash notifications. If you let them be in their normal state, you get about 15 per order:
Your order went through to the store
They are working on your order
Your order has been picked up
Your delivery person is approaching
Your order has been dropped off
For each of these notifications, you get it 3 times: email, in-app, and text. If you modify them and shut them off, you end up with just 6 notifications, I believe. And that’s if you “turn off” notifications. Someone from Doordash responded and told me why. The hard part of the Doordash model is the delivery drivers. If you don’t notice your delivery happened, and get it 15 minutes after it’s been sitting outside, your food is cold and you are mad and if I remember correctly, maybe the driver gets rated low on the delivery quality, which affects their pay.
So Doordash nags you to death with notifications to avoid that problem. It’s a dark pattern because it’s more beneficial to Doordash and delivery drivers than to customers.
I bring this up because, I believe AI will create dark patterns far beyond what we’ve seen today. In particular, I expect the botomation that is coming - everything turning into a bot - will contribute to this. Why? Because automated bots are great at gathering information that companies will want. Thus, the incentive will be for the bots to nag you, and since it isn’t a human who actually has to feel bad about it, it will be easier to justify because no human is in the loop to complain.
Think about how annoying it is that Alexa is always asking extra questions and offering stuff you don’t need but that Amazon wants to force on you. I just ask for today’s weather and Alexa tells me something about a new artist releasing an album I like or something I bought in the past is now on sale. I just wanted the weather.
As bots take over everything, they will have so much information at their fingertips, and the incentives to gather more. I’m sure bot designers will justify it by saying things like “imagine if it can just anticipate your thoughts and give you want you want before you even know it.” Call me old-fashioned but, I like thinking. I don’t want something to short-circuit my thoughts and predict everything for me. I already live my life in ways that try to minimize my reliance on algorithms (I personally hate recommendation engines of all types) and I dread the world that botomation will most likely create.
I’ve written before how tech constantly creates problems that weren’t expected. We obsess over lowering the friction to certain activities, missing the idea that maybe friction is sometimes a good thing. And we ignore possible second order effects of our decisions. Anytime you think about something that’s a neat hack, you should think about what life is like when everyone does that hack at scale. If you are designing bots and thinking about the future of tech, I hope you will think about more than how to manipulate users to keep up engagement and growth in data sets. Businesses should think holistically about their products and their impact on the world. That’s about best chance of avoiding the dark patterns of the coming botomation.
Thanks for reading.