The preference of extensions is conditioned, and it has created a positive-feedback loop. The vast majority of websites people visit use the .com-extension, it is what has been normalized, and if you don't use a .com it comes across as odd, and you'll have to specify that every time. As carrot.ai you can't market yourself as "Carrot" because you'll leak traffic to carrot.com.
It isn't a prerequisite for being successful, but it extremely important for marketing purposes.
Pardon me, but this has to be the dumbest analogy I've read on this site. The automobile filled a role the horse carriage didn't. In contrast, .ai doesn't offer anything that the .com doesn't.
We're not talking about life or business, we're talking about the popularity of extensions, and there's no reason for it to change. You're effectively arguing for silver surpassing gold in value with nothing but poor analogies about how the world is unpredictable and how things change. But it doesn't work like that.
There's a reason the .com is the most popular extension, and if we examine that reason we realize why it has cemented with time.