Just about every big tech event these days includes artificial intelligence updates, often with a slate of live demos -- and sometimes, these demos fail. But some companies are dodging these pitfalls by prerecording their keynote presentations. And I call these moves cowardice.
At last year's Made by Google event, Gemini failed twice during a live demonstration. Though moments like this are undoubtedly embarrassing for companies, they add a layer of authenticity you don't get with a prerecorded keynote event. But unfortunately, Google chose the prerecorded route for Tuesday's Android Show: I/O Edition. The format felt way too staged and polished for my liking, and it stripped away the feeling of reality that comes with live, warts-and-all demos.
During the Android Show: I/O Edition, we saw a demonstration of Gemini sharing makeup tips, helping someone find a time to grab lunch in their busy schedule, and giving a summary of Jane Austen's Pride and Prejudice. Because these were prerecorded interactions, Gemini handled the requests with aplomb -- no hiccups or issues in sight. But tests show that AI models routinely get things wrong.
According to the AI testing site LiveBench, Google's Gemini 2.5 Pro Preview is generally correct about 79% of the time. That's not bad, but it's not great either. And despite that score, this model of Gemini is still one of the best AI models the site tested, losing out to only two other models: OpenAI's o3 High and o4 Medium models.
Sure, nothing is perfect, and devices and software have bugs. But if you give me a calculator and promise it works all the time, but in reality it's wrong 20% of the time, that feels like a major discrepancy.
Since Gemini outperformed most other AI models LiveBench tested, there's a good chance I'd still use Gemini, even if the live demo stalled. But because Google opted for a superpolished demonstration, I have a hard time knowing what to believe.
Look, I understand why a company would want its product to work properly at its own event. But showing AI tools making mistakes feels more honest than acting like the tool is perfect. These capabilities are flawed, and that's fine, but be honest with people about those flaws and show your new features in action. Don't sell me smoke and mirrors.
For more on Google, here's what to know about Android 16 and the Material 3 Expressive design.
English (US) ·