Google’s new AI Overviews aim to provide quick answers at the top of search results. However, many have noticed these summaries can be humorously or dangerously incorrect. Mistakes from existing AI chatbots can be problematic, but the impact is greater when these errors appear prominently in Google search results.
A Google spokesperson explained that most AI Overviews deliver quality information, with links for further exploration. However, some errors occur, often in response to uncommon queries.
Many errors fall into a few categories. For example, the AI sometimes fails to recognize jokes or sarcasm, like suggesting “non-toxic glue” to keep cheese on pizza or recommending “blinker fluid” for a turn signal issue. These humorous posts would usually be buried in regular search results but are highlighted by AI Overviews.
In some cases, Google’s AI uses sources that aren’t reliable, such as fan fiction or troll posts, without indicating their dubious nature. This includes claiming a 2022 remake of “2001: A Space Odyssey” by Spielberg and Lucas, sourced from an unreliable wiki.
The AI can also answer a different question than asked. For example, when asked about the southernmost point in mainland Alaska, the AI gave information about an Aleutian island instead. Similarly, it provided responses about dogs participating in promotional activities instead of playing in the NHL or NBA.
Simple factual errors also appear, like incorrect math or geography answers. For instance, the AI claimed a cumulative price decrease between 2000 and 2023, which is incorrect. It also confused people with similar names, mixing up President Harry S. Truman with a man who died in the Mount St. Helens eruption.
Google’s AI sometimes produces odd but technically correct answers, like identifying NBA players nicknamed “Duck.” These errors are being fixed gradually. A Google spokesperson stated that extensive testing was done before launch and that the company is refining the system based on these issues.
Despite the errors, many AI Overviews work well, providing accurate information. However, given the importance of Google search, these mistakes highlight the need for careful improvement.
