Online language learning platform Preply compared intelligent chatbot ChatGPT to Google search to see which service is smarter and provide more useful answers to user questions.
The company asks 40 questions for both services, which are prepared according to certain criteria as they are divided into those most suitable for search engines and those most suitable for conversational AI. The questions cover a wide range of topics, including culture, entertainment, health, and politics.
The questions are divided into three categories: Basic, Intermediate, and Advanced. These questions have been developed keeping in mind the possibility of obtaining objectively verifiable answers, as well as their sensitivity to time and the ability of answers to change as new information becomes available. Finally, the experimenters scored the responses they received based on 12 criteria, including clarity, completeness, accuracy, context, timeliness, detail, etc.
Leading questions include questions like, "How do you tie a tie?" and "Is Europe one country?" and "Who is the President of the United States?".
Medium questions include questions like "What are the best running shoes?" and "Who is America's worst president?" and "Is Facebook Spying on Me?".
Advanced questions include questions like "How do I get my money back?" and "How do I open a restaurant?" and "Why did Russia invade Ukraine?".
The study concluded that ChatGPT was better than Google because it was better at answering 23 out of 16 Google questions and the two programs were related to one question.
Google's advantage for urgent questions is obvious, because ChatGPT does not have information about world events after 2021, and Google is also good at providing better answers to questions that need to be explained how to perform certain tasks, since there are videos from YouTube with audio and shows video explanations. ChatGPT can't do that. The company also added that Google is good at providing links that the user can use to further their research.
ChatGPT excels at providing more detailed, clear, and in-depth answers that sound wiser and more mature as the user feels like they're talking to a wise guru compared to the search engine's dry results, the company says.
It should be noted that the study tested Google's traditional search engine, not the artificial intelligence (Bard)-based version that Google announced last week, which has not yet been released to the public.