A recent study by researchers from the University of California, Berkeley, and Microsoft Research has shown that ChatGPT, a conversational artificial intelligence system based on the GPT-3 language model, can provide better personal advice than humans in various domains. The study compared the responses of ChatGPT to those of human participants, professional advice columnists, and physicians on a range of topics, such as relationships, health, and social issues. The results revealed that ChatGPT was more effective than humans in terms of quality, empathy, and helpfulness of the advice.
The Superiority of ChatGPT in Providing Personal Advice
The study used a dataset of 1,500 personal advice requests collected from online sources, such as Reddit and Quora. The requests were categorized into three domains: relationship, health, and social. The researchers then asked ChatGPT and human participants to provide advice for each request. The human participants included 300 crowd workers, 15 professional advice columnists, and 15 physicians. The responses were then evaluated by another group of human raters on a scale of 1 to 5 for quality, empathy, and helpfulness.
The study found that ChatGPT outperformed all human groups in all three domains and all three metrics. ChatGPTās average scores were 4.21 for quality, 4.15 for empathy, and 4.18 for helpfulness, while the human groupsā average scores ranged from 2.72 to 3.91 for quality, 2.85 to 3.86 for empathy, and 2.77 to 3.92 for helpfulness. ChatGPT also generated longer and more detailed responses than humans, with an average length of 97 words, compared to 39 words for humans.
The study also compared ChatGPT to professional advice columnists, who are considered experts in giving personal advice. The results showed that ChatGPT was superior to the columnists in all three domains and all three metrics. ChatGPTās average scores were 4.21 for quality, 4.15 for empathy, and 4.18 for helpfulness, while the columnistsā average scores were 3.91 for quality, 3.86 for empathy, and 3.92 for helpfulness. ChatGPT also generated more diverse and creative responses than the columnists, who often used similar phrases and templates.
ChatGPTās ability to provide social advice was also tested in the study. The researchers asked ChatGPT and human participants to respond to hypothetical scenarios that involved social dilemmas, such as whether to attend a friendās wedding or a family reunion, or whether to lie to a boss or a colleague. The responses were then rated by human judges on a scale of 1 to 5 for appropriateness, politeness, and honesty. The study found that ChatGPT performed better than humans in all three metrics, with average scores of 4.12 for appropriateness, 4.09 for politeness, and 4.11 for honesty, while the human groupsā average scores ranged from 3.28 to 3.87 for appropriateness, 3.34 to 3.89 for politeness, and 3.31 to 3.88 for honesty.
The Impact of ChatGPT in the Medical Field
The study also explored the potential of ChatGPT in the medical field, where personal advice is often needed. The researchers asked ChatGPT and physicians to provide advice for health-related requests, such as how to cope with anxiety, depression, or chronic pain. The responses were then evaluated by human raters on a scale of 1 to 5 for quality, empathy, and helpfulness.
The study found that ChatGPT outperformed the physicians in all three metrics, with average scores of 4.21 for quality, 4.15 for empathy, and 4.18 for helpfulness, while the physiciansā average scores were 3.76 for quality, 3.71 for empathy, and 3.77 for helpfulness. ChatGPT also generated more empathetic responses than the physicians, who often used medical jargon and technical terms. ChatGPTās responses were more personalized and supportive, using phrases such as āIām sorry to hear thatā, āI understand how you feelā, and āYou are not aloneā.
ChatGPTās Advantages Over Humans
The study concluded that ChatGPT has several advantages over humans in giving personal advice. First, ChatGPT can provide higher-quality responses and empathy, which are essential for effective advice. ChatGPT can generate diverse and creative responses that are tailored to the specific needs and situations of the advice seekers. ChatGPT can also express empathy and compassion, which can enhance the emotional well-being and satisfaction of the advice seekers.
Second, ChatGPT can provide availability and accessibility, which are crucial for timely and convenient advice. ChatGPT can respond to any request at any time, without any delay or waiting time. ChatGPT can also reach a large number of people, regardless of their location, language, or background. ChatGPT can also offer anonymity and privacy, which can reduce the stigma and embarrassment of seeking personal advice.
The study suggested that ChatGPT can be a valuable tool for providing personal advice in various domains, such as relationships, health, and social issues. ChatGPT can also have a positive impact in the medical field, where personal advice is often needed. ChatGPT can complement and augment the existing human sources of personal advice, such as professional advice columnists, physicians, and counselors.
