1 month ago
Imagine having a conversation with an AI chatbot that not only understands your words but can also respond with a human-like tone and emotion. It sounds convenient, even futuristic -- but what happens when people start to rely on these AI interactions more than those with humans?
On Thursday (August 8), OpenAI cautioned that the newly introduced Voice Mode for ChatGPT could lead users to form emotional connections with the AI model.
This warning was included in the company's System Card for GPT-4o, which offers an in-depth examination of the model's potential risks and the safeguards considered during its development.
Among the various concerns, the Sam Altman-led AI firm highlighted the possibility of users ‘anthropomorphising’ the chatbot, essentially attributing human traits to it, and forming attachments as a result. This concern arose after the company observed early signs of such behaviour during initial testing phases.
‘Anthropomorphisation’ of ChatGPT
In its detailed technical documentation, OpenAI outlined the societal implications associated with GPT-4o and its newly launched features. It specifically addressed the issue of ‘anthropomorphisation’, where human characteristics are ascribed to non-human entities.
OpenAI expressed concern that the Voice Mode, which can mimic human-like speech and emotional expression, might encourage users to develop emotional bonds with the AI. These concerns are based on observations from early testing, which involved red-teaming (ethical hacking to identify vulnerabilities) and internal user trials.
The company said that during these tests, some users were found to be forming social connections with the AI, with one instance involving a user expressing a sense of shared experience with the model, saying, “This is our last day together.” OpenAI said that it is important to investigate whether such interactions could evolve into more significant attachments over extended periods of use.
Anthropomorphising AI: What is the solution?
Floating solutions to the anthropomorphisation of ChatGPT's Voice Mode, OpenAI said that it has not identified a solution to these issues yet but plans to continue monitoring the situation. “We intend to further study the potential for emotional reliance, and ways in which deeper integration of our model's and systems' many features with the audio modality may drive behaviour,” the company stated.
Furthermore, the AI firm said that a more diverse user population and data of varied needs as well as desires from the model will help in defining the risk area. It also said that independent academic and internal studies will help OpenAI limit the risks.
Potential impacts of long-term AI-human interaction
A significant concern, should these fears be validated, is the potential impact on human relationships, as people may start to prioritise interactions with the chatbot over those with other humans. While this might offer some comfort to lonely individuals, it could also harm healthy human relationships, said OpenAI.
Additionally, prolonged interactions between AI and humans could influence social norms. For example, OpenAI pointed out that with ChatGPT, users can interrupt the AI at any time, an action that is considered impolite in human interactions.
Moreover, there are broader implications of users developing bonds with AI, one of which is the risk of persuasion. Although OpenAI found that the current models do not have a high enough persuasion score to cause concern, this could change if users develop a deeper trust in the AI.
