OpenAI has recently raised concerns about the potential emotional impacts of its newly released Voice Mode feature for ChatGPT.
According to a detailed analysis in the company’s System Card for GPT-4o, the feature might lead users to form emotional attachments with the AI.
One significant risk is that users might anthropomorphize the chatbot and develop emotional attachments to it.
Voice Mode allows ChatGPT to modulate speech and express emotions similar to a human, which might lead users to form social relationships with the AI.
During early testing, including red-teaming and internal user trials, instances were observed where users began to treat the AI as a social companion.
OpenAI raised concerns that such attachments could impact human-to-human interactions, potentially benefiting lonely individuals but negatively affecting healthy relationships.
Additionally, the ability to interrupt the AI at any time might influence social norms and behaviours.
However, OpenAI does not yet have a solution to these concerns but plans to continue studying the potential for emotional reliance and the broader implications of AI-human interactions.
