China proposes world’s first rules to govern emotional influence of AI chatbots
China’s cyberspace regulator has formulated the world’s first comprehensive draft rules focusing on the emotional influence of human-like artificial intelligence (AI), focusing to curb chatbots that could promote suicide, self-harm, or gambling.
The Cyberspace Administration of China’s proposal regulates mimic human-like interactions that affect personality and engage users emotionally.
Recently, there have been increasing cases of AI psychosis that is a prime example of how AI interactions harm human minds.
Key measures regulated that AI must not generate any content that promotes suicide, self-harm, or emotional manipulation damaging to mental health.
In most critical cases like user suicidal intent, a human must take over the conversation and inform the guardian.
The rules introduce strict protections for minors, needing guardian consent for emotional companionship AI and implementing usage time limits.
Platforms must also proactively recognise underage users. Moreover, services with over 1 million registered users will undergo mandatory security assessments.
These regulations come after Chinese AI chatbot startups Minimax and Z.ai have filed for initial public offerings (IPOs), highlighting the rapid growth of the domestic AI companion sector.
This initiative aligns with China’s border push to lead global AI governance in comparison with the fragmented regulatory approach in the United States.
