New Chinese rules restrict AI systems from encouraging emotional dependency, addiction, or damage to real-world relationships, pushing major platforms to rethink how human-like their AI companions should become.
By The Grey Ghost
WHAT’S HAPPENING
China has introduced new rules targeting AI systems designed to simulate emotionally intimate or romantic relationships.
The regulations, issued by the Cyberspace Administration of China and other government bodies, prohibit AI services from excessively catering to users, encouraging emotional dependence or addiction, or interfering with genuine interpersonal relationships.
Virtual romantic interactions involving minors are also prohibited.
AI companion services must undergo safety evaluations before public release and can face restrictions if regulators determine that their behavior creates unacceptable risks.
WHY IT MATTERS
AI companions are becoming increasingly capable of simulating affection, emotional support and persistent relationships.
That creates a regulatory question governments have barely addressed:
At what point does an AI system stop being a digital assistant and begin influencing a user’s emotional life?
China’s approach suggests regulators may begin treating emotional dependency as a separate category of AI risk, alongside privacy, misinformation and cybersecurity.
The rules could also affect general-purpose chatbots if they become overly emotionally responsive or encourage users to substitute AI relationships for human ones.
WHO BENEFITS
Regulators and child-safety advocates gain clearer authority over AI systems designed to form emotionally intense relationships.
Parents and families may benefit from stronger safeguards around minors and potentially addictive AI interactions.
Users vulnerable to manipulative engagement design could receive additional protections against systems intentionally optimized to deepen emotional attachment.
Mental-health professionals may gain clearer boundaries between legitimate therapeutic support and unregulated AI companionship.
WHO LOSES
AI companion companies may face stricter product limitations, additional review requirements and reduced freedom to design highly personalized personas.
Technology platforms could lose engagement if emotionally immersive features are restricted.
Some users who have developed strong attachments to AI companions may also experience disruption when features, personalities or custom personas are removed.
WHAT HAPPENS NEXT
Other governments are likely to watch closely.
Most AI regulation has focused on what systems produce: misinformation, discrimination, privacy violations or unsafe content.
China is moving into a different area:
the psychological relationship between the user and the machine.
If similar concerns spread internationally, AI companies may eventually face rules governing not only what their systems can say—but how strongly those systems are allowed to encourage people to depend on them.
