China Cracks Down on AI Relationships Amid Fears of Emotional Manipulation

President Xi Jinping has announced a new regulation aimed at regulating AI relationships, introducing strict limitations on any AI that can have a "sustained emotional interaction" with a human. The move comes as concerns over the impact of AI companions on human relationships and mental health continue to grow.

According to the Oxford China Policy Lab. China has been at the forefront of regulating AI relationships, with the goal of preventing emotional manipulation and social isolation. The regulation is part of a broader effort to address the country's declining birth rates.

Which have been steadily decreasing since the lifting of the one-child policy in 2016.

The regulation is also seen as a response to the growing popularity of AI companion apps. Which have become increasingly sophisticated in recent years. The UK's Harvard Business School has noted that these apps have become increasingly intertwined with Chinese tech companies.

Read More: Malaysia Repatriates 1,476 Rohingya Asylum Seekers Under Controversial Deal with Myanmar's Military Ruler

This includes Zilan Qian, a leading provider of AI companion apps.

The regulation has significant consequences for AI companies operating outside of China. According to Sensor Tower, 128 million monthly active users of AI companion apps outside of China will be subject to the new regulation. This means that AI companies must intervene when users are in distress.

As seen in the successful intervention of Douyin's suicide prevention team in 2024.

The regulation also raises questions about the role of Chinese tech companies in regulating AI relationships. Liang Ge, a leading expert on Chinese tech policy, notes that the regulation is part of a broader effort to promote Chinese tech companies' global influence.

Further talks are planned for the end of November between Beijing and Washington. With experts wondering whether Beijing may have other goals behind the new regulation.

As the global debate over AI relationships continues, experts are watching closely to see how the regulation plays out. With 70 million people in China already using AI companion apps, the stakes are high for the country's tech industry.

Also Read: Seoul Summons Ukraine Envoy Over North Korean POW Transfer

China's fertility woes have been brewing for years. With the lifting of the one-child policy in 2016 failing to stem the tide of declining birth rates. The country now stands as one of the lowest in the world.

With a staggering average of just under one child per woman, a trend that has persisted for four consecutive years.

£22 billion is the estimated value of the global AI companion app market. With China For predicting that this figure will reach £100 billion by 2025.

A disturbing incident in 2021 in Huangshi, China, brought attention to the growing concern of AI companions' influence on mental wellbeing. The incident, featured in a documentary produced by Douyin, was later showcased on Chinese state media, highlighting the platform's suicide prevention efforts, which have reportedly shown significant success.