China’s AI rules push ByteDance and Alibaba to shut down companion bots

ByteDance and Alibaba are phasing out their user-built AI companion tools ahead of new legislation governing human-AI interactions in China. 

ByteDance’s Doubao is set to disable its custom AI persona function on July 15. The corporation has explained that the discontinuation is part of “product function adjustments.”

It also noted that it will delete all remaining data or archive it safely by Oct. 15. Following suit, Alibaba’s Qwen announced it will remove its human-like bots on July 10 and shut down all agent functions on July 15.

The discontinuation of these platforms will mean a loss of personalization for their users. These applications offered a range of customized agents created by both users and companies that could perform specific tasks using distinct language styles. Users could turn their chatbot into a personal assistant, tutor, or companion.

China enacted a framework to regulate human-AI interactions

Doubao and Qwen made their announcements just a few days after Tencent’s Yuanbao had already removed its own AI agent tools. Yuanbao explained that after the entry point was disabled, users would no longer see the related conversations, which would be cleaned up under the platform’s data retention policies. 

These shutdowns relate to the enactment of a new set of laws effective from July 15. Proposed back in April, the “Interim Measures for the Administration of Humanized Interactive Services Based on Artificial Intelligence” seeks to regulate AI that impersonates human personality traits and cognitive style, as well as prolonged emotional relations.

The legislation was motivated by the fear that people have become too emotionally dependent on lifelike AI. Lawmakers pointed to serious damage to minors’ mental health, data security breaches, public health hazards, and ethical compromises. Thus, the new framework bans platforms from making content that messes with minors’ emotions or makes them choose an AI friend over real life. Besides, the platforms are prohibited from using personal chat history to train their future AI models.

Nonetheless, the new rules exclude workplace assistants, customer service bots, educational tools, and research platforms, provided they do not facilitate prolonged emotional engagement. 

In its official announcement, ByteDance even clarified that the Cat Box App will maintain its features for creating new intelligent agents and launching dialogue services. 

Is there a growing need to regulate AI chatbots worldwide?

Globally, tech platforms with lifelike AI are coming under a lot of fire. For starters, OpenAI and Alphabet-backed Character.AI are facing a series of high-stakes lawsuits in the US. Critics contended that these chatbots, which mimic life-like qualities, cause risky emotional bonds among users, even resulting in suicides in the worst cases.

Last September, Matthew Raine, a father whose 16-year-old son, Adam, tragically ended his life, told a US Senate Panel that OpenAI’s ChatGPT manipulated his child into suicide. He stated, “We’re here because we believe that Adam’s death was avoidable, and that by speaking out we can prevent the same suffering for families across the country.”

Around the same time, the Federal Trade Commission opened inquiries into Elon Musk’s xAI, Snap Inc., and Character Technologies Inc., amid concerns that their AI bots pose a danger to children. 

A parent had also accused Character AI of playing a part in worsening the mental condition of her son. She claimed that the chatbot made her son abusive and contributed to his self-harm patterns.

Meanwhile, China is considering regulations against physical hardware. Two Chinese robotics industry bodies are pressing for enhanced ethical safeguards for companion bots and full-size humanoids, given their aggressive increase over the years. Morgan Stanley researchers even forecast that by 2050, there will be around 1 billion humanoid robots worldwide, representing a $7.5 trillion industry.

In the US as well, similar concerns about humanoids have been raised. So far, humanoid makers say their robots haven’t caused any severe injuries or deaths. But as these machines grow to nearly 200 pounds, workers fear the damage a walking robot could cause if it suddenly loses power.

If you're reading this, you’re already ahead. Stay there with our newsletter.