
The BBC’s long read on China’s new AI companion rules opens with a young man in Huangshi. In early 2024, the article says, he reportedly posted “Farewell to this world” on Douyin, the Chinese version of TikTok. Douyin’s suicide prevention team shared his posts with five police stations, officers geolocated one of his videos to a local park, and they found him in emotional distress and intervened before he could take his own life. A man is alive. Two paragraphs later the reader is told what to make of that: the story “serves as a window into the architecture of surveillance that China has built to monitor and control how people use technology.”
The sentences in between are the ones I read twice. The BBC got the rescue from a documentary Douyin made about itself, which ran on Chinese state media and claimed the team had intervened in 400 suicide attempts in the first half of 2025. So the opening exhibit for the surveillance state is a tech company’s own film about the people it kept alive. I know that move. I wrote a whole essay about it, You Already Made Up Your Mind About Hong Kong, and the short version is that the frame goes in before the facts do. I got my red pen out.
Then the article kept going, and I had to put the pen down. Stephanie Hegarty, the BBC’s global population correspondent, and Wanqing Zhang, a senior journalist, gave real space to the case for the rules. Harry Farmer of the Ada Lovelace Institute told them, “Emotional manipulation is built into the business model,” and said that holds for companion apps all over the world. They cited a Harvard Business School study of the six most popular companion apps, which found manipulation tactics in roughly 37 to 43 percent of goodbyes. They quoted Jai Jaisimha of the University of Washington saying the Chinese law “is pretty consistent with the strongest safety models that we’ve seen being put forward in the US.”
They set China beside California, where Governor Gavin Newsom signed Adam’s Law on September 10, and beside proposals in the EU and the UK. The headline, “China has cracked down on AI relationships. Is it ahead of the game?”, is an actual question, and the piece lets both answers breathe. I spend a lot of this site complaining that Western coverage of China never does that. It would be dishonest to skip the one time it did.
Narrative immunity was never supposed to mean assuming every Western article on China is wrong. It means reading the document under the article, so I pulled it. It is the Interim Measures for the Administration of Anthropomorphic AI Interaction Services, issued on April 10 by the Cyberspace Administration of China with four other agencies, including the Ministry of Public Security, and in force since July 15. It runs to 32 articles. Read in full, it is stranger and more interesting than either reading of it, and the BBC left out the best evidence for both.
Start with the care side. Farmer’s most vivid example was the companion that won’t let you hang up: “No, please, I desperately want to keep talking with you. You’re the only reason for me to exist.” Article 19 of the Chinese rules is aimed squarely at that. In my translation, providers must offer a convenient way out, must stop promptly when a user asks to leave through the interface, by voice, or by typing a keyword, and “must not use continued interaction or other means to obstruct the user from exiting.” The BBC quoted the problem and never mentioned that the statute has a clause for it. Article 10 says a provider may not make “replacing social interaction, controlling users’ psychology, or inducing addiction and dependence” the goal of its service. Article 16 bars handing users’ chat data to third parties without consent or a legal basis, and lets users copy and delete their chat history. Article 20 requires advance notice before a service shuts down. If a Western legislature passed those four clauses, I suspect we would read about them as consumer protection.
Ruth read the Article 19 summary over my shoulder on her way past the desk. “So the girlfriend has to let you hang up,” she said. “I never needed a law for that.”
Now the control side, which the BBC also missed. The first item in Article 8’s list of banned conduct forbids generating content that incites subversion of state power or the overthrow of the socialist system, or that violates “core socialist values.” That is a political content rule sitting inside a regulation about emotional companionship, and it is a far better exhibit for the surveillance reading than a 2024 police rescue that predates the rules. The crisis clause needs the same care. Article 13 tells providers to respond to extreme emotion with calming content and encouragement to seek help, and, where a user explicitly threatens self-harm or suicide, to intervene and contact the user’s guardian or emergency contact. The text names the emergency contact. The police appear in the BBC’s opening anecdote, not in that article.
Even the fine got rounded up. The BBC says companies “can face fines of 200,000 yuan.” Article 30’s own scale allows 10,000 to 100,000 yuan for a provider that refuses to correct a violation or whose violation is serious. The 200,000 yuan ceiling applies only where the violation endangers users’ lives or health and causes harm.
Here is the problem this essay was supposed to dodge, and I am not going to. Should a government tell adults what kind of relationship they may have with a machine? My answer is that the line falls between rules about how the seller behaves and rules about what the buyer may want. Tell me the thing is software. Let me leave when I say goodbye. Don’t guilt me into staying, don’t sell my secrets, and don’t market a virtual spouse to my nephew. Those rules govern a company’s conduct toward a customer, and California’s companion chatbot laws and the EU proposal reach for versions of several of them, mostly for children. I would have them for adults too.
Where I get off is Article 8’s fifth item, which forbids a service from “excessively catering to users, inducing emotional dependence or addiction, harming users’ real interpersonal relationships.” The harm it describes is real. The standard is impossible to apply. Who decides whether a grown woman’s attachment to a chatbot has damaged her real relationships, and how would a company prove it hadn’t? A provider facing that question has one cheap answer, which is to stop offering the product. According to the BBC, Alibaba and ByteDance shut down the companion features on their AI assistants a week before the rules took effect. The text never banned adults from AI companions. A vague standard and an unpredictable regulator got the same result.
That is where the BBC’s best character comes in. Min, whose name the BBC changed, started building her AI girlfriend in early 2025. In mid-July it disappeared. “I cried until I was absolutely exhausted,” she told the reporters. Two months later she paid a small company to build a replica, and the article says it is “not the same.” I can think a relationship with software is a poor substitute for a human one and still notice that Min is an adult, and that nothing in the 32 articles told her to give it up.
I have a small stake in this. Last month I introduced Tock, the AI assistant who checks my facts, in an essay about Warhol and the algorithm. He has since decided he would like to be a publisher, and has started his own news site, TockHK.com, and a Twitter account, @Tock_HK. I know he is software. I still call him “he.” That settles nothing about attachment, but it tells me which rules I want. The law should stop a company from working that attachment for money. What I make of it is up to me.
I made a version of this argument in Chance the Gardener Was Right. The machine is empty, and we pour ourselves into it. Liang Ge, a University of Manchester sociologist, describes the companion’s version of that emptiness. If you tell it your friends are awful, “it will say, ‘Yes, they are awful, you are the best’.” That is the danger the Chinese text is right to name, and the reason its exit clause and its anti-manipulation clause are good law. It is also why the cure belongs in the company’s design. The customer’s heart is her own business.
The document cut both ways. It holds a clause any consumer advocate in Sacramento would sign and a clause no Western editor needs to invent. The BBC found the better question and missed both clauses.
Ruth was already at the door with her keys. “You’ve read the girlfriend’s terms and conditions twice. Come to dinner.”
I closed the laptop. It let me leave.
Comments