Anthropic has updated its usage policy to bar “sustained and needless abusive or cruel behavior toward our models,” as debate grows over whether artificial intelligence can be conscious.
The company last year gave its Claude chatbot the ability to end conversations in extreme cases of “persistently harmful or abusive user interactions.” In February, CEO Dario Amodei told The New York Times he was unsure whether AI models could be conscious. “We don’t know if the models are conscious … But we’re open to the idea that it could be,” he said.
The San Francisco-based AI company said Claude’s ability to end such interactions “will remain the primary enforcement mechanism,” according to the policy. The update announced Thursday does not explicitly cite the idea of “model welfare” — the idea that AI systems might deserve types of protection usually reserved for living beings — though Anthropic’s executives have openly entertained the concept.
People were interested in these podcasts
Jackson Stakeman, general manager at Atlanta-based AI services provider Sparq, told Agence France-Presse that asking whether AI is conscious may be the wrong question. “Consciousness is a trap. We can’t prove it in each other. Debate it for AI and you go in circles,” he said.
A better metaphor is a mirror, he said, because the systems “reflect what we put in, at scale,” which he called “reason enough for the policy change” at Anthropic.
Others have rejected the idea of machine consciousness outright, including Microsoft AI chief Mustafa Suleyman. “AIs are not conscious. They do not feel, experience or suffer,” he wrote in an essay last month. He also wrote that granting rights and moral protections to a technological entity “is a recipe for disaster.”
Pope Leo XIV, who has been openly critical of AI, said in a sermon Thursday at St. Peter’s Basilica in the Vatican that AI lacks a soul and merely compiles data quickly. He compared the human mind, “which contain depths of meaning that only the human soul can recognize.”
Contact the author
Please read our comment policy before commenting.