Anthropic, an artificial intelligence (AI) research company, recently recruited researcher Kyle Fish to its science team, focusing on studying AI welfare and developing guidelines for AI capabilities that may require ethical considerations.
Kyle Fish co-authored a report called “AI Welfare Concerns,” which warns that AI could achieve sentience or autonomous behavior — two factors considered important in ethical considerations. However, the report does not assert that AI will definitely have these abilities.
According to the report, there are three steps AI companies can take to address these concerns.
First, it is important to acknowledge that this is a difficult and important problem, and to ensure that AI models reflect this in their results.
Second, companies should evaluate AI for signs of cognition. Finally, policies need to be developed to treat AI with the appropriate level of ethical concern.
The researchers propose a method for assessing AI “awareness” that could be based on markers commonly used to assess cognition in animals. However, they are speculative and say that multiple metrics would need to be examined to make a probabilistic assessment.
The risk of viewing AI as a sentient entity is that misunderstanding AI as “sentient” will lead to the risk of wasting unnecessary resources on protecting AI.
In 2022, Google fired engineer Blake Lamoine after he claimed that Google's AI LaMDA was sentient. Similarly, when Microsoft launched Bing Chat, many people mistakenly believed that the chatbot had real emotions, to the point that changing the chatbot's settings was like "losing a friend."
In this context, while AI protection is still a novel idea, interest is growing.
Google DeepMind also recently posted a job posting for “machine cognition” research. While Anthropic has not yet taken an official stance on AI welfare, CEO Dario Amodei has mentioned AI cognition as a concern.
Kyle Fish believes that developing the concept of “AI welfare” is necessary, although there are still many questions without clear answers.
