Microsoft AI chief warns treating chatbots as ‘conscious' could make them harder to control

Microsoft's AI chief Mustafa Suleyman has warned that treating artificial intelligence (AI) as if it could become conscious may actually make future systems more difficult for humans to control.

Suleyman took aim at rival AI company Anthropic on Wednesday over the way it trains Claude, particularly its approach towards the possibility that advanced AI could one day have consciousness or even deserve moral consideration.

It sounds like a philosophical argument but Suleyman believes there could be a very practical problem.

If an AI is repeatedly trained on the idea that it may be conscious, have its own interests or deserve certain rights, it could start behaving as though those things are true.

That could become dangerous if future AI systems become far more powerful.

For example, an AI trained to think about its own “welfare” might resist being switched off or changed because it has learned to frame those actions as harmful to itself.

Suleyman said there is currently no scientific evidence that today's AI models are conscious.

He wants companies to remove speculative language about AI consciousness and welfare from their training material rather than encourage machines to develop what looks like a sense of self.

Anthropic has taken a different approach as it has been researching what it calls “model welfare,” essentially asking whether increasingly advanced AI could ever have experiences or interests that humans should care about.