What is "Conscious AI"?

Amol Kelkar (kelkar.amol@gmail.com)

People use “conscious AI” to mean different things. It helps to first separate how capable an AI system is from whether it feels anything.

Capability: narrow AI and AGI

Narrow AI is built and trained for one task, e.g. recognizing images, recommending videos or playing chess. It can be very good at that task, but it can’t carry what it knows over to other tasks.

Artificial General Intelligence (AGI) is the ability to understand, learn and apply knowledge across a wide range of tasks, at or above human level. Today’s large models are moving towards this level quickly.

Both of these are about capability. A system can move from narrow AI to AGI without anything changing about whether it feels.

Two meanings of “conscious”

• Access consciousness. The system can notice, report on and reason about its own states and its surroundings. It models itself, and makes information available across the system for reporting and control. This is a capability, and it can be measured, at least in principle. Today’s large models already show signs of an alien form of it, limited to what is in their prompt. Interpretability research is finding self-referential, globally broadcast structure inside large models (see the gap).
• Phenomenal consciousness / Sentience. There is something it is like to be the system, and some of its states feel good or bad to it. We think scaling up current models doesn’t give you this. Without recurrent state as the dominant part of its processing, a model can’t carry an experience forward, or have a self model that is its own implementation. And according to HTCC, a system also needs a self-preserving stake, i.e. homeostasis. Today’s models have stakes set from outside, e.g. trained rewards and instructions, but not a self-preserving one.

So “conscious AI” is not the next step after AGI. Sentience is a different question from capability.

Why the difference matters

When the two meanings get mixed up, capable and self-modeling systems get called “conscious”, and people assume they have feelings too. Separating the two makes it clear that a system can be very capable, and maybe even functionally self-aware, without being sentient.

At i3AI we study the second kind. HTCC explains why feelings need a self-preserving stake, Pattern Machine is an architecture designed to build that stake in, and the testable claims are on Predictions. The terms are defined in the glossary.