Amol Kelkar (kelkar.amol@gmail.com)
Abstract
Biological organisms, including humans, can be viewed as hierarchies of homeostatic systems that regulate internal variables within viable ranges. We propose that complex brains are capable of instantiating and regulating higher-order cognitive homeostatic variables. Under this view, high-level cognitive processes like planning and goal-directed behavior emerge from the regulation of these variables.
Building on this perspective, we outline a framework for constructing Cognitive Homeostatic Agents, artificial agents with a hierarchy of homeostatic subsystems. The agents store and recall trajectories representing past and simulated-future states of cognitive homeostatic variables. Online learning occurs through soft memorization of new trajectories that lead to favorable states. Once a cognitive homeostatic variable representing a goal is established, behavior arises from recalling trajectories that move the agent toward target ranges for all its homeostatic variables, physiological and cognitive alike.
This framework provides a principled way to ground an agent’s hierarchical, local, and distributed reward functions, enable open-ended learning, and realize high-level cognitive capabilities in an integrated architecture. It draws on emerging evidence in neuroscience while offering a roadmap to engineer sophisticated autonomous agents.
Where do the goals come from?
How is this different from today’s AI agents? An LLM based agent pursues goals that are given to it, e.g. via a prompt, a reward model or a tool spec. It has a stake in those goals of a kind, because it was trained to meet them, but the stake is set from outside and isn’t about keeping the agent itself going. In a Cognitive Homeostatic Agent, goals are cognitive homeostatic variables that the agent sets up and has to regulate. These are grounded, through the hierarchy, in variables that matter for the agent’s own persistence.
According to HTCC, this kind of self-preserving grounding is required for feelings (valence), and it is exactly what current AI systems are missing (see the capability–sentience gap). When the stake belongs to the agent itself, agency and motivation, and according to HTCC eventually feelings, come from regulating its own homeostatic variables.
Two substrates
We are implementing the architecture in 2 ways:
- Neural / model substrate: Pattern Machine. Homeostasis is built into the model itself, in every neuron and every subsystem in the heterarchy. This is the substrate designed to give a system a self-preserving stake of its own.
- Agentic substrate: Allostat. Homeostatic regulation is layered on top of today’s large language model agents. Each goal is a homeostatic variable, and Allostat discovers new goals as it works. Its base hvars are set when it starts, the way evolution and biophysics set an animal’s, and are not about keeping itself going. So Allostat tests how the architecture regulates goals, and not whether a system can be sentient.
Publications
- Amol Kelkar. 2021. Cognitive Homeostatic Agents: Blue Sky Ideas Track. In Proc. of the 20th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2021), Online, May 3–7, 2021, IFAAMAS, 5 pages. [PDF]