An agent is something that acts in an environment; it does something. Agents include worms, dogs, thermostats, airplanes, robots, humans, companies, and countries.
from Poole and Mackworth as quoted by Willison
For anyone working with language models on complex projects, a certain weirdness starts to become salient. In one sense, they're astonishingly human, capable of complex output that suggests a nuanced intelligence. The biggest difference is once they've finished streaming output and you're looking at a blinking cursor again. Whatever you're interacting with is quiescent, totally reactive.
Agentic AI systems - agents to help you code, agents to help you book travel, agents connected via MCP to practically anything - change this dynamic. There isn't a commonly agreed on definition of agent as applied to AI, although certain features (an LLM in a loop, tool use) are pretty universal. It's helpful to think about it as a spectrum: at one end is an LLM that just answers questions, in the middle are AI agents as we know them today, at the far end is something like a human being.
What is increasing as we go from the chatbot end of the spectrum to the human end? Let's start from just observing behavior to try to differentiate between humans and AIs. To keep things fair, let's just imagine observing a browser controlled by a human and a browser controlled by an AI agent. The AI agent gets an instruction - a goal - and pursues a set of steps to try to achieve it. The human being, on the other hand, has forty tabs open related to different projects, interests and sudden whims, and switches between them fluidly. The human being has a much richer set of goals, like an orchestra of motivations that blend together to produce behavior in a given moment. Some of these goals are conscious, like accomplishing a specific goal. Some are basically implicit, like survival.
I don't mean to suggest that the AI agent has just a single goal. Even if it's given a single explicit goal, it will come up with sub-goals that need to be accomplished to reach its main goal. It can change approaches, replacing some sub-goals with other sub-goals. Still, in terms of goal complexity it's orders of magnitude away from a human being.
Adaptivity implies sense-making, which is behaviour or conduct in relation to norms of interaction that the system itself brings forth on the basis of its adaptive autonomy. An adaptive autonomous system produces and sustains its own identity in precarious conditions, registered as better or worse, and thereby establishes a perspective from which interactions with the world acquire a normative status.
from Could all life be sentient?
Could human behavior be simulated with a large enough context window? Maybe. Given a large enough network it's hard to prove that something isn't happening at some level of the network. I'm pretty confident, though, that the current approach for building agentic systems - giving explicit instructions - isn't going to produce equivalently complex goal-oriented behavior. We are constantly evaluating our environment with reference to a much richer goal set than we're able to process consciously. Those goals cause us to assign a hedonic valence to new input - and new input can also cause us to adjust which goals we think are salient.
As AI pioneer Hans Moravec put it, abstract thought “is a new trick, perhaps less than 100 thousand years old….effective only because it is supported by this much older and much more powerful, though usually unconscious, sensorimotor knowledge.
from Catalyzing next-generation Artificial Intelligence through NeuroAI
I think the way we assign valence is sort of a lower level version of emotions. Academics eventually settled on calling emotions a combination of a physical sensation and an interpretation of that sensation. The way we register valence similarly has a physical component. That physical component suggests that emotions and valence are tied to our sensorimotor system.
I'm less confident about the physical component of assessing hedonic valence than I am about the need for a much richer approach to goal-oriented behavior. Maybe if we just scale up context windows enough, we'll get AI agents that display goal-oriented behavior just as rich as human beings. If you think that emotions are linked to hedonic valence, though, and you acknowledge that we experience emotions using a different system than language processing, you'll have reason to believe that our motivational system is meaningfully different than our system for language processing. Call it memory, call it something else, but I think we'll need a breakthrough on the level of neural networks to achieve adaptive goal-oriented behavior at a human level.