How AI Agents Actually Work: Beyond the Hype, Into the Infrastructure

In the previous article we opened the machine and named its parts. This one explains what happens when the machine runs. Not because every executive needs the engineering, but because a small amount of mechanical understanding changes how you buy, budget, and govern this technology. Most of the poor decisions we see in this market, overpaying for the wrong thing, underestimating running costs, expecting instant responses from systems doing deep work, trace back to a mental model of AI that stopped at the chatbot.
So here is the machinery, in plain language, and what each part means for the operating decisions you will actually face.
An Agent Is a Loop, Not an Answer
A chatbot is a single exchange: input arrives, output returns, the transaction ends. An agent is a loop that runs until a goal is reached. It observes the current state of the work, reasons about the next step, takes an action, observes the result, and goes again. Reconciling one invoice might take a dozen cycles of that loop. Monitoring a compliance register takes an unending stream of them.
This single mechanical fact explains most of the economic difference between the two categories, and it is worth internalizing before any budgeting conversation. A chatbot’s cost scales with how often people talk to it. An agent’s cost scales with how much work exists.
The loop is also why agents can recover from failure in a way scripts cannot: when a step fails, the loop observes the failure and reasons about an alternative, rather than halting.
That resilience is bought with computation, and computation is the next subject.
Tokens: The Unit of Machine Work
Every cycle of the loop consumes inference, and inference is measured in tokens, small fragments of text the model reads and writes. Tokens are to the agentic economy what kilowatt-hours are to the electrical one: the unit in which work is denominated, priced, and capacity-planned.
This is not our metaphor. It is now how capacity is described at national scale. When wenoted in the first article of this series that G42 frames its planned compute hub as producing up to one hundred trillion tokens per day, the choice of unit was the message: the UAE is measuring national AI capacity in units of work performed, not hardware installed.
For a buyer, tokens change the shape of cost in two practical ways. First, cost follows volume and complexity, not headcount: an agent that processes double the documents costs roughly double to run, which makes agent economics elastic in the way we described in the article on chatbots and the bottom line. Second, cost is sensitive to design quality. A well-engineered agent reads only the context each step requires. A carelessly built one rereads everything, every cycle, and produces a bill several times larger for identical output.
When comparing vendors, cost per completed outcome is the honest metric. Cost per seat, inherited from the software era, measures nothing an agent does.
Tool Calls: How Software Reaches Into the World
Reasoning becomes action through a mechanism called tool calling. The model does not press buttons or move a cursor. It emits a precisely structured request, create this record, send this message, query this database, and the surrounding system executes that request against the real software through its interfaces, then feeds the result back into the loop.
Two consequences matter at the executive level. The first is that an agent can only act where it has been deliberately connected. Integration is not an afterthought to the intelligence; in production it is most of the work, and in this region it includes the channel reality we described in the GCC article, because an agent that cannot operate where Gulf business actually communicates is connected to the wrong systems. The second consequence is reassuring: every action passes through a defined, permissioned, loggable interface. The tool call is precisely where the guardrails from the previous article attach.
There is no mysterious hand inside your systems. There is a narrow doorway you built, with a lock you control and a camera you installed.
Memory: Why Agents Forget, and What Is Done About It
Models have a working memory called the context window: the information they can hold in view at one moment. It is finite, and nothing outside it exists for the model at that moment.
Left alone, an agent would begin every task with total amnesia.
Production systems solve this with an external memory architecture. Durable facts, policies, precedents, and history live in databases and document stores, and a retrieval layer fetches exactly what the current step needs into the context window, then lets it go. The agent does not remember your business the way an employee does. It looks things up, extremely fast, every single cycle.The executive consequence is direct and frequently discovered too late: an agent is only as good as what it can retrieve. Organizations whose knowledge lives in tidy systems of record get capable agents quickly. Organizations whose truth lives in inboxes and personal folders discover that the first phase of an agent programme is, in substance, an information hygiene programme. That work is unglamorous and it is also where a large share of the value is created, because it outlasts any single system.
The Infrastructure Bill Arrives
Assemble the mechanics, a loop that never ends, denominated in tokens, acting through integrations, retrieving context continuously, and one strategic conclusion follows: agents convert AI from an occasional expense into a continuous industrial input.
A chatbot consumes inference when someone is talking to it. A portfolio of agents consumes inference around the clock, every day, at a volume that grows with the business.
That is a different demand curve, and it is exactly the curve national infrastructure planners in this region saw coming. The sovereign compute build-out we covered in the first article, the gigawatt-scale capacity rising in Abu Dhabi and its counterparts across the Gulf, is the supply response to precisely this demand. It also carries the compliance dimension from our regulation article: where the loop physically runs determines where your data physically goes, which in this market is not a detail but a deployment criterion.
For an enterprise, the planning implication is simple to state. Treat agent capacity the way you treat any industrial input: budget it against output, secure it where your regulators require it to live, and design your systems to use it efficiently, because at continuous volumes, engineering quality shows up directly on the invoice.
The Mental Model to Keep
If you retain one paragraph from this article, keep this one. An AI agent is a loop that observes, reasons, and acts; its work is denominated in tokens; it touches your business only through doorways you construct; it knows only what it can retrieve; and it consumes computation continuously, like a machine on a production line rather than a consultant on a call. Every operating decision, cost, governance, integration priority, infrastructure location, follows from those five facts.
None of this diminishes what the technology is. It relocates the achievement to where it belongs: not magic in the model, but disciplined engineering around it. Which is, not coincidentally, where the durable competitive advantage in this market will sit.
At Red Rock Technology we design, deploy, and operate governed AI agents for enterprises from our base in Dubai Internet City, under agreements governed by DIFC law. We publisharticles like this one because clients who understand the machinery make better decisions, and better-informed clients are the market we want to build in.
Next in this series: From Free Zones to AI Zones: How Dubai Is Engineering Agentic Advantage.