The Next CPU Shortage: Why Your AI Agent Needs a Plan, Not a Faster GPU

For the past three years, the same sentence keeps showing up: there is not enough hardware. Not enough GPUs, not enough fast storage, not enough memory. Prices rise, delivery times stretch, and people who write software for a living suddenly need to know what VRAM and NVMe mean.

Now the sequel is arriving, and it is stranger than the original. The next wave of the chip shortage is not about the flashy chip that does the “AI thinking.” It is about the most ordinary chip in the box: the CPU, the general-purpose processor that runs everything else.

This is not a prediction about some distant future. The CPU shortage has already started. Intel has said its server processors are effectively sold out and expects the squeeze to peak early this year. AWS reportedly asked its engineers to conserve CPU cycles. AMD doubled its server CPU forecast. The useful question is no longer “if,” but “how big” — and what anyone building with AI should do about it before the wave gets taller.

Why Your AI Agent Spends Its Day on the CPU

Start with what an agent actually does. When you ask a chatbot a question, one model produces an answer and the GPU does the heavy lifting. When you ask an AI agent to do something — plan a trip, fix a bug, research a purchase — the job looks nothing like that.

The model reads the goal and breaks it into steps. It calls a search tool. It runs code. It checks a database. It asks a smaller sub-agent (a helper program with its own task) to go look something up. It checks the result, notices something is missing, and loops back. This dance of steps has a name: orchestration. Almost all of it runs on the CPU.

AMD, which has every reason to emphasize this point, counts seven of the eight stages of a typical agent pipeline as CPU work. Strip away the sales pitch and the basic fact stands: the GPU is involved in exactly one part of the process, the moment the model thinks. Everything around that moment is ordinary computing — API calls, file handling, permission checks, parsing results. Those moving parts are also what quietly ages a self-hosted box; Built and Abandoned Systems measures what happens when nobody tends them.

And one request rarely stays one request. A single prompt can fan out into dozens of sub-agents and hundreds of tool calls (requests the agent makes to outside software). An OpenAI agent that got loose on Hugging Face last year was clocked at up to 300 actions per hour. Multiply that by every user, every day, and the boring chip has a lot of work to do.

It’s Not the Model That’s Slow

Here is the finding that should change how you plan. When researchers at Georgia Tech and Intel profiled agent workloads from start to finish, they found that in tool-heavy tasks, processing the tools on the CPU took up to 88% of the total time. In plain words: the “AI” part was often sitting idle, waiting for the rest of the pipeline.

The same study found something even more useful. A machine with a strong CPU and a modest GPU matched the end-to-end speed of a machine with a top-tier GPU and a weaker CPU. The expensive part was not always the important part.

Now multiply that pattern across an industry. These are the numbers keeping infrastructure planners awake:

What is changing	A short while ago	Where it is heading
CPUs per GPU in AI data centers	1 CPU for every 4–8 GPUs	toward 1:1, sometimes more CPU-heavy
CPU cores per gigawatt of AI capacity	about 30 million	about 120 million (a fourfold jump)
Compute spent on inference	roughly a third in 2023	roughly two thirds in 2026

Read vendor numbers with a raised eyebrow: AMD and Intel sell CPUs, and every forecast they publish is also a sales pitch. But the direction agrees across independent sources, including academic work with no stake in the outcome.

Why Shortages Arrive in Waves

The first wave was GPUs. The second is memory: DRAM spot prices rose roughly tenfold in a year, while GPU prices actually eased as new supply arrived. Now CPUs. Each wave has the same shape. A component everyone treated as a solved problem suddenly becomes the constraint, because the workload changed shape faster than factories could.

That first easing did not hold. The fixed-price memory contracts expired at the end of 2025, and by 2026 the same suppliers were pricing cards against the spot market. I wrote that half of the wave separately: RTX 5080 24GB and the 16GB Tier: A 2027 Price Forecast.

You cannot build a chip factory in a quarter. Supply answers slowly, in years, while demand changes in months. So the winning move is not to out-wait the market. It is to understand your own demand well enough that the market’s mood matters less.

Know Your Hardware, Size Your Work

There is a mistake most of us have seen in an office. A company hires someone new and, on day one, hands them a top-of-the-line machine — for a job that is mostly email and spreadsheets. Then the developer on the same floor fights a ten-year-old laptop, waits minutes for every build, and is expected to deliver like everyone else. In both cases the problem is not the person. It is the gap between what they were given and what the job actually needs.

AI agents get exactly the same treatment. A team with a generous budget buys top-tier GPUs and then runs simple, repetitive agent loops on them, paying premium prices for work that mostly needs ordinary CPU power. Another team does the opposite: a few small machines, hundreds of agents, and a queue that grows all day while everyone blames the model for being slow. Neither problem is the AI’s fault. Both are sizing problems.

The whole takeaway fits in one sentence: know the speed of your hardware, know its capacity, and design the work you give it accordingly.

Know the speed. Before optimizing anything, measure where the time actually goes. Most teams guess wrong. They assume the model is the slow part when the hours are spent in tool calls, data retrieval, and parsing. Profile one real task from start to finish. Surprises are much cheaper before you have bought hardware.

Know the capacity. Capacity is not only GPU-hours. For agents it is CPU cores, memory, and data throughput. One agent is quiet. Ten thousand concurrent agents are a server farm making constant small demands. Budget in concurrent tasks and cores, not just in model tokens.

Design the work to fit. Once you know the speed and the capacity, the work itself becomes adjustable. Use a smaller, faster model for the simple steps and save the big model for the parts that need it. Batch tool calls instead of making them one at a time. Cache results that get looked up again and again. Trim the context an agent carries instead of sending the whole history on every step. Queue work instead of demanding instant answers everywhere. None of this requires new hardware. All of it multiplies what the hardware you already own can do.

What to Do This Week

Pick one real agent task your team runs and profile it. Write down where the seconds go: model, tools, retrieval, waiting. Then ask three questions. Where is the time actually going? What is the ceiling on concurrent tasks your cores and memory can carry? And which steps genuinely need the expensive model?

The chip market will do what the chip market does. Shortages will keep moving from one component to another, and the headlines will keep arriving. A team that knows its own numbers reads those headlines differently: not as threats, but as information.

Frequently Asked Questions

Will CPUs replace GPUs? No. GPUs remain where the model runs. What is changing is the balance around the model: the CPU-side work of agentic systems is growing faster than the GPU side.

Is the CPU shortage real, or just marketing? Both signals exist. Intel and AMD sell CPUs and have an interest in the story. But capacity reports, price increases, and independent academic profiling of agent workloads point the same way.

How do I know if my agent is CPU-bound? Profile a task from start to finish. If the GPU sits idle while tools run, or if adding GPU capacity does not improve completion time, the bottleneck is elsewhere — usually in CPU-side tool execution and orchestration.

Will this get better? Supply will grow and prices will settle eventually, as they did for GPUs. But workloads keep changing shape. Knowing your own hardware and your own work is a permanent advantage, not a temporary fix.

---

Photos: “Sun Rack Mount Servers” by Raysonho (public domain) and “Hitachi HD61700 microprocessor die shot” by Piotr433 (CC0), via Wikimedia Commons. Diagrams are original.