Infrastructure for the Agent Era

Dozens of AI agents per person. Not per company. Per person.That’s the future that we built Dragonfly.

My co-founder Roman and I have argued for years that legacy in-memory architectures cannot scale to meet high-volume, modern workloads. Over the past year, that thesis has reached a critical tipping point. Driven by the explosive rise of AI agents, we are now starting to witness an infrastructure shift far larger than the commercial internet or the mobile revolution. Let me start with a striking observation from Cloudflare's recent announcement on their Agent Cloud:

"Existing infrastructure cannot scale to a world where every user and employee has dozens of personal agents running simultaneously."

Read that again: dozens of agents per person. Not per company - per person.

The math simply doesn't work anymore

Modern cloud infrastructure was designed to support a scale that can be represented in a simple equation: (Number of Humans) × (Hours Spent Online).

In the desktop era, that equation was capped by the few hours a day we sat at a PC. The mobile revolution unlocked a ~10x leap by allowing us to stay connected anywhere, but traffic was still bounded by human attention and physical waking hours.

The agentic era flips this equation on its head: (Dozens of Agents) × (Number of Humans) × (24 Hours a Day). ~100x leap. This requires a massive scale of infrastructure - which is why cloud titans are committing hundreds of billions to infrastructure expansion.

Data infrastructure is already broken

Hardware capacity is only half the equation. Data infrastructure was also engineered strictly around human-scale interactions and workloads.

In-memory data stores like Redis were designed to handle short-lived, human-centric state - things like active shopping carts, recent search filters, or user session tokens.

Redis was created 17 years ago as a beautifully simple, single-threaded system. It was revolutionary for 2009. But because Redis is single-threaded, it cannot scale vertically. It can only scale horizontally across complex clusters, usually capping out at a few hundred shards. This creates a hard ceiling on both memory capacity and total operations per second. Even in the human-scale era, engineering teams frequently struggle to manage and scale Redis under peak traffic.

Now, the agentic shift is demanding an entirely new baseline requirement: 100x the scale.

Why we built Dragonfly and why it matters now

When Roman and I founded Dragonfly four years ago, we were driven by a fundamental engineering principle to utilize modern hardware to its full capacity. We saw a growing disconnect between modern multi-core CPUs and legacy in-memory data stores, watching teams strain to manage complex clusters just to handle a few terabytes or a few million requests per second.

We engineered Dragonfly from the ground up to push modern multi-core cloud hardware to its physical limits. Dragonfly utilizes all available CPU cores on a machine, enabling a single node to sustain millions of requests per second. It also transparently extends RAM onto SSDs to hold terabytes of data while maintaining sub-millisecond performance.

Four years ago, that architectural choice was about extreme efficiency and simplifying cluster operations. Today, in the agentic era, those exact design principles have become an essential requirement to sustain the coming 100x growth.

The Inflection Point

Smartphones took years to reach global adoption. In that human-driven era, infrastructure teams had time to adapt and scale alongside demand. The Agent Era will grant no such ramp-up time. It is arriving faster and at a much larger scale.

AWS, Azure, and Google are committing hundreds of billions to scale infrastructure to support mass agent deployments. The real-time data layer remains overlooked. High-throughput, real-time data flow is the single critical bottleneck every business must address. That is the gap.

We didn't build Dragonfly to be an incremental caching speedup. We built it to scale.

Is your data layer ready for the agent boom?