SSD Data Tiering Is Generally Available

Five months ago, we shipped the initial release of Dragonfly SSD Data Tiering. Since then, dozens of teams have deployed it in production. Today we're calling it generally available.

This isn't a milestone we're marking for formality. GA means the architecture is stable, the edge cases are handled, and we're confident putting it in front of workloads that can't afford surprises. The production deployments bear that out.

SSD Data Tiering is now GA in the Dragonfly Community Edition -- get started by visiting the docs. It is also available in beta for Dragonfly Cloud; if you want early access, get in touch here.

What It Does

SSD Data Tiering extends Dragonfly's in-memory store with local NVMe SSDs. Hot data stays in RAM with the same sub-millisecond latency you'd expect. Warm and cold data gets offloaded to disk automatically, with keys and metadata always kept in memory so operations like EXISTS, TTL, and key-space scans never touch the SSD.

The result: you can operate datasets several times larger than your available DRAM on a single Dragonfly instance, without resharding, without a cluster topology change, and without a line of application code changing.

The Only Source-Available Redis-Compatible Store With SSD Tiering

Redis and Valkey do not have SSD tiering. When your dataset outgrows available DRAM, they leave you with three options: pay for more RAM, reshard across more nodes, or accept eviction. None of them are good answers. Dragonfly is the only source-available, Redis-compatible in-memory data store that lets you extend capacity to local NVMe SSDs -- transparently, without API changes, and without touching your application.

That alone is a meaningful difference. But the architecture behind how Dragonfly does it matters just as much as the fact that it does.

Why Dragonfly’s Implementation is Different

The core challenge with SSD tiering is that disk I/O is orders of magnitude slower than RAM. A naive implementation serializes that latency directly into your request path. Four architectural decisions separate Dragonfly's tiering from what a single-threaded store could ever do:

Single-threaded stores block on disk I/O

Redis and Valkey process commands on a single thread. If Dragonfly were built the same way, an SSD fetch would stall the entire request queue until the read completed. Dragonfly's fiber-based execution model suspends the waiting request, handles other work, and resumes when the disk read is done. Disk latency becomes a background event, not a wall.

io_uring for non-blocking I/O 

Dragonfly uses the Linux io_uring interface (kernel 5.19+) for all disk operations. This provides a non-blocking interface for I/O submission and completion, allowing Dragonfly to batch many disk operations concurrently and fully saturate NVMe bandwidth. Conventional blocking syscalls can't do this.

Page cache bypass via O_DIRECT

 Dragonfly manages its own caching layer. Letting the Linux page cache also cache tiered data would mean competing for the same RAM and paying the CPU cost of redundant kernel-level copies. Dragonfly uses O_DIRECT to skip the page cache entirely, keeping full control over what lives in memory.

A three-state data model, not just LRU

 Dragonfly tracks data in three states: hot (RAM only), cold (SSD only, pointer in RAM), and cooled (written to SSD, RAM copy temporarily retained). The cooled state handles a real problem: when a value is written and immediately read -- common in service-to-service patterns -- Dragonfly serves it from the in-memory copy while the disk copy is already verified safe. The RAM copy is dropped as soon as memory pressure requires it. This prevents write spikes from causing OOM conditions and avoids unnecessary SSD reads for recently written values.

Keys always stay in RAM. In tiered mode, only values are offloaded to the SSD. Keys, TTLs, and metadata always remain in Dragonfly's DASH table at 20-30 bytes per entry. This means operations like EXISTS, TTL, and key-space scans are always sub-millisecond regardless of where the value lives.

Meesho: 300GB Working Set, 70% Cost Reduction

Meesho, one of India's largest e-commerce platforms, serves 175 million annual transacting users and runs some of their most critical caching infrastructure on Dragonfly. One of their P0 workloads -- a memory-heavy cache with a 300GB working set running at ~30K ops/sec -- was the proving ground for SSD tiering.

Before tiering, running that workload as a pure in-memory store meant paying for a large memory footprint while leaving CPU underutilized. They had previously adopted Redis on Flash (only available with an enterprise license) for the SSD capability. When Dragonfly's tiering support became available, they tested the same P0 workload on it and found 20-25% lower latencies at equivalent load, alongside roughly 70% cost savings for the same cache footprint. They've been running that workload on Dragonfly's tiering-based cache for two months with consistent performance throughout.

The Meesho workload illustrates something important: SSD tiering isn't only for datasets that can't fit in RAM. It's for workloads where most data is warm or cold — where paying DRAM prices for infrequently accessed keys is simply waste. A 300GB working set at 30K ops/sec fits comfortably on NVMe for the inactive portion, with RAM reserved for what's actually hot.

"Setup was genuinely straightforward -- no complex flags, no fiddly configuration. Dragonfly delivered on both performance and economics without the operational tax we expected."

-- Yash Shah, SDE 4 @ Meesho

Current Capabilities and Constraints

Tiering is optimized for String values larger than 64 bytes. Values smaller than 64 bytes stay in RAM -- writing small values to 4KB SSD blocks causes write amplification that isn't worth the disk management overhead at that size.

A few other constraints to be aware of:

- Keys and metadata always remain in DRAM. Only values are offloaded.

- HyperLogLog and BITOP operations require data to be in memory.

- Other data types (hashes, lists, sets, sorted sets) remain RAM-only for now. Tiering support for elements of hashes, lists, and streams is on the roadmap.

- Durability is the same as standard Dragonfly -- tiering is a capacity extension, not a persistence layer. Use snapshots or HA replicas for durability.

To run Dragonfly with tiering:

 ./dragonfly --maxmemory=20G \
              --tiered_prefix=/mnt/fast-ssd/dragonfly-tiered-file \
              --tiered_offload_threshold=0.2

 This starts Dragonfly with a 20GB RAM limit and begins offloading aggressively when memory utilization crosses 80%. The tiering engine manages the rest.

Full configuration options and monitoring guidance: https://www.dragonflydb.io/docs/managing-dragonfly/tiering

The Architecture Behind It

Three additional design decisions keep the implementation efficient at scale.

DASH table for memory-efficient indexing. Dragonfly's segmented hash table uses constant-size segments rather than large contiguous allocations. In tiered mode, keys, TTLs, and pointers stay in the DASH table at 20-30 bytes per entry. The memory index stays small and predictable regardless of how much data is on disk.

Small value compaction. SSDs operate on 4KB blocks. Grouping small values into 4KB pages before writing prevents write amplification. A background compaction process identifies fragmented pages over time and repacks valid data into dense pages, keeping the write and space amplification factor below 2x.

Write-back pressure. When the SSD offload queue reaches a critical depth, Dragonfly throttles incoming writes rather than allowing memory to overflow. This prevents OOM errors during write spikes without requiring manual tuning.

Get Started

SSD Data Tiering is GA today in Dragonfly Community Edition -- visit the docs to get started.

SSD tiering is in beta for Dragonfly Cloud on AWS and GCP. If you want early access, get in touch.

If you're running a workload where dataset size has become the binding constraint -- feature stores, session stores, recommendation caches, large leaderboards -- this is the path to scaling vertically without a cluster overhaul.

Want to go deeper? The engineers who built SSD tiering are hosting a live technical walkthrough where they'll cover the architecture in detail and take questions. Register for the webinar here.

Try it and let us know what you find. The community is on Discord and GitHub.