How Dailymotion consolidated Redis Sentinel, Memcached, and Aerospike onto a single engine across seven regions, cutting node counts 72% to 75% per workload and saving about $38K a month.

At a glance

Dailymotion is a French video platform founded in 2005 that distributes news, entertainment, and culture from creators and publishers worldwide. It reaches about 400 million monthly users and 4 billion video views a month across screens. Their platform runs entirely on Kubernetes, spread across seven regions on AWS and GCP.

Moving three engines to Dragonfly

Dailymotion runs about 600 microservices, and roughly 300 of them need a key-value store. In 2024 those services were spread across three engines, each with its own operational model, monitoring, scaling procedure, and bill.

Aerospike powered the video catalog and the recommender workload. Redis Sentinel handled catalogs, cache, queues, and the ML data stores. Memcached served legacy applications, including the GraphQL API, through McRouter. Redis also had historical usage in the Prebid ad-tech stack.

Running all of it fell to the platform team. Developers could not provision or manage their own stores, so every new use case queued behind SRE bandwidth.

Each engine brought its own problems. Redis and Redis Sentinel had no official open source Kubernetes operator, and RDB backups were complicated, which left the team without a consistent recovery story. McRouter needed its own build pipelines and was hard to upgrade. Aerospike was oversized for a dataset of about 100 GB, and it lacked the async Python client the developers wanted.

The recommendation store made the cost of that sprawl even more apparent. The team moved it off Aerospike onto Redis Sentinel, planned for 6 replicas, and had to scale to 9 during traffic events (and later up to 11). And it was unstable. The setup hit load problems during the Ballon d'Or, the annual France Football award for the best player of the year, which draws one of the largest traffic spikes Dailymotion sees all year.

Discovering Dragonfly, one engine for both protocols

Dragonfly speaks both the Redis and the Memcached wire protocols. That meant a single engine could cover the catalog, the recommendation API, and the API cache, and the three-engine split could collapse into one. It also shipped a Kubernetes operator, which mattered for an environment that deploys only on Kubernetes.

The architecture fit the hardware model as well. Dragonfly uses every core on a node, so throughput scales with CPUs instead of stopping at one core. That let the team size instances by compute and use them fully, rather than spreading a workload over many small nodes just to get enough network capacity. Native snapshotting to S3 and GCS gave the team the unified backup story it never had with Redis.

Evaluation and migration

The migration ran in three phases. In 2024 the team moved the recommendation store and all new use cases. In 2025 it migrated the rest of the existing stack. In 2026 it moved moderation, the API, and what remained. The goals were set up front: more traffic at the same or better performance, lower cost, better operability, and one backup and recovery system.

Because Dragonfly speaks both protocols, most migrations were a hostname-and-secret swap. The team ran old and new configurations side by side and cut over in production without a maintenance window.

The recommendation store was the hardest one, and it ended up the cleanest result. It went from 3 Aerospike nodes, to 11 Redis Sentinel nodes, to 3 Dragonfly nodes. The only application change was swapping the Redis Sentinel client for a plain Redis client. That setup has not changed since 2024 and now serves a larger video catalog than it did then.

Results: fewer nodes, lower latency, backups everywhere

"We were surprised to see 30% in-memory dataset compression when we migrated to Dragonfly, and the latency after each migration went two to five times lower than Redis."
Cyril Corbon, Principal DevOps Architect, Dailymotion

The recommendation stack dropped from 11 Redis nodes to 3 Dragonfly nodes, a 72.7% reduction, taking monthly spend from $8K to about $2K. The GraphQL cache dropped from 8 Memcached nodes to 2 Dragonfly nodes, a 75% reduction. Across the migration, Dailymotion saves about $36K per month in 2026. The original target was up to 70% cost reduction, and the team beat it.

Performance also moved in the positive direction. Latency after each migration was 2 to 5 times lower than on Redis, and the in-memory dataset shrank 30%. For the recommendation use case, Dragonfly outperformed Aerospike on the same workload that had needed three times the nodes on Redis Sentinel.

The catalog store backs every video shown on the Dailymotion app and website, serving between 150K and 500K commands per second depending on load. Across the fleet, Dailymotion runs roughly half a million operations per second at about 90% cache hit rate, with average command latency in the tens of microseconds.

Operationally, the change was just as significant. Every store now has native backups to GCS, S3, or local disk, which was not true before. Upgrades roll through the operator, which replaces replicas first and then hands over from the master. Moving a store between Kubernetes clusters is now a restore from backup: start the new instance with the right IAM permissions and it comes up with data. Developers deploy their own Dragonfly instances through Flux and GitOps, and the repo ships skills so they can bootstrap and configure Dragonfly with their own AI agents. The platform team is no longer the bottleneck.

Dailymotion now runs about 40 Dragonfly instances, each dedicated to one data type (catalog, cache, feature store, metadata, pacing) to keep pressure isolated. Datasets range from about 2 GB to 200 GB, and the team caps each instance around 40 GB before asking developers to split to a new one. The fleet runs on ARM, Graviton on AWS and Axion on GCP.

No Memcached or standalone Redis remains on the platform. Another 13 Redis Sentinel clusters are queued for migration in 2026, worth an estimated $5K per month in further savings. One Aerospike cluster stays for the ad-tech stack, where it fits the use case.

"In 2026 we saved around $36K per month with the Dragonfly migration. It was really worth it."
Cyril Corbon, Principal DevOps Architect, Dailymotion

Upgrading to Dragonfly 2.0

In October, Dailymotion upgraded its whole Dragonfly fleet from v1.40 to v2.0.2 in a one-line, zero-incident change. It cut latency on their reco cache in half, the cache behind Dailymotion’s video recommendations, and made commands faster on seven more busy instances, shrank small instances' memory by about 20%, let the team drop a long-standing S3 workaround, and every backup kept succeeding. The upgrade to Dragonfly 2.0 saves them about $1,700 a month (about $20K a year). This comes on top of the original $36K a month from moving to Dragonfly.

Scope

Dragonfly is now Dailymotion's Redis- and Memcached-compatible in-memory store. In prod the team runs about 40 instances through the dragonfly-operator: on EKS in four regions (eu-west-1, us-east-2, us-west-2, ap-south-1) and on GKE (data-platform, data-backbone, ci, adtech, jx3). They range from 20 MB rate-limit stores to 70 GB recommendation caches serving close to 100K commands per second.

All prod instances moved to v2.0.2 on 2026-10-12 with a single change to the shared base manifest. To avoid noise from pod restarts, the team compared daily Datadog metrics and two matched windows: Saturday to Monday morning before (Oct 10-12) and after (Oct 17-19). Latency is measured inside Dragonfly, per command.

A one-line rollout

Because every instance inherits the same base Dragonfly manifest, the major version bump was one image tag change for the whole fleet.

The upgrade also let the team remove a workaround. Since v1.39 the team had forced the legacy S3 client (--s3_use_helio_client=false) because of an IRSA credential refresh bug. That bug is fixed upstream, so v2 runs on the default native S3 client again.

The team added one tuning flag. v2 keeps only 5 s of replication backlog by default. On Dailymotion's three largest instances (55-100 GB), the team keeps 60 s (--shard_repl_backlog_time_ms=60000). A briefly disconnected replica then catches up with a partial sync instead of copying the full dataset again.

reco: latency cut in half

reco, the cache behind Dailymotion’s video recommendations, is Dailymotion's second-busiest instance: about 95K commands per second on 60 GB of data. After the upgrade, its average command latency fell from 14.7 μs to 8.1 μs, while it served 3% more traffic on the same CPU.

Datadog · dragonfly_commands_duration_seconds / dragonfly_commands, cmd get and mget · prod-gke-data-platform-eu-west4, dragonfly-reco · daily averages

The drop came in the two days after the rollout. MGET has held flat since; GET is drifting back up slowly (1.8 → 2.4 μs) and is worth watching. MGET accounts for about half of reco's total command time, and its latency fell from about 97 μs to 48 μs.

A smaller footprint for small instances

Most of Dailymotion's instances are small: 52 of them hold less than 2 GB of data. On v2 they need about 20% less memory: the master pods' total went from 14.9 GB to 11.9 GB. 18 of the 52 dropped by more than a fifth, and none grew.

Datadog · max of dragonfly_used_memory_rss_bytes per instance, daily median · prod, Oct 9-11 vs Oct 13-15, 2026 · the 12 largest drops of 52 small instances

The drop happened the day of the upgrade and survived later pod restarts, so it is not a fresh-process effect. The data stored did not change; the saving is in memory the process kept for itself. The team's memory requests have not changed yet, so this is headroom the team can now reclaim when right-sizing.

Faster commands beyond reco

reco is not alone. Comparing restart-free days, seven other busy instances on EKS and GKE answer faster on v2, from 7% on ads-ranking to 73% on context-segment in ap-south-1.

Datadog · dragonfly_commands_duration_seconds / dragonfly_commands, daily median · prod, Oct 9-11 vs Oct 13-15, 2026 · traffic = commands per second before the upgrade

The gains are largest where traffic is real: geo-cache in eu-west-1 (18K commands per second) is 36% faster, and ads-ranking improved in three of its four regions. The team left out instances below about 1,000 commands per second, whose averages move a lot from day to day. Not every instance improved: core-api in eu-west-1 went from 25 to 30 μs, still far below what its clients can notice.

Steady everywhere else

For a major version, the rest of the fleet held steady, which is exactly what the team wanted. catalog and memcached-westeros, Dailymotion's other two busiest instances, stayed within normal day-to-day variation: catalog average latency went from 5.4 to 6.2 μs, westeros from 5.0 to 4.9 μs. Across prod, commands per CPU core moved from 33.9K to 34.9K (+3%), too small to credit to v2 alone. Large instances keep the same memory footprint.

Snapshots: just as reliable, one workaround fewer

Backups stayed fully reliable through the upgrade: about 1,218 snapshots a day to S3, GCS or disk, and zero failures from Oct 6 to Oct 20, now on the native S3 client.

Conclusion

Dragonfly 2.0 was a low-risk upgrade: one line of config, no incidents, no failed backups. It halved latency on reco, sped up commands on seven more busy instances, cut the memory of small instances by about 20%, and removed a workaround the team had carried since v1.39. Above all, it showed that a major version of Dailymotion's main in-memory store can now be a routine change.

Advice for platform engineering leaders

Reflecting on the project, Cyril Corbon offered guidance for teams running Dragonfly on Kubernetes.

• Enable Dragonfly's exposed metrics and wire them into your monitoring before you scale up.
• Remember that maxmemory counts the dataset only. Leave headroom for pipelines and connection overhead or you will hit problems.
• Give developers repo access and skills so they can deploy alongside the platform team instead of queuing behind it.
• Back up to a persistent volume claim for the fastest recovery. When you snapshot to GCS or S3, right-size the instance, because the network load is heavy.
• Split data types across dedicated instances rather than loading one store with mixed traffic.