Walmart Apache Druid

Walmart — 1B Events/Day, Batch to Real-Time

Walmart migrated 1 billion events/day (2TB raw data) from batch analytics to Apache Druid, dramatically reducing query latencies.

Architecture diagram: Walmart — 1B Events/Day, Batch to Real-Time

Scale

~1 billion events/day (2TB raw data)

Before

Conventional batch analytics systems — high query latencies, stale data

After

Apache Druid with dramatically reduced query latencies and near-real-time analytics

Key Insight

Classic 'we outgrew batch' migration. Use this to explain why streaming OLAP engines exist as a category.

In a Snowflake Conversation

Classic 'we outgrew batch' migration — use this to explain why streaming OLAP engines exist as a category. Every large retailer eventually hits this wall.

My Read

Practitioner commentary coming soon.

Apache Druid batch to real-time 1B events/day analytics migration

Relevant Conversations

Streaming OLAP Kafka & Flink