As Didi’s unstructured data grew rapidly, HDFS metadata pressure became a major scalability challenge. Apache Ozone was introduced as the next-generation storage engine to address these limitations, and it has now been running in production at Didi for more than two years, supporting over 100 PB of data and tens of billions of files.

This session gives an introductory overview of Didi’s Ozone journey, including why Ozone was chosen, how the platform evolved in production, and what engineering work was required to improve scalability, performance, and reliability. We will share practical experience in multi-cluster routing, follower-read optimization for S3 workloads, read performance tuning, erasure coding adoption, large-scale deletion and migration, observability, and availability improvements. The session will also summarize lessons learned from operating Ozone at scale and contributing improvements back to the Apache Ozone community.

Speakers:

Shilun Fan: Storage Engineer at DiDi, Apache Ozone Contributor

Shilun Fan is an Apache Hadoop PMC Member, an Apache Auron Committer, and an Apache Ozone Contributor. He works on large-scale distributed storage systems and data infrastructure at DiDi. He has been deeply involved in the production deployment and optimization of Apache Ozone, with a focus on scalability, performance tuning, erasure coding, and reliability. His work focuses on building scalable, reliable, and cost-efficient storage platforms for massive data workloads.

Ming Wei: Storage Engineer at DiDi

Apache Ozone Contributor

Hongbing Wang: Storage Engineer at DiDi

Apache Ozone Committer