How Celeborn Makes Shuffle in AlibabaCloud Serverless Spark Comfortable, Reassuring, and Worry-Free

How AlibabCloud EMR Serverless Spark integrates Apache Celeborn to overcome traditional Spark Shuffle limitations (coupled storage/compute, connection storms, random I/O, and single-point failures), delivering a robust production experience.

Speakers:

Zhentao Shuai: Alibaba Cloud

Software Development Engineer at Alibaba Cloud Celeborn committer

Jiashu Xiong: Alibaba Cloud, Senior Development Engineer

Apache Celeborn PMC member, mainly focused on the optimization of Apache Celeborn and the integration of Apache Celeborn with engines such as Flink and Spark