Abstract: After a decade of evolution, Apache Flink has solidified its position as the de facto standard in stream processing, delivering sub-second latency for real-time data processing. However, emerging trends in cloud-native architectures, data lakes, and artificial intelligence have introduced new challenges and requirements. In response, Flink has undergone continuous innovation and architectural upgrades to provide more user-friendly, cost-effective, and scalable real-time solutions, further enhancing the integration of Data and AI. This session will detail Flink’s core functional and architectural evolution for the AI era, including its new state storage and computing separation architecture, continuous advancement of unified stream and batch processing capabilities, in-depth analysis of unified stream and batch Materialized Table, deep integration with Lakehouse for real-time data lakes, and explorations and practices in AI for real-time AI applications.

Key Takeaways:

1. Discover the latest advancements in Flink’s core architecture and its roadmap for cloud-native scalability14.
2. Learn how Flink empowers real-time data lakes through seamless integration with Lakehouse ecosystems.
3. Explore how Materialized Tables unify batch and streaming data warehouse development paradigms2.
4. Gain insights into Flink’s role in AI-driven applications, including real-time feature computation and model serving

Speakers:

Apache Flink Committer, Apache Flink 2.1 Release Manager. I’ve been focusing on big data, with extensive experience in streaming, batch computing, and storage.