Iceberg v3 introduces performance enhancements through the Puffin specification and Deletion Vectors. This session details the engineering effort to implement a high-performance C++ Puffin reader/writer within Apache Impala. We will walk through the migration path from v2 to v3, highlighting the technical challenges of bringing these features to a C++ engine. By implementing the Puffin reader/writer, Impala also gains the ability to read and write Apache Datasketches statistics. The core of the discussion focuses on the architectural shift from position deletes to v3 deletion vectors. We will compare these two approaches in-depth, sharing implementation details and the performance gains observed during the implementation process. We will present real-world benchmarks and discuss plans for additional v3 features for Impala. This includes how Puffin-based deletion vectors improve scan performance and how Impala can profit from engine-agnostic statistics. Attendees will leave with a clear understanding of how to leverage Iceberg v3 for high-throughput, write-heavy workloads.

Speakers:

Péter Rózsa: Cloudera, Software Engineer

Péter Rózsa, a Software Engineer at Cloudera since 2020, started contributing to Apache Impala in 2022. He’s been working on different parts of the Impala project, recently on Iceberg integrations

Zoltán Borók-Nagy: Cloudera, Principal Engineer

Zoltán is a software engineer at Cloudera, working on Apache Impala. He is also a PMC member on the project. Currently he is leading the Impala/Iceberg integration efforts. Before Cloudera, Zoltán worked on C++ static analysis tools. He is interested in distributed, massively parallel systems, databases, and performance engineering.