Born in the Hadoop era, Apache Parquet is now over a decade old. As AI workloads demand ultra-wide tables, high-dimensional vector embeddings, and complex semi-structured data, new contenders like Lance, Vortex, and Nimble have emerged, positioning themselves as “Parquet killers.” Is the industry’s most ubiquitous columnar format finally reaching its twilight?

In this session, we will go beyond the hype to dissect the current state of Parquet and its ambitious roadmap for 2026. We will explore why Parquet’s ecosystem remains its “unbeatable moat”—spanning every major query engine to AI hubs like Hugging Face—and, more importantly, how it is undergoing a fundamental technical rebirth to meet modern challenges.

The presentation is structured into three core pillars:

• The Challenger Landscape: We will analyze the performance gaps that modern formats like Lance (random access) and Nimble (meta-data efficiency) aim to fill, acknowledging the limitations of “classic” Parquet in AI-centric workloads.
• The “Self-Evolution” of Parquet: We will dive deep into the latest architectural breakthroughs within the Apache Parquet community: (1) New Primitive Power: Exploring the Variant type for JSON-like flexibility with columnar performance, and Geometry types for specialized analytics. (2) Next-Gen Encodings: A look at ALP (Adaptive Lossless floating-point) and FSST (Fast Static Symbol Table) for extreme compression and string acceleration. (3) Architectural Modernization: Transitioning from Thrift to FlatBuffers to solve the metadata bottleneck in tables with tens of thousands of columns.
• Cross-Project Synergy: How Parquet is collaborating with Apache Iceberg to support native column-level updates, secondary indexing, and unstructured data storage.

Speakers:

Gang Wu: Software Engineer at Singdata

Gang is a software engineer at Singdata working on the lakehouse storage. He is a PMC member of Apache Arrow, Apache ORC, Apache Parquet, and a committer of Apache Iceberg. He is also an ASF member. Prior to Singdata, he has worked at Uber on Apache Spark and Alibaba on the MaxCompute cloud data warehouse, respectively.

Zehua Zou: Software Engineer at Singdata

I’m a contributor of Arrow, Parquet, Iceberg and so on.