With the rise of AI and vector-based workloads, Lance is gaining traction as a high-performance data format for analytical and ML use cases. However, managing Lance datasets across systems remains challenging due to the lack of a unified metadata layer.

In this talk, we introduce how Apache Gravitino enables centralized metadata management for Lance through its Lance REST service integration. We will walk through how Lance datasets can be modeled using a unified catalog abstraction, making them easier to manage, discover, and access across different engines.

We will also share the design considerations behind this integration, demonstrate how to create and access Lance datasets via Gravitino, and discuss how this approach helps build a more consistent and extensible data platform for both traditional analytics and emerging AI workload.

Speakers:

Qi Yu: Data infrastructure engineer, Apache Gravitino PMC

Engineer at Datastato and PMC member of Apache Gravitino, working on big data infrastructure, metadata management, and database systems, with a focus on scalable data platform development.