In distributed data processing engines, responsibilities such as coordination, communication, and state management are often tightly coupled within a single framework. This can make the system difficult to evolve, extend, or replace individual components over time.

In this talk, I will share my experience working on the SeaTunnel engine, where we explored introducing clearer abstractions to reduce dependency on a single distributed framework. I will discuss why this kind of decoupling is necessary, the approach we considered, and the challenges we encountered along the way.

Rather than focusing on a specific technology choice, this talk will highlight the design questions and trade-offs involved in separating core responsibilities in a distributed engine.

Attendees will gain practical insights into how to think about abstraction boundaries and how to approach architectural evolution in real-world distributed systems.

Speakers:

Doyeon Kim: Apache SeaTunnel Committer

Doyeon Kim is an Apache SeaTunnel Committer and a student with a strong interest in data engineering and distributed systems. She has contributed to SeaTunnel in areas such as connector development, engine improvements, and architectural discussions. Her recent work has focused on engine internals, dependency decoupling, and design trade-offs in distributed systems. Through this talk, she shares lessons learned from exploring clearer abstractions in the SeaTunnel engine.