The explosive growth of data driven by AI has significantly increased Kafka workloads and operational costs. This session focuses on how to optimize Kafka costs, covering the following key aspects:

1. The composition of Kafka costs and their relationship to system architecture
2. How to kick off Kafka cost optimization, e.g., the motivation behind KIP-1227 (exposing RackID)
3. Recent cost-related KIPs and PRs (e.g., KAFKA-19909, KIP-1123, KIP-1241, KIP-1150) and how they help reduce costs
4. Beyond Kafka itself, additional approaches to lowering costs (e.g., non-technical efforts, modifications to KIP-405 tiered storage plugins)
5. Common pitfalls and challenges encountered in real-world cost optimization
6. The role and features of AI in cost optimization

Target Audience Engineers, architects, and technical leaders who are concerned with Kafka cost management and optimization practices or those who want to learn Kafka architecture from a cost perspective

Expected Takeaways Gain a clear understanding of Kafka cost structure, optimization strategies, and common pitfalls, along with the latest technical advancements and AI-driven practices. These methodologies can also be applied to cost optimization in other software systems.

Speakers:

Jian Fu: Zoom-Engineering Manager

Engineering Manager at Zoom, formerly at Cisco, leading the development and operations of Zoom’s messaging queue systems, with 15+ years of experience building high-performance, highly available, and elastic systems.

Research: 1 Patent in USA , 10+ Published paper

Certification: Network Engineer Information Systems Project Manager

Contributor of open sources: Netty Apache kafka Influxdb/Influxdb-java Redis/Jedis Spring/Spring-data-redis Jenkins Apache zookeeper

Author of the books: authored: Netty原理剖析与应用(Netty author Trustin Lee recommended） co-authored: (微服务之道：度量驱动开发) (完美测试：软件测试系列最佳实践)

Author of the Kafka improvement proposals: Four Kafka improvement proposals (KIP) Accept and implemented

Author of the online courses: 极客时间: (Spring 编程常见错误50例) (Netty 源码剖析与实战)