Choose Your Own Topology, A Kafka Streams Adventure with Anna McDonald | Saint Louis Apache Kafka® Meetup by Confluent
Designing the perfect Kafka Streams topology for your events can be a challenge -- How should you configure your input Kafka topics? Is your target topology stateless or does it have to be stateful? What keys should you use for partitioning data and how will that selection impact join, groupBy and other operations? How should you manage state store size in RocksDB? What processing semantics do you need? Should you use session or windowed operations? Those are just some of the questions that need to be answered when designing a new Kafka Streams topology. In this presentation I will discuss the information needed to make major design decisions in Kafka Streams along with the associated pitfalls. The resulting decision map will serve as a basic guide and help lead the way on your journey to find the perfect topology!