← All courses

🪵 Apache Kafka

A log, not a queue — topics and partitions, consumer groups and offsets, ordering guarantees and what exactly-once really costs. Plus Connect and Kafka Streams, which are how data people actually meet it.

No lessons published yet — this is the roadmap, so you can see exactly what the course will cover before it lands.

Before you start: ETL & Data Pipelines — Series 1, and Series 3 · 42. ETL & Data Pipelines teaches what a load must guarantee and, at Series 3 · 42, what bounded and unbounded data actually mean. This track is the transport that carries the unbounded case, and it assumes you already know why that is a different problem.

Short on time? The 14 episodes marked core are the shortest path to being productive at work. Start there and pick up the rest whenever a job asks for them.

Series 1 · The Log

topics, partitions, offsets — and why nothing is deleted when you read it

  1. 01

    A Log, Not a Queue — Why Reading a Record Does Not Remove It

    coreintermediatecoming soon
  2. 02

    Partitions and Keys — Ordering Is Only Guaranteed Within a Partition

    coreintermediatecoming soon
  3. 03

    Offsets and Consumer Groups — Why a Fifth Consumer Sat Idle

    coreintermediatecoming soon
  4. 04

    Retention and Log Compaction — Turning a Topic Into a Table

    coreintermediatecoming soon
  5. 05

    Serialisation and Schema Evolution — The Field That Broke Every Consumer

    coreintermediatecoming soon
  6. 06

    Project — Design the Topics and Keys for a Real Event Stream

    coreintermediate#projectcoming soon

Series 2 · Guarantees

what you actually get, and what it costs

  1. 01

    acks and min.insync.replicas — The Pair That Decides Whether You Lose Data

    coreadvancedcoming soon
  2. 02

    Delivery Semantics — At-Least-Once, Exactly-Once, and the Edge of Kafka

    coreadvancedcoming soon
  3. 03

    Replication and ISR — What a Broker Failure Does to Your Data

    advancedcoming soon
  4. 04

    Consumer Lag and Poison Records — Why the Consumer Keeps Restarting

    coreadvancedcoming soon
  5. 05

    Project — A Consumer That Does Not Lose or Duplicate a Row

    coreadvanced#projectcoming soon

Series 3 · Kafka for Data Teams

Connect, Streams, and the database at the other end

  1. 01

    Kafka Connect and Debezium — Getting a Database In and Out Without Writing Code

    coreadvancedcoming soon
  2. 02

    Kafka Streams — KStream, KTable, and Processing Without Another Cluster

    advancedcoming soon
  3. 03

    Kafka to the Warehouse — The Shape Most Data Platforms Actually Have

    coreadvancedcoming soon
  4. 04

    What Kafka Is Bad At — and the Table With a Timestamp That Would Have Done

    coreadvancedcoming soon
  5. 05

    Project — CDC From a Database to an Analytical Store, Reconciled

    coreadvanced#projectcoming soon

Where this leads

Not here, on purpose

  • Event time, watermarks and windowing as concepts — covered in ETL & Data Pipelines, Series 3 · 42, and applied in Apache Flink
  • Running the cluster — broker sizing, KRaft or ZooKeeper operations, rebalancing, security — covered in Data Ops for the operational discipline. Here you learn what a PRODUCER AND CONSUMER AUTHOR must know
  • Choosing Kafka over a database or a queue — covered in ETL & Data Pipelines, Series 3 · 42, which makes the reader decide before reaching for any of this