← All courses

🐿️ Apache Flink

Stream processing where the time that matters is when the event happened, not when it arrived — event time and watermarks, windows, keyed state, and checkpointing.

No lessons published yet — this is the roadmap, so you can see exactly what the course will cover before it lands.

Before you start: ETL & Data Pipelines — Series 3 · 42. That episode defines bounded against unbounded data, event time against processing time, and what a watermark is for. This track is the implementation of all three, and it does not re-teach them.

Before you start: Apache Kafka — Series 1. Almost every Flink job reads from and writes to Kafka. You need topics, partitions and offsets before the source and sink connectors make sense.

Short on time? The 12 episodes marked core are the shortest path to being productive at work. Start there and pick up the rest whenever a job asks for them.

Series 1 · Time and State

watermarks, windows, and what the job remembers

  1. 01

    A Job That Never Finishes — Sources, Operators, Sinks and Parallelism

    coreadvancedcoming soon
  2. 02

    Watermarks — Deciding an Answer Is Final, and the Source That Never Advances

    coreadvancedcoming soon
  3. 03

    Windows — Tumbling, Sliding and Session, in Event Time and Processing Time

    coreadvancedcoming soon
  4. 04

    Keyed State — What the Job Remembers, and the Key Space That Grows Forever

    coreadvancedcoming soon
  5. 05

    Checkpoints and Savepoints — Restarting Without Losing What the Job Knew

    coreadvancedcoming soon

Series 2 · Writing Jobs

Flink SQL first, then the DataStream API

  1. 01

    Flink SQL — A Stream as a Changelog of a Table, With WATERMARK in the DDL

    coreadvancedcoming soon
  2. 02

    Streaming Joins — Interval and Temporal, and the Regular Join That Never Forgets

    coreadvancedcoming soon
  3. 03

    The DataStream API — ProcessFunction and Timers, for What SQL Cannot Say

    advancedcoming soon
  4. 04

    Connectors — Kafka, JDBC and CDC, and the Guarantee Each One Actually Gives

    coreadvancedcoming soon
  5. 05

    Backpressure — Reading the UI to Find Which Operator Is Actually Slow

    advancedcoming soon
  6. 06

    Project — A Windowed Aggregate That Survives a Restart and Late Data

    coreadvanced#projectcoming soon

Series 3 · In Production

evolving a job that is always running

  1. 01

    Upgrading a Running Job — Operator UIDs, and the Restore That Fails Without Them

    coreadvancedcoming soon
  2. 02

    Testing — Driving Event Time by Hand Instead of Waiting for a Clock

    advancedcoming soon
  3. 03

    When Flink Is Worth It — the Bill, and the Batch Job That Would Have Done

    coreadvancedcoming soon
  4. 04

    Project — Kafka In, Windowed Result Out, Upgraded Without Losing State

    coreadvanced#projectcoming soon

Not here, on purpose

  • Deciding whether you need streaming at all — covered in ETL & Data Pipelines, Series 3 · 42 — and the honest answer there is usually no
  • Event time and watermarks as concepts — covered in ETL & Data Pipelines, Series 3 · 42. Here they are configured and debugged rather than introduced
  • Running a Flink cluster — JobManager sizing, HA, Kubernetes operators — covered in Data Ops for the operational discipline. Here you learn what a JOB AUTHOR must know
  • The transport itself — covered in Apache Kafka