← All courses

☸️ Kubernetes for Data Workloads

The subset a data person actually meets — Jobs and CronJobs rather than Deployments and Ingress, why the pod was OOMKilled with nothing in the logs, where a batch is supposed to write, the Spark and Postgres operators, and an honest episode on when cron on one box wins.

No lessons published yet — this is the roadmap, so you can see exactly what the course will cover before it lands.

Before you start: Data Ops — Series 1, and Series 1 · 37 in particular. Everything here runs a container, and this course does not teach one. You need images, volumes, environment variables and logs to already mean something, and Data Ops Series 1 · 37 is where that happens.

Short on time? The 29 episodes marked core are the shortest path to being productive at work. Start there and pick up the rest whenever a job asks for them.

Series 1 · The Objects You Will Actually Touch

pods, jobs, and why the container died

  1. 01

    Pods, Nodes and the Control Loop — Declaring What You Want, Not What to Do

    coreintermediatecoming soon
  2. 02

    kubectl — describe, logs, exec, and Reading What the Cluster Tells You

    coreintermediatecoming soon
  3. 03

    Job and CronJob — Your Nightly Load as a Kubernetes Object

    coreintermediatecoming soon
  4. 04

    Deployments and Services — The Long-Running Half You Have to Read

    intermediatecoming soon
  5. 05

    Requests, Limits and OOMKilled — Why the Pod Died and the Log Said Nothing

    coreintermediatecoming soon
  6. 06

    ConfigMaps and Secrets — Where the Connection String Comes From, and Why base64 Is Not Encryption

    coreintermediatecoming soon
  7. 07

    Project — Run a Nightly Load as a CronJob and Prove It Recovers

    coreintermediate#projectcoming soon

Series 2 · Data Has State, Kubernetes Assumes It Does Not

where the output goes, and why it vanished

  1. 01

    Ephemeral by Default — Why the Job Succeeded and the Output Is Gone

    coreadvancedcoming soon
  2. 02

    PersistentVolumeClaims and StorageClasses — Asking the Cluster for Somewhere to Write

    coreadvancedcoming soon
  3. 03

    StatefulSets — and the Honest Argument About Databases in Kubernetes

    advancedcoming soon
  4. 04

    Object Storage Instead of a Volume — the Pattern That Usually Wins

    coreadvancedcoming soon
  5. 05

    Project — Give a Job Somewhere to Write, Twice, and Compare

    coreadvanced#projectcoming soon

Series 3 · Running Data Workloads Well

scheduling, operators, and reading a failure

  1. 01

    Scheduling, Taints and Tolerations — Getting the Big Job onto the Big Machine

    advancedcoming soon
  2. 02

    The Autoscalers — Why HPA Does Nothing for a Batch Job

    advancedcoming soon
  3. 03

    Operators and CRDs — Spark, Flink and Postgres as Objects the Cluster Understands

    advancedcoming soon
  4. 04

    Helm — Installing Something You Did Not Write, and Reading values.yaml First

    advancedcoming soon
  5. 05

    Pending, CrashLoopBackOff, Evicted — Reading a Failure Without Guessing

    coreadvancedcoming soon
  6. 06

    When Not to Use Kubernetes — and Why cron on One Box Often Wins

    coreadvancedcoming soon
  7. 07

    Project — Diagnose Three Broken Workloads From Their Events Alone

    coreadvanced#projectcoming soon

Series 4 · Longhorn — Replicated Block, the Simple Way

volumes made of files on your nodes, and the node drain that eats one

  1. 01

    Replicas Are Files on Your Nodes — Why You Can Debug This One

    coreadvancedcoming soon
  2. 02

    Installing It — open-iscsi on Every Node, and the Check That Saves an Afternoon

    coreadvancedcoming soon
  3. 03

    Replicas, Data Locality and Expansion — Where the Copies Actually Land

    coreadvancedcoming soon
  4. 04

    Snapshots and Backups — the Only Copy That Leaves the Cluster

    coreadvancedcoming soon
  5. 05

    Draining a Node — the Rebuild a Routine kubectl Command Starts

    coreadvancedcoming soon
  6. 06

    Project — Run a Database on Longhorn, Kill a Node, and Keep the Data

    coreadvanced#projectcoming soon

Series 5 · OpenEBS — Local Volumes, and Not Replicating Twice

fast local disks with a lifecycle, and the argument for using them

  1. 01

    Local PV Against Replicated — Two Products Under One Name

    coreadvancedcoming soon
  2. 02

    Local PV on LVM or ZFS — a Real Lifecycle on One Node's Disk

    coreadvancedcoming soon
  3. 03

    Double Replication — If the Database Already Does It, the Storage Should Not

    coreadvancedcoming soon
  4. 04

    Mayastor — Replicated PV When You Genuinely Need It, and Its Prerequisites

    advancedcoming soon
  5. 05

    Project — Put a Self-Replicating Database on Local PV and Prove the Failure Path

    coreadvanced#projectcoming soon

Series 6 · Rook, and Choosing Between the Three

Ceph operated by Kubernetes, and the minimum that gets you what you need

  1. 01

    Rook Is an Operator, Not a Storage System — the Storage Is Ceph

    coreadvancedcoming soon
  2. 02

    The CephCluster Resource — Declaring a Cluster, and Why It Found No Disks

    coreadvancedcoming soon
  3. 03

    CephBlockPool, CephFilesystem and CephObjectStore — Storage as Manifests

    coreadvancedcoming soon
  4. 04

    Operating Rook — Two Sets of Logs, and Two Version Ladders

    coreadvancedcoming soon
  5. 05

    Longhorn, OpenEBS or Rook — and the Case for None of Them

    coreadvancedcoming soon
  6. 06

    Project — Choose a Storage Backend for Three Workloads and Defend Each

    coreadvanced#projectcoming soon

Not here, on purpose

  • Building the container image — covered in Data Ops, Series 1 · 37, which is Docker itself. Here the image already exists and gets scheduled
  • Provisioning the cluster — nodes, networking, ingress, service mesh, RBAC administration, upgrades — not covered; this is platform engineering and a different job. Terraform & Pulumi owns describing infrastructure as code, and even that course stops at the cluster's edge rather than operating one
  • Expressing dependencies between pipeline steps — covered in ETL & Data Pipelines, Series 2 · 45 as a concept and in Apache Airflow as a tool. A CronJob runs one thing on a clock; it has no idea what ran before it
  • Whether to use an orchestrator at all, and which one — covered in Data Ops, Series 3 · 00. This course assumes the cluster is already the answer somebody gave
  • Ceph itself, ZFS, erasure coding and the S3 protocol — covered in Storage for Data, which is the physical layer. In this course, Series 6 is Rook — the OPERATOR that runs Ceph inside Kubernetes — and it assumes that course rather than re-teaching CRUSH
  • Continuous delivery of the manifests — covered in Continuous Delivery, which owns the deployment pipeline as a subject. In this course, Series 3 · 15 names the GitOps tools and does not teach them