Skip to content
Course contents

Encoding and Evolution

Data outlives the code that wrote it. How to change a schema on a Tuesday without taking the site down.

4 lessons1 lab1 hr 41 min

What you will be able to do

  • Explain why a rolling upgrade requires compatibility in both directions, not just forwards
  • Compare JSON, binary JSON variants and schema-based formats on size, speed and what they let you change safely
  • Describe how a field tag lets a reader skip a field it has never heard of
  • Predict what happens when a new writer talks to an old reader, for a given schema change
  • Reason about the same compatibility problem in three places: through a database, through a service, and through a message queue

An application is never a single version. It is an old version and a new version running side by side for the twenty minutes of a deploy, and, if the data is durable, a new version reading bytes that a version from three years ago wrote and nobody remembers.

That makes schema change the quiet reason deploys go wrong. This module is about encoding — how a structure in memory becomes bytes on a wire or a disk — and the compatibility rules that follow from it. It is the shortest module and it is the one whose lessons you will apply the soonest, because everyone changes a field name in their first month.

The lab at the end has you write a binary encoder with field tags, then break compatibility on purpose, so you can watch the exact failure mode you are being warned about.

Read alongsideChapter 4 of Designing Data-Intensive Applications.

In this module5

Start module 4