Version Your Serializables

A serializable is an object you flatten into bytes so that it can travel, whether that trip is across an API or an RPC, onto a queue, into a cache, into object storage, or across any other system boundary you care to name. Serialization is the act of shaping it for that journey. Every one of those boundaries is a contract, and in my experience engineers treat those contracts very differently depending on which system happens to be on the other side.

Most teams version their APIs and their RPCs, and if they are being thorough, the events they publish onto a bus. You keep a tidy history of migrations for your database schemas, which is not quite versioning but is close enough to count. Through all of that care, you almost certainly never think about versioning your cache at all.

The reasoning I hear most often is that a cache is “internal”, and “ephemeral anyway”, so whatever you lose you can always get back easily. That assumption is exactly the one that breaks on the next deploy, and it does not break quietly.

How this usually breaks

I have been building in the Java ecosystem for the better part of a decade, and this particular incident has found me more than once. It has found the teams around me more often than that. What makes it worth writing about is that the decision underneath it is not really about a contract at all. It reaches further, into whether the thing you cached is still delivering the value you cached it for, and into how you are willing to interact with that cache once it stops.

Which brings me to Redisson. If you write Java and you talk to Redis or Valkey, there is a good chance you are already using it, because it has quietly become the default choice on a lot of backends//NOTE 01Redisson is a Redis and Valkey client for Java: https://redisson.pro/docs/. Its own overview counts roughly sixty objects and services, more than thirty of which exist only in Redisson: distributed locks, semaphores, maps, queues, topics, rate limiters and remote services.. It goes a long way past the plain data structures and trivial caching that people first come for. You get distributed locks, semaphores, queues, topics, rate limiters and a good deal more, all wearing the interfaces you already know from the standard library.

So what does any of that have to do with data model contracts? It comes down to the codec Redisson picks for you when it turns your cached object into bytes, and that choice is what the rest of this article puts under scrutiny. Move a field, reorder a class, or add a single new one on an otherwise unremarkable afternoon, and the thing you rebuild on the way out is no longer the thing you put in. On the next deploy that mismatch introduces itself to you personally, in the shape of a KryoBufferUnderflowException.

Consider this data model for a cached object representing a user’s profile you probably fetch frequently:

public class UserProfile {
    public String userId;
    public String name;
    public String email;
    public String role;   // exactly one role, which felt like enough at the time
}

Then the product grows up, as products do, and one role per user stops being enough. A user needs to hold several. The obvious move is to reach into the class and change the field.

public class UserProfile {
    public String userId;
    public String name;
    public String email;
    public List<String> roles;   // was: String role
}

That change is a violation no matter which library you are using, because your new code can no longer read the data your old code wrote.※※ BACKWARDS = NEW CODE READS OLD DATA. FORWARDS = OLD CODE READS NEW DATA. This is the first place the term earns its keep, so it is worth naming properly: what you have just broken is backwards compatibility, the ability of new code to read old data. None of this is rocket science, and both of us can see the fix from here. You leave the old field where it is, you add a new one for the list, and you write a small adapter that maps the old shape onto the new one until the old shape drains away.

public class UserProfile {
    public String userId;
    public String name;
    public String email;
    public String role;          // kept, and left alone
    public List<String> roles;   // added alongside it
}

Congratulations, you have just derived the first principle of schema evolution from scratch. Unfortunately, the author of the Redisson library has other plans for you. You ship that change, you have existing cached profiles sitting in Redis or Valkey being read back through this very library, and the first read after the deploy hands you the exception.

The read did not come back with a wrong value. It threw outright, and the exception has a name worth remembering: KryoBufferUnderflowException, whose message is a curt two words, “Buffer underflow.” What that means is that the reader ran out of bytes before it ran out of fields to fill. Somewhere along the way, the bytes and the class had stopped agreeing with each other.

The reason is that Redisson hands your object to the Kryo 5 codec//NOTE 02Redisson's codec catalogue lists Kryo5Codec as the default: https://redisson.pro/docs/data-and-services/data-serialization/. Kryo itself lives at https://github.com/EsotericSoftware/kryo., and Kryo writes only the values. There are no field names on the wire, and no tag anywhere saying “this one is the email.” Kryo works out the order by taking your class’s fields and sorting them alphabetically by name, writing the values in that order, and expecting the reader to compute exactly the same ordering from its own copy of the class. Kryo’s own documentation is blunt about what follows: it does not support adding, removing, or changing the type of a field without invalidating everything you already wrote, and renaming a field is safe only when the new name lands in the same alphabetical slot. Add a single field and every value that sorts after it shifts by one, so the reader walks off the end of the buffer while it is still asking for more.

It helps to see which mutations survive that arrangement and which do not.

// v1, the bytes already sitting in your cache.
// Kryo sorts by field name, so the wire order is: email, name, role, userId.
class UserProfile { String userId; String name; String email; String role; }

// ✅ reorder the declarations: the alphabetical order is unchanged, so the bytes still read
class UserProfile { String name; String userId; String email; String role; }

// ❌ add a field: "roles" sorts between "role" and "userId", so everything after it shifts
class UserProfile { String userId; String name; String email; String role; List<String> roles; }

// ❌ remove a field: the reader expects a value that is no longer on the wire
class UserProfile { String userId; String name; String role; }

// ❌ rename across an alphabetical boundary: "access" jumps to the front of the order
class UserProfile { String userId; String name; String email; String access; }

// ❌ change a type: the reader decodes the bytes as the wrong type and misaligns
class UserProfile { String userId; String name; String email; int role; }

// ✅ change nothing at all
class UserProfile { String userId; String name; String email; String role; }

The part that stings is that nobody sat down and chose Kryo. It is simply the default. If you use Redisson and never call setCodec, this is what you get.

Config config = new Config();
// setCodec is never called, so the default stands.
RedissonClient redisson = Redisson.create(config); // codec == new Kryo5Codec()

Kryo became that default in December 2022, in a single line of a changelog//NOTE 03Redisson 3.19.0, released December 2022, made Kryo5Codec the out-of-the-box default. The prior default was MarshallingCodec, and FstCodec before that. that nobody reads. The buffer underflow it armed goes off much later, on somebody else’s deploy.

Let me be fair to the choice before I take it apart, because it was not a stupid one. Kryo 5 is fast, it produces a very compact binary, and it asks nothing of you in the way of configuration. If your objects never change shape then it is a genuinely good default, and it has the pleasant side effect of making the library benchmark beautifully the moment anyone tries it.

What is harder to forgive is that the switch itself broke the contract. The library changed the expected behaviour of its own default by swapping codecs underneath everyone, and the new codec cannot read what the old ones wrote. Imagine upgrading a dependency and watching your entire read path fill with exceptions for reasons that have nothing to do with anything you wrote. Ouch.

More fundamentally, you cannot put a narrow default on an open tool. Redisson is not a thin, honest client that hands you Redis and gets out of the way, because that is Jedis//NOTE 04Jedis is a thin, low-level Redis client: you ask for Redis, you get Redis. Redisson is a whole framework on top of it (distributed locks, maps, queues, services). People reach for it precisely because it does more than Redis does.. Redisson is the framework people reach for as a portal into everything the Redis ecosystem can do, and “just a cache” is the smallest of those things. They arrive expecting Redis-like freedom, and then the default quietly forbids the most ordinary thing software does, which is change. Shipping a codec that punishes evolution, as the default, in a place where evolution is guaranteed, is beyond me. And it is not only me. The Redisson tracker carries a steady row of issues telling the same story, where a schema changed, Kryo 5 was underneath, and deserialization came apart.

The cache did not betray you. The default did. And you accepted the default.

What “version your serializables” actually means

Schema evolution is a grand name for a very ordinary fact, which is that your data changes shape over time. You add a field, you drop another, you rename a third and retype a fourth, and any object that outlives a single deploy will go through some version of that eventually. It is the normal case, not the edge case.

Compatibility, though, has a direction, and there are two of them worth keeping straight.

Being backwards compatible means new code can read old data, so your v2 reader meets a v1 payload and quietly fills in whatever is missing. Being forwards compatible means old code can read new data, so a v1 instance that is still running meets a v2 payload and ignores the fields it has never heard of. You need both, in different places and for different reasons.

A rolling deploy is the case that needs forwards compatibility, because for a few minutes v1 and v2 run side by side and the old instances have to survive whatever the new ones are writing. A cache is the case that needs backwards compatibility, because your new code has to make sense of v1 bytes that were already sitting in Redis before you deployed anything.

Now the humbling part. Adding a field is the gentlest evolution there is, since old readers can ignore it and new readers can default it, which makes it the lowest bar in the entire subject. Plain JSON clears that bar for free//NOTE 05One Jackson setting does most of the work: objectMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false). Unknown fields are dropped instead of throwing, and a missing field stays at its default., without being asked. Kryo does not clear it at all.

So versioning your serializables was never about stamping a number on a payload for its own sake. It means working out which direction of compatibility you actually need, and then choosing an encoding that gives it to you. A forgiving codec will carry you past the trivial gotchas on its own, but an explicit version stamp buys you something it cannot: the ability to let two or three generations of your data coexist deliberately while you migrate between them.

The codecs that survive

Three common formats handle a field change without any drama. JSON is self-describing, so it matches by name. Protobuf pins every field to a number that never gets reused. Avro resolves the writer’s schema against the reader’s as it reads, though the schema itself travels out of band rather than inside the bytes. Kryo can be made tolerant too, using a name-based serializer instead of the positional one//NOTE 06Kryo's CompatibleFieldSerializer tracks fields by name instead of position, which survives add and remove. The catch: Redisson's Kryo5Codec does not expose a hook to swap it in, so you cannot reach it through Redisson without subclassing the codec., but Redisson does not hand you that lever. And while plain JSON will always be slower than the binary format Kryo was chosen for, the gap narrows considerably once you add the right compression. On modern hardware you can have both the performance and the ability to evolve.

I did not want to settle this from a table in somebody’s documentation, so I measured it//NOTE 07The full experiment, the code, the raw numbers, and every caveat live in the repo: github.com/vertebraeker/pramana, under experiments/2026-06-29-redisson-codec-compression.. The object under test is a nested Order, the kind of record a real service caches, seeded so its high-entropy fields like ids and a free-text note sit next to low-entropy ones like enums and timestamps.

public class Order {
    public String orderId;          // UUID, high entropy
    public OrderStatus status;      // enum, low entropy
    public long createdAtEpochMs;   // timestamp, low entropy
    public List<LineItem> items;    // nested line items
    public Address shipping;        // nested address
    public String note;             // free text, high entropy
    // ... a few more fields in the same spirit
}

The codecs are Kryo 5, Protobuf, Avro and JSON, each tried on its own and each wrapped in one of four compressors, LZ4, ZSTD, Gzip or Brotli. The whole matrix runs end to end through a real Redisson against a real Redis, on two machines with very different paths to it, and each codec is put through one added field to see whether it survives the change.

The compatibility gate itself is simple enough to state in one breath: serialize a v1 object, add a field, then try to read the old bytes back with the v2 reader. Here is that check in miniature, taking the Kryo path from the experiment.

class ProfileV1 { public String name; public int age; }
class ProfileV2 { public String name; public int age; public String email; } // one field added

Kryo kryo = new Kryo();
kryo.setRegistrationRequired(false);

Output out = new Output(4096);
kryo.writeObject(out, new ProfileV1("Ada", 36)); // the v1 bytes now in your cache
byte[] oldBytes = out.toBytes();

kryo.readObject(new Input(oldBytes), ProfileV2.class);
// -> com.esotericsoftware.kryo.io.KryoBufferUnderflowException: Buffer underflow

Run every codec through that same mutation and the split is clean. Kryo 5 breaks with the buffer underflow, while JSON and Avro read the old bytes without complaining. Protobuf is safe by its wire format too, though this live check ran only the other three.

Buffer underflow happens to be the failure we hit, but it is not the only shape the break can take//NOTE 08A Kryo schema drift can also surface as 'Encountered unregistered class ID', a missing no-arg constructor, a ClassCastException, or silent field corruption. The common thread is the same: the bytes and the class disagree.. The thread running through all of them is the same. The bytes and the class disagree, and Kryo has no way of noticing in time to tell you.

What surprised me is how little that safety costs. On size the whole decision is roughly a factor of two, and that factor lives entirely between JSON and the binary formats rather than among the binary formats themselves. Avro comes in smallest at 703 bytes per order, Kryo 5 sits at 861, and JSON is the outlier at 1608, which puts the three binary encodings close enough together that size alone will not choose between them.

The part I did not expect is that Kryo, a positional format that writes no field names at all, comes out no smaller than Protobuf, which tags every field with a number. Kryo still writes a full class name on the wire for each class it has not been told about, and Redisson registers only a handful of primitives, so a single Order pays for the names of Order, Address, LineItem and the two collection types it nests. That is about 190 of its 861 bytes, which is why a positional format ends up no smaller than a tagged one here. A number turns out to be cheaper than a class name.

//FIGUREFIG.03 · BYTES PER OBJECT, NO COMPRESSIONEXPAND

JSON also happens to be the one format that compresses well. Per value it shrinks by roughly half under ZSTD, while the binary formats barely move at all, for the simple reason that a packed binary encoding has almost no redundancy left in it to squeeze. Compress the JSON, and do not waste cycles compressing the binary.

//FIGUREFIG.04 · COMPRESSION RATIO, PER VALUEEXPAND

Which compressor you reach for is a smaller decision with its own shape. LZ4 is the quickest and packs the least, Brotli packs the tightest and takes the longest doing it, and ZSTD sits comfortably between them.

There is also a crossover worth knowing about. JSON’s bulk is mostly repeated structure, and repeated structure is exactly what compressors are good at, so somewhere past roughly fifteen to thirty line items compressed JSON becomes smaller than uncompressed binary. Small payloads favour binary. Large ones favour compressed JSON.

//FIGUREFIG.05 · WIRE BYTES AS THE PAYLOAD GROWSEXPAND

//FIGUREFIG.06 · AT 100 LINE ITEMSEXPAND

On speed the honest finding is that the format you choose barely matters, and the thing that does matter is not the format at all. Running the full matrix end to end through Redisson against a live Redis, seven times over with the order shuffled on every pass, the binary codecs land within about three percent of each other on both machines. Uncompressed JSON is the one format that breaks from that pack, and only on the slower machine, where it trails the rest by about a fifth because it carries the most bytes. What separates the two machines is not the codec but the round trip around it. When Redis sits behind a virtualised container network the hop runs about two milliseconds, heavy enough to bury whatever a codec spends on top of it. When the container shares the kernel the same hop drops to about a third of a millisecond, and with the hop out of the way the CPU a codec spends is what is left to measure. That cost was there on both machines all along, and the slow hop was simply hiding it.

//FIGUREFIG.07 · WHEN COMPRESSION PAYS, AND WHEN IT COSTSEXPAND

Whether compressing the JSON is worth it flips completely between the two machines. Compression always costs some CPU and always removes some bytes. When the network hop is the heavy part of the round trip, the bytes it removes matter more than the CPU it costs, and compressing the JSON bought between fourteen and seventeen percent. When the hop is cheap those same bytes are worth almost nothing, so the CPU the compression spent is what shows up, and the same compression now gives back five to ten percent of the throughput, with Redisson’s own ZSTD codec//NOTE 09Redisson's ZStdCodec builds a new zstd stream for every value instead of reusing one, so it allocates about 262 KiB of heap per round trip to move a 572-byte payload, and gives back roughly a quarter of the throughput on small values on a fast path. Its LZ4Codec uses the one-shot API and does not. The algorithm is not the problem, the codec around it is. Still the case in 4.6.1. giving back a full quarter. LZ4 is the exception, fast enough that it costs almost nothing either way.

But throughput is only one of the things compression changes, and reading it on its own is what makes compression look like a loss on a fast path. The bytes a codec removes are not only bytes that cross the wire. Redis keeps every value in memory, so they are also bytes you pay to store, and on a managed cache that memory is most of the bill. Across a zone or out of a cloud provider they are bytes you pay to move as well. Neither of those savings changes with the network, so neither one shows up in a throughput number at all.

//FIGUREFIG.08 · WHAT COMPRESSING JSON BUYS, AND WHAT IT COSTSEXPAND

Measured against Redis’s own memory accounting, compressing the JSON gives back about half the RAM each key costs, from a third under LZ4 to fifty-two percent under Brotli, and about the same fraction of the bytes on the wire. One round trip here is a write and a read, so every value crosses twice, and at the corpus average raw JSON moves roughly 3.2 gigabytes per million round trips against about 1.6 for the same JSON under Brotli. On a single host that traffic is free, which is the whole reason compression does not pay for itself there. Across a zone or out of a provider it is billed by the gigabyte in both directions. So even on a fast local path, where compression costs a few percent of throughput, the saving on the other side does not shrink with it: about half your cache memory and about half your transferred bytes, and with LZ4 the throughput cost is almost nothing.

For the purely additive case, then, the pragmatic default is short enough to say in a sentence: anything but Kryo 5. If your team already keeps Protobuf or Avro schemas then use them, and Avro will be the smallest of the lot. Otherwise reach for JSON and wrap it in LZ4, which costs almost nothing on any path and still takes about a third off both the memory and the bytes.

// JSON, wrapped in LZ4, the safe default. Both are real Redisson codecs.
config.setCodec(new LZ4Codec(new TypedJsonJacksonCodec(UserProfile.class)));

When JSON is not enough

Not every evolution is purely additive, and this is where “just use JSON” quietly runs out of road.

Some changes are genuinely breaking. You change timeout from seconds to milliseconds, which keeps the same type while inventing an entirely new meaning. You split name into firstName and lastName. You retype a price from the string “9.99” into an integer count of cents, at which point it is really a different field under the old name. On the changes it can still parse, JSON will not throw at all. It hands your code a value that is wrong, which is considerably worse than a crash, because nothing anywhere tells you it happened.

A breaking change of that kind needs a version stamped onto the payload and code that branches on it, so version 1 is read one way and version 2 another. Avro and Protobuf are both built for exactly this. Protobuf pins every field to a number you promise never to reuse, so an old reader and a new one always agree about what field 3 is.

message UserProfileV2 {
  string user_id = 1;
  repeated string roles = 2;   // v1 had a single `role`; the number is the contract
  string payment_tier = 3;     // new in v2
}

Then you guard the seam between producer and consumer on that version, letting the consumer read the stamp and run whichever adapter belongs to it, whether that means wrapping a single value into a list, splitting a name, converting units or backfilling a default.

Envelope e = readEnvelope(bytes);
UserProfile p = switch (e.schemaVer()) {
    case 1  -> migrateV1toV2(e.payload()); // wrap role -> roles, backfill new fields
    case 2  -> parseV2(e.payload());
    default -> throw new IllegalStateException("unknown schema version " + e.schemaVer());
};

Avro reaches the same result from a different angle, and it is worth showing because of where you will most likely meet it. In the Kafka ecosystem Avro is the conventional default, to the point where Confluent’s own documentation still calls it the original default format, and the Schema Registry API treats a response with no declared schema type as Avro by definition. Protobuf and JSON Schema have been first-class there since 2020, but they are the formats you have to ask for. Rather than making you branch on a version by hand, Avro resolves the schema the data was written with against the one your code holds now, so a field that did not exist when the bytes were written simply arrives carrying the default you declared for it. The bytes themselves carry neither field names nor types, which is exactly why the writer’s schema has to reach the reader some other way, and exactly what a schema registry is for.

{
  "type": "record",
  "name": "UserProfile",
  "fields": [
    { "name": "userId", "type": "string" },
    { "name": "roles",  "type": { "type": "array", "items": "string" },
      "default": [] },
    { "name": "paymentTier", "type": "string", "default": "unknown" }
  ]
}

Notice the v2 lives in the schema name.

// Avro's binary encoding carries no field names and no schema. The writer's schema
// has to reach you some other way: a registry lookup, or an Avro file header.
Schema writerSchema = schemaRegistry.getSchemaById(idFromEnvelope);  // out of band
Schema readerSchema = UserProfile.SCHEMA$;                           // what your code wants now

GenericDatumReader<GenericRecord> reader = new GenericDatumReader<>(writerSchema, readerSchema);
GenericRecord profile = reader.read(null, DecoderFactory.get().binaryDecoder(oldBytes, null));
// paymentTier resolves to "unknown" rather than throwing, because the reader's schema declared a default

None of this is exotic. It is the same trick you already use everywhere else: you are versioning a behaviour and choosing the implementation at runtime, which is the strategy pattern//NOTE 10The Gang of Four strategy pattern: define a family of interchangeable behaviors and select one at runtime. Versioned deserialization is the same shape, keyed on a schema version instead of a config flag. pointed at bytes instead of at business logic.

You can carry that version in an envelope wrapped around the payload. Or you can bake it into the cache key itself//NOTE 11Key namespacing (v2:user:9872) isolates versions completely: v1 code reads the v1 keyspace, v2 code the v2 keyspace, and old keys expire on their own TTL. It trades a little cache duplication for zero cross-version collisions., as something like v2:user:9872, so the two generations never share a slot in the first place.

None of this is really about caching, or about Redis. Everything that crosses a system boundary is a contract, and the same move works whichever boundary you are standing on. In Kafka you reach for Avro or Protobuf with a schema registry//NOTE 12A schema registry (Confluent Schema Registry and friends) stores each schema centrally and hands producers and consumers a compatibility check at publish time. Confluent's own docs are explicit that the schema never travels with the data: the serializer 'does not include the message schema. Instead, it includes the schema ID (in addition to a magic byte) followed by the normal binary encoding of the data itself.', and the only thing that really moves is where the compression lives, which is per batch at the broker rather than per value at the client.

Wherever the boundary sits, the discipline is the same: pick an encoding that can still be read after the shape has changed, stamp a version onto it when the change is too large to absorb quietly, and the objects you serialize stay yours to evolve rather than a deploy you have to survive.