Kafka Replay Strategy Without Duplicate Events
Replaying a Kafka topic re-delivers events, so duplicates are guaranteed unless consumers are idempotent. The safe replay playbook: dedup, offsets, and isolation.
Guide · 8-part series
A connected series on the distributed-systems patterns you reach for again and again: how to replay events without duplicates, make operations idempotent, and keep systems stable under load.
Replaying a Kafka topic re-delivers events, so duplicates are guaranteed unless consumers are idempotent. The safe replay playbook: dedup, offsets, and isolation.
Pick database sharding strategies before traffic forces your hand. Shard keys, hash vs range vs directory, online resharding, and the traps that bite early.
Idempotency keys make retried requests safe so a timed-out payment or duplicate POST applies exactly once. The design, storage, and TTL decisions that matter.
A timeout budget is one deadline split across a service chain so the whole request fails fast instead of piling up doomed work. How to set and propagate it.
Backpressure keeps real-time systems alive under load by making producers slow down instead of drowning consumers. Strategies, tradeoffs, and a checklist.
WebSocket capacity planning for social products: budget memory, file descriptors, and fan-out per connection so you scale before connections, not CPU, break first.
Notification fanout comes down to fan-out-on-write vs on-read, and the celebrity problem that breaks the naive choice. The hybrid design that scales, explained.
Designing leaderboards at scale: why a sorted database query dies under load, how a Redis sorted set fixes it, and how to shard ranking past one node.
Newsletter
New posts in this series and others, when there is something worth saying. No spam.