Engineering Strategy
Alerts That Don't Wake You at 3 AM
Most on-call pain is bad alerts, not bad systems. Alert on symptoms users feel, page only on what needs a human now, and kill the rest. The rules.
#observability
9 articles
Distributed Systems
Structured Logging at Scale
Plain-text logs are ungreppable at scale. Structured logging makes logs queryable, but volume and cost bite back. How to log structured, useful, and affordable.
Engineering Strategy
SLOs and Error Budgets That Hold Up
An SLO you never enforce is a number on a wiki. How to set SLOs from user experience, turn them into an error budget, and actually act on the burn rate.
Distributed Systems
OpenTelemetry Sampling Strategies
Tracing everything is too expensive; tracing 1% randomly drops the traces you need. OpenTelemetry sampling strategies, head vs tail, and how to choose.
Distributed Systems
Taming Prometheus Cardinality
High-cardinality labels are what kill Prometheus, not query volume. What causes a cardinality explosion, how to spot it, and the label rules that prevent it.
Kubernetes
Do You Actually Need a Service Mesh?
A service mesh gives mTLS, traffic control, and observability, at real complexity cost. When a mesh is worth it, when it isn't, and the lighter alternatives.
Engineering Strategy
Microservice Incident Response That Works
In microservices, the hard part of incident response is locating the fault across services. The triage order, the tools, and how to stop cascades fast.
Engineering Strategy
Grafana Dashboards for Operators, Not Executives
Most Grafana dashboards are decoration. An operator dashboard answers one question fast during an incident. How to design dashboards that speed up debugging.
Distributed Systems
Jaeger Tracing for Cross-Service Debugging
Jaeger turns a slow request across many services into one visual trace. How distributed tracing works, what to instrument, and the sampling tradeoff that bites.