Crispfern
News and analysis from the world of production systems.
What Good Observability Actually Looks Like
May 23, 2026
Dashboards multiply like rabbits and answer questions nobody asked this month. Good observability starts from the opposite direction: the alert fired, a human is awake, and the tooling should answer 'what changed' within a minute.
In practice that means a small set of high-cardinality traces on the request path, deployment markers on every graph that matters, and logs structured enough that you can pivot from a customer ID to a timeline without five grep commands.
Managing Secrets Without Losing Sleep
October 1, 2026
There are exactly two ages of secrets management: 'we keep them in an encrypted file' and 'we were audited'. The distance between them is covered by rotation policies, access trails, and the gradual realisation that humans should read production credentials roughly never.…
Email Deliverability: Authentication Beyond SPF
May 14, 2026
Traditional SPF checking fails to address modern deliverability challenges on its own. While an envelope IP check might succeed, mismatched headers trigger spam filters that prioritize cryptographic DKIM validations and broader DMARC policy rules.…
Cache Invalidation Patterns That Survive Traffic
September 16, 2026
Everyone quotes the two hard things joke; fewer people ship caches engineered against stampede. The pattern that matters most is request collapsing: when a hot key expires, exactly one worker rebuilds while everyone else serves slightly stale data, which turns a cache miss from a database incident i…
A Field Guide to Graceful Degradation
August 26, 2026
Complex platforms require predetermined shedding tiers when capacity degrades. Personalized feeds should drop before transactional checkout; suggestion dropdowns should degrade before primary queries; media thumbnails should omit before raw assets. Documenting and enforcing these boundaries via circ…
More reading
- When to Choose a Queue Over a Request — Engineering, May 23, 2026
- A Practical Guide to API Rate Limiting — Engineering, August 17, 2026
- Practical Notes on Postgres Connection Pooling — Data, June 15, 2026
About us
Our contributors have spent years on-call for large platforms. This site collects the playbooks, postmortems and reference material we wish someone had handed us earlier.