Skip to content
Back
2024CompletedFeatured

Atlas Ingest

A back-pressure aware ingestion service that survives its own bad days.

RoleDesign and implementation

  • Go
  • Kafka
  • ClickHouse
  • Kubernetes

The problem

The original pipeline dropped events silently under load. Nobody found out until a customer asked why a dashboard disagreed with their own numbers.

The approach

Atlas Ingest makes back-pressure a first-class concept. Producers get an explicit signal instead of a timeout, and every batch carries an idempotency key so a retry is always safe.

Result

180M events/day at a steady p99 of 40ms, with zero silent drops in eighteen months of operation.