In this article, the author walks us through the clever engineering tradeoffs made in Bluesky’s timeline system, where embracing a bit of imperfection has led to major performance gains. Instead of striving for perfect consistency across every user’s following feed, Bluesky intentionally introduces "lossy timelines" that probabilistically drop some write operations when a user follows an unusually high number of accounts. This means that while the timelines may not be perfectly chronological, they’re fast and scalable—reducing the P99 latency dramatically by over 90% and slashing full post fanout times from minutes to seconds. As the article puts it, “Knowing where it’s okay to be imperfect lets you trade consistency for other desirable aspects of your systems and scale ever higher.”

Key points:

• System Tradeoffs: Traditional systems aim for perfect consistency, availability, and low latency, but achieving all at once is impractical. Bluesky makes strategic sacrifices for improved scalability.
• Timeline Fanout Mechanism: When a post is created, it’s fanned out to the timelines of all followers, which, if handled sequentially, would be prohibitively slow.
• The Hot Shard Problem: Users following hundreds of thousands of accounts can overload a shard, causing disproportionate latency for many co-located timelines.
• Lossy Timelines Concept: For heavy-follow users, Bluesky introduces a "loss_factor" (calculated as min(reasonable_limit/num_follows, 1)) that probabilistically drops writes, thereby preventing any one timeline from overburdening the system.
• Caching Strategy: High-follow account data is cached using Redis, avoiding additional load on the primary database and keeping follow count lookups lightning fast.
• Performance Gains: Implementation of these tradeoffs significantly reduced latency—cutting the P99 duration for timeline fanout by over 96% and nearly eliminating hot shards.
• Scalable Design: The approach allows Bluesky to handle more than one million timeline writes per second while keeping the system responsive and efficient.
• Opportunities for Future Work: The article hints at additional architectural improvements in the pipeline and encourages interested engineers to check out job opportunities at Bluesky.

This innovative approach demonstrates that sometimes, perfection isn’t necessary to deliver a responsive, high-throughput service. Enjoy the blend of practicality and creativity as Bluesky scales new heights by embracing imperfection!
Link to Article