21 November 2026
48 new posts across 7 practice and leadership sources — 6 worth your time.
Presentation: Scaling LLM-Based Ranking Systems for Latency-Critical Search & Recommendation Workloads
LinkedIn details how it met strict P99 latency SLAs for LLM ranking by distilling a 7B parameter model down to 400M and compressing job contexts into single-token embeddings. It also highlights eliminating underlying system bottlenecks using custom flash attention masks, gc.freeze, Python GIL workarounds, and non-blocking CUDA memory copies.
InfoQ Architecture · 21 November 2026 · Read the original →
Uber Cuts Translation Cache Memory by up to 90% While Keeping P99 Latency Below 1 MS
Uber redesigned its Go-based translation cache into a hybrid model that keeps keys and hot values in memory while offloading remaining entries to disk. This design reduced initial memory consumption by over 90% while maintaining hit rates above 99.9% and disk-backed miss P99 latency under 100 microseconds.
InfoQ Architecture · 21 November 2026 · Read the original →
Deterministic Simulation Testing
Unmesh Joshi explains how deterministic simulation testing overcomes non-deterministic failures in distributed systems by swapping out real clocks, networks, and disk storage for simulated versions. Controlled by a single-threaded execution loop, this approach allows teams to predictably reproduce and debug complex race conditions and fault scenarios.
Martin Fowler · 21 November 2026 · Read the original →
Fragments: November 20
Marcus Borg conducted an experiment across 465 pull requests in 246 open-source repositories to measure how code quality impacts LLM code reviews. The results showed that reviewing unhealthy codebases (CodeHealth 2.8) consumed 22% to 35% more LLM tokens than reviewing healthy codebases (CodeHealth 9.7).
Martin Fowler · 21 November 2026 · Read the original →
The Pulse: What can we learn from the Railway hack?
Gergely Orosz breaks down the recent security incident at cloud platform Railway, detailing how the compromise occurred and how the team responded. The post analyzes technical vulnerabilities exploited during the breach and extracts security operational lessons for software leaders.
Pragmatic Engineer · 21 November 2026 · Read the original →
Lessons from Railway Hack
A recent security incident involving a hack on the Railway platform is analyzed, detailing the attack vectors, exploited vulnerabilities, and the broader impact on users and the platform's operations.
newsletter.pragmaticengineer.com · 21 November 2026 · Read the original →
Sub-millisecond latency just gives us more time to reflect on our choices.
6 stories, every Saturday
Published here every week. Follow by RSS to get it as it lands.