03 November 2026
74 new posts across 9 database and data engineering sources — 8 worth your time.
Vadim Yatsenko: Zero-Copy I/O for the Shared Storage Daemon: Removing the Last Copy from the WAL Path
In PolarDB, every database commit requires the Write-Ahead Log (WAL) record to reach shared storage, involving data movement from WAL buffers through a shared-storage daemon to the device. This article details how the last data copy in this critical WAL path was eliminated to improve performance.
Planet PostgreSQL · 03 November 2026 · Read the original →
Vadim Yatsenko: Commit Sequence Number (CSN) MVCC: Lock-Free Snapshot Acquisition
PostgreSQL's Multi-Version Concurrency Control (MVCC) determines row visibility by checking if a writing transaction was committed before a query's snapshot. This article describes Commit Sequence Number (CSN) MVCC, a technique for achieving lock-free snapshot acquisition, improving concurrency.
Planet PostgreSQL · 03 November 2026 · Read the original →
Igor Suhorukov: pg_vexec one month on: ahead of ClickHouse on Parquet, close behind on MergeTree
This project ported Greenplum's ORCA analytical query planner to PostgreSQL 19, isolating minimal core patches to create a vectorized, high-performance DBMS. After one month, this database fork shows faster performance, outperforming ClickHouse on Parquet and nearing its MergeTree performance.
Planet PostgreSQL · 03 November 2026 · Read the original →
Designing a planned switchover for minimum disruption
This post details the process of designing a planned switchover for a Neki shard, which consists of a Postgres primary and multiple replicas. The goal is to safely promote a replica to primary with minimum disruption, covering scenarios like pod replacement, kernel upgrades, or moving the primary.
PlanetScale · 03 November 2026 · Read the original →
How Virgin Media O2 cut compute costs 30% by giving dbt a memory
Virgin Media O2 achieved significant cost and performance improvements by adopting dbt v2 and dbt State at scale. This resulted in a 30% reduction in dbt compute costs and a 35% decrease in model run times.
dbt · 03 November 2026 · Read the original →
Why an AI agent's revenue query was $3.5 million wrong, and how AWS fixed it with dbt Semantic Layer
An AI agent generated a plausible revenue query that was incorrect by $3.5 million. AWS addressed this by using the dbt Semantic Layer to provide essential business context, preventing such errors in AI-driven analytics.
dbt · 03 November 2026 · Read the original →
How OpenRouter observes, understands, and acts on 740 million LLM requests a day with ClickHouse Cloud
OpenRouter scaled its customer analytics, model rankings, internal AI agents, and routing across 500+ models using ClickHouse Cloud, handling 740 million LLM requests daily and growing monthly requests 33x to 19.7 billion. It serves as their single store for these diverse workloads.
ClickHouse · 03 November 2026 · Read the original →
pg_vexec Ahead of ClickHouse on Parquet
Igor Suhorukov's experimental `pg_vexec` extension for PostgreSQL, which adds vectorized execution, outperformed ClickHouse on Parquet data by 13x and performed closely on MergeTree data in recent benchmarks. This demonstrates PostgreSQL's growing analytical capabilities.
postgresql.org · 03 November 2026 · Read the original →
Zero-copy WAL optimizations on one end, AI agents guessing the schema on the other.
8 stories, every Tuesday
Published here every week. Follow by RSS to get it as it lands.