Newsdesk
Engineering
Data & AI
Industries
Enterprise Systems
Go-to-Market
Longform
DesignIndiaAll stories

Databases & Data

10 November 2026

83 new posts across 10 database and data engineering sources — 9 worth your time.

One week of Databases & Data, 9 stories, as published.

Postgres

Nikolay Samokhvalov: UUIDv4 vs UUIDv7: what a random primary key costs in Postgres

This post benchmarks UUIDv4, UUIDv7, and ordered primary keys on Postgres 18 using a 100M row dataset. It quantifies the performance penalty of random keys, showing UUIDv4 inserts at roughly 65% the rate of ordered keys while generating up to 1.9x the WAL volume per row.

Why it matters — Random primary keys like UUIDv4 create significant write amplification and cache churn; switching to time-ordered UUIDv7 can recover ~35% of insert throughput and nearly halve WAL generation.

Planet PostgreSQL · 10 November 2026 · Read the original →

Performance

Chris van Eijk: json vs jsonb in PostgreSQL: size and speed, measured

A detailed comparison of json and jsonb storage in Postgres 17 and 18 across six different document structures. The results show that while jsonb is often faster to query, it can be up to 41% larger than raw json depending on the document shape.

Why it matters — The common assumption that jsonb is always more efficient is false; document shape dictates whether you gain a 23% size reduction or suffer a 41% storage penalty.

Planet PostgreSQL · 10 November 2026 · Read the original →

Reliability

Fencing a Postgres primary that might still be alive

This technical deep dive explains the 'fencing' problem in database failover, where a promoted standby must ensure the old primary cannot accept writes. It details specific implementation strategies for bare metal, AWS, and GCP, and explains the rationale behind a fail-open design.

Why it matters — Promoting a standby without a robust fencing mechanism for the old primary risks split-brain scenarios and data corruption during network partitions.

ClickHouse · 10 November 2026 · Read the original →

Storage Internals

Lakebase Postgres is built on object storage, but 99.99999% of reads never touch it

The post examines the architecture of running Postgres on top of object storage like S3. It explains how a multi-tiered caching strategy allows 99.99999% of reads to be served from local memory or NVMe, bypassing the high latency of the object store.

Why it matters — Architectural caching can make high-latency, low-cost object storage viable for primary database workloads without sacrificing millisecond-level read performance.

Neon · 10 November 2026 · Read the original →

Performance

The hidden latency inside your database

An analysis of the 'hidden' network latency that exists inside cloud-managed database clusters, specifically between the compute engine and network-attached storage (NAS). It argues that this internal hop is often a larger bottleneck than the distance between the application and the database.

Why it matters — Optimizing application-to-database latency is futile if the database itself is bottlenecked by the millisecond-scale network hop to its own storage volumes.

PlanetScale · 10 November 2026 · Read the original →

Practice

Richard Yen: An Index Scan Is Only Half the Story

A migration case study where a query's performance degraded from tens of milliseconds to over a second despite identical schemas and indexes. It explores how infrastructure environment differences and query planner behavior can diverge during cloud migrations.

Why it matters — Identical SQL and indexing do not guarantee performance parity; environmental factors like underlying hardware and planner statistics can cause 100x latency regressions during migrations.

Planet PostgreSQL · 10 November 2026 · Read the original →

Streaming & CDC

What I learned from a month with WalShadow on ClickHouse Managed Postgres

A report on a month-long test of WalShadow for Postgres-to-ClickHouse replication. It provides a direct comparison of replication latency against standard pipes and evaluates how the system handles schema changes and data recovery.

Why it matters — Real-world replication testing reveals the trade-offs between managed 'pipes' and custom CDC implementations regarding latency, schema drift, and operational recovery.

ClickHouse · 10 November 2026 · Read the original →

Data Engineering

Snowflake Delivers Autonomous Data Engineering

Snowflake announced new autonomous data engineering and real-time analytics innovations at Expedition 2026. These capabilities leverage AI agents to automate recurring data pipeline fixes, enabling end-to-end real-time analytics with a median data-readiness latency of 3 seconds for dashboards, applications, and AI agents.

Why it matters — Data engineers can leverage Snowflake's new AI agents to automate pipeline fixes, reducing manual effort and achieving 3-second data readiness for real-time analytics.

Snowflake · 10 November 2026 · Read the original →

Streaming & CDC

Confluent Introduces Streamhouse Architecture

Confluent introduced its Streamhouse architecture, a new data architecture designed to provide real-time business context for AI agents and applications. With Lightning Tables now generally available on AWS, the platform offers low-latency SQL and API access to live streaming data, with changes available within hundreds of milliseconds and queries returning in 10-100 ms at over 1,000 queries per second.

Why it matters — Teams building AI agents and real-time applications can now query live streaming data directly in Confluent with 10-100ms latency, eliminating separate ETL and databases.

confluent.io · 10 November 2026 · Read the original →

Still micro-optimizing Postgres while the vendors invent new words for databases.

9 stories, every Tuesday

Published here every week. Follow by RSS to get it as it lands.

← Previous issue Next issue →