Newsdesk
Engineering
Data & AI
Industries
Enterprise Systems
Go-to-Market
Longform
DesignIndiaAll stories

Databases & Data

01 September 2026

55 new posts across 9 database and data engineering sources — 9 worth your time.

One week of Databases & Data, 9 stories, as published.

Reliability

How one connection kills a database

Analyzes a classic database outage scenario where an uncommitted transaction holding an exclusive lock on a table blocks a pending schema change. This block cascades, preventing all subsequent queries from executing and ultimately taking down the database.

Why it matters — An uncommitted transaction combined with a pending DDL lock request can completely block all read and write traffic, making lock timeout configurations critical.

PlanetScale · 01 September 2026 · Read the original →

Performance

Andrew Atkinson: PostgreSQL 18: 23x Faster Inserts With UUID V7

Shares a real-world migration from UUID v1 and v4 primary keys to UUID v7, resulting in 23x faster inserts. It discusses the performance benefits of sequential keys and the locking trade-offs of altering column defaults.

Why it matters — Switching to sequential UUID v7 keys restores B-tree index locality and drastically improves write performance, but altering column defaults requires an exclusive table lock.

Planet PostgreSQL · 01 September 2026 · Read the original →

Postgres

Radim Marek: Read your own writes, off the primary

Addresses the common issue of replication lag where a user writes to the primary database but immediately reads stale data from an asynchronous replica. It explores architectural patterns to ensure read-your-own-writes consistency.

Why it matters — Relying on asynchronous replicas for immediate subsequent reads leads to transient data invisibility unless the application routes post-write reads back to the primary or tracks replication lag.

Planet PostgreSQL · 01 September 2026 · Read the original →

Performance

semab tariq: How to Calculate Fillfactor for Your Tables

Provides a practical guide and mathematical formula to calculate the optimal fillfactor for PostgreSQL tables. Adjusting this setting leaves free space on pages to accommodate in-place (HOT) updates, reducing write amplification.

Why it matters — Using the formula 100 - (average row size / 8192 * 100) to set fillfactor enables Heap-Only Tuple (HOT) updates, significantly cutting down WAL volume and disk I/O.

Planet PostgreSQL · 01 September 2026 · Read the original →

Storage Internals

How DuckDB Runs Recursive CTEs Faster

Deep dives into DuckDB's query engine internals to show how optimizing the scope of reusable runtime state speeds up recursive CTEs. By keeping state active across iterations instead of rebuilding it, the engine avoids redundant evaluation overhead.

Why it matters — Scoping runtime evaluation state across iterations in recursive CTEs prevents redundant work and significantly accelerates graph traversal queries.

DuckDB · 01 September 2026 · Read the original →

Streaming & CDC

Ensuring reliable OpenTelemetry ingestion at scale

Explains the architecture and engineering decisions behind ClickHouse Cloud's ingestion pipeline, which processes 50 million OpenTelemetry events per second. It focuses on maintaining reliability and performance at extreme scale.

Why it matters — Scaling telemetry ingestion to 50 million events per second requires robust buffering and batching strategies to prevent backpressure from destabilizing the storage layer.

ClickHouse · 01 September 2026 · Read the original →

Practice

Problems with large tables in Postgres

Examines how tables that grow excessively large, wide, or 'fat' (containing oversized out-of-line values) degrade database performance in distinct, predictable ways. It outlines the specific operational bottlenecks associated with each type of growth.

Why it matters — Table growth degrades performance differently depending on whether it is driven by row count, column count, or oversized values, requiring distinct mitigation strategies like partitioning or normalization.

PlanetScale · 01 September 2026 · Read the original →

Performance

GPU-Accelerated Spark on EMR for Faster Analytics

Amazon EMR on EKS now runs Apache Spark workloads up to 3.7x faster and 31% cheaper on Amazon EC2 G7 instances with NVIDIA RTX PRO 4500 GPUs compared to comparable CPU instances. In TPC-DS 3 TB benchmarks, G7 instances completed the 103-query power run in 4.7 minutes, saving 750 seconds per run.

Why it matters — Data engineers can significantly reduce Spark job run times and operational costs for large-scale ETL and ML pipelines without modifying existing code.

aws.amazon.com · 01 September 2026 · Read the original →

Streaming & CDC

Oracle XStream CDC to Kafka for High-Volume Streaming

The Oracle XStream CDC Source connector for Confluent Platform captures changes from Oracle databases to Apache Kafka topics, supporting high-volume scenarios of over 150 GB of CDC data per hour using Oracle's XStream API. It offers two topologies, including downstream capture to offload capture workload from the source OLTP database, ensuring production performance is not affected.

Why it matters — Teams can implement robust, high-throughput real-time data synchronization from Oracle databases to Kafka, minimizing impact on production OLTP systems.

Confluent · 01 September 2026 · Read the original →

We will do absolutely anything to avoid just buying a larger database instance.

9 stories, every Tuesday

Published here every week. Follow by RSS to get it as it lands.

← Previous issue Next issue →