Newsdesk
Engineering
Data & AI
Industries
Enterprise Systems
Go-to-Market
Longform
DesignIndiaAll stories

Databases & Data

20 October 2026

50 new posts across 7 database and data engineering sources — 7 worth your time.

One week of Databases & Data, 7 stories, as published.

Performance

Álvaro Hernández: Open-source Citus matches PlanetScale's Neki at 10M QPS on half the data nodes

This post benchmarks open-source Citus against PlanetScale's published Neki benchmarks on distributed PostgreSQL workloads. It analyzes query throughput and node efficiency, demonstrating Citus achieving 10 million QPS on half the data nodes required by Neki.

Why it matters — Sharding overhead varies significantly across distributed SQL implementations; Citus shows higher per-node throughput efficiency compared to Neki at 10M QPS scale.

Planet PostgreSQL · 20 October 2026 · Read the original →

Storage Internals

What Is Inside an Iceberg Table?

The article unpacks the physical storage architecture of Apache Iceberg tables, detailing the role of top-level JSON metadata, mid-level Avro manifest files, and underlying Parquet data files. It demonstrates practical SQL techniques in DuckDB to directly query and inspect each layer of the Iceberg metadata tree when debugging query performance or state corruption.

Why it matters — Lakehouse metadata can be inspected and debugged directly using standard SQL engine functions on underlying JSON, Avro, and Parquet layers without custom SDKs.

DuckDB · 20 October 2026 · Read the original →

Practice

Hubert 'depesz' Lubaczewski: Old tech is sometimes better – a case study based on blogpost from Now-Next…

The post tests the operational tradeoffs of audit logging using PostgreSQL triggers by comparing full-row snapshot logs against diff-only column updates. Through practical experiments, it measures how storing only changed columns yields a 5x reduction in log table storage size at the cost of higher trigger CPU execution time.

Why it matters — Diff-based trigger logging reduces audit table storage by 5x but introduces query execution overhead that can degrade transaction throughput.

Planet PostgreSQL · 20 October 2026 · Read the original →

Postgres

Chris van Eijk: CREATE POLICY IF NOT EXISTS in PostgreSQL: what works

This piece explores how to execute idempotent Row Level Security (RLS) policy changes in PostgreSQL despite the lack of native CREATE POLICY IF NOT EXISTS syntax. It evaluates three migration patterns across PostgreSQL 17 and 18, analyzing lock acquisition, execution gaps, and race conditions.

Why it matters — Idempotent policy deployment requires explicit lock management or defensive block execution to prevent race conditions during schema migrations.

Planet PostgreSQL · 20 October 2026 · Read the original →

Analytics & OLAP

How Ridgeline unified Iceberg and ClickHouse into one analytics layer for investment management

Ridgeline details its production architecture overhaul, pairing ClickHouse Cloud with historical data stored in Apache Iceberg tables. This hybrid setup allows fast point queries over recent data alongside cold analytics over decades of historical records, scaling the platform to support customers 10x larger than previous limits.

Why it matters — Coupling object-storage Iceberg tables for cold historical data with ClickHouse for active analytical state allows querying multi-decade datasets without duplicating data.

ClickHouse · 20 October 2026 · Read the original →

Storage Internals

Sequences and other volatiles in Neki

This article examines the semantics and execution mechanics of sequence generators and volatile functions inside distributed SQL query planners like Neki. It delves into the trade-offs of handling side-effects and non-deterministic functions across sharded execution layers.

Why it matters — Volatile functions and sequence generation require centralized coordination or strict pushdown boundaries to prevent non-deterministic evaluation across sharded execution nodes.

PlanetScale · 20 October 2026 · Read the original →

Data Engineering

From JSON Strings to Typed Columns

The post explains how to apply dynamic JSON schema inference and row parsing to raw text columns already stored inside relational or file-based datasets. It details how DuckDB's read_json mechanics can be repurposed to process stringified event payloads into structured, typed relational columns.

Why it matters — Schema inference algorithms built for file ingestion can be applied directly to unparsed text columns to unpack JSON strings without manual schema mapping.

DuckDB · 20 October 2026 · Read the original →

Eventually every modern architecture turns back into a relational database with extra steps.

7 stories, every Tuesday

Published here every week. Follow by RSS to get it as it lands.

← Previous issue Next issue →