pg_duckdb
pg_duckdb
pg_duckdb : DuckDB Embedded in Postgres
Overview
| ID | Extension | Package | Version | Category | License | Language |
|---|---|---|---|---|---|---|
| 2430 | pg_duckdb | pg_duckdb | 1.1.0 | OLAP | MIT | C++ |
| Attribute | Has Binary | Has Library | Need Load | Has DDL | Relocatable | Trusted |
|---|---|---|---|---|---|---|
--sLd-- | No | Yes | Yes | Yes | no | no |
| Relationships | |
|---|---|
| Need By | pg_mooncake |
| See Also | pg_mooncake duckdb_fdw pg_analytics pg_parquet columnar citus citus_columnar orioledb |
conflict with duckdb_fdw
Packages
| Type | Repo | Version | PG Major Compatibility | Package Pattern | Dependencies |
|---|---|---|---|---|---|
| EXT | PIGSTY | 1.1.0 | 18 17 16 15 14 13 | pg_duckdb | - |
| RPM | PIGSTY | 1.1.0 | 18 17 16 15 14 13 | pg_duckdb_$v* | - |
| DEB | PIGSTY | 1.1.0 | 18 17 16 15 14 13 | postgresql-$v-pg-duckdb | - |
| Linux / PG | PG18 | PG17 | PG16 | PG15 | PG14 | PG13 |
|---|---|---|---|---|---|---|
el8.x86_64 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | MISS |
el8.aarch64 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | MISS |
el9.x86_64 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | MISS |
el9.aarch64 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | MISS |
el10.x86_64 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | MISS |
el10.aarch64 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | MISS |
d12.x86_64 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | MISS |
d12.aarch64 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | MISS |
d13.x86_64 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | MISS |
d13.aarch64 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | MISS |
u22.x86_64 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | MISS |
u22.aarch64 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | MISS |
u24.x86_64 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | MISS |
u24.aarch64 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | PIGSTY 1.1.0 | MISS |
Source
pig build pkg pg_duckdb; # build rpm / deb with pigInstall
Make sure PGDG and PIGSTY repo available:
pig repo add pgdg pigsty -u # add both repo and update cacheInstall this extension with pig:
pig install pg_duckdb; # install via package name, for the active PG version
pig install pg_duckdb -v 18; # install for PG 18
pig install pg_duckdb -v 17; # install for PG 17
pig install pg_duckdb -v 16; # install for PG 16
pig install pg_duckdb -v 15; # install for PG 15
pig install pg_duckdb -v 14; # install for PG 14Config this extension to shared_preload_libraries:
shared_preload_libraries = 'pg_duckdb';Create this extension with:
CREATE EXTENSION pg_duckdb;Usage
| Topic | Description |
|---|---|
| Functions | Complete reference for all available functions |
| Syntax Guide & Gotchas | Quick reference for common SQL patterns and things to know |
| Types | Supported data types and type mappings |
| Extensions | DuckDB extension installation and usage |
| Settings | Configuration options and parameters |
| Transactions | Transaction behavior and limitations |
Quick Setup
Install pg_duckdb with pig:
pig repo set
pig install pg_duckdbEdit postgresql.conf, then restart to take effect
shared_preload_libraries = 'pg_duckdb'
duckdb.allow_community_extensions = trueAccelerate Query
You can use DuckDB to query existing PostgreSQL table without modifying them:
-- pgbench -is 1000 # init some test workloads with pgbench
CREATE EXTENSION pg_duckdb;
-- default behavior, common postgres engine
SET duckdb.force_execution = false;
EXPLAIN ANALYZE SELECT count(*) FROM pgbench_accounts;
-- now the query goes to pg_duckdb
SET duckdb.force_execution = true;
EXPLAIN ANALYZE SELECT count(*) FROM pgbench_accounts;The result would be 8s -> 4s on 4c VM on local laptop) :
postgres@el9:5432/postgres=# SET duckdb.force_execution = true;
EXPLAIN ANALYZE SELECT count(*) FROM pgbench_accounts;
SET
Time: 0.206 ms
QUERY PLAN
------------------------------------------------------------------------------------------------------
Custom Scan (DuckDBScan) (cost=0.00..0.00 rows=0 width=0) (actual time=0.001..0.001 rows=0 loops=1)
DuckDB Execution Plan:
┌─────────────────────────────────────┐
│┌───────────────────────────────────┐│
││ Query Profiling Information ││
│└───────────────────────────────────┘│
└─────────────────────────────────────┘
EXPLAIN ANALYZE SELECT count(*) AS count FROM pgduckdb.public.pgbench_accounts
┌────────────────────────────────────────────────┐
│┌──────────────────────────────────────────────┐│
││ Total Time: 3.89s ││
│└──────────────────────────────────────────────┘│
└────────────────────────────────────────────────┘
┌───────────────────────────┐
│ QUERY │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│ EXPLAIN_ANALYZE │
│ ──────────────────── │
│ 0 rows │
│ (0.00s) │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│ UNGROUPED_AGGREGATE │
│ ──────────────────── │
│ Aggregates: │
│ count_star() │
│ │
│ 1 row │
│ (0.00s) │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│ TABLE_SCAN │
│ ──────────────────── │
│ Table: │
│ pgbench_accounts │
│ │
│ 100,000,000 rows │
│ (3.88s) │
└───────────────────────────┘Data Lake
Let’s play with a local minio instance:
SELECT duckdb.create_simple_secret(
type := 'S3', key_id := 's3user_data', secret := 'S3User.Data',
endpoint := 'https://sss.pigsty:9000', url_style := 'path'
);Last updated on