pg_duckdb

pg_duckdb

DuckDB Embedded in Postgres

Overview

IDExtensionPackageVersionCategoryLicenseLanguage
2430
pg_duckdb
pg_duckdb
1.1.0
OLAP
MIT
C++
AttributeHas BinaryHas LibraryNeed LoadHas DDLRelocatableTrusted
--sLd--
No
Yes
Yes
Yes
no
no
Relationships
Need By
pg_mooncake
See Also
pg_mooncake
duckdb_fdw
pg_analytics
pg_parquet
columnar
citus
citus_columnar
orioledb

conflict with duckdb_fdw

Packages

TypeRepoVersionPG Major AvailabilityPackage PatternDependencies
EL
PIGSTY
1.1.0
18
17
16
15
14
13
pg_duckdb_$v*-
Debian
PIGSTY
1.1.0
18
17
16
15
14
13
postgresql-$v-pg-duckdb-
Linux / PGPG18PG17PG16PG15PG14PG13
el8.x86_64
PIGSTY 1.1.0
PIGSTY 1.1.0
PIGSTY 1.1.0
PIGSTY 1.1.0
PIGSTY 1.1.0
MISS
el8.aarch64
PIGSTY 1.1.0
PIGSTY 1.1.0
PIGSTY 1.1.0
PIGSTY 1.1.0
PIGSTY 1.1.0
MISS
el9.x86_64
PIGSTY 1.1.0
PIGSTY 1.1.0
PIGSTY 1.1.0
PIGSTY 1.1.0
PIGSTY 1.1.0
MISS
el9.aarch64
PIGSTY 1.1.0
PIGSTY 1.1.0
PIGSTY 1.1.0
PIGSTY 1.1.0
PIGSTY 1.1.0
MISS
el10.x86_64
PIGSTY 1.1.0
PIGSTY 1.1.0
PIGSTY 1.1.0
PIGSTY 1.1.0
PIGSTY 1.1.0
MISS
el10.aarch64
PIGSTY 1.1.0
PIGSTY 1.1.0
PIGSTY 1.1.0
PIGSTY 1.1.0
PIGSTY 1.1.0
MISS
d12.x86_64
MISS
PIGSTY 0.3.1
PIGSTY 0.3.1
PIGSTY 0.3.1
PIGSTY 0.3.1
MISS
d12.aarch64
MISS
PIGSTY 0.3.1
PIGSTY 0.3.1
PIGSTY 0.3.1
PIGSTY 0.3.1
MISS
d13.x86_64
MISS
MISS
MISS
MISS
MISS
MISS
d13.aarch64
MISS
MISS
MISS
MISS
MISS
MISS
u22.x86_64
MISS
PIGSTY 0.3.1
PIGSTY 0.3.1
PIGSTY 0.3.1
PIGSTY 0.3.1
MISS
u22.aarch64
MISS
PIGSTY 0.3.1
PIGSTY 0.3.1
PIGSTY 0.3.1
PIGSTY 0.3.1
MISS
u24.x86_64
MISS
PIGSTY 0.3.1
PIGSTY 0.3.1
PIGSTY 0.3.1
PIGSTY 0.3.1
MISS
u24.aarch64
MISS
PIGSTY 0.3.1
PIGSTY 0.3.1
PIGSTY 0.3.1
PIGSTY 0.3.1
MISS
PackageVersionOSORGSIZEFile URL
pg_duckdb_181.1.0el8.x86_64pigsty15.4 MiBpg_duckdb_18-1.1.0-1PIGSTY.el8.x86_64.rpm
pg_duckdb_181.1.0el8.aarch64pigsty13.3 MiBpg_duckdb_18-1.1.0-1PIGSTY.el8.aarch64.rpm
pg_duckdb_181.1.0el9.x86_64pigsty15.8 MiBpg_duckdb_18-1.1.0-1PIGSTY.el9.x86_64.rpm
pg_duckdb_181.1.0el9.aarch64pigsty14.2 MiBpg_duckdb_18-1.1.0-1PIGSTY.el9.aarch64.rpm
pg_duckdb_181.1.0el10.x86_64pigsty15.0 MiBpg_duckdb_18-1.1.0-1PIGSTY.el10.x86_64.rpm
pg_duckdb_181.1.0el10.aarch64pigsty13.3 MiBpg_duckdb_18-1.1.0-1PIGSTY.el10.aarch64.rpm
PackageVersionOSORGSIZEFile URL
pg_duckdb_171.1.0el8.x86_64pigsty15.4 MiBpg_duckdb_17-1.1.0-1PIGSTY.el8.x86_64.rpm
pg_duckdb_171.1.0el8.aarch64pigsty13.3 MiBpg_duckdb_17-1.1.0-1PIGSTY.el8.aarch64.rpm
pg_duckdb_171.1.0el9.x86_64pigsty15.8 MiBpg_duckdb_17-1.1.0-1PIGSTY.el9.x86_64.rpm
pg_duckdb_170.3.1el9.x86_64pigsty14.2 MiBpg_duckdb_17-0.3.1-1PIGSTY.el9.x86_64.rpm
pg_duckdb_171.1.0el9.aarch64pigsty14.2 MiBpg_duckdb_17-1.1.0-1PIGSTY.el9.aarch64.rpm
pg_duckdb_170.3.1el9.aarch64pigsty12.9 MiBpg_duckdb_17-0.3.1-1PIGSTY.el9.aarch64.rpm
pg_duckdb_171.1.0el10.x86_64pigsty15.0 MiBpg_duckdb_17-1.1.0-1PIGSTY.el10.x86_64.rpm
pg_duckdb_171.1.0el10.aarch64pigsty13.3 MiBpg_duckdb_17-1.1.0-1PIGSTY.el10.aarch64.rpm
postgresql-17-pg-duckdb0.3.1d12.x86_64pigsty12.3 MiBpostgresql-17-pg-duckdb_0.3.1-1PIGSTY~bookworm_amd64.deb
postgresql-17-pg-duckdb0.3.1d12.aarch64pigsty10.7 MiBpostgresql-17-pg-duckdb_0.3.1-1PIGSTY~bookworm_arm64.deb
postgresql-17-pg-duckdb0.3.1u22.x86_64pigsty14.2 MiBpostgresql-17-pg-duckdb_0.3.1-1PIGSTY~jammy_amd64.deb
postgresql-17-pg-duckdb0.3.1u22.aarch64pigsty13.0 MiBpostgresql-17-pg-duckdb_0.3.1-1PIGSTY~jammy_arm64.deb
postgresql-17-pg-duckdb0.3.1u24.x86_64pigsty14.1 MiBpostgresql-17-pg-duckdb_0.3.1-1PIGSTY~noble_amd64.deb
postgresql-17-pg-duckdb0.3.1u24.aarch64pigsty13.0 MiBpostgresql-17-pg-duckdb_0.3.1-1PIGSTY~noble_arm64.deb
PackageVersionOSORGSIZEFile URL
pg_duckdb_161.1.0el8.x86_64pigsty15.4 MiBpg_duckdb_16-1.1.0-1PIGSTY.el8.x86_64.rpm
pg_duckdb_161.1.0el8.aarch64pigsty13.3 MiBpg_duckdb_16-1.1.0-1PIGSTY.el8.aarch64.rpm
pg_duckdb_161.1.0el9.x86_64pigsty15.8 MiBpg_duckdb_16-1.1.0-1PIGSTY.el9.x86_64.rpm
pg_duckdb_160.3.1el9.x86_64pigsty14.2 MiBpg_duckdb_16-0.3.1-1PIGSTY.el9.x86_64.rpm
pg_duckdb_161.1.0el9.aarch64pigsty14.2 MiBpg_duckdb_16-1.1.0-1PIGSTY.el9.aarch64.rpm
pg_duckdb_160.3.1el9.aarch64pigsty12.9 MiBpg_duckdb_16-0.3.1-1PIGSTY.el9.aarch64.rpm
pg_duckdb_161.1.0el10.x86_64pigsty15.0 MiBpg_duckdb_16-1.1.0-1PIGSTY.el10.x86_64.rpm
pg_duckdb_161.1.0el10.aarch64pigsty13.3 MiBpg_duckdb_16-1.1.0-1PIGSTY.el10.aarch64.rpm
postgresql-16-pg-duckdb0.3.1d12.x86_64pigsty12.3 MiBpostgresql-16-pg-duckdb_0.3.1-1PIGSTY~bookworm_amd64.deb
postgresql-16-pg-duckdb0.3.1d12.aarch64pigsty10.7 MiBpostgresql-16-pg-duckdb_0.3.1-1PIGSTY~bookworm_arm64.deb
postgresql-16-pg-duckdb0.3.1u22.x86_64pigsty14.1 MiBpostgresql-16-pg-duckdb_0.3.1-1PIGSTY~jammy_amd64.deb
postgresql-16-pg-duckdb0.3.1u22.aarch64pigsty13.0 MiBpostgresql-16-pg-duckdb_0.3.1-1PIGSTY~jammy_arm64.deb
postgresql-16-pg-duckdb0.3.1u24.x86_64pigsty14.1 MiBpostgresql-16-pg-duckdb_0.3.1-1PIGSTY~noble_amd64.deb
postgresql-16-pg-duckdb0.3.1u24.aarch64pigsty12.9 MiBpostgresql-16-pg-duckdb_0.3.1-1PIGSTY~noble_arm64.deb
PackageVersionOSORGSIZEFile URL
pg_duckdb_151.1.0el8.x86_64pigsty15.4 MiBpg_duckdb_15-1.1.0-1PIGSTY.el8.x86_64.rpm
pg_duckdb_151.1.0el8.aarch64pigsty13.3 MiBpg_duckdb_15-1.1.0-1PIGSTY.el8.aarch64.rpm
pg_duckdb_151.1.0el9.x86_64pigsty15.8 MiBpg_duckdb_15-1.1.0-1PIGSTY.el9.x86_64.rpm
pg_duckdb_150.3.1el9.x86_64pigsty14.2 MiBpg_duckdb_15-0.3.1-1PIGSTY.el9.x86_64.rpm
pg_duckdb_151.1.0el9.aarch64pigsty14.2 MiBpg_duckdb_15-1.1.0-1PIGSTY.el9.aarch64.rpm
pg_duckdb_150.3.1el9.aarch64pigsty12.9 MiBpg_duckdb_15-0.3.1-1PIGSTY.el9.aarch64.rpm
pg_duckdb_151.1.0el10.x86_64pigsty15.1 MiBpg_duckdb_15-1.1.0-1PIGSTY.el10.x86_64.rpm
pg_duckdb_151.1.0el10.aarch64pigsty13.4 MiBpg_duckdb_15-1.1.0-1PIGSTY.el10.aarch64.rpm
postgresql-15-pg-duckdb0.3.1d12.x86_64pigsty12.4 MiBpostgresql-15-pg-duckdb_0.3.1-1PIGSTY~bookworm_amd64.deb
postgresql-15-pg-duckdb0.3.1d12.aarch64pigsty10.7 MiBpostgresql-15-pg-duckdb_0.3.1-1PIGSTY~bookworm_arm64.deb
postgresql-15-pg-duckdb0.3.1u22.x86_64pigsty14.2 MiBpostgresql-15-pg-duckdb_0.3.1-1PIGSTY~jammy_amd64.deb
postgresql-15-pg-duckdb0.3.1u22.aarch64pigsty13.0 MiBpostgresql-15-pg-duckdb_0.3.1-1PIGSTY~jammy_arm64.deb
postgresql-15-pg-duckdb0.3.1u24.x86_64pigsty14.1 MiBpostgresql-15-pg-duckdb_0.3.1-1PIGSTY~noble_amd64.deb
postgresql-15-pg-duckdb0.3.1u24.aarch64pigsty13.0 MiBpostgresql-15-pg-duckdb_0.3.1-1PIGSTY~noble_arm64.deb
PackageVersionOSORGSIZEFile URL
pg_duckdb_141.1.0el8.x86_64pigsty15.4 MiBpg_duckdb_14-1.1.0-1PIGSTY.el8.x86_64.rpm
pg_duckdb_141.1.0el8.aarch64pigsty13.3 MiBpg_duckdb_14-1.1.0-1PIGSTY.el8.aarch64.rpm
pg_duckdb_141.1.0el9.x86_64pigsty15.8 MiBpg_duckdb_14-1.1.0-1PIGSTY.el9.x86_64.rpm
pg_duckdb_140.3.1el9.x86_64pigsty14.2 MiBpg_duckdb_14-0.3.1-1PIGSTY.el9.x86_64.rpm
pg_duckdb_141.1.0el9.aarch64pigsty14.2 MiBpg_duckdb_14-1.1.0-1PIGSTY.el9.aarch64.rpm
pg_duckdb_140.3.1el9.aarch64pigsty12.9 MiBpg_duckdb_14-0.3.1-1PIGSTY.el9.aarch64.rpm
pg_duckdb_141.1.0el10.x86_64pigsty15.1 MiBpg_duckdb_14-1.1.0-1PIGSTY.el10.x86_64.rpm
pg_duckdb_141.1.0el10.aarch64pigsty13.4 MiBpg_duckdb_14-1.1.0-1PIGSTY.el10.aarch64.rpm
postgresql-14-pg-duckdb0.3.1d12.x86_64pigsty12.4 MiBpostgresql-14-pg-duckdb_0.3.1-1PIGSTY~bookworm_amd64.deb
postgresql-14-pg-duckdb0.3.1d12.aarch64pigsty10.7 MiBpostgresql-14-pg-duckdb_0.3.1-1PIGSTY~bookworm_arm64.deb
postgresql-14-pg-duckdb0.3.1u22.x86_64pigsty14.1 MiBpostgresql-14-pg-duckdb_0.3.1-1PIGSTY~jammy_amd64.deb
postgresql-14-pg-duckdb0.3.1u22.aarch64pigsty13.0 MiBpostgresql-14-pg-duckdb_0.3.1-1PIGSTY~jammy_arm64.deb
postgresql-14-pg-duckdb0.3.1u24.x86_64pigsty14.1 MiBpostgresql-14-pg-duckdb_0.3.1-1PIGSTY~noble_amd64.deb
postgresql-14-pg-duckdb0.3.1u24.aarch64pigsty13.0 MiBpostgresql-14-pg-duckdb_0.3.1-1PIGSTY~noble_arm64.deb

Source

pig build get pg_duckdb; # get pg_duckdb source code
pig build dep pg_duckdb; # install build dependencies
pig build pkg pg_duckdb; # build extension rpm or deb
pig build ext pg_duckdb; # build extension rpms

Install

To add the required PGDG / PIGSTY upstream repository, use:

pig repo add pgsql -u   # add PGDG + Pigsty repo and update cache (leave existing repos)

Install this extension with:

pig ext install pg_duckdb; # install by extension name, for the current active PG version
pig ext install pg_duckdb; # install via package alias, for the active PG version
pig ext install pg_duckdb -v 18;   # install for PG 18
pig ext install pg_duckdb -v 17;   # install for PG 17
pig ext install pg_duckdb -v 16;   # install for PG 16
pig ext install pg_duckdb -v 15;   # install for PG 15
pig ext install pg_duckdb -v 14;   # install for PG 14

Create this extension with:

CREATE EXTENSION pg_duckdb;

Usage

TopicDescription
FunctionsComplete reference for all available functions
Syntax Guide & GotchasQuick reference for common SQL patterns and things to know
TypesSupported data types and type mappings
ExtensionsDuckDB extension installation and usage
SettingsConfiguration options and parameters
TransactionsTransaction behavior and limitations

Quick Setup

Install pg_duckdb with pig:

pig repo add pigsty -u
pig install pg_duckdb

Edit postgresql.conf, then restart to take effect

shared_preload_libraries = 'pg_duckdb,pg_mooncake'
duckdb.allow_community_extensions = true
wal_level = logical

Accelerate Query

You can use DuckDB to query existing PostgreSQL table without modifying them:

-- pgbench -is 1000  # init some test workloads with pgbench
CREATE EXTENSION pg_duckdb;

-- default behavior, common postgres engine
SET duckdb.force_execution = false;
EXPLAIN ANALYZE SELECT count(*) FROM pgbench_accounts;

-- now the query goes to pg_duckdb
SET duckdb.force_execution = true;
EXPLAIN ANALYZE SELECT count(*) FROM pgbench_accounts;

The result would be 8s -> 4s on 4c VM on local laptop) :

postgres@el9:5432/postgres=# SET duckdb.force_execution = true;
EXPLAIN ANALYZE SELECT count(*) FROM pgbench_accounts;
SET
Time: 0.206 ms
                                              QUERY PLAN
------------------------------------------------------------------------------------------------------
 Custom Scan (DuckDBScan)  (cost=0.00..0.00 rows=0 width=0) (actual time=0.001..0.001 rows=0 loops=1)
   DuckDB Execution Plan:

 ┌─────────────────────────────────────┐
 │┌───────────────────────────────────┐│
 ││    Query Profiling Information    ││
 │└───────────────────────────────────┘│
 └─────────────────────────────────────┘
 EXPLAIN ANALYZE  SELECT count(*) AS count FROM pgduckdb.public.pgbench_accounts
 ┌────────────────────────────────────────────────┐
 │┌──────────────────────────────────────────────┐│
 ││               Total Time: 3.89s              ││
 │└──────────────────────────────────────────────┘│
 └────────────────────────────────────────────────┘
 ┌───────────────────────────┐
 │           QUERY           │
 └─────────────┬─────────────┘
 ┌─────────────┴─────────────┐
 │      EXPLAIN_ANALYZE      │
 │    ────────────────────   │
 │           0 rows          │
 │          (0.00s)          │
 └─────────────┬─────────────┘
 ┌─────────────┴─────────────┐
 │    UNGROUPED_AGGREGATE    │
 │    ────────────────────   │
 │        Aggregates:        │
 │        count_star()       │
 │                           │
 │           1 row           │
 │          (0.00s)          │
 └─────────────┬─────────────┘
 ┌─────────────┴─────────────┐
 │         TABLE_SCAN        │
 │    ────────────────────   │
 │           Table:          │
 │      pgbench_accounts     │
 │                           │
 │      100,000,000 rows     │
 │          (3.88s)          │
 └───────────────────────────┘

Data Lake

Let’s play with a local minio instance:

SELECT duckdb.create_simple_secret(
    type := 'S3', key_id := 's3user_data', secret := 'S3User.Data',
    endpoint := 'https://sss.pigsty:9000', url_style := 'path' 
);
Last updated on