pg_fact_loader

pg_fact_loader

pg_fact_loader : build fact tables with Postgres

Overview

IDExtensionPackageVersionCategoryLicenseLanguage
9820
pg_fact_loader
pg_fact_loader
2.0.1
ETL
MIT
C
AttributeHas BinaryHas LibraryNeed LoadHas DDLRelocatableTrusted
--s-d--
No
Yes
No
Yes
no
no
Relationships
Schemasfact_loader
See Also
pg_cron
pg_partman
pg_jobmon
mimeo
timescaledb
citus
tablefunc
pg_bulkload

Packages

TypeRepoVersionPG Major CompatibilityPackage PatternDependencies
EXT
PGDG
2.0.1
18
17
16
15
14
pg_fact_loader-
RPM
PGDG
2.0.1
18
17
16
15
14
pg_fact_loader_$v-
DEB
PGDG
2.0.1
18
17
16
15
14
postgresql-$v-pg-fact-loader-
Linux / PGPG18PG17PG16PG15PG14
el8.x86_64
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
el8.aarch64
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
el9.x86_64
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
el9.aarch64
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
el10.x86_64
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
el10.aarch64
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
d12.x86_64
MISS
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
d12.aarch64
MISS
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
d13.x86_64
MISS
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
d13.aarch64
MISS
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
u22.x86_64
MISS
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
u22.aarch64
MISS
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
u24.x86_64
MISS
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
u24.aarch64
MISS
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
PGDG 2.0.1
PackageVersionOSORGSIZEFile URL
pg_fact_loader_182.0.1el8.x86_64pgdg36.2 KiBpg_fact_loader_18-2.0.1-3PGDG.rhel8.noarch.rpm
pg_fact_loader_182.0.1el8.aarch64pgdg36.1 KiBpg_fact_loader_18-2.0.1-3PGDG.rhel8.noarch.rpm
pg_fact_loader_182.0.1el9.x86_64pgdg34.6 KiBpg_fact_loader_18-2.0.1-3PGDG.rhel9.noarch.rpm
pg_fact_loader_182.0.1el9.aarch64pgdg34.5 KiBpg_fact_loader_18-2.0.1-3PGDG.rhel9.noarch.rpm
pg_fact_loader_182.0.1el10.x86_64pgdg35.1 KiBpg_fact_loader_18-2.0.1-3PGDG.rhel10.noarch.rpm
pg_fact_loader_182.0.1el10.aarch64pgdg35.0 KiBpg_fact_loader_18-2.0.1-3PGDG.rhel10.noarch.rpm
PackageVersionOSORGSIZEFile URL
pg_fact_loader_172.0.1el8.x86_64pgdg36.2 KiBpg_fact_loader_17-2.0.1-3PGDG.rhel8.noarch.rpm
pg_fact_loader_172.0.1el8.x86_64pgdg36.0 KiBpg_fact_loader_17-2.0.1-2PGDG.rhel8.noarch.rpm
pg_fact_loader_172.0.1el8.aarch64pgdg36.1 KiBpg_fact_loader_17-2.0.1-3PGDG.rhel8.noarch.rpm
pg_fact_loader_172.0.1el8.aarch64pgdg36.0 KiBpg_fact_loader_17-2.0.1-2PGDG.rhel8.noarch.rpm
pg_fact_loader_172.0.1el9.x86_64pgdg34.6 KiBpg_fact_loader_17-2.0.1-3PGDG.rhel9.noarch.rpm
pg_fact_loader_172.0.1el9.x86_64pgdg34.5 KiBpg_fact_loader_17-2.0.1-2PGDG.rhel9.noarch.rpm
pg_fact_loader_172.0.1el9.aarch64pgdg34.5 KiBpg_fact_loader_17-2.0.1-3PGDG.rhel9.noarch.rpm
pg_fact_loader_172.0.1el9.aarch64pgdg34.4 KiBpg_fact_loader_17-2.0.1-2PGDG.rhel9.noarch.rpm
pg_fact_loader_172.0.1el10.x86_64pgdg35.1 KiBpg_fact_loader_17-2.0.1-3PGDG.rhel10.noarch.rpm
pg_fact_loader_172.0.1el10.aarch64pgdg35.0 KiBpg_fact_loader_17-2.0.1-3PGDG.rhel10.noarch.rpm
postgresql-17-pg-fact-loader2.0.1d12.x86_64pgdg40.5 KiBpostgresql-17-pg-fact-loader_2.0.1-5.pgdg120+1_amd64.deb
postgresql-17-pg-fact-loader2.0.1d12.aarch64pgdg40.5 KiBpostgresql-17-pg-fact-loader_2.0.1-5.pgdg120+1_arm64.deb
postgresql-17-pg-fact-loader2.0.1d13.x86_64pgdg40.5 KiBpostgresql-17-pg-fact-loader_2.0.1-5.pgdg130+1_amd64.deb
postgresql-17-pg-fact-loader2.0.1d13.aarch64pgdg40.5 KiBpostgresql-17-pg-fact-loader_2.0.1-5.pgdg130+1_arm64.deb
postgresql-17-pg-fact-loader2.0.1u22.x86_64pgdg40.6 KiBpostgresql-17-pg-fact-loader_2.0.1-5.pgdg22.04+1_amd64.deb
postgresql-17-pg-fact-loader2.0.1u22.aarch64pgdg40.6 KiBpostgresql-17-pg-fact-loader_2.0.1-5.pgdg22.04+1_arm64.deb
postgresql-17-pg-fact-loader2.0.1u24.x86_64pgdg40.5 KiBpostgresql-17-pg-fact-loader_2.0.1-5.pgdg24.04+1_amd64.deb
postgresql-17-pg-fact-loader2.0.1u24.aarch64pgdg40.5 KiBpostgresql-17-pg-fact-loader_2.0.1-5.pgdg24.04+1_arm64.deb
PackageVersionOSORGSIZEFile URL
pg_fact_loader_162.0.1el8.x86_64pgdg36.2 KiBpg_fact_loader_16-2.0.1-3PGDG.rhel8.noarch.rpm
pg_fact_loader_162.0.1el8.x86_64pgdg36.0 KiBpg_fact_loader_16-2.0.1-2PGDG.rhel8.noarch.rpm
pg_fact_loader_162.0.1el8.x86_64pgdg36.0 KiBpg_fact_loader_16-2.0.1-1PGDG.f42.noarch.rpm
pg_fact_loader_162.0.1el8.aarch64pgdg36.1 KiBpg_fact_loader_16-2.0.1-3PGDG.rhel8.noarch.rpm
pg_fact_loader_162.0.1el8.aarch64pgdg36.0 KiBpg_fact_loader_16-2.0.1-2PGDG.rhel8.noarch.rpm
pg_fact_loader_162.0.1el9.x86_64pgdg34.6 KiBpg_fact_loader_16-2.0.1-3PGDG.rhel9.noarch.rpm
pg_fact_loader_162.0.1el9.x86_64pgdg34.5 KiBpg_fact_loader_16-2.0.1-2PGDG.rhel9.noarch.rpm
pg_fact_loader_162.0.1el9.aarch64pgdg34.5 KiBpg_fact_loader_16-2.0.1-3PGDG.rhel9.noarch.rpm
pg_fact_loader_162.0.1el9.aarch64pgdg34.3 KiBpg_fact_loader_16-2.0.1-2PGDG.rhel9.noarch.rpm
pg_fact_loader_162.0.1el10.x86_64pgdg35.1 KiBpg_fact_loader_16-2.0.1-3PGDG.rhel10.noarch.rpm
pg_fact_loader_162.0.1el10.aarch64pgdg35.0 KiBpg_fact_loader_16-2.0.1-3PGDG.rhel10.noarch.rpm
postgresql-16-pg-fact-loader2.0.1d12.x86_64pgdg40.5 KiBpostgresql-16-pg-fact-loader_2.0.1-5.pgdg120+1_amd64.deb
postgresql-16-pg-fact-loader2.0.1d12.aarch64pgdg40.5 KiBpostgresql-16-pg-fact-loader_2.0.1-5.pgdg120+1_arm64.deb
postgresql-16-pg-fact-loader2.0.1d13.x86_64pgdg40.5 KiBpostgresql-16-pg-fact-loader_2.0.1-5.pgdg130+1_amd64.deb
postgresql-16-pg-fact-loader2.0.1d13.aarch64pgdg40.5 KiBpostgresql-16-pg-fact-loader_2.0.1-5.pgdg130+1_arm64.deb
postgresql-16-pg-fact-loader2.0.1u22.x86_64pgdg40.6 KiBpostgresql-16-pg-fact-loader_2.0.1-5.pgdg22.04+1_amd64.deb
postgresql-16-pg-fact-loader2.0.1u22.aarch64pgdg40.6 KiBpostgresql-16-pg-fact-loader_2.0.1-5.pgdg22.04+1_arm64.deb
postgresql-16-pg-fact-loader2.0.1u24.x86_64pgdg40.5 KiBpostgresql-16-pg-fact-loader_2.0.1-5.pgdg24.04+1_amd64.deb
postgresql-16-pg-fact-loader2.0.1u24.aarch64pgdg40.5 KiBpostgresql-16-pg-fact-loader_2.0.1-5.pgdg24.04+1_arm64.deb
PackageVersionOSORGSIZEFile URL
pg_fact_loader_152.0.1el8.x86_64pgdg36.2 KiBpg_fact_loader_15-2.0.1-3PGDG.rhel8.noarch.rpm
pg_fact_loader_152.0.1el8.x86_64pgdg36.0 KiBpg_fact_loader_15-2.0.1-2PGDG.rhel8.noarch.rpm
pg_fact_loader_152.0.1el8.x86_64pgdg36.0 KiBpg_fact_loader_15-2.0.1-1PGDG.f42.noarch.rpm
pg_fact_loader_152.0.1el8.aarch64pgdg36.1 KiBpg_fact_loader_15-2.0.1-3PGDG.rhel8.noarch.rpm
pg_fact_loader_152.0.1el8.aarch64pgdg36.0 KiBpg_fact_loader_15-2.0.1-2PGDG.rhel8.noarch.rpm
pg_fact_loader_152.0.1el9.x86_64pgdg34.6 KiBpg_fact_loader_15-2.0.1-3PGDG.rhel9.noarch.rpm
pg_fact_loader_152.0.1el9.x86_64pgdg34.5 KiBpg_fact_loader_15-2.0.1-2PGDG.rhel9.noarch.rpm
pg_fact_loader_152.0.1el9.aarch64pgdg34.6 KiBpg_fact_loader_15-2.0.1-3PGDG.rhel9.noarch.rpm
pg_fact_loader_152.0.1el9.aarch64pgdg34.3 KiBpg_fact_loader_15-2.0.1-2PGDG.rhel9.noarch.rpm
pg_fact_loader_152.0.1el10.x86_64pgdg35.1 KiBpg_fact_loader_15-2.0.1-3PGDG.rhel10.noarch.rpm
pg_fact_loader_152.0.1el10.aarch64pgdg35.0 KiBpg_fact_loader_15-2.0.1-3PGDG.rhel10.noarch.rpm
postgresql-15-pg-fact-loader2.0.1d12.x86_64pgdg40.5 KiBpostgresql-15-pg-fact-loader_2.0.1-5.pgdg120+1_amd64.deb
postgresql-15-pg-fact-loader2.0.1d12.aarch64pgdg40.5 KiBpostgresql-15-pg-fact-loader_2.0.1-5.pgdg120+1_arm64.deb
postgresql-15-pg-fact-loader2.0.1d13.x86_64pgdg40.5 KiBpostgresql-15-pg-fact-loader_2.0.1-5.pgdg130+1_amd64.deb
postgresql-15-pg-fact-loader2.0.1d13.aarch64pgdg40.5 KiBpostgresql-15-pg-fact-loader_2.0.1-5.pgdg130+1_arm64.deb
postgresql-15-pg-fact-loader2.0.1u22.x86_64pgdg40.6 KiBpostgresql-15-pg-fact-loader_2.0.1-5.pgdg22.04+1_amd64.deb
postgresql-15-pg-fact-loader2.0.1u22.aarch64pgdg40.6 KiBpostgresql-15-pg-fact-loader_2.0.1-5.pgdg22.04+1_arm64.deb
postgresql-15-pg-fact-loader2.0.1u24.x86_64pgdg40.5 KiBpostgresql-15-pg-fact-loader_2.0.1-5.pgdg24.04+1_amd64.deb
postgresql-15-pg-fact-loader2.0.1u24.aarch64pgdg40.5 KiBpostgresql-15-pg-fact-loader_2.0.1-5.pgdg24.04+1_arm64.deb
PackageVersionOSORGSIZEFile URL
pg_fact_loader_142.0.1el8.x86_64pgdg36.2 KiBpg_fact_loader_14-2.0.1-3PGDG.rhel8.noarch.rpm
pg_fact_loader_142.0.1el8.x86_64pgdg36.0 KiBpg_fact_loader_14-2.0.1-2PGDG.rhel8.noarch.rpm
pg_fact_loader_142.0.1el8.x86_64pgdg36.0 KiBpg_fact_loader_14-2.0.1-1PGDG.f42.noarch.rpm
pg_fact_loader_142.0.1el8.aarch64pgdg36.1 KiBpg_fact_loader_14-2.0.1-3PGDG.rhel8.noarch.rpm
pg_fact_loader_142.0.1el8.aarch64pgdg36.0 KiBpg_fact_loader_14-2.0.1-2PGDG.rhel8.noarch.rpm
pg_fact_loader_142.0.1el9.x86_64pgdg34.6 KiBpg_fact_loader_14-2.0.1-3PGDG.rhel9.noarch.rpm
pg_fact_loader_142.0.1el9.x86_64pgdg34.5 KiBpg_fact_loader_14-2.0.1-2PGDG.rhel9.noarch.rpm
pg_fact_loader_142.0.1el9.aarch64pgdg34.5 KiBpg_fact_loader_14-2.0.1-3PGDG.rhel9.noarch.rpm
pg_fact_loader_142.0.1el9.aarch64pgdg34.3 KiBpg_fact_loader_14-2.0.1-2PGDG.rhel9.noarch.rpm
pg_fact_loader_142.0.1el10.x86_64pgdg35.1 KiBpg_fact_loader_14-2.0.1-3PGDG.rhel10.noarch.rpm
pg_fact_loader_142.0.1el10.aarch64pgdg35.0 KiBpg_fact_loader_14-2.0.1-3PGDG.rhel10.noarch.rpm
postgresql-14-pg-fact-loader2.0.1d12.x86_64pgdg40.5 KiBpostgresql-14-pg-fact-loader_2.0.1-5.pgdg120+1_amd64.deb
postgresql-14-pg-fact-loader2.0.1d12.aarch64pgdg40.5 KiBpostgresql-14-pg-fact-loader_2.0.1-5.pgdg120+1_arm64.deb
postgresql-14-pg-fact-loader2.0.1d13.x86_64pgdg40.5 KiBpostgresql-14-pg-fact-loader_2.0.1-5.pgdg130+1_amd64.deb
postgresql-14-pg-fact-loader2.0.1d13.aarch64pgdg40.5 KiBpostgresql-14-pg-fact-loader_2.0.1-5.pgdg130+1_arm64.deb
postgresql-14-pg-fact-loader2.0.1u22.x86_64pgdg40.6 KiBpostgresql-14-pg-fact-loader_2.0.1-5.pgdg22.04+1_amd64.deb
postgresql-14-pg-fact-loader2.0.1u22.aarch64pgdg40.6 KiBpostgresql-14-pg-fact-loader_2.0.1-5.pgdg22.04+1_arm64.deb
postgresql-14-pg-fact-loader2.0.1u24.x86_64pgdg40.5 KiBpostgresql-14-pg-fact-loader_2.0.1-5.pgdg24.04+1_amd64.deb
postgresql-14-pg-fact-loader2.0.1u24.aarch64pgdg40.5 KiBpostgresql-14-pg-fact-loader_2.0.1-5.pgdg24.04+1_arm64.deb

Source

Install

Make sure PGDG repo available:

pig repo add pgdg -u    # add pgdg repo and update cache

Install this extension with pig:

pig install pg_fact_loader;		# install via package name, for the active PG version

pig install pg_fact_loader -v 18;   # install for PG 18
pig install pg_fact_loader -v 17;   # install for PG 17
pig install pg_fact_loader -v 16;   # install for PG 16
pig install pg_fact_loader -v 15;   # install for PG 15
pig install pg_fact_loader -v 14;   # install for PG 14

Create this extension with:

CREATE EXTENSION pg_fact_loader;

Usage

pg_fact_loader: build fact tables with Postgres

Build and maintain fact tables using queue-based change data capture. Processes audit/change log tables to incrementally update fact tables.

Enabling

CREATE EXTENSION pg_fact_loader;

Optionally with pglogical for replica-based setup:

CREATE EXTENSION pglogical;
CREATE EXTENSION pglogical_ticker;
CREATE EXTENSION pg_fact_loader;

Workflow

  1. Replicate source tables to a reporting database (via pglogical or other means)
  2. Create audit/change log tables on the OLTP system for source tables
  3. Create a fact table structure for aggregated data
  4. Create a merge function that takes a key ID and returns one row of the fact table
  5. Configure pg_fact_loader to wire queue tables to fact tables
  6. Backfill the fact table initially
  7. Schedule the worker to process changes continuously

Configuration Tables

-- Register a fact table
INSERT INTO fact_loader.fact_tables (fact_table_relid, fact_table_agg_proid, ...)
VALUES ('public.customers_fact'::regclass, 'customers_fact_merge'::regproc, ...);

-- Register queue (audit) tables
INSERT INTO fact_loader.queue_tables (queue_table_relid, queue_of_base_table_relid, ...)
VALUES ('audit.customers_audit'::regclass, 'public.customers'::regclass, ...);

-- Connect queue tables to fact tables with merge functions
INSERT INTO fact_loader.queue_table_deps
    (fact_table_id, queue_table_id, insert_merge_proid, update_merge_proid, delete_merge_proid)
VALUES (1, 1, 'customers_fact_merge'::regproc, 'customers_fact_merge'::regproc, 'customers_fact_merge'::regproc);

-- Define how to retrieve the key from queue entries
INSERT INTO fact_loader.key_retrieval_sequences
    (queue_table_dep_id, return_columns, is_fact_key)
VALUES (1, '{customer_id}', true);

Running the Worker

-- Process pending changes
SELECT fact_loader.worker();

-- Schedule this to run periodically (e.g., every few seconds via pg_cron)

Initial Backfill

-- Run the merge function for every existing row
SELECT customers_fact_merge(customer_id) FROM customers;

Adding Batch ID Fields

SELECT fact_loader.add_batch_id_fields();

Key Features

  • Queue-based incremental fact table updates
  • Supports insert, update, and delete events
  • Handles multi-level key retrieval (joins through multiple tables)
  • Fact table dependency chains (child facts updated after parent)
  • Checks replication lag before processing (when used with pglogical)
Last updated on