mimeo

mimeo : Extension for specialized, per-table replication between PostgreSQL instances

Overview

IDExtensionPackageVersionCategoryLicenseLanguage
9700
mimeo
mimeo
1.5.1
ETL
PostgreSQL
SQL
AttributeHas BinaryHas LibraryNeed LoadHas DDLRelocatableTrusted
----d--
No
No
No
Yes
no
no
Relationships
Requires
dblink
See Also
pg_jobmon
postgres_fdw
pglogical
pg_cron
pg_partman
repmgr
pg_fact_loader
pg_failover_slots

name conflict with pg_partman

Packages

TypeRepoVersionPG Major CompatibilityPackage PatternDependencies
EXT
MIXED
1.5.1
18
17
16
15
14
mimeodblink
RPM
PIGSTY
1.5.1
18
17
16
15
14
mimeo_$v-
DEB
PGDG
1.5.1
18
17
16
15
14
postgresql-$v-mimeo-
Linux / PGPG18PG17PG16PG15PG14
el8.x86_64
PIGSTY 1.5.1
PIGSTY 1.5.1
PIGSTY 1.5.1
PIGSTY 1.5.1
PIGSTY 1.5.1
el8.aarch64
PIGSTY 1.5.1
PIGSTY 1.5.1
PIGSTY 1.5.1
PIGSTY 1.5.1
PIGSTY 1.5.1
el9.x86_64
PIGSTY 1.5.1
PIGSTY 1.5.1
PIGSTY 1.5.1
PIGSTY 1.5.1
PIGSTY 1.5.1
el9.aarch64
PIGSTY 1.5.1
PIGSTY 1.5.1
PIGSTY 1.5.1
PIGSTY 1.5.1
PIGSTY 1.5.1
el10.x86_64
PIGSTY 1.5.1
PIGSTY 1.5.1
PIGSTY 1.5.1
PIGSTY 1.5.1
PIGSTY 1.5.1
el10.aarch64
PIGSTY 1.5.1
PIGSTY 1.5.1
PIGSTY 1.5.1
PIGSTY 1.5.1
PIGSTY 1.5.1
d12.x86_64
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
d12.aarch64
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
d13.x86_64
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
d13.aarch64
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
u22.x86_64
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
u22.aarch64
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
u24.x86_64
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
u24.aarch64
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1
PGDG 1.5.1

Source

pig build pkg mimeo;		# build rpm

Install

Make sure PGDG and PIGSTY repo available:

pig repo add pgsql -u   # add both repo and update cache

Install this extension with pig:

pig install mimeo;		# install via package name, for the active PG version

pig install mimeo -v 18;   # install for PG 18
pig install mimeo -v 17;   # install for PG 17
pig install mimeo -v 16;   # install for PG 16
pig install mimeo -v 15;   # install for PG 15
pig install mimeo -v 14;   # install for PG 14

Create this extension with:

CREATE EXTENSION mimeo CASCADE; -- requires dblink

Usage

mimeo: Extension for specialized, per-table replication between PostgreSQL instances

Provides per-table replication between PostgreSQL instances with snapshot (full copy), incremental (timestamp/id based), and DML (insert/update/delete) modes.

Enabling

CREATE SCHEMA mimeo;
CREATE EXTENSION mimeo SCHEMA mimeo;

Requires the dblink extension. Optionally install pg_jobmon for monitoring.

Setting Up a Data Source

-- Create a dblink connection to the source database
SELECT mimeo.dblink_mapping_create(
    p_mapping_name := 'source_db',
    p_data_source := 'host=sourcehost dbname=sourcedb user=replicator password=secret',
    p_superuser := true
);

Snapshot Replication (Full Table Copy)

Copies the entire source table each time it runs:

SELECT mimeo.snapshot_maker(
    p_src_table := 'public.my_table',
    p_dblink_id := 1  -- from dblink_mapping
);

-- Refresh the snapshot
SELECT mimeo.refresh_snap('public.my_table');

Incremental Replication (Timestamp-Based)

Replicates rows based on an incrementing timestamp column:

SELECT mimeo.inserter_maker(
    p_src_table := 'public.events',
    p_control := 'created_at',  -- timestamp column
    p_dblink_id := 1
);

-- Refresh incrementally
SELECT mimeo.refresh_inserter('public.events');

For tables with updates (not just inserts):

SELECT mimeo.updater_maker(
    p_src_table := 'public.orders',
    p_control := 'updated_at',
    p_dblink_id := 1
);

SELECT mimeo.refresh_updater('public.orders');

DML Replication (Insert/Update/Delete)

Full DML tracking via triggers on the source:

SELECT mimeo.dml_maker(
    p_src_table := 'public.accounts',
    p_dblink_id := 1
);

SELECT mimeo.refresh_dml('public.accounts');

Scheduling Refreshes

Use pg_jobmon or cron to schedule periodic calls to the appropriate refresh_* function.

Key Features

  • Three replication modes: snapshot, incremental, DML
  • Per-table replication (no need to replicate entire database)
  • Works between different PostgreSQL versions
  • Built on dblink for cross-database communication
Last updated on