wal2mongo

wal2mongo

wal2mongo : PostgreSQL logical decoding output plugin for MongoDB

Overview

IDExtensionPackageVersionCategoryLicenseLanguage
9640
wal2mongo
wal2mongo
1.0.7
ETL
Apache-2.0
C
AttributeHas BinaryHas LibraryNeed LoadHas DDLRelocatableTrusted
--s----
No
Yes
No
No
no
no
Relationships
See Also
mongo_fdw
wal2json
decoderbufs
decoder_raw
documentdb
pglogical
test_decoding
pgoutput

Packages

TypeRepoVersionPG Major CompatibilityPackage PatternDependencies
EXT
PIGSTY
1.0.7
18
17
16
15
14
wal2mongo-
RPM
PIGSTY
1.0.7
18
17
16
15
14
wal2mongo_$v-
DEB
PIGSTY
1.0.7
18
17
16
15
14
postgresql-$v-wal2mongo-
Linux / PGPG18PG17PG16PG15PG14
el8.x86_64
MISS
MISS
PGDG 1.0.7
PGDG 1.0.7
PGDG 1.0.7
el8.aarch64
MISS
MISS
PGDG 1.0.7
PGDG 1.0.7
PGDG 1.0.7
el9.x86_64
MISS
MISS
PGDG 1.0.7
PGDG 1.0.7
PGDG 1.0.7
el9.aarch64
MISS
MISS
PGDG 1.0.7
PGDG 1.0.7
PGDG 1.0.7
el10.x86_64
MISS
MISS
PGDG 1.0.7
PGDG 1.0.7
PGDG 1.0.7
el10.aarch64
MISS
MISS
PGDG 1.0.7
PGDG 1.0.7
PGDG 1.0.7
d12.x86_64
MISS
MISS
PIGSTY 1.0.7
PIGSTY 1.0.7
PIGSTY 1.0.7
d12.aarch64
MISS
MISS
PIGSTY 1.0.7
PIGSTY 1.0.7
PIGSTY 1.0.7
d13.x86_64
MISS
MISS
PIGSTY 1.0.7
PIGSTY 1.0.7
PIGSTY 1.0.7
d13.aarch64
MISS
MISS
PIGSTY 1.0.7
PIGSTY 1.0.7
PIGSTY 1.0.7
u22.x86_64
MISS
MISS
PIGSTY 1.0.7
PIGSTY 1.0.7
PIGSTY 1.0.7
u22.aarch64
MISS
MISS
PIGSTY 1.0.7
PIGSTY 1.0.7
PIGSTY 1.0.7
u24.x86_64
MISS
MISS
PIGSTY 1.0.7
PIGSTY 1.0.7
PIGSTY 1.0.7
u24.aarch64
MISS
MISS
PIGSTY 1.0.7
PIGSTY 1.0.7
PIGSTY 1.0.7

Source

pig build pkg wal2mongo;		# build deb

Install

Make sure PGDG and PIGSTY repo available:

pig repo add pgsql -u   # add both repo and update cache

Install this extension with pig:

pig install wal2mongo;		# install via package name, for the active PG version

pig install wal2mongo -v 16;   # install for PG 16
pig install wal2mongo -v 15;   # install for PG 15
pig install wal2mongo -v 14;   # install for PG 14

This extension does not need CREATE EXTENSION DDL command

Usage

wal2mongo: PostgreSQL logical decoding output plugin for MongoDB

A logical decoding output plugin that formats PostgreSQL WAL changes as MongoDB-compatible commands, enabling data replication from PostgreSQL to MongoDB.

Configuration

In postgresql.conf:

wal_level = logical
max_replication_slots = 10

Using with SQL Functions

-- Create a logical replication slot
SELECT * FROM pg_create_logical_replication_slot('w2m_slot', 'wal2mongo');

-- Perform DML operations
CREATE TABLE books (id SERIAL PRIMARY KEY, title VARCHAR(100), author VARCHAR(100));
INSERT INTO books (id, title, author) VALUES (123, 'My Book', 'Author');

-- Peek at changes (MongoDB format)
SELECT * FROM pg_logical_slot_peek_changes('w2m_slot', NULL, NULL);
-- Output: db.books.insertOne( { id:123, title:"My Book", author:"Author" } )

-- Consume changes
SELECT * FROM pg_logical_slot_get_changes('w2m_slot', NULL, NULL);

-- Drop the slot
SELECT pg_drop_replication_slot('w2m_slot');

Using with pg_recvlogical

pg_recvlogical -d postgres --slot w2m_slot --create-slot -P wal2mongo
pg_recvlogical -d postgres --slot w2m_slot --start -f -

Replicating to MongoDB

The output can be applied directly in the MongoDB shell:

// Copy the output from pg_logical_slot_get_changes
db.books.insertOne( { id:123, title:"My Book", author:"Author" } )

Or save to a .js file and import:

mongo < changes.js

Output Format

  • INSERT: db.<table>.insertOne( { <columns> } )
  • UPDATE: db.<table>.updateOne( { <key> }, { $set: { <changes> } } )
  • DELETE: db.<table>.deleteOne( { <key> } )

Tables need a primary key or replica identity for UPDATE/DELETE operations to be captured.

Last updated on