pgq
pgq
pgq : Generic queue for PostgreSQL
Overview
| ID | Extension | Package | Version | Category | License | Language |
|---|---|---|---|---|---|---|
| 2890 | pgq | pgq | 3.5.1 | FEAT | ISC | C |
| Attribute | Has Binary | Has Library | Need Load | Has DDL | Relocatable | Trusted |
|---|---|---|---|---|---|---|
--s-d-- | No | Yes | No | Yes | no | no |
| Relationships | |
|---|---|
| Schemas | pg_catalog |
| See Also | age hll rum pg_graphql pg_jsonschema jsquery pg_hint_plan hypopg |
Packages
| Type | Repo | Version | PG Major Compatibility | Package Pattern | Dependencies |
|---|---|---|---|---|---|
| EXT | PGDG | 3.5.1 | 18 17 16 15 14 | pgq | - |
| RPM | PGDG | 3.5.1 | 18 17 16 15 14 | pgq_$v | - |
| DEB | PGDG | 3.5.1 | 18 17 16 15 14 | postgresql-$v-pgq3 | - |
| Linux / PG | PG18 | PG17 | PG16 | PG15 | PG14 |
|---|---|---|---|---|---|
el8.x86_64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
el8.aarch64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
el9.x86_64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
el9.aarch64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
el10.x86_64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
el10.aarch64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
d12.x86_64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
d12.aarch64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
d13.x86_64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
d13.aarch64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
u22.x86_64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
u22.aarch64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
u24.x86_64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
u24.aarch64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
Source
Install
Make sure PGDG repo available:
pig repo add pgdg -u # add pgdg repo and update cacheInstall this extension with pig:
pig install pgq; # install via package name, for the active PG version
pig install pgq -v 18; # install for PG 18
pig install pgq -v 17; # install for PG 17
pig install pgq -v 16; # install for PG 16
pig install pgq -v 15; # install for PG 15
pig install pgq -v 14; # install for PG 14Create this extension with:
CREATE EXTENSION pgq;Usage
PgQ is a PostgreSQL extension that provides a generic, high-performance lockless queue with a simple SQL function API. It uses a producer-consumer model with batch-based event processing.
CREATE EXTENSION pgq;Core Concepts
- Queue: A named event stream. Events are inserted by producers and consumed in batches.
- Consumer: A named subscriber registered to a queue. Each consumer tracks its own position.
- Batch: A group of events retrieved together. Consumers process events batch by batch.
- Ticker: A background process that creates batch boundaries (ticks) at regular intervals.
Queue Management
-- Create a queue
SELECT pgq.create_queue('myqueue');
-- Drop a queue
SELECT pgq.drop_queue('myqueue');
-- Get queue info
SELECT * FROM pgq.get_queue_info();
SELECT * FROM pgq.get_queue_info('myqueue');Consumer Registration
-- Register a consumer on a queue
SELECT pgq.register_consumer('myqueue', 'myconsumer');
-- Unregister a consumer
SELECT pgq.unregister_consumer('myqueue', 'myconsumer');
-- Get consumer info
SELECT * FROM pgq.get_consumer_info('myqueue');Producing Events
-- Insert an event into a queue
SELECT pgq.insert_event('myqueue', 'event_type', 'event_data');
-- Insert with extra fields
SELECT pgq.insert_event('myqueue', 'event_type', 'event_data',
'extra1', 'extra2', 'extra3', 'extra4');Consuming Events
-- Get the next batch of events (returns batch_id or NULL if no new batches)
SELECT pgq.next_batch('myqueue', 'myconsumer');
-- Get events from the batch
SELECT * FROM pgq.get_batch_events(:batch_id);
-- Retry a failed event (will reappear after the specified interval)
SELECT pgq.event_retry(:batch_id, :event_id, :retry_seconds);
-- Mark batch as done
SELECT pgq.finish_batch(:batch_id);Typical Consumer Loop
-- 1. Get next batch
SELECT pgq.next_batch('myqueue', 'myconsumer') AS batch_id;
-- 2. If batch_id is not NULL, get events
SELECT * FROM pgq.get_batch_events(:batch_id);
-- 3. Process events, retry failures
SELECT pgq.event_retry(:batch_id, :event_id, 60);
-- 4. Finish the batch
SELECT pgq.finish_batch(:batch_id);Maintenance
PgQ requires a ticker daemon (pgqd) to run in the background for creating batch boundaries and performing maintenance tasks like table rotation and retry event processing.
Key Functions
| Function | Description |
|---|---|
pgq.create_queue(name) | Create a new queue |
pgq.drop_queue(name) | Remove a queue |
pgq.register_consumer(queue, consumer) | Register a consumer |
pgq.unregister_consumer(queue, consumer) | Unregister a consumer |
pgq.insert_event(queue, type, data, ...) | Insert an event |
pgq.next_batch(queue, consumer) | Get next batch ID |
pgq.get_batch_events(batch_id) | Get events from a batch |
pgq.event_retry(batch_id, event_id, seconds) | Schedule event retry |
pgq.finish_batch(batch_id) | Mark batch as processed |
pgq.get_queue_info([name]) | Get queue statistics |
pgq.get_consumer_info(queue) | Get consumer statistics |
Last updated on