Overview
PostgreSQL is a powerful, open-source relational database known for its reliability, extensibility, and standards compliance. In Zenskar, you can use PostgreSQL as a data source to import transactional, usage, or customer data for metering, billing, and analytical reporting.Prerequisites
Create a dedicated read-only PostgreSQL user for replicating data. Alternatively, you can use an existing user:You must re-run the above command for each schema from which you expect to replicate data.
Set up a PostgreSQL data-source connector via Zenskar dashboard
- Log into your Zenskar dashboard.
- In the left side bar, click Usage > Data Sources.
- In the top-right corner, click + ADD DATA SOURCE.
- On the Add New Data Source page, configure the PostgreSQL connector.
- Click on the SAVE SOURCE button.
Connector configuration
General configuration
Connector configuration
Set up a PostgreSQL data-source connector via API
Refer to the create data-source connector API reference.
The
connector_config object is the only part of the request that differs across connector types (such as BigQuery, Snowflake, or Redshift). The create data-source connector API reference provides a generic overview, while this document explains the PostgreSQL-specific structure of connector_config.Request example
PostgreSQL-specific connector configuration
Theconnector_config object contains configuration fields specific to the PostgreSQL data source.
Addendum
SSL modes in PostgreSQL
If SSL mode is set to Require, the connection will fail if the PostgreSQL instance doesn’t support SSL.
Data replication
Change Data Capture (CDC) replication
Zenskar uses logical replication of PostgreSQL’s write-ahead log (WAL) to incrementally capture data changes, including deletions. You should configure the PostgreSQL data source with CDC when:- You need a record of deletions.
- Your database is large (500 GB or more).
- Your table has a primary key but lacks a suitable incremental cursor field (for example,
updated_at).
Set up CDC
-
Grant
REPLICATIONprivileges to the read-only user: -
Enable logical replication on your PostgreSQL database.
-
For self-hosted or VM environments (bare metal, EC2, GCE, Docker):
Edit the
postgresql.conffile: -
For AWS RDS or Aurora:
- Go to the Configuration tab of your DB cluster.
- Locate your cluster parameter group and set
rds.logical_replication = 1. - Restart your instance manually or during the next maintenance window.
-
For Azure Database for PostgreSQL:
-
Change the replication mode to
logicalfrom the Azure portal, or run:
-
Change the replication mode to
-
For self-hosted or VM environments (bare metal, EC2, GCE, Docker):
Edit the
-
Create a dedicated replication slot for Zenskar using the
pgoutputplugin:The output will include the slot name to be filled in the Zenskar setup page. -
Define replication identities for all tables you want to replicate:
Use
FULLfor tables with large or TOASTable fields: -
Create a publication including all tables to be replicated:
The publication name is customizable.
The Zenskar UI allows table selection for CDC, but if a table is not part of the publication, it will not be replicated. If a table lacks a replication identity, Zenskar can create it automatically if the user has sufficient permissions.
Xmin replication
The xmin method is a cursorless replication mode that tracks inserts and updates using PostgreSQL’s internalxmin system column.
Use xmin when:
- There’s no suitable cursor field (for example,
updated_at). - You want to replace a full-refresh sync.
- You are replicating PostgreSQL tables smaller than 500 GB.
- You are not replicating non-materialized views (these are unsupported by xmin).