RDS Postgres source setup guide

[Edit this page](https://github.com/ClickHouse/clickhouse-docs/blob/main/docs/integrations/data-ingestion/clickpipes/postgres/source/rds.md)

## Supported Postgres versions[​](#supported-postgres-versions "Direct link to Supported Postgres versions")

ClickPipes supports Postgres version 12 and later.

## Enable logical replication[​](#enable-logical-replication "Direct link to Enable logical replication")

You can skip this section if your RDS instance already has the following settings configured:

- `rds.logical_replication = 1`
- `wal_sender_timeout = 0`

These settings are typically pre-configured if you previously used another data replication tool.

```
postgres=> SHOW rds.logical_replication ;
 rds.logical_replication
-------------------------
 on
(1 row)

postgres=> SHOW wal_sender_timeout ;
 wal_sender_timeout
--------------------
 0
(1 row)
```

If not already configured, follow these steps:

1. Create a new parameter group for your Postgres version with the required settings:
   - Set `rds.logical_replication` to 1
   - Set `wal_sender_timeout` to 0

2. Apply the new parameter group to your RDS Postgres database

3. Reboot your RDS instance to apply the changes

## Configure database user[​](#configure-database-user "Direct link to Configure database user")

Connect to your RDS Postgres instance as an admin user and execute the following commands:

1. Create a dedicated user for ClickPipes:

   ```
   CREATE USER clickpipes_user PASSWORD 'some-password';
   ```
2. Grant schema-level, read-only access to the user you created in the previous step. The following example shows permissions for the `public` schema. Repeat these commands for each schema containing tables you want to replicate:

   ```
   GRANT USAGE ON SCHEMA "public" TO clickpipes_user;
   GRANT SELECT ON ALL TABLES IN SCHEMA "public" TO clickpipes_user;
   ALTER DEFAULT PRIVILEGES IN SCHEMA "public" GRANT SELECT ON TABLES TO clickpipes_user;
   ```
3. Grant replication privileges to the user:

   ```
   GRANT rds_replication TO clickpipes_user;
   ```
4. Create a [publication](https://www.postgresql.org/docs/current/logical-replication-publication.html) with the tables you want to replicate. We strongly recommend only including the tables you need in the publication to avoid performance overhead.

   Note

   Any table included in the publication must either have a **primary key** defined *or* have its **replica identity** configured to `FULL`. See the [Postgres FAQs](/docs/integrations/clickpipes/postgres/faq#how-should-i-scope-my-publications-when-setting-up-replication) for guidance on scoping.

   - To create a publication for specific tables:

     ```
     CREATE PUBLICATION clickpipes FOR TABLE table_to_replicate, table_to_replicate2;
     ```
   - To create a publication for all tables in a specific schema:

     ```
     CREATE PUBLICATION clickpipes FOR TABLES IN SCHEMA "public";
     ```

   The `clickpipes` publication will contain the set of change events generated from the specified tables, and will later be used to ingest the replication stream.

## Configure network access[​](#configure-network-access "Direct link to Configure network access")

### IP-based access control[​](#ip-based-access-control "Direct link to IP-based access control")

If you want to restrict traffic to your RDS instance, please add the [documented static NAT IPs](/docs/integrations/clickpipes#list-of-static-ips) to the `Inbound rules` of your RDS security group.

### Private Access via AWS PrivateLink[​](#private-access-via-aws-privatelink "Direct link to Private Access via AWS PrivateLink")

To connect to your RDS instance through a private network, you can use AWS PrivateLink. Follow our [AWS PrivateLink setup guide for ClickPipes](/docs/knowledgebase/aws-privatelink-setup-for-clickpipes) to set up the connection.

### Workarounds for RDS Proxy[​](#workarounds-for-rds-proxy "Direct link to Workarounds for RDS Proxy")

RDS Proxy doesn't support logical replication connections. If you have dynamic IP addresses in RDS and can't use DNS name or a lambda, here are some alternatives:

1. Using a cron job, resolve the RDS endpoint's IP periodically and update the NLB if it has changed.
2. Using RDS Event Notifications with EventBridge/SNS: Trigger updates automatically using AWS RDS event notifications
3. Stable EC2: Deploy an EC2 instance to act as a polling service or IP-based proxy
4. Automate IP address management using tools like Terraform or CloudFormation.

## What's next?[​](#whats-next "Direct link to What's next?")

You can now [create your ClickPipe](/docs/integrations/clickpipes/postgres) and start ingesting data from your Postgres instance into ClickHouse Cloud.
Make sure to note down the connection details you used while setting up your Postgres instance as you will need them during the ClickPipe creation process.

- [Supported Postgres versions](#supported-postgres-versions)- [Enable logical replication](#enable-logical-replication)- [Configure database user](#configure-database-user)- [Configure network access](#configure-network-access)
        - [IP-based access control](#ip-based-access-control)- [Private Access via AWS PrivateLink](#private-access-via-aws-privatelink)- [Workarounds for RDS Proxy](#workarounds-for-rds-proxy)- [What's next?](#whats-next)

Was this page helpful?