ClickPipes for Postgres FAQ

Once you identify the blocking query, you can decide to either wait for it to finish or cancel it if it's not critical. After the blocking query is resolved, the replication slot creation should proceed, allowing the snapshot to start and data to flow in.

- [Are transaction rollbacks replicated to ClickHouse?](#are-transaction-rollbacks-replicated)- [Can I keep data in ClickHouse longer than in my source Postgres?](#retain-data-longer-in-clickhouse)- [How can I enrich data as it flows from Postgres to ClickHouse?](#data-enrichment)- [Can I replicate from multiple Postgres instances into one or more ClickHouse services?](#multi-region-multi-source)- [How does idling affect my Postgres CDC ClickPipe?](#how-does-idling-affect-my-postgres-cdc-clickpipe)- [How are TOAST columns handled in ClickPipes for Postgres?](#how-are-toast-columns-handled-in-clickpipes-for-postgres)- [How are generated columns handled in ClickPipes for Postgres?](#how-are-generated-columns-handled-in-clickpipes-for-postgres)- [Do tables need to have primary keys to be part of Postgres CDC?](#do-tables-need-to-have-primary-keys-to-be-part-of-postgres-cdc)- [Do you support partitioned tables as part of Postgres CDC?](#do-you-support-partitioned-tables-as-part-of-postgres-cdc)- [Can I connect Postgres databases that don't have a public IP or are in private networks?](#can-i-connect-postgres-databases-that-dont-have-a-public-ip-or-are-in-private-networks)- [How do you handle UPDATEs and DELETEs?](#how-do-you-handle-updates-and-deletes)- [Can I update primary key columns in PostgreSQL?](#can-i-update-primary-key-columns-in-postgresql)- [Do you support schema changes?](#do-you-support-schema-changes)- [What are the costs for ClickPipes for Postgres CDC?](#what-are-the-costs-for-clickpipes-for-postgres-cdc)- [My replication slot size is growing or not decreasing; what might be the issue?](#my-replication-slot-size-is-growing-or-not-decreasing-what-might-be-the-issue)- [How are Postgres data types mapped to ClickHouse?](#how-are-postgres-data-types-mapped-to-clickhouse)- [Can I define my own data type mapping while replicating data from Postgres to ClickHouse?](#can-i-define-my-own-data-type-mapping-while-replicating-data-from-postgres-to-clickhouse)- [How are JSON and JSONB columns replicated from Postgres?](#how-are-json-and-jsonb-columns-replicated-from-postgres)- [What happens to inserts when a mirror is paused?](#what-happens-to-inserts-when-a-mirror-is-paused)- [Can ClickPipe creation be automated or done via API or CLI?](#can-clickpipe-creation-be-automated-or-done-via-api-or-cli)- [How do I speed up my initial load?](#how-do-i-speed-up-my-initial-load)- [How should I scope my publications when setting up replication?](#how-should-i-scope-my-publications-when-setting-up-replication)- [Recommended `max_slot_wal_keep_size` settings](#recommended-max_slot_wal_keep_size-settings)- [I'm seeing a ReceiveMessage EOF error in the logs. What does it mean?](#im-seeing-a-receivemessage-eof-error-in-the-logs-what-does-it-mean)- [My replication slot is invalidated. What should I do?](#my-replication-slot-is-invalidated-what-should-i-do)- [I am seeing out of memory (OOMs) on ClickHouse while my ClickPipe is ingesting data. Can you help?](#i-am-seeing-out-of-memory-ooms-on-clickhouse-while-my-clickpipe-is-ingesting-data-can-you-help)- [I am seeing an `invalid snapshot identifier` during the initial load. What should I do?](#i-am-seeing-an-invalid-snapshot-identifier-during-the-initial-load-what-should-i-do)- [What happens if I drop a publication in Postgres?](#what-happens-if-i-drop-a-publication-in-postgres)- [What if I am seeing `Unexpected Datatype` errors or `Cannot parse type XX ...`](#what-if-i-am-seeing-unexpected-datatype-errors)- [I'm seeing errors like `invalid memory alloc request size <XXX>` during replication/slot creation](#postgres-invalid-memalloc-bug)- [I need to maintain a complete historical record in ClickHouse, even when the data is deleted from the source Postgres database. Can I completely ignore DELETE and TRUNCATE operations from Postgres in ClickPipes?](#ignore-delete-truncate)- [Why can I not replicate my table which has a dot in it?](#replicate-table-dot)- [Initial load completed but there is no/missing data on ClickHouse. What could be the issue?](#initial-load-issue)- [Can I have the ClickPipe create a replication slot with failover enabled?](#failover-slot)- [I am seeing errors like `Internal error encountered during logical decoding of aborted sub-transaction`](#transient-logical-decoding-errors)- [I am seeing errors like `error converting new tuple to map` or `error parsing logical message` during CDC replication](#logical-message-processing-errors)- [Can I include columns I initially excluded from replication?](#include-excluded-columns)- [I am noticing that my ClickPipe has entered Snapshot but data isn't flowing in, what could be the issue?](#snapshot-no-data-flow)

Was this page helpful?