title_wip

[Edit this page](https://github.com/ClickHouse/clickhouse-docs/blob/main/docs/integrations/data-ingestion/clickpipes/mongodb/faq.md)

### Can I query for individual fields in the JSON datatype?[​](#can-i-query-for-individual-fields-in-the-json-datatype "Direct link to Can I query for individual fields in the JSON datatype?")

For direct field access, such as `{"user_id": 123}`, you can use **dot notation**:

```
SELECT doc.user_id as user_id FROM your_table;
```

For direct field access of nested object fields, such as `{"address": { "city": "San Francisco", "state": "CA" }}`, use the `^` operator:

```
SELECT doc.^address.city AS city FROM your_table;
```

For aggregations, cast the field to the appropriate type with the `CAST` function or `::` syntax:

```
SELECT sum(doc.shipping.cost::Float32) AS total_shipping_cost FROM t1;
```

To learn more about working with JSON, see our [Working with JSON guide](/docs/integrations/clickpipes/mongodb/quickstart).

### How do I flatten the nested MongoDB documents in ClickHouse?[​](#how-do-i-flatten-the-nested-mongodb-documents-in-clickhouse "Direct link to How do I flatten the nested MongoDB documents in ClickHouse?")

MongoDB documents are replicated as JSON type in ClickHouse by default, preserving the nested structure. You have several options to flatten this data. If you want to flatten the data to columns, you can use normal views, materialized views, or query-time access.

1. **Normal Views**: Use normal views to encapsulate flattening logic.
2. **Materialized Views**: For smaller datasets, you can use refreshable materialized with the [`FINAL` modifier](/docs/sql-reference/statements/select/from#final-modifier) to periodically flatten and deduplicate data. For larger datasets, we recommend using incremental materialized views without `FINAL` to flatten the data in real-time, and then deduplicate data at query time.
3. **Query-time Access**: Instead of flattening, use dot notation to access nested fields directly in queries.

For detailed examples, see our [Working with JSON guide](/docs/integrations/clickpipes/mongodb/quickstart).

### Can I connect MongoDB databases that don't have a public IP or are in private networks?[​](#can-i-connect-mongodb-databases-that-dont-have-a-public-ip-or-are-in-private-networks "Direct link to Can I connect MongoDB databases that don't have a public IP or are in private networks?")

We support AWS PrivateLink for connecting to MongoDB databases that don't have a public IP or are in private networks. Azure Private Link and GCP Private Service Connect are currently not supported.

### What happens if I delete a database/table from my MongoDB database?[​](#what-happens-if-i-delete-a-database-table-from-my-mongodb-database "Direct link to What happens if I delete a database/table from my MongoDB database?")

When you delete a database/table from MongoDB, ClickPipes will continue running but the dropped database/table will stop replicating changes. The corresponding tables in ClickHouse is preserved.

### How does MongoDB CDC Connector handle transactions?[​](#how-does-mongodb-cdc-connector-handle-transactions "Direct link to How does MongoDB CDC Connector handle transactions?")

Each document change within a transaction is processed individually to ClickHouse. Changes are applied in the order they appear in the oplog; and only committed changes are replicated to ClickHouse. If a MongoDB transaction is rolled back, those changes won't appear in the change stream.

For more examples, see our [Working with JSON guide](/docs/integrations/clickpipes/mongodb/quickstart).

### How do I handle `resume of change stream was not possible, as the resume point may no longer be in the oplog.` error?[​](#resume-point-may-no-longer-be-in-the-oplog-error "Direct link to resume-point-may-no-longer-be-in-the-oplog-error")

This error typically occurs when the oplog is truncated and ClickPipe is unable to resume the change stream at the expected point. To resolve this issue, [resync the ClickPipe](/docs/integrations/clickpipes/mongodb/resync). To avoid this issue from recurring, we recommend increasing the oplog retention period. See instructions for [MongoDB Atlas](/docs/integrations/clickpipes/mongodb/source/atlas#enable-oplog-retention), [self-managed MongoDB](/docs/integrations/clickpipes/mongodb/source/generic#enable-oplog-retention), or [Amazon DocumentDB](/docs/integrations/clickpipes/mongodb/source/documentdb#configure-change-stream-log-retention).

### How is replication managed?[​](#how-is-replication-managed "Direct link to How is replication managed?")

We use MongoDB's native Change Streams API to track changes in the database. Change Streams API provides a resumable stream of database changes by leveraging MongoDB's oplog (operations log). ClickPipe uses MongoDB's resume tokens to track the position in the oplog and ensure every change is replicated to ClickHouse.

### Which read preference should I use?[​](#which-read-preference-should-i-use "Direct link to Which read preference should I use?")

Which read preference to use depends on your specific use case. If you want to minimize the load on your primary node, we recommend using `secondaryPreferred` read preference. If you want to optimize ingestion latency, we recommend using `primaryPreferred` read preference. For more details, see [MongoDB documentation](https://www.mongodb.com/docs/manual/core/read-preference/#read-preference-modes-1).

### Does the MongoDB ClickPipe support Sharded Cluster?[​](#does-the-mongodb-clickpipe-support-sharded-cluster "Direct link to Does the MongoDB ClickPipe support Sharded Cluster?")

Yes, the MongoDB ClickPipe supports both Replica Set and Sharded Cluster.

### Does MongoDB ClickPipe support Amazon DocumentDB?[​](#documentdb-support "Direct link to Does MongoDB ClickPipe support Amazon DocumentDB?")

Yes, MongoDB ClickPipe supports Amazon DocumentDB 5.0. See [Amazon DocumentDB source setup guide](/docs/integrations/clickpipes/mongodb/source/documentdb) for details.

### Does MongoDB ClickPipe support PrivateLink?[​](#privatelink-support "Direct link to Does MongoDB ClickPipe support PrivateLink?")

We support PrivateLink for MongoDB (and DocumentDB) cluster in AWS only.

Note that unlike single-node relational database, MongoDB client requires successful replica set discovery to be able to respect the configured `ReadPreference`. This requires setting up PrivateLink with all the nodes in the cluster so the MongoDB client can successfully establish replica set connection, as well as redirect to another node when the connected node goes down.

If you prefer to connect to a single node in your cluster, you can skip replica set discovery by specifying `/?directConnection=true` in the connection string during ClickPipes setup. The PrivateLink setup in this case will be similar to a single-node relational database, and is the simplest option for PrivateLink support.

For replica set connection, you can set up PrivateLink for MongoDB with either VPC Resource or VPC Endpoint Service. If you go with VPC Resource, you would need to create a `GROUP` resource configuration, plus a `CHILD` resource configuration for each node in the cluster. If you go with VPC Endpoint Service, you would need to create a separate Endpoint Service (and a separate NLB) for each node in the cluster.

See [AWS PrivateLink for ClickPipes](/docs/integrations/clickpipes/aws-privatelink) documentation for more details. Please reach out to ClickHouse support for assistance.