Monitoring MongoDB Logs with ClickStack

[Edit this page](https://github.com/ClickHouse/clickhouse-docs/blob/main/docs/use-cases/observability/clickstack/ingesting-data/integration-examples/mongodb-logs.md)

TL;DR

Collect and visualize MongoDB server logs (4.4+ JSON format) in ClickStack using the OTel `filelog` receiver. Includes a demo dataset and pre-built dashboard.

## Integration with existing MongoDB[​](#existing-mongodb "Direct link to Integration with existing MongoDB")

This section covers configuring your existing MongoDB installation to send logs to ClickStack by modifying the ClickStack OTel collector configuration.
If you would like to test the MongoDB integration before configuring your own existing setup, you can test with our preconfigured setup and sample data in the ["Demo dataset"](/docs/use-cases/observability/clickstack/integrations/mongodb-logs#demo-dataset) section.

### Prerequisites[​](#prerequisites "Direct link to Prerequisites")

- ClickStack instance running
- Existing self-managed MongoDB installation (version 4.4 or newer)
- Access to MongoDB log files

Verify MongoDB logging configuration

#### Verify MongoDB logging configuration[​](#verify-mongodb "Direct link to Verify MongoDB logging configuration")

MongoDB 4.4+ outputs structured JSON logs by default. Check your log file location:

```
cat /etc/mongod.conf | grep -A 5 systemLog
```

Common MongoDB log locations:

- **Linux (apt/yum)**: `/var/log/mongodb/mongod.log`
- **macOS (Homebrew)**: `/usr/local/var/log/mongodb/mongo.log`
- **Docker**: Often logged to stdout, but can be configured to write to `/var/log/mongodb/mongod.log`

If MongoDB is logging to stdout, configure it to write to a file by updating `mongod.conf`:

```
systemLog:
  destination: file
  path: /var/log/mongodb/mongod.log
  logAppend: true
```

After changing the configuration, restart MongoDB:

```
# For systemd
sudo systemctl restart mongod

# For Docker
docker restart <mongodb-container>
```

Create a custom OTel collector configuration for MongoDB

#### Create a custom OTel collector configuration for MongoDB[​](#custom-otel "Direct link to Create a custom OTel collector configuration for MongoDB")

ClickStack allows you to extend the base OpenTelemetry Collector configuration by mounting a custom configuration file and setting an environment variable. The custom configuration is merged with the base configuration managed by HyperDX via OpAMP.

Create a file named `mongodb-monitoring.yaml` with the following configuration:

```
receivers:
  filelog/mongodb:
    include:
      - /var/log/mongodb/mongod.log
    start_at: beginning
    operators:
      - type: json_parser
        parse_from: body
        parse_to: attributes
        timestamp:
          parse_from: attributes.t.$$date
          layout: '2006-01-02T15:04:05.000-07:00'
          layout_type: gotime
        severity:
          parse_from: attributes.s
          overwrite_text: true
          mapping:
            fatal: F
            error: E
            warn: W
            info: I
            debug:
              - D1
              - D2
              - D3
              - D4
              - D5

      - type: move
        from: attributes.msg
        to: body

      - type: add
        field: attributes.source
        value: "mongodb"

      - type: add
        field: resource["service.name"]
        value: "mongodb-production"

service:
  pipelines:
    logs/mongodb:
      receivers: [filelog/mongodb]
      processors:
        - memory_limiter
        - transform
        - batch
      exporters:
        - clickhouse
```

Note

- You only define new receivers and pipelines in the custom config. The processors (`memory_limiter`, `transform`, `batch`) and exporters (`clickhouse`) are already defined in the base ClickStack configuration — you just reference them by name.
- This configuration uses `start_at: beginning` to read all existing logs when the collector starts. For production deployments, change to `start_at: end` to avoid re-ingesting logs on collector restarts.

Configure ClickStack to load custom configuration

#### Configure ClickStack to load custom configuration[​](#load-custom "Direct link to Configure ClickStack to load custom configuration")

To enable custom collector configuration in your existing ClickStack deployment, you must:

1. Mount the custom config file at `/etc/otelcol-contrib/custom.config.yaml`
2. Set the environment variable `CUSTOM_OTELCOL_CONFIG_FILE=/etc/otelcol-contrib/custom.config.yaml`
3. Mount your MongoDB log directory so the collector can read them

- Docker Compose- Docker Run (All-in-One Image)

Update your ClickStack deployment configuration:

```
services:
  clickstack:
    # ... existing configuration ...
    environment:
      - CUSTOM_OTELCOL_CONFIG_FILE=/etc/otelcol-contrib/custom.config.yaml
      # ... other environment variables ...
    volumes:
      - ./mongodb-monitoring.yaml:/etc/otelcol-contrib/custom.config.yaml:ro
      - /var/log/mongodb:/var/log/mongodb:ro
      # ... other volumes ...
```

If you're using the all-in-one image with docker, run:

```
docker run --name clickstack \
  -p 8080:8080 -p 4317:4317 -p 4318:4318 \
  -e CUSTOM_OTELCOL_CONFIG_FILE=/etc/otelcol-contrib/custom.config.yaml \
  -v "$(pwd)/mongodb-monitoring.yaml:/etc/otelcol-contrib/custom.config.yaml:ro" \
  -v /var/log/mongodb:/var/log/mongodb:ro \
  clickhouse/clickstack-all-in-one:latest
```

Note

Ensure the ClickStack collector has appropriate permissions to read the MongoDB log files. In production, use read-only mounts (`:ro`) and follow the principle of least privilege.

Verify Logs in HyperDX

#### Verify Logs in HyperDX[​](#verifying-logs "Direct link to Verify Logs in HyperDX")

Once configured, log into HyperDX and verify that logs are flowing:

## Demo dataset[​](#demo-dataset "Direct link to Demo dataset")

Test the MongoDB integration with a pre-generated sample dataset before configuring your production systems.

Download the sample dataset

#### Download the sample dataset[​](#download-sample "Direct link to Download the sample dataset")

Download the sample log file:

```
curl -O https://datasets-documentation.s3.eu-west-3.amazonaws.com/clickstack-integrations/mongodb/mongod.log
```

Create test collector configuration

#### Create test collector configuration[​](#test-config "Direct link to Create test collector configuration")

Create a file named `mongodb-demo.yaml` with the following configuration:

```
cat > mongodb-demo.yaml << 'EOF'
receivers:
  filelog/mongodb:
    include:
      - /tmp/mongodb-demo/mongod.log
    start_at: beginning
    operators:
      - type: json_parser
        parse_from: body
        parse_to: attributes
        timestamp:
          parse_from: attributes.t.$$date
          layout: '2006-01-02T15:04:05.000-07:00'
          layout_type: gotime
        severity:
          parse_from: attributes.s
          overwrite_text: true
          mapping:
            fatal: F
            error: E
            warn: W
            info: I
            debug:
              - D1
              - D2
              - D3
              - D4
              - D5

      - type: move
        from: attributes.msg
        to: body

      - type: add
        field: attributes.source
        value: "mongodb-demo"

      - type: add
        field: resource["service.name"]
        value: "mongodb-demo"

service:
  pipelines:
    logs/mongodb-demo:
      receivers: [filelog/mongodb]
      processors:
        - memory_limiter
        - transform
        - batch
      exporters:
        - clickhouse
EOF
```

Run ClickStack with demo configuration

#### Run ClickStack with demo configuration[​](#run-demo "Direct link to Run ClickStack with demo configuration")

Run ClickStack with the demo logs and configuration:

```
docker run --name clickstack-demo \
  -p 8080:8080 -p 4317:4317 -p 4318:4318 \
  -e CUSTOM_OTELCOL_CONFIG_FILE=/etc/otelcol-contrib/custom.config.yaml \
  -v "$(pwd)/mongodb-demo.yaml:/etc/otelcol-contrib/custom.config.yaml:ro" \
  -v "$(pwd)/mongod.log:/tmp/mongodb-demo/mongod.log:ro" \
  clickhouse/clickstack-all-in-one:latest
```