Monitoring PostgreSQL Logs with ClickStack

Once configured, log into HyperDX and verify logs are flowing:

1. Navigate to the search view
2. Set source to Logs
3. Filter by `source:postgresql` to see PostgreSQL-specific logs
4. You should see structured log entries with fields like `user_name`, `database_name`, `error_severity`, `message`, `query`, etc.

## Demo dataset[​](#demo-dataset "Direct link to Demo dataset")

For users who want to test the PostgreSQL logs integration before configuring their production systems, we provide a sample dataset of pre-generated PostgreSQL logs with realistic patterns.

Download the sample dataset

#### Download the sample dataset[​](#download-sample "Direct link to Download the sample dataset")

Download the sample log file:

```
curl -O https://datasets-documentation.s3.eu-west-3.amazonaws.com/clickstack-integrations/postgres/postgresql.log
```

Create test collector configuration

#### Create test collector configuration[​](#test-config "Direct link to Create test collector configuration")

Create a file named `postgres-logs-demo.yaml` with the following configuration:

```
cat > postgres-logs-demo.yaml << 'EOF'
receivers:
  filelog/postgres:
    include:
      - /tmp/postgres-demo/postgresql.log
    start_at: beginning  # Read from beginning for demo data
    multiline:
      line_start_pattern: '^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}'
    operators:
      - type: csv_parser
        parse_from: body
        parse_to: attributes
        header: 'log_time,user_name,database_name,process_id,connection_from,session_id,session_line_num,command_tag,session_start_time,virtual_transaction_id,transaction_id,error_severity,sql_state_code,message,detail,hint,internal_query,internal_query_pos,context,query,query_pos,location,application_name,backend_type,leader_pid,query_id'
        lazy_quotes: true
        
      - type: time_parser
        parse_from: attributes.log_time
        layout: '%Y-%m-%d %H:%M:%S.%L %Z'
      
      - type: add
        field: attributes.source
        value: "postgresql-demo"
      
      - type: add
        field: resource["service.name"]
        value: "postgresql-demo"

service:
  pipelines:
    logs/postgres-demo:
      receivers: [filelog/postgres]
      processors:
        - memory_limiter
        - transform
        - batch
      exporters:
        - clickhouse
EOF
```

Run ClickStack with demo configuration

#### Run ClickStack with demo configuration[​](#run-demo "Direct link to Run ClickStack with demo configuration")

Run ClickStack with the demo logs and configuration:

```
docker run --name clickstack-demo \
  -p 8080:8080 -p 4317:4317 -p 4318:4318 \
  -e CUSTOM_OTELCOL_CONFIG_FILE=/etc/otelcol-contrib/custom.config.yaml \
  -v "$(pwd)/postgres-logs-demo.yaml:/etc/otelcol-contrib/custom.config.yaml:ro" \
  -v "$(pwd)/postgresql.log:/tmp/postgres-demo/postgresql.log:ro" \
  clickhouse/clickstack-all-in-one:latest
```

Verify logs in HyperDX

#### Verify logs in HyperDX[​](#verify-demo-logs "Direct link to Verify logs in HyperDX")

Once ClickStack is running:

1. Open [HyperDX](http://localhost:8080/) and log in to your account (you may need to create an account first)
2. Navigate to the Search view and set the source to `Logs`
3. Set the time range to **2025-11-09 00:00:00 - 2025-11-12 00:00:00**

Timezone Display

HyperDX displays timestamps in your browser's local timezone. The demo data spans **2025-11-10 00:00:00 - 2025-11-11 00:00:00 (UTC)**. The wide time range ensures you'll see the demo logs regardless of your location. Once you see the logs, you can narrow the range to a 24-hour period for clearer visualizations.

## Dashboards and visualization[​](#dashboards "Direct link to Dashboards and visualization")

To help you get started monitoring PostgreSQL with ClickStack, we provide essential visualizations for PostgreSQL logs.

Download the dashboard configuration

#### [Download](/docs/examples/postgres-logs-dashboard.json) the dashboard configuration[​](#download "Direct link to download")

Import the pre-built dashboard

#### Import the pre-built dashboard[​](#import-dashboard "Direct link to Import the pre-built dashboard")

1. Open HyperDX and navigate to the Dashboards section
2. Click **Import Dashboard** in the upper right corner under the ellipses

3. Upload the `postgresql-logs-dashboard.json` file and click **Finish Import**

View the dashboard

#### View the dashboard[​](#created-dashboard "Direct link to View the dashboard")

The dashboard will be created with all visualizations pre-configured:

Note

For the demo dataset, set the time range to **2025-11-10 00:00:00 - 2025-11-11 00:00:00 (UTC)** (adjust based on your local timezone). The imported dashboard won't have a time range specified by default.

## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting")

### Custom config not loading[​](#troubleshooting-not-loading "Direct link to Custom config not loading")

Verify the environment variable is set:

```
docker exec <container-name> printenv CUSTOM_OTELCOL_CONFIG_FILE
```

Check the custom config file is mounted and readable:

```
docker exec <container-name> cat /etc/otelcol-contrib/custom.config.yaml | head -10
```

### No logs appearing in HyperDX[​](#no-logs "Direct link to No logs appearing in HyperDX")

Check the effective config includes your filelog receiver:

```
docker exec <container> cat /etc/otel/supervisor-data/effective.yaml | grep -A 10 filelog
```

Check for errors in the collector logs:

```
docker exec <container> cat /etc/otel/supervisor-data/agent.log | grep -i postgres
```

If using the demo dataset, verify the log file is accessible:

```
docker exec <container> cat /tmp/postgres-demo/postgresql.log | wc -l
```

## Next steps[​](#next-steps "Direct link to Next steps")

- Set up [alerts](/docs/use-cases/observability/clickstack/alerts) for critical events (connection failures, slow queries, error spikes)
- Correlate logs with [PostgreSQL metrics](/docs/use-cases/observability/clickstack/integrations/postgresql-metrics) for comprehensive database monitoring
- Create custom dashboards for application-specific query patterns
- Configure `log_min_duration_statement` to identify slow queries specific to your performance requirements

## Going to production[​](#going-to-production "Direct link to Going to production")

This guide extends ClickStack's built-in OpenTelemetry Collector for quick setup. For production deployments, we recommend running your own OTel Collector and sending data to ClickStack's OTLP endpoint. See [Sending OpenTelemetry data](/docs/use-cases/observability/clickstack/ingesting-data/opentelemetry) for production configuration.

- [Integration with existing PostgreSQL](#existing-postgres)- [Demo dataset](#demo-dataset)- [Dashboards and visualization](#dashboards)- [Troubleshooting](#troubleshooting)
        - [Custom config not loading](#troubleshooting-not-loading)- [No logs appearing in HyperDX](#no-logs)- [Next steps](#next-steps)- [Going to production](#going-to-production)

Was this page helpful?