Monitoring MySQL Logs with ClickStack

## Demo dataset[​](#demo-dataset "Direct link to Demo dataset")

For users who want to test the MySQL logs integration before configuring their production systems, we provide a sample dataset of pre-generated MySQL logs with realistic patterns.

Download the sample dataset

#### Download the sample dataset[​](#download-sample "Direct link to Download the sample dataset")

Download the sample log files:

```
# Download error log
curl -O https://datasets-documentation.s3.eu-west-3.amazonaws.com/clickstack-integrations/mysql/error.log

# Download slow query log
curl -O https://datasets-documentation.s3.eu-west-3.amazonaws.com/clickstack-integrations/mysql/mysql-slow.log
```

The dataset includes:

- Error log entries (startup messages, warnings, connection errors, InnoDB messages)
- Slow queries with realistic performance characteristics
- Connection lifecycle events
- Database server startup and shutdown sequences

Create test collector configuration

#### Create test collector configuration[​](#test-config "Direct link to Create test collector configuration")

Create a file named `mysql-logs-demo.yaml` with the following configuration:

```
cat > mysql-logs-demo.yaml << 'EOF'
receivers:
  filelog/mysql_error:
    include:
      - /tmp/mysql-demo/error.log
    start_at: beginning  # Read from beginning for demo data
    multiline:
      line_start_pattern: '^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}'
    operators:
      - type: regex_parser
        parse_from: body
        parse_to: attributes
        regex: '^(?P<timestamp>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{6}[+-]\d{2}:\d{2})\s+(?P<thread_id>\d+)\s+\[(?P<level>[^\]]+)\]\s+(\[(?P<error_code>[^\]]+)\]\s+)?(?P<message>.*)$'
      - type: time_parser
        parse_from: attributes.timestamp
        layout_type: gotime
        layout: '2006-01-02T15:04:05.999999-07:00'
        parse_to: body
      - type: add
        field: attributes.source
        value: "mysql-demo-error"
      - type: add
        field: resource["service.name"]
        value: "mysql-demo"

  filelog/mysql_slow:
    include:
      - /tmp/mysql-demo/mysql-slow.log
    start_at: beginning  # Read from beginning for demo data
    multiline:
      line_start_pattern: '^# Time:'
    operators:
      - type: regex_parser
        parse_from: body
        parse_to: attributes
        regex: '^# Time: (?P<timestamp>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d+Z)\n# User@Host: (?P<user>[^\[]+)\[(?P<user_host>[^\]]*)\]\s+@\s+(?P<host>[^\[]*)\[(?P<ip>[^\]]*)\].*\n# Query_time: (?P<query_time>[\d.]+)\s+Lock_time: (?P<lock_time>[\d.]+)\s+Rows_sent: (?P<rows_sent>\d+)\s+Rows_examined: (?P<rows_examined>\d+)'
      - type: time_parser
        parse_from: attributes.timestamp
        layout_type: gotime
        layout: '2006-01-02T15:04:05.999999Z'
        parse_to: body
      - type: add
        field: attributes.source
        value: "mysql-demo-slow"
      - type: add
        field: resource["service.name"]
        value: "mysql-demo"

service:
  pipelines:
    logs/mysql-demo:
      receivers: [filelog/mysql_error, filelog/mysql_slow]
      processors:
        - memory_limiter
        - transform
        - batch
      exporters:
        - clickhouse
EOF
```

Run ClickStack with demo configuration

#### Run ClickStack with demo configuration[​](#run-demo "Direct link to Run ClickStack with demo configuration")

Run ClickStack with the demo logs and configuration:

```
docker run --name clickstack-demo \
  -p 8080:8080 -p 4317:4317 -p 4318:4318 \
  -e CUSTOM_OTELCOL_CONFIG_FILE=/etc/otelcol-contrib/custom.config.yaml \
  -v "$(pwd)/mysql-logs-demo.yaml:/etc/otelcol-contrib/custom.config.yaml:ro" \
  -v "$(pwd)/error.log:/tmp/mysql-demo/error.log:ro" \
  -v "$(pwd)/mysql-slow.log:/tmp/mysql-demo/mysql-slow.log:ro" \
  clickhouse/clickstack-all-in-one:latest
```

Verify logs in HyperDX

#### Verify logs in HyperDX[​](#verify-demo-logs "Direct link to Verify logs in HyperDX")

Once ClickStack is running:

1. Wait a few moments for ClickStack to fully initialize (typically 30-60 seconds)
2. Open [HyperDX](http://localhost:8080/) and log in to your account (you may need to create an account first)
3. Navigate to the Search view and set the source to `Logs`
4. Set the time range to **2025-11-13 00:00:00 - 2025-11-16 00:00:00**
5. You should see 40 logs total (30 error logs with `source:mysql-demo-error` + 10 slow queries with `source:mysql-demo-slow`)

Note

If you don't see all 40 logs immediately, wait about a minute for the collector to finish processing. If logs still don't appear after waiting, run `docker restart clickstack-demo` and check again after another minute. This is a known issue with the OpenTelemetry filelog receiver when bulk-loading pre-existing files with `start_at: beginning`. Production deployments using `start_at: end` process logs as they're written in real-time and don't experience this issue.

Timezone Display

HyperDX displays timestamps in your browser's local timezone. The demo data spans **2025-11-14 00:00:00 - 2025-11-15 00:00:00 (UTC)**. The wide time range ensures you'll see the demo logs regardless of your location. Once you see the logs, you can narrow the range to a 24-hour period for clearer visualizations.

## Dashboards and visualization[​](#dashboards "Direct link to Dashboards and visualization")

To help you get started monitoring MySQL with ClickStack, we provide essential visualizations for MySQL logs.

Download the dashboard configuration

#### [Download](/docs/examples/mysql-logs-dashboard.json) the dashboard configuration[​](#download "Direct link to download")

Import the pre-built dashboard

#### Import the pre-built dashboard[​](#import-dashboard "Direct link to Import the pre-built dashboard")

1. Open HyperDX and navigate to the Dashboards section
2. Click **Import Dashboard** in the upper right corner under the ellipses

3. Upload the `mysql-logs-dashboard.json` file and click **Finish Import**

View the dashboard

#### View the dashboard[​](#created-dashboard "Direct link to View the dashboard")

The dashboard will be created with all visualizations pre-configured.

Note

For the demo dataset, set the time range to **2025-11-14 00:00:00 - 2025-11-15 00:00:00 (UTC)** (adjust based on your local timezone). The imported dashboard won't have a time range specified by default.

## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting")

### Custom config not loading[​](#troubleshooting-not-loading "Direct link to Custom config not loading")

Verify the environment variable is set:

```
docker exec <container-name> printenv CUSTOM_OTELCOL_CONFIG_FILE
```

Check the custom config file is mounted and readable:

```
docker exec <container-name> cat /etc/otelcol-contrib/custom.config.yaml | head -10
```

### No logs appearing in HyperDX[​](#no-logs "Direct link to No logs appearing in HyperDX")

Check the effective config includes your filelog receiver:

```
docker exec <container> cat /etc/otel/supervisor-data/effective.yaml | grep -A 10 filelog
```

Check for errors in the collector logs:

```
docker exec <container> cat /etc/otel/supervisor-data/agent.log | grep -i mysql
```

If using the demo dataset, verify the log files are accessible:

```
docker exec <container> cat /tmp/mysql-demo/error.log | wc -l
docker exec <container> cat /tmp/mysql-demo/mysql-slow.log | wc -l
```

### Slow query logs not appearing[​](#no-slow-queries "Direct link to Slow query logs not appearing")

Verify slow query log is enabled in MySQL:

```
SHOW VARIABLES LIKE 'slow_query_log';
SHOW VARIABLES LIKE 'long_query_time';
```

Check if MySQL is writing slow queries:

```
tail -f /var/log/mysql/mysql-slow.log
```

Generate a test slow query:

```
SELECT SLEEP(2);
```

### Logs not parsing correctly[​](#logs-not-parsing "Direct link to Logs not parsing correctly")

Verify your MySQL log format matches the expected format. The regex patterns in this guide are designed for MySQL 5.7+ and 8.0+ default formats.

Check a few lines from your error log:

```
head -5 /var/log/mysql/error.log
```

Expected format: