Allocation profiling

## Additional options for the profiler[​](#additional-options-for-profiler "Direct link to Additional options for the profiler")

`jemalloc` has many different options available, which are related to the profiler. They can be controlled by modifying the `MALLOC_CONF` environment variable.
For example, the interval between allocation samples can be controlled with `lg_prof_sample`.
If you want to dump the heap profile every N bytes you can enable it using `lg_prof_interval`.

It is recommended to check `jemalloc`s [reference page](https://jemalloc.net/jemalloc.3.html) for a complete list of options.

## Other resources[​](#other-resources "Direct link to Other resources")

ClickHouse/Keeper expose `jemalloc` related metrics in many different ways.

Warning

It's important to be aware that none of these metrics are synchronized with each other and values may drift.

### System table `asynchronous_metrics`[​](#system-table-asynchronous_metrics "Direct link to system-table-asynchronous_metrics")

```
SELECT *
FROM system.asynchronous_metrics
WHERE metric LIKE '%jemalloc%'
FORMAT Vertical
```

[Reference](/docs/operations/system-tables/asynchronous_metrics)

### System table `jemalloc_bins`[​](#system-table-jemalloc_bins "Direct link to system-table-jemalloc_bins")

Contains information about memory allocations done via the jemalloc allocator in different size classes (bins) aggregated from all arenas.

[Reference](/docs/operations/system-tables/jemalloc_bins)

### System table `jemalloc_stats` (26.2+)[​](#system-table-jemalloc_stats "Direct link to system-table-jemalloc_stats")

Returns the full output of `malloc_stats_print()` as a single string. Equivalent to the `SYSTEM JEMALLOC STATS` command.

```
SELECT * FROM system.jemalloc_stats
```

### Prometheus[​](#prometheus "Direct link to Prometheus")

All `jemalloc` related metrics from `asynchronous_metrics` are also exposed using the Prometheus endpoint in both ClickHouse and Keeper.

[Reference](/docs/operations/server-configuration-parameters/settings#prometheus)

### `jmst` 4LW command in Keeper[​](#jmst-4lw-command-in-keeper "Direct link to jmst-4lw-command-in-keeper")

Keeper supports the `jmst` 4LW command which returns [basic allocator statistics](https://github.com/jemalloc/jemalloc/wiki/Use-Case%3A-Basic-Allocator-Statistics):

```
echo jmst | nc localhost 9181
```

- [Sampling allocations](#sampling-allocations)- [Storing jemalloc samples in `system.trace_log`](#storing-jemalloc-samples-in-system-trace-log)
    - [Example: analyzing memory usage of a query](#example-analyzing-memory-usage-trace-log)- [Jemalloc web UI](#jemalloc-web-ui)- [Fetching heap profiles from SQL](#fetching-heap-profiles-from-sql)
        - [Output format](#output-format)- [Additional settings](#fetching-heap-profiles-settings)- [Example: generating a flame graph from SQL](#example-flamegraph-from-sql)- [Flushing heap profiles to disk](#flushing-heap-profiles)- [Analyzing heap profile files with `jeprof`](#analyzing-heap-profile-files-with-jeprof)
            - [Symbolized heap profiles](#symbolized-heap-profiles)- [Examples](#examples)- [Generating a flame graph](#generating-flame-graph)- [Additional options for the profiler](#additional-options-for-profiler)- [Other resources](#other-resources)
                - [System table `asynchronous_metrics`](#system-table-asynchronous_metrics)- [System table `jemalloc_bins`](#system-table-jemalloc_bins)- [System table `jemalloc_stats` (26.2+)](#system-table-jemalloc_stats)- [Prometheus](#prometheus)- [`jmst` 4LW command in Keeper](#jmst-4lw-command-in-keeper)

Was this page helpful?