Core dump

A
core dump
is a file containing a process's address space (memory) when the process terminates unexpectedly. Core dumps may be produced on-demand (such as by a
debugger
), or automatically upon termination. Core dumps are triggered by the kernel in response to program crashes, and may be passed to a helper program (such as
systemd-coredump(8)
) for further processing. A core dump is not typically used by an average user, but developers could use it as a post-mortem snapshot of the program's state at the time of the crash, especially if the fault is hard to reliably reproduce.
Warning
Core dumps should be shared only with trusted parties as they may contain sensitive data (such as passwords or cryptographic keys).
Disabling automatic core dumps
Users may wish to disable automatic core dumps for a number of reasons:
Performance: generating core dumps for memory-heavy processes can waste system resources and delay the cleanup of memory.
Disk space: core dumps of memory-heavy processes may consume disk space equal to, if not greater, than the process's memory footprint if not compressed.
Security: core dumps, although typically readable only by root, may contain sensitive data (such as passwords or cryptographic keys), which are written to disk following a crash.
Using sysctl
sysctl
can be used to set the
kernel.core_pattern
to nothing to disable core dump handling. Create this file
/etc/sysctl.d/50-coredump.conf
kernel.core_pattern=|/bin/false
To apply the setting immediately, use
sysctl
:
# sysctl -p /etc/sysctl.d/50-coredump.conf
Using systemd
systemd
's default behavior is defined in
/usr/lib/sysctl.d/50-coredump.conf
, which sets
kernel.core_pattern
to call
systemd-coredump
. It generates core dumps for all processes in
/var/lib/systemd/coredump
.
systemd-coredump
behavior can be overridden by creating a configuration snippet in the
/etc/systemd/coredump.conf.d/
directory with the following content (See
coredump.conf(5) § DESCRIPTION
,
[1]
):
/etc/systemd/coredump.conf.d/custom.conf
[Coredump]
Storage=none
ProcessSizeMax=0
Note
Do not forget to include the
[Coredump]
section name, otherwise this option will be ignored.
Then reload the systemd manager configuration with
daemon-reload
.
See
systemd-coredump(8) § Disabling coredump processing
.
Using PAM limits
See
limits.conf#core
.
Using ulimit
Command-line shells
such as
bash
or
zsh
provide a builtin
ulimit
command which can be used to report or set resource limits of the shell and the processes started by the shell. See
bash(1) § SHELL BUILTIN COMMANDS
or
zshbuiltins(1)
for details.
To disable core dumps in the current shell:
$ ulimit -c 0
If the system is setup to pipe coredumps into a program such as
systemd-coredump
using
kernel.core_pattern
, the Linux kernel itself ignores the
ulimit
setting (see
core(5)
), so then it depends on the program the dump gets piped to whether this setting is respected or not (
systemd-coredump
will still use it).
For programs not using the
ulimit
setting of the crashed process,
dumpable
prctl(2)
can be used to disable coredump processing for selected processes.
Making a core dump
To generate a core dump of an arbitrary process, first
install
the
gdb
package. Then attach to this process by following
Debugging/Getting traces#Attaching to an existing process
.
Then at the
(gdb)
prompt:
(gdb) generate-core-file
Saved corefile core.2071
(gdb) quit
Now you have a coredump file called
core.2071
.
Where do they go?
The
kernel.core_pattern
sysctl
decides where automatic core dumps go. By default, core dumps are sent to
systemd-coredump
which can be configured in
/etc/systemd/coredump.conf
. By default, all core dumps are stored in
/var/lib/systemd/coredump
(due to
Storage=external
) and they are compressed with
zstd
(due to
Compress=yes
). Additionally, various size limits for the storage can be configured.
Note
The default value for
kernel.core_pattern
is set in
/usr/lib/sysctl.d/50-coredump.conf
. This file may be masked or overridden to use a different setting following normal
sysctl.d(5)
rules. Setting
kernel.core_pattern=core
will make core dumps be stored in plain
core.
number
files in the working directory of the crashed process.
To retrieve a core dump from the journal, see
coredumpctl(1)
.
Managing the core dump files
Use
coredumpctl
to find the corresponding dump. Note that regular users can run
coredumpctl
without special privileges to manage core dumps of their processes.
# coredumpctl list
Cleanup of core dump files
The core dump files stored in
/var/lib/systemd/coredump/
will be automatically cleaned by
systemd-tmpfiles --clean
, which is triggered daily with
systemd-tmpfiles-clean.timer
. Core dumps are configured to persist for at least 2 weeks, see
systemd-tmpfiles --cat-config
.
See
Journal#Clean journal files manually
to remove the entries.
Analyzing a core dump
First, you need to uniquely identify the relevant dump. This is possible by specifying a
PID
, name of the executable, path to the executable or a
journalctl
predicate (see
coredumpctl(1)
and
journalctl(1)
for details). To see details of the core dumps:
# coredumpctl info
match
Pay attention to "Signal" row, that helps to identify crash cause. For the analysis one usually examine the backtrace using a debugger (
gdb(1)
by default):
# coredumpctl debug
match
When
gdb
is started, use the
bt
command to print the full backtrace:
(gdb) thread apply all backtrace full
In many cases, the output will contain question marks as placeholders for missing debugging symbols. See
Debugging/Getting traces
for how to obtain them.
See also
american fuzzy lop
- A tool for automated tests of the kernel and programs
Filesystem fuzzing
- LWN article about testing filesystems for bugs