# pcp-atop

## NAME



           **pcp-atop** - Advanced System and Process Monitor
    


## SYNOPSIS



           Interactive Usage:
    
           **pcp**[_pcp options_] **atop**[**-aABcCdDfFgGHmMnNopRsuvxyY1**] [**-L** _linelen_]
           [**-P** _label_[,_label_]... [**-Z**]] [_interval_ [_samples_]]
    
           Writing and reading PCP archive folios:
    
           **pcp**[_pcp options_] **atop -w** _folio_ [**-a**] [**-S**] [_interval_ [_samples_]]
           **pcp**[_pcp options_] **atop -r** _folio_ [**-AcCdDfFgGmMnNopRsuvxy1**] [**-b**[_yy-_
           _mm-dd_] _hh:mm_] [**-e** _yy-mm-dd_] _hh:mm_] [**-L** _linelen_]
           [**-P** _label_[,_label_]... [**-Z**]] [_interval_ [_samples_]]
    


## DESCRIPTION



           The program **pcp-atop** is an interactive monitor to view various
           aspects of load on a system.  Every _interval_ seconds (default: 10
           seconds) information is gathered about the resource occupation on
           system level of the most critical hardware resources (from a
           performance point of view), i.e. CPUs, memory, disks and network
           interfaces. Besides, information is gathered about the processes
           (or threads) that are responsible for the utilization of the CPUs,
           memory and disks.  Network load per process is shown only when the
           optional pmdabpf(1) or pmdabcc(1) metrics have been installed and
           configured.
    


## BAR GRAPH MODE



           When running **pcp-atop** you can choose to view the system load in
           bar graph mode or in text mode.  In bar graph mode the resource
           utilization of CPUs, memory, disks and network interfaces is shown
           via (character-based) bar graphs, but only on system level.  When
           you want to view more detailed information on system level or when
           you want to view the resource consumption on process or thread
           level, you can switch to text mode by pressing the 'B' key.
           Alternatively, you can use the 'B' key (again) to switch from text
           mode to bar graph mode.
           By default, **pcp-atop** starts in text mode unless the _-B_ flag is
           used or unless 'B' has been configured as a default flag in the
           _.atoprc_ file (for further information about default flags, refer
           to the pcp-atoprc(5) man page).
    
           In bar graph mode the terminal will be subdivided into four
           character-based windows, i.e. one window for each hardware
           resource:
    
           **Processors**
                The first bar shows the average busy percentage of all CPUs
                with the bar label 'Avg' (might be abbreviated to 'Av' or
                even just 'A').  The subsequent bars show the busy
                percentages of single CPUs.
                When there is not enough horizontal space to show all CPUs,
                only the most busy CPUs per sample will be shown after the
                width of each bar has been reduced to a minimum.
    
                By default, the categories of CPU consumption are shown by
                different colors in the bars, marked with a character 'S'
                (system mode), 'U' (user mode), 'I' (interrupt handling), 's'
                (steal) and 'G' (guest, i.e. consumed by virtual machines).
                The top of the bar might consist of an unmarked color
                representing a 'neutral' category. Suppose that the scale
                unit is 5% per line and the total busy percentage is 54%
                consisting of two categories of 27%.  The two categories will
                be rounded to 25% (5 lines of 5% each) but the total busy
                percentage will be rounded to 55% (11 lines of 5%).  Then the
                top line will represent a 'neutral' category.
                By pressing the 'H' key or by starting **pcp-atop** with the _-H_
                flag, no categories are shown.
    
                A red line is drawn in the bar graph as critical threshold.
                By default this value is 90% and can be modified by the
                'cpucritperc' option in the configuration file (see separate
                pcp-atoprc(5) man page).  When this value is set to zero, no
                threshold line will be drawn.
    
           **Memory and swap space**
                Memory is presented as a column in which the specific
                categories of memory consumption are shown. These categories
                are (code, data and  stack of) processes/kernel, slab caches
                (i.e. dynamically allocated kernel memory), shared memory,
                tmpfs, static huge pages, page cache and free memory.
                Swap space (if present) is also presented as a column in
                which the categories processes/tmpfs, shared memory and free
                space are shown.
    
                At the right side memory-related event counters are shown.
                The bottom three counters are colored green when there is no
                memory pressure.  When considerable activity is noticed such
                counter might be colored orange and with high activity red.
                When memory pressure starts, usually memory page scanning
                will be activated first. When pressure increases, memory
                pages of processes might be swapped out to swap space (if
                present).
                The 'oomkills' counter (Out Of Memory killing) is most
                serious: it reflects the number of processes that are killed
                due to lack of memory (and swap). Therefore this counter
                shows the absolute number (not per second) of processes being
                killed during the last interval and will immediately be
                colored red when it is 1 or more.  Besides, after **pcp-atop**
                has noticed OOM killing the 'oomkills' counter remains orange
                for the next 15 minutes, just in case that you have missed
                the OOM killing event itself.
                When there is enough vertical space in the memory window,
                event counters are shown about the number of memory pages
                being swapped in, the number of memory pages paged out to
                block devices and the number of memory pages paged in from
                block devices.
    
                Memory and swap space consumption will preferably be shown in
                a character-based window that vertically uses the entire
                screen for optimal granularity. However, when there are a lot
                of disks and/or network interfaces the memory and swap space
                consumption will be shown in a character-based window that
                only uses the upper half of the screen.
    
           **Disks**
                For each disk the busy percentage is shown as a bar.
                When there is not enough horizontal space to show all disks,
                only the most busy disks per sample will be shown.
    
                By default, categories of disk consumption are shown by
                different colors in the bars, marked with a character 'R'
                (read) and 'W' (write).
                The top of the bar might consist of an unmarked color
                representing a 'neutral' category. Suppose that the scale
                unit is 5% per line and the total busy percentage is 54%
                consisting of two categories of 27%.  The two categories will
                be rounded to 25% (5 lines of 5% each) but the total busy
                percentage will be rounded to 55% (11 lines of 5%).  Then the
                top line will represent a 'neutral' category.
                By pressing the 'H' key or by starting **pcp-atop** with the _-H_
                flag, no categories are shown.
    
                A red line is drawn in the bar graph as critical threshold.
                By default this value is 90% and can be modified by the
                'dskcritperc' option in the configuration file (see separate
                **atoprc** man page). When this value is set to zero, no
                threshold line will be drawn.
    
           **Interfaces**
                For each non-virtual network interface a double bar graph is
                shown with a dedicated scale that reflects the traffic rate.
                One of the bars shows the transmit rate ('TX') and the other
                bar the receive rate ('RX').  The traffic scale of each
                network interface remains at its highest level.  All
                interface scales can be reset during the measurement by
                pressing the 'L' key.
    
                Most often the real speed (maximum bandwidth) of network
                interfaces is not known, e.g. in case of the network
                interfaces of virtual machines.  Therefore it is not possible
                to show the interface utilization as a percentage. However,
                when the real speed of an interface is known it will be shown
                underneath the concerning bar graph.
    
                When there is not enough horizontal space to show all network
                interfaces, only the most busy interfaces per sample will be
                shown.
    
           Usually the bar graphs will not be sorted on busy percentage when
           there is enough horizontal space. However, after switching from
           text mode to bar graph mode the bar graphs might have been sorted
           because this was needed for the presentation in text mode. The
           next interval in bar graph mode shows the bars unsorted again
           unless the window width is insufficient for all bars.
    
           The remaining part of this manual page mainly describes the
           information shown in text mode.  When certain descriptions also
           apply to bar graph mode it will be mentioned explicitly.
    


## TEXT MODE IN GENERAL



           The initial screen in text mode shows if **pcp-atop** runs with
           restricted view (unprivileged user) or unrestricted view
           (privileged user).  In case of restricted view **pcp-atop** does not
           have the privileges (root identity or necessary capabilities) to
           retrieve all counter values on system level and on process level.
           does not have the privileges (no root identity nor the necessary
           capabilities) to retrieve all counter values on system level and
           on process level.
    
           With every interval information is shown about the resource
           occupation on system level (CPU, memory, disks and network
           layers), followed by a list of processes which have been active
           during the last interval.  Notice that all processes that were
           unchanged during the last interval re not shown, unless the key
           'a' has been pressed or unless sorting on memory occupation is
           done (then inactive processes are relevant as well).  If the list
           of active processes does not entirely fit on the screen, only the
           top of the list is shown (sorted in order of activity).
           The intervals are repeated till the number of _samples_ (specified
           as command argument) is reached, or till the key 'q' is pressed in
           interactive mode.
    
           When invoked via the pcp(1) command, the PCPIntro(1) options
           **-A** /**--align** , **-a** /**--archive** , **-h** /**--host** , **-O** /**--origin** , **-S** /**--start** ,
           **-s** /**--samples** , **-T** /**--finish** , **-t** /**--interval** , **-v** /**--version** ,
           **-z** /**--hostzone** and **-z** /**--timezone** become indirectly available.
           Additionally, the **--hotproc** option can be used to request the per-
           process PCP metrics be used instead of the default proc metrics
           from pmdaproc(1).
    
           When **pcp-atop** is started, it checks whether the standard output
           channel is connected to a screen, or to a file/pipe.  In the first
           case it produces screen control codes (via the ncurses library)
           and behaves interactively; in the second case it produces flat
           text output.
    
           In interactive mode, the output of **pcp-atop** scales dynamically to
           the current dimensions of the screen/window.
           If the window is resized horizontally, columns will be added or
           removed automatically. For this purpose, every column has a par‐
           ticular weight.  The columns with the highest weights that fit
           within the current width will be shown.
           If the window is resized vertically, lines of the process/thread
           list will be added or removed automatically.
    
           In interactive mode the output of **pcp-atop** can be controlled by
           pressing particular keys.  However it is also possible to specify
           such key as **flag** on the command line.  In that case **pcp-atop**
           switches to the indicated mode on beforehand.  This mode can be
           modified again interactively.  Specifying such key as flag is es‐
           pecially useful when running **pcp-atop** with output to a pipe or
           file (non-interactively).  These flags are the same as the keys
           that can be pressed in interactive mode (see section INTERACTIVE
           COMMANDS).
           Additional flags are available to support storage of **pcp-atop** data
           in PCP archive format (see section PCP DATA STORAGE).
    


## COLORS



           For the resource consumption on system level, **pcp-atop** uses colors
           in text mode to indicate that a critical occupation percentage has
           been (almost) reached.  A critical occupation percentage means
           that is likely that this load causes a noticeable negative perfor‐
           mance influence for applications using this resource.  The criti‐
           cal percentage depends on the type of resource: e.g. the perfor‐
           mance influence of a disk with a busy percentage of 80% might be
           more noticeable for applications/users than a CPU with a busy per‐
           centage of 90%.
           Currently **pcp-atop** uses the following default values to calculate
           a weighted percentage per resource:
    
            **Processor**
                A busy percentage of 90% or higher is considered 'critical'
                (also in bar graph mode).
    
            **Disk**
                A busy percentage of 90% or higher is considered 'critical'.
    
            **Network**
                A busy percentage of 90% or higher for the load of an inter‐
                face is considered 'critical'.
    
            **Memory**
                An occupation percentage of 90% is considered 'critical'.
                Notice that this occupation percentage is the accumulated
                memory consumption of the kernel (including slab) and all
                processes. The memory for the page cache ('cache' and 'buff'
                in the MEM-line) and the reclaimable part of the slab
                ('slrec') is not implied!
                If the number of pages swapped out ('swout' in the PAG-line)
                is larger than 10 per second, the memory resource is consid‐
                ered 'critical'.  A value of at least 1 per second is consid‐
                ered 'almost critical'.
                If the committed virtual memory exceeds the limit ('vmcom'
                and 'vmlim' in the SWP-line), the SWP-line is colored due to
                overcommitting the system.
    
            **Swap**
                An occupation percentage of 80% is considered 'critical' be‐
                cause swap space might be completely exhausted in the near
                future.  It is not critical from a performance point-of-view.
    
           These default values can be modified in the configuration file
           (see separate pcp-atoprc(5) man page).
    
           When a resource exceeds its critical occupation percentage, the
           concerning values in the screen line are colored red by default.
           When a resource exceeds (by default) 80% of its critical percent‐
           age (so it is almost critical), the concerning values in the
           screen line are colored cyan by default.  This 'almost critical
           percentage' (one value for all resources) can be modified in the
           configuration file (see separate pcp-atoprc(5) man page).
           The default colors red and cyan can be modified in the configura‐
           tion file as well (see separate man-page of pcp-atoprc(5)).
    
           With the key 'x' (or flag **-x**), the use of colors can be suppressed
           in text mode.  The use of colors is however mandatory in case of
           bar graph mode.
    


## NETATOP BPF MODULE



           Per-process and per-thread network activity can be measured by the
           _netatop_ BPF module that can be separately installed with
           pmdabpf(1).  or pmdabcc(1).
           When **pcp-atop** gathers counters for a new interval, it verifies if
           the eBPF module is currently active. If so, **pcp-atop** obtains the
           relevant network counters from this module and shows the number of
           sent and received packets per process/thread in the generic
           screen. Besides, detailed counters can be requested by pressing
           the 'n' key.
    


## GPU STATISTICS GATHERING



           GPU statistics can be gathered by pmdanvidia(1) which is a sepa‐
           rate data collection daemon process.  It gathers cumulative uti‐
           lization counters of every Nvidia GPU in the system, as well as
           utilization counters of every process that uses a GPU.  When **pcp-**
           **atop** notices that the daemon is active, it reads these GPU uti‐
           lization counters with every interval.
    
           Find a description about the utilization counters in the section
           OUTPUT DESCRIPTION.
    


## INTERACTIVE COMMANDS



           When running **pcp-atop** interactively (no output redirection), keys
           can be pressed to control the output.  In general, lower case keys
           can be used to show other information for the active processes
           while certain upper case keys can be used to influence the sort
           order of the active process/thread list. Some of these keys can
           also be used to switch from bar graph mode to particular detailed
           process information in text mode.
    
           **g** Show generic output (default).
    
                Per process the following fields are shown in case of a win‐
                dow-width of 80 positions: process-id, CPU consumption during
                the last interval in system and user mode, the virtual and
                resident memory growth of the process.
                The data transfer per process for read/write on disk can only
                be shown when **pcp-atop** accesses metrics with root privileges.
                When the optional pmdabpf(1) or pmdabcc(1) module **netatop** is
                loaded, the data transfer for send/receive of network packets
                is shown for each process.
                The last columns contain the state, the occupation percentage
                for the chosen resource (default: CPU) and the process name.
    
                When more than 80 positions are available, other information
                is added.
    
           **m** Show memory related output.
    
                Per process the following fields are shown in case of a win‐
                dow width of 80 positions: process-id, minor and major memory
                faults, size of virtual shared text, total virtual process
                size, total resident process size, virtual and resident
                growth during last interval, memory occupation percentage and
                process name.
    
                When more than 80 positions are available, other information
                is added.
    
                For memory consumption, always all processes are shown (also
                the processes that were not active during the interval).
    
           **d** Show disk-related output.
    
                When **pcp-atop** runs with root privileges, the following fields
                are shown: process-id, amount of data read from disk, amount
                of data written to disk, amount of data that was written but
                has been withdrawn again (WCANCL), disk occupation percentage
                and process name.
    
           **n** Show network related output.
    
                Per process the following fields are shown in case of a win‐
                dow width of 80 positions: process-id, thread-id, total band‐
                width for received packets, total bandwidth for sent packets,
                number of received TCP packets with the average size per
                packet (in bytes), number of sent TCP packets with the aver‐
                age size per packet (in bytes), number of received UDP pack‐
                ets with the average size per packet (in bytes), number of
                sent UDP packets with the average size per packet (in bytes),
                the network occupation percentage and process name.
                This information can only be shown when the optional
                pmdabpf(1) or pmdabcc(1) module _netatop_ is installed.
    
                When more than 80 positions are available, other information
                is added.
    
           **s** Show scheduling characteristics.
    
                Per process the following fields are shown in case of a win‐
                dow width of 80 positions: process-id, number of threads in
                state 'running' (R), number of threads in state 'interrupt‐
                ible sleeping' (S), number of threads in state 'uninterrupt‐
                ible sleeping' (D), number of threads in state 'idle' (I),
                scheduling policy (normal timesharing, realtime round-robin,
                realtime fifo), nice value, priority, realtime priority, cur‐
                rent processor, status, exit code, state, the occupation per‐
                centage for the chosen resource and the process name.
    
                When more than 80 positions are available, other information
                is added.
    
           **v** Show various process characteristics.
    
                Per process the following fields are shown in case of a win‐
                dow width of 80 positions: process-id, user name and group,
                start date and time, status (e.g. exit code if the process
                has finished), state, the occupation percentage for the cho‐
                sen resource and the process name.
    
                When more than 80 positions are available, other information
                is added.
    
           **c** Show the command line of the process.
    
                Per process the following fields are shown: process-id, the
                occupation percentage for the chosen resource and the command
                line including arguments.
    
           **X** Show cgroup v2 information.
    
                Per process the following fields are shown: process-id, the
                command name, and the cgroup path name (horizontally scrol‐
                lable).
    
           **e** Show GPU utilization.
    
                Per process at least the following fields are shown: process-
                id, range of GPU numbers on which the process currently runs,
                GPU busy percentage on all GPUs, memory busy percentage (i.e.
                read and write accesses on memory) on all GPUs, memory occu‐
                pation at the moment of the sample, average memory occupation
                during the sample, and GPU percentage.
    
                When the **pmdanvidia** daemon does not run with root privileges,
                the GPU busy percentage and the memory busy percentage are
                not available on process level.  In that case, the GPU per‐
                centage on process level reflects the GPU memory occupation
                instead of the GPU busy percentage (which is preferred).
    
           **o** Show the user-defined line of the process.
    
                In the configuration file the keyword _ownprocline_ can be
                specified with the description of a user-defined output-line.
                Refer to the man-page of pcp-atoprc(5) for a detailed de‐
                scription.
    
           **y** Show the individual threads within a process (toggle).
    
                Single-threaded processes are still shown as one line.
                For multi-threaded processes, one line represents the process
                while additional lines show the activity per individual
                thread (in a different color).  Depending on the option 'a'
                (all or active toggle), all threads are shown or only the
                threads that were active during the last interval.  Depending
                on the option 'Y' (sort threads), the threads per process
                will be sorted on the chosen sort criterion or not.
                Whether this key is active or not can be seen in the header
                line.
    
           **Y** Sort the threads per process when combined with option 'y'
                (toggle).
    
           **u** Show the process activity accumulated per user.
    
                Per user the following fields are shown: number of processes
                active or terminated during last interval (or in total if
                combined with command 'a'), accumulated CPU consumption dur‐
                ing last interval in system and user mode, the current virtu‐
                al and resident memory space consumed by active processes (or
                all processes of the user if combined with command 'a').
                When **pcp-atop** access metrics with root privileges, the accu‐
                mulated read and write throughput on disk is shown.  When the
                optional pmdabpf(1) or pmdabcc(1) module 'netproc' has been
                installed, the number of receive and send network calls are
                shown.
                The last columns contain the accumulated occupation percent‐
                age for the chosen resource (default: CPU) and the user name.
    
           **p** Show the process activity accumulated per program (i.e.
                process name).
    
                Per program the following fields are shown: number of
                processes active or terminated during last interval (or in
                total if combined with command 'a'), accumulated CPU consump‐
                tion during last interval in system and user mode, the cur‐
                rent virtual and resident memory space consumed by active
                processes (or all processes of the user if combined with com‐
                mand 'a').
                When **pcp-atop** access metrics with root privileges, the accu‐
                mulated read and write throughput on disk is shown.  When the
                pmdabcc(1) module 'netproc' has been installed, the number of
                receive and send network calls are shown.
                The last columns contain the accumulated occupation percent‐
                age for the chosen resource (default: CPU) and the program
                name.
    
           **j** Show the process activity accumulated per container/pod.
    
                Per container (e.g. Docker/Podman) or pod (e.g. Kubernetes)
                the following fields are shown: number of processes active or
                terminated during last interval (or in total if combined with
                command 'a'), accumulated CPU consumption during last inter‐
                val in system and user mode, the current virtual and resident
                memory space consumed by active processes (or all processes
                of the user if combined with command 'a').
                When **pcp-atop** access metrics with root privileges, the accu‐
                mulated read and write throughput on disk is shown.  When the
                pmdabpf(1) or pmdabcc(1) module 'netproc' has been installed,
                the number of receive and send network calls are shown.
                The last columns contain the accumulated occupation percent‐
                age for the chosen resource (default: CPU) and the contain‐
                er/pod name (CID/POD).
    
           **C** Sort the current list in the order of CPU consumption (de‐
                fault).  The one-but-last column changes to 'CPU'.
    
           **E** Sort the current list in the order of GPU utilization (pre‐
                ferred, but only applicable when the **pmdanvidia** daemon runs
                under root privileges) or the order of GPU memory occupa‐
                tion).  The one-but-last column changes to 'GPU'.
    
           **M** Sort the current list in the order of resident memory con‐
                sumption.  The one-but-last column changes to 'MEM'.  In case
                of sorting on memory, the full process list will be shown
                (not only the active processes).
    
           **D** Sort the current list in the order of disk accesses issued.
                The one-but-last column changes to 'DSK'.
    
           **N** Sort the current list in the order of network bandwidth (re‐
                ceived and transmitted).  The one-but-last column changes to
                'NET'.
    
           **A** Sort the current list automatically in the order of the most
                busy system resource during this interval.  The one-but-last
                column shows either 'ACPU', 'AMEM', 'ADSK' or 'ANET' (the
                preceding 'A' indicates automatic sorting-order).  The most
                busy resource is determined by comparing the weighted busy-
                percentages of the system resources, as described earlier in
                the section COLORS.
                This option remains valid until another sorting-order is ex‐
                plicitly selected again.
                A sorting order for disk is only possible when **pcp-atop** runs
                with root privileges.  A sorting-order for network is only
                possible when the pmdabpf(1) or pmdabcc(1) module 'netproc'
                has been installed.
    
           Miscellaneous interactive commands:
    
           **?** Request for help information (also the key 'h' can be
                pressed).
    
           **V** Request for version information (version number and date).
    
           **R** Gather and calculate the proportional set size of processes
                (toggle).  Gathering of all values that are needed to calcu‐
                late the PSIZE of a process is a very time-consuming task, so
                this key should only be active when analyzing the resident
                memory consumption of processes.
    
           **W** Get the WCHAN per thread (toggle).  Gathering of the WCHAN
                string per thread is a relatively time-consuming task, so
                this key should only be made active when analyzing the reason
                for threads to be in sleep state.
    
           **x** Suppress colors to highlight critical resources (toggle).
                Whether this key is active or not can be seen in the header
                line.
    
           **z** The pause key can be used to freeze the current situation in
                order to investigate the output on the screen.  While **pcp-**
                **atop** is paused, the keys described above can be pressed to
                show other information about the current list of processes.
                Whenever the pause key is pressed again, **pcp-atop** will con‐
                tinue with the next sample.
                The pause key can be used in text mode and bar graph mode.
    
           **i** Modify the interval timer (default: 10 seconds).  If an in‐
                terval timer of 0 is entered, the interval timer is switched
                off.  In that case a new sample can only be triggered manual‐
                ly by pressing the key 't'.
                The interval can be modified in text mode and bar graph mode.
    
           **t** Trigger a new sample manually.  This key can be pressed if
                the current sample should be finished before the timer has
                exceeded, or if no timer is set at all (interval timer de‐
                fined as 0).  In the latter case **pcp-atop** can be used as a
                stopwatch to measure the load being caused by a particular
                application transaction, without knowing on beforehand how
                many seconds this transaction will last.
                This key can be used in text mode and bar graph mode.
    
                When viewing the contents of an archive folio, this key can
                be used to show the next sample from the folio.
    
           **T** When viewing the contents of an archive folio, this key can
                be used to show the previous sample from the folio.
                This key can be used in text mode and bar graph mode.
    
           **b** When viewing the contents of an archive folio, this key can
                be used to move to a certain timestamp within the file (ei‐
                ther forward or backward).
                This key can be used in text mode and bar graph mode.
    
           **r** Reset all counters to zero to see the system and process ac‐
                tivity since boot again.
                This key can be used in text mode and bar graph mode.
    
                When viewing the contents of an archive, this key can be used
                to rewind to the beginning of the file again.
    
           **U** Specify a search string for specific user names as a regular
                expression.  From now on, only (active) processes will be
                shown from a user which matches the regular expression.  The
                system statistics are still system wide.  If the Enter-key is
                pressed without specifying a name, (active) processes of all
                users will be shown again.
                Whether this key is active or not can be seen in the header
                line.
    
           **I** Specify a list with one or more PIDs to be selected.  From
                now on, only processes will be shown with a PID which matches
                one of the given list.  The system statistics are still sys‐
                tem wide.  If the Enter-key is pressed without specifying a
                PID, all (active) processes will be shown again.
                Whether this key is active or not can be seen in the header
                line.
    
           **P** Specify a search string for specific process names as a regu‐
                lar expression.  From now on, only processes will be shown
                with a name which matches the regular expression.  The system
                statistics are still system wide.  If the Enter-key is
                pressed without specifying a name, all (active) processes
                will be shown again.
                Whether this key is active or not can be seen in the header
                line.
    
           **/** Specify a specific command line search string as a regular
                expression.  From now on, only processes will be shown with a
                command line which matches the regular expression.  The sys‐
                tem statistics are still system wide.  If the Enter-key is
                pressed without specifying a string, all (active) processes
                will be shown again.
                Whether this key is active or not can be seen in the header
                line.
    
           **J** Specify a container id (e.g. Docker or Podman) or pod name
                (e.g. Kubernetes) of maximum 15 characters. In case the name
                is longer, the last 15 characters are expected.  From now on,
                only processes will be shown that run in that specific con‐
                tainer or pod.  The system statistics are still system wide.
                If the Enter-key is pressed without specifying a container id
                or pod name, all (active) processes will be shown again.
                Whether this key is active or not can be seen in the header
                line.
    
           **Q** Specify a comma-separated list of process state characters.
                From now on, only processes will be shown that are in those
                specific process states.  Accepted states are: R (running), S
                (sleeping), D (disk sleep), T (stopped), t (tracing stop), X
                (dead), Z (zombie) and P (parked).  The system statistics are
                still system wide.  If the Enter-key is pressed without spec‐
                ifying a state, all (active) processes will be shown again.
                Whether this key is active or not can be seen in the header
                line.
    
           **S** Specify search strings for specific logical volume names,
                specific disk names and specific network interface names.
                All search strings are interpreted as a regular expressions.
                From now on, only those system resources are shown that match
                the concerning regular expression.  If the Enter-key is
                pressed without specifying a search string, all (active) sys‐
                tem resources of that type will be shown again.
                Whether this key is active or not can be seen in the header
                line.
    
           **a** The 'all/active' key can be used to toggle between only show‐
                ing/accumulating the processes that were active during the
                last interval (default) or showing/accumulating all process‐
                es.
                Whether this key is active or not can be seen in the header
                line.
    
           **G** By default, **pcp-atop** shows/accumulates the processes that are
                alive and the processes that are exited during the last in‐
                terval.  With this key (toggle), showing/accumulating the
                processes that are exited can be suppressed.
                Whether this key is active or not can be seen in the header
                line.
    
           **f** Show a fixed (maximum) number of header lines for system re‐
                sources (toggle).  By default only the lines are shown about
                system resources (CPUs, paging, logical volumes, disks, net‐
                work interfaces) that really have been active during the last
                interval.  With this key you can force **pcp-atop** to show lines
                of inactive resources as well.
                Whether this key is active or not can be seen in the header
                line.
    
           **F** Suppress sorting of system resources (toggle).  By default
                system resources (CPUs, logical volumes, disks, network in‐
                terfaces) are sorted on utilization.
                Whether this key is active or not can be seen in the header
                line.
    
           **1** Show relevant counters as an average per second (in the for‐
                mat '..../s') instead of as a total during the interval (tog‐
                gle).
                Whether this key is active or not can be seen in the header
                line.
    
           **l** Limit the number of system level lines for the counters per-
                cpu, the active disks and the network interfaces.  By default
                lines are shown of all CPUs, disks and network interfaces
                which have been active during the last interval.  Limiting
                these lines can be useful on systems with huge number CPUs,
                disks or interfaces in order to be able to run **pcp-atop** on a
                screen/window with e.g. only 24 lines.
                For all mentioned resources the maximum number of lines can
                be specified interactively. When using the flag **-l** the maxi‐
                mum number of per-cpu lines is set to 0, the maximum number
                of disk lines to 5 and the maximum number of interface lines
                to 3.  These values can be modified again in interactive
                mode.
    
           **k** Send a signal to an active process (a.k.a. kill a process).
    
           **q** Quit the program.
                This key can be used in text mode and bar graph mode.
    
           **PgDn** Show the next page of the process/thread list.
                With the arrow-down key the list can be scrolled downwards
                with single lines.
    
           **^F** Show the next page of the process/thread list (forward).
                With the arrow-down key the list can be scrolled downwards
                with single lines.
    
           **PgUp** Show the previous page of the process/thread list.
                With the arrow-up key the list can be scrolled upwards with
                single lines.
    
           **^B** Show the previous page of the process/thread list (backward).
                With the arrow-up key the list can be scrolled upwards with
                single lines.
    
           **^L** Redraw the screen.
    


## PCP DATA STORAGE



           In order to store system and process level statistics for long-
           term analysis (e.g. to check the system load and the active
           processes running yesterday between 3:00 and 4:00 PM), **pcp-atop**
           can store the system and process level statistics in the PCP
           archive format, as an archive folio (see mkaf(1)).
           All information about processes and threads is stored in the
           archive.
           The interval (default: 10 seconds) and number of samples (default:
           infinite) can be passed as last arguments.  Instead of the number
           of samples, the flag **-S** can be used to indicate that **pcp-atop**
           should finish anyhow before midnight.
    
           A PCP archive can be read and visualized again with the **-r** option.
           The argument is a comma-separated list of names, each of which may
           be the base name of an archive or the name of a directory contain‐
           ing one or more archives.  If no argument is specified, the file
           **$PCP_LOG_DIR/pmlogger/HOST/YYYYMMDD** is opened for input (where
           _YYYYMMDD_ are digits representing the current date, and HOST is the
           hostname of the machine being logged).  If a filename is specified
           in the format YYYYMMDD (representing any valid date), the file
           **$PCP_LOG_DIR/pmlogger/HOST/YYYYMMDD** is opened.  If a filename with
           the symbolic name **y** is specified, yesterday's daily logfile is
           opened (this can be repeated so 'yyyy' indicates the logfile of
           four days ago).
           The samples from the file can be viewed interactively by using the
           key 't' to show the next sample, the key 'T' to show the previous
           sample, the key 'b' to branch to a particular time or the key 'r'
           to rewind to the beginning of the file.  These keys can be used in
           text mode as well as in bar graph mode.
           When output is redirected to a file or pipe, **pcp-atop** prints all
           samples in plain ASCII.  The default line length is 80 characters
           in that case. With the flag **-L** followed by an alternate line
           length, more (or less) columns will be shown.
           With the flag **-b**(begin time) and/or **-e**(end time) followed by a
           time argument of the form [YYYYMMDD]hhmm[ss], a certain time peri‐
           od within the archive can be selected.
    


## OUTPUT DESCRIPTION



           The first sample shows the system level activity since boot (the
           elapsed time in the header shows the time since boot).
    
           In text mode, **pcp-atop** first shows the lines related to system
           level activity for every sample.  If a particular system resource
           has not been used during the interval, the entire line related to
           this resource is suppressed. So the number of system level lines
           may vary for each sample.
           After that a list is shown of processes which have been active
           during the last interval.  This list is by default sorted on CPU
           consumption, but this order can be changed by the keys which are
           previously described.
    
           If values have to be shown by **pcp-atop** which do not fit in the
           column width, another format is used. If e.g. a CPU consumption of
           233216 milliseconds should be shown in a column width of 4 posi‐
           tions, it is shown as '233s' (in seconds).  For large memory fig‐
           ures, another unit is chosen if the value does not fit (Mb instead
           of Kb, Gb instead of Mb, Tb instead of Gb, etcetera).  For other
           values, a kind of exponent notation is used (value 123456789 shown
           in a column of 5 positions gives 123e6).
    


## OUTPUT DESCRIPTION - SYSTEM LEVEL



           The system level information in text mode consists of the follow‐
           ing output lines:
    
           **PRC** Process and thread level totals.
                This line contains the total CPU time consumed in system mode
                ('sys') and in user mode ('user'), the total number of
                processes present at this moment ('#proc'), the total number
                of threads present at this moment in state 'running'
                ('#trun'), 'sleeping interruptible' ('#tslpi'), 'sleeping un‐
                interruptible' ('#tslpu') and 'idle' ('#tidle'), the number
                of zombie processes ('#zombie'), the number of clone system
                calls ('clones'), and the number of processes that ended dur‐
                ing the interval ('#exit') when process accounting is used.
                Instead of '#exit' the last column may indicate that process
                accounting could not be activated ('no procacct').
                If the screen width does not allow all of these counters, on‐
                ly a relevant subset is shown.
    
           **CPU** CPU utilization.
                At least one line is shown for the total occupation of all
                CPUs together.
                In case of a multi-processor system, an additional line is
                shown for every individual processor (with 'cpu' in lower
                case), sorted on activity.  Inactive CPUs will not be shown
                by default.  The lines showing the per-cpu occupation contain
                the CPU number in the field combined with the wait percent‐
                age.
    
                Every line contains the percentage of CPU time spent in ker‐
                nel mode by all active processes ('sys'), the percentage of
                cpu time consumed in user mode ('user') for all active
                processes (including processes running with a nice value
                larger than zero), the percentage of CPU time spent for in‐
                terrupt handling ('irq') including softirq, the percentage of
                unused CPU time while no processes were waiting for disk I/O
                ('idle'), and the percentage of unused CPU time while at
                least one process was waiting for disk I/O ('wait').
                In case of per-cpu occupation, the CPU number and the wait
                percentage ('w') for that CPU.  The number of lines showing
                the per-cpu occupation can be limited.
    
                For virtual machines, the steal-percentage ('steal') shows
                the percentage of CPU time stolen by other virtual machines
                running on the same hardware.
                For physical machines hosting one or more virtual machines,
                the guest-percentage ('guest') shows the percentage of CPU
                time used by the virtual machines.  Notice that this percent‐
                age overlaps the user-percentage!
    
                When PMC performance monitoring counters are supported by the
                CPU and the kernel (and pmdaperfevent(1) runs with root priv‐
                ileges), the number of instructions per CPU cycle ('ipc') is
                shown.  The first sample always shows the value 'initial',
                because the counters are just activated at the moment that
                **pcp-atop** is started.
                When the _CPU busy percentage is high_ and the IPC is less than
                1.0, it is likely that the CPU is frequently waiting for mem‐
                ory access during instruction execution (larger CPU caches or
                faster memory might be helpful to improve performance).  When
                the _CPU busy percentage is high_ and the IPC is greater than
                1.0, it is likely that the CPU is instruction-bound
                (more/faster cores might be helpful to improve performance).
                Furthermore, per CPU the effective number of cycles ('cycl')
                is shown.  This value can reach the current CPU frequency if
                such CPU is 100% busy.  When an idle CPU is halted, the num‐
                ber of effective cycles can be (considerably) lower than the
                current frequency.
                Notice that the _average_ instructions per cycle and number of
                cycles is shown in the CPU line for all CPUs.
                See also:
                _http://www.brendangregg.com/blog/2017-05-09/cpu-utilization-is-wrong.html _
    
                In case of frequency scaling, all previously mentioned CPU
                percentages are relative to the used scaling of the CPU dur‐
                ing the interval.  If a CPU has been active for e.g. 50% in
                user mode during the interval while the frequency scaling of
                that CPU was 40%, only 20% of the full capacity of the CPU
                has been used in user mode.
                In case that the kernel module 'cpufreq_stats' is active (af‐
                ter issuing 'modprobe cpufreq_stats'), the _average_ frequency
                ('avgf') and the _average_ scaling percentage ('avgscal') is
                shown. Otherwise the _current_ frequency ('curf') and the _cur‐_
                _rent_ scaling percentage ('curscal') is shown at the moment
                that the sample is taken.  Notice that _average_ values for
                frequency and scaling are shown in the CPU line for every
                CPU.
                Frequency scaling statistics are only gathered for systems
                with maximum 8 CPUs, since gathering of these values per CPU
                is very time consuming.
    
                If the screen-width does not allow all of these counters, on‐
                ly a relevant subset is shown.
    
           **CPL** CPU load information.
                This line contains the load average figures reflecting the
                number of threads that are available to run on a CPU (i.e.
                part of the runqueue) or that are waiting for disk I/O. These
                figures are averaged over 1 ('avg1'), 5 ('avg5') and 15
                ('avg15') minutes.
                Furthermore the number of context switches ('csw'), the num‐
                ber of serviced interrupts ('intr') and the number of avail‐
                able CPUs are shown.
    
                If the screen-width does not allow all of these counters, on‐
                ly a relevant subset is shown.
    
           **GPU** GPU utilization (Nvidia).
                Read the section GPU STATISTICS GATHERING in this document to
                find the details about the activation of the **pmdanvidia** dae‐
                mon.
    
                In the first column of every line, the bus-id (last nine
                characters) and the GPU number are shown.  The subsequent
                columns show the percentage of time that one or more kernels
                were executing on the GPU ('gpubusy'), the percentage of time
                that global (device) memory was being read or written ('mem‐
                busy'), the occupation percentage of memory ('memocc'), the
                total memory ('total'), the memory being in use at the moment
                of the sample ('used'), the average memory being in use dur‐
                ing the sample time ('usavg'), the number of processes being
                active on the GPU at the moment of the sample ('#proc'), and
                the type of GPU.
    
                If the screen-width does not allow all of these counters, on‐
                ly a relevant subset is shown.
                The number of lines showing the GPUs can be limited.
    
           **MEM** Memory occupation (two lines).
                These lines contain the total amount of physical memory
                ('tot'), the amount of memory which is currently free
                ('free'), the amount of memory that is available for new
                workloads without pushing the system into swap ('avail'), the
                amount of memory in use as page cache including the total
                resident shared memory ('cache'), the amount of memory within
                the page cache that has to be flushed to disk ('dirty'), the
                amount of memory used for filesystem meta data ('buff'), the
                amount of memory being used for kernel mallocs ('slab'), the
                amount of slab memory that is reclaimable ('slrec'), the res‐
                ident size of SYSV shared memory including tmpfs but exclud‐
                ing static huge pages ('shmem'), the resident size of SYSV
                shared memory including static huge pages ('shrss'), the
                amount of SYSV shared memory that is currently swapped
                ('shswp'), the amount of memory that is currently used for
                page tables ('pgtab'), the number of NUMA nodes in this sys‐
                tem ('numnode'), the amount of memory that is currently
                claimed by vmware's balloon driver ('vmbal'), the amount of
                memory that is currently claimed by the ARC (cache) of ZFSon‐
                linux ('zfarc'), the amount of memory for anonymous transpar‐
                ent huge pages ('anthp'), the amount of memory that is
                claimed for huge pages ('hptot'), the amount of huge page
                memory that is really in use ('hpuse'), the amount of memory
                that is used for TCP sockets ('tcps'), and the amount of mem‐
                ory that is used for UDP sockets ('udps').
    
                If the screen-width does not allow all of these counters, on‐
                ly a relevant subset is shown.
    
           **SWP** Swap occupation and overcommit info.
                This line contains the total amount of swap space on disk
                ('tot') and the amount of free swap space ('free'), the size
                of the swap cache ('swcac'), the size of compressed storage
                used for zswap ('zswap'), the real (decompressed) size of the
                pages stored in zswap ('zstor'), the total size of the memory
                used for KSM ('ksuse', i.e. shared), and the total size of
                the memory saved (deduped) by KSM ('kssav', i.e. sharing).
                Furthermore the committed virtual memory space ('vmcom') and
                the maximum limit of the committed space ('vmlim', which is
                by default swap size plus 50% of memory size) is shown.  The
                committed space is the reserved virtual space for all alloca‐
                tions of private memory space for processes.  The kernel only
                verifies whether the committed space exceeds the limit if
                strict overcommit handling is configured (vm.overcommit_memo‐
                ry is 2).
    
           **LLC** Last-Level Cache of CPU info.
                This line contains the total memory bandwidth of LLC ('tot'),
                the bandwidth of the local NUMA node ('loc'), and the per‐
                centage of LLC in use ('LLCXX YY%').
    
                Note that this feature depends on the 'resctrl' pseudo
                filesystem.  Be sure that the kernel is built with the rele‐
                vant config and take care that the pseudo-filesystem is
                mounted:
    
                  **mount -t resctrl resctrl -o mba_MBps /sys/fs/resctrl**(on
                Intel)
                  **mount -t resctrl resctrl -o cdp      /sys/fs/resctrl**(on
                AMD)
    
           **NUM** Memory utilization per NUMA node (not shown for single NUMA
                node).
                This line shows the total amount of physical memory of this
                node ('tot'), the amount of free memory ('free'), the amount
                of memory for cached file data ('file'), modified cached file
                data ('dirty'), recently used memory ('activ'), less recently
                used memory ('inact'), memory being used for kernel mallocs
                ('slab'), the amount of slab memory that is reclaimable
                ('slrec'), shared memory including tmpfs ('shmem'), total
                huge pages ('hptot'), used huge pages('hpuse'), and the frag‐
                mentation percentage ('frag').
    
           **NUC** CPU utilization per NUMA node (not shown for single NUMA
                node).
                This line shows the utilization percentages of all CPUs re‐
                lated to this NUMA node, categorized for  system mode
                ('sys'), user mode ('user'), user mode for niced processes
                ('niced'), idle mode ('idle'), wait mode ('w' preceded by the
                node number), irq mode ('irq'), softirq mode ('sirq'), steal
                mode ('steal'), and guest mode ('guest') overlapping user
                mode.
    
           **PAG** Paging frequency.
                This line contains the number of scanned pages ('scan') due
                to the fact that free memory drops below a particular thresh‐
                old, the number of times that the kernel tries to reclaim
                pages due to an urgent need ('stall'), the number of process
                stalls to run memory compaction to allocate huge pages ('com‐
                pact'), the number of NUMA pages migrated ('numamig'), and
                the total number of memory pages migrated successfully e.g.
                between NUMA nodes or for compaction ('migrate') are shown.
                Also the number of memory pages the system read from block
                devices ('pgin'), the number of memory pages the system wrote
                to block devices ('pgout'), the number of memory pages
                swapped in from zswap ('zswin'), the number of memory pages
                swapped out to zswap ('zswout'), the number of memory pages
                the system read from swap space ('swin'), the number of memo‐
                ry pages the system wrote to swap space ('swout'), and the
                number of out-of-memory kills ('oomkill').
    
           **PSI** Pressure Stall Information.
                This line contains percentages about resource pressure relat‐
                ed to CPU, memory and I/O. Certain percentages refer to
                'some' meaning that some processes/threads were delayed due
                to resource overload. Other percentages refer to 'full' mean‐
                ing a loss of overall throughput due to resource overload.
                The values 'cpusome', 'memsome', 'memfull', 'iosome' and 'io‐
                full' show the pressure percentage during the entire inter‐
                val.
                The values 'cs' (cpu some), 'ms' (memory some), 'mf' (memory
                full), ´is' (I/O some) and 'if' (I/O full) each show three
                percentages separated by slashes: pressure percentage over
                the last 10, 60 and 300 seconds.
    
           **LVM/MDD/DSK**
                Logical volume/multiple device/disk utilization.
                Per active unit one line is produced, sorted on unit activi‐
                ty.  Such line shows the name (e.g. VolGroup00-lvtmp for a
                logical volume or sda for a hard disk), the percentage of
                elapsed time during which I/O requests were issued to the de‐
                vice ('busy') (note that for devices serving requests in par‐
                allel, such as RAID arrays, SSD and NVMe, this number does
                not reflect their performance limits), the number of read re‐
                quests issued ('read'), the number of write requests issued
                ('write'), the number of discard requests issued ('discrd')
                if supported by kernel version, the number of KiBytes per
                read ('KiB/r'), the number of KiBytes per write ('KiB/w'),
                the number of KiBytes per discard ('KiB/d') if supported by
                kernel version, the number of MiBytes per second throughput
                for reads ('MBr/s'), the number of MiBytes per second
                throughput for writes ('MBw/s'), requests issued to the de‐
                vice driver but not completed ('inflt'), the average queue
                depth ('avq') and the average number of milliseconds needed
                by a request ('avio') for seek, latency and data transfer.
                If the screen-width does not allow all of these counters, on‐
                ly a relevant subset is shown.
    
                The number of lines showing the units can be limited per
                class (LVM, MDD or DSK) with the 'l' key or statically (see
                separate man-page of pcp-atoprc(5)).  By specifying the value
                0 for a particular class, no lines will be shown any more for
                that class.
    
           **NFM** Network Filesystem (NFS) mount at the client side.
                For each NFS-mounted filesystem, a line is shown that con‐
                tains the mounted server directory, the name of the server
                ('srv'), the total number of bytes physically read from the
                server ('read') and the total number of bytes physically
                written to the server ('write').  Data transfer is subdivided
                in the number of bytes read via normal read() system calls
                ('nread'), the number of bytes written via normal read() sys‐
                tem calls ('nwrit'), the number of bytes read via direct I/O
                ('dread'), the number of bytes written via direct I/O
                ('dwrit'), the number of bytes read via memory mapped I/O
                pages ('mread'), and the number of bytes written via memory
                mapped I/O pages ('mwrit').
    
           **NFC** Network Filesystem (NFS) client side counters.
                This line contains the number of RPC calls issues by local
                processes ('rpc'), the number of read RPC calls ('read') and
                write RPC calls ('rpwrite') issued to the NFS server, the
                number of RPC calls being retransmitted ('retxmit') and the
                number of authorization refreshes ('autref').
    
           **NFS** Network Filesystem (NFS) server side counters.
                This line contains the number of RPC calls received from NFS
                clients ('rpc'), the number of read RPC calls received
                ('cread'), the number of write RPC calls received ('cwrit'),
                the number of Megabytes/second returned to read requests by
                clients ('MBcr/s'), the number of Megabytes/second passed in
                write requests by clients ('MBcw/s'), the number of network
                requests handled via TCP ('nettcp'), the number of network
                requests handled via UDP ('netudp'), the number of reply
                cache hits ('rchits'), the number of reply cache misses
                ('rcmiss') and the number of uncached requests ('rcnoca').
                Furthermore some error counters indicating the number of re‐
                quests with a bad format ('badfmt') or a bad authorization
                ('badaut'), and a counter indicating the number of bad
                clients ('badcln').
    
           **NET** Network utilization (TCP/IP).
                One line is shown for activity of the transport layer (TCP
                and UDP), one line for the IP layer and one line per active
                interface.
                For the transport layer, counters are shown concerning the
                number of received TCP segments including those received in
                error ('tcpi'), the number of transmitted TCP segments ex‐
                cluding those containing only retransmitted octets ('tcpo'),
                the number of UDP datagrams received ('udpi'), the number of
                UDP datagrams transmitted ('udpo'), the number of active TCP
                opens ('tcpao'), the number of passive TCP opens ('tcppo'),
                the number of TCP output retransmissions ('tcprs'), the num‐
                ber of TCP input errors ('tcpie'), the number of TCP output
                resets ('tcpor'), the number of UDP no ports ('udpnp'), the
                number of UDP input errors ('udpie'), and the number of TCP
                incorrect checksums ('csumie').
                If the screen-width does not allow all of these counters, on‐
                ly a relevant subset is shown.
                These counters are related to IPv4 and IPv6 combined.
    
                For the IP layer, counters are shown concerning the number of
                IP datagrams received from interfaces, including those re‐
                ceived in error ('ipi'), the number of IP datagrams that lo‐
                cal higher-layer protocols offered for transmission ('ipo'),
                the number of received IP datagrams which were forwarded to
                other interfaces ('ipfrw'), the number of IP datagrams which
                were delivered to local higher-layer protocols ('deliv'), the
                number of received ICMP datagrams ('icmpi'), and the number
                of transmitted ICMP datagrams ('icmpo').
                If the screen-width does not allow all of these counters, on‐
                ly a relevant subset is shown.
                These counters are related to IPv4 and IPv6 combined.
    
                For every active network interface one line is shown, sorted
                on the interface activity.  Such line shows the name of the
                interface and its busy percentage in the first column.  The
                busy percentage for half duplex is determined by comparing
                the interface speed with the number of bits transmitted and
                received per second; for full duplex the interface speed is
                compared with the highest of either the transmitted or the
                received bits.  When the interface speed can not be deter‐
                mined (e.g. for the loopback interface), '---' is shown in‐
                stead of the percentage.
                Furthermore the number of received packets ('pcki'), the num‐
                ber of transmitted packets ('pcko'), the line speed of the
                interface ('sp'), the effective amount of bits received per
                second ('si'), the effective amount of bits transmitted per
                second ('so'), the number of collisions ('coll'), the number
                of received multicast packets ('mlti'), the number of errors
                while receiving a packet ('erri'), the number of errors while
                transmitting a packet ('erro'), the number of received pack‐
                ets dropped ('drpi'), and the number of transmitted packets
                dropped ('drpo').
                If the screen-width does not allow all of these counters, on‐
                ly a relevant subset is shown.
                The number of lines showing the network interfaces can be
                limited.
    
           **IFB** Infiniband utilization.
                For every active Infiniband port one line is shown, sorted on
                activity.  Such line shows the name of the port and its busy
                percentage in the first column.  The busy percentage is de‐
                termined by taking the highest of either the transmitted or
                the received bits during the interval, multiplying that value
                by the number of lanes and comparing it against the maximum
                port speed.
                Furthermore the number of received packets divided by the
                number of lanes ('pcki'), the number of transmitted packets
                divided by the number of lanes ('pcko'), the maximum line
                speed ('sp'), the effective amount of bits received per sec‐
                ond ('si'), the effective amount of bits transmitted per sec‐
                ond ('so'), and the number of lanes ('lanes').
                If the screen-width does not allow all of these counters, on‐
                ly a relevant subset is shown.
                The number of lines showing the Infiniband ports can be lim‐
                ited.
    


## OUTPUT DESCRIPTION - PROCESS LEVEL



           Following the system level information, a list of processes is
           shown in text mode from which the resource utilization has changed
           during the last interval.  These processes might have used CPU
           time or issued disk or network requests.  However a process is al‐
           so shown if part of it has been paged out due to lack of memory
           (while the process itself was in sleep state).
    
           Per process the following fields may be shown (in alphabetical or‐
           der), depending on the current output mode as described in the
           section INTERACTIVE COMMANDS and depending on the current width of
           your window:
    
           **AVGRSZ** The average size of one read-action on disk.
    
           **AVGWSZ** The average size of one write-action on disk.
    
           **BANDWI** Total bandwidth for received TCP and UDP packets consumed
                    by this process (bits-per-second).  This value can be
                    compared with the value 'si' on interface level (used
                    bandwidth per interface).
                    This information will only be shown when the optional
                    pmdabpf(1) or pmdabcc(1) module 'netproc' has been in‐
                    stalled.
    
           **BANDWO** Total bandwidth for sent TCP and UDP packets consumed by
                    this process (bits-per-second).  This value can be com‐
                    pared with the value 'so' on interface level (used band‐
                    width per interface).
                    This information will only be shown when the optional
                    pmdabpf(1) or pmdabcc(1) module 'netproc' has been in‐
                    stalled.
    
           **BDELAY** Aggregated block I/O delay, i.e. time waiting for disk
                    I/O.
    
           **CGROUP** Path name of the cgroup (version 2) to which this process
                    belongs.  This path name is relative to the cgroup root
                    directory, which is usually '/sys/fs/cgroup'.
    
           **CID/POD** Container id (e.g. Docker or Podman) or pod name (e.g.
                    Kubernetes) referring to the container/pod in which the
                    process/thread is running.  When a pod name is longer
                    than 15 characters, only the last 15 characters are
                    shown.
    
                    If a process has been started and finished during the
                    last interval, a '?' is shown because the container id or
                    pod name is not part of the standard process accounting
                    record.
    
                    This column will only be shown when _atop_ runs with supe‐
                    ruser privileges and when at least one containerized
                    process is detected.
    
           **CMD** The name of the process.  This name can be surrounded by
                    "less/greater than" signs ('<name>') which means that the
                    process has finished during the last interval. A single
                    accounting record is written for the entire process on
                    termination of the last thread in the process. When the
                    main thread exits, the process name is changed to the
                    thread name.
                    Behind the abbreviation 'CMD' in the header line, the
                    current page number and the total number of pages of the
                    process/thread list are shown.
    
           **COMMAND-LINE**
                    The full command line of the process (including argu‐
                    ments). If the length of the command line exceeds the
                    length of the screen line, the arrow keys -> and <- can
                    be used for horizontal scroll.
    
                    The '-z <regex>' command line option can be used to
                    prepend matching environment variables to the displayed
                    command line. POSIX Extended Regular Expression syntax
                    are used (see regex(3)). When a matching environment
                    variable is too long (exceeding the buffer that should
                    contain the command line), it will be truncated.
                    Behind the verb 'COMMAND-LINE' in the header line, the
                    current page number and the total number of pages of the
                    process/thread list are shown.
    
           **CPU** The occupation percentage of this process related to the
                    available capacity for this resource on system level.
    
           **CPUNR** The identification of the CPU the (main) thread is run‐
                    ning on or has recently been running on.
    
           **CTID** Container ID (OpenVZ).  If a process has been started and
                    finished during the last interval, a '?' is shown because
                    the container ID is not part of the standard process ac‐
                    counting record.
    
           **DSK** The occupation percentage of this process related to the
                    total load that is produced by all processes (i.e. total
                    disk accesses by all processes during the last interval).
                    This information is shown when per process "storage ac‐
                    counting" is active in the kernel.
    
           **EGID** Effective group-id under which this process executes.
    
           **ENDATE** Date that the process has been finished.  If the process
                    is still running, this field shows 'active'.
    
           **ENTIME** Time that the process has been finished.  If the process
                    is still running, this field shows 'active'.
    
           **ENVID** Virtual environment identified (OpenVZ only).
    
           **EUID** Effective user-id under which this process executes.
    
           **EXC** The exit code of a terminated process (second position of
                    column 'ST' is E) or the fatal signal number (second po‐
                    sition of column 'ST' is S or C).
    
           **FSGID** Filesystem group-id under which this process executes.
    
           **FSUID** Filesystem user-id under which this process executes.
    
           **GPU** When the **pmdanvidia** daemon does not run with root privi‐
                    leges, the GPU percentage reflects the GPU memory occupa‐
                    tion percentage (memory of all GPUs is 100%).
                    When the **pmdanvidia** daemon runs with root privileges, the
                    GPU percentage reflects the GPU busy percentage.
    
           **GPUBUSY** Busy percentage on all GPUs (one GPU is 100%).
                    When the **pmdanvidia** daemon does not run with root privi‐
                    leges, this value is not available.
    
           **GPUNUMS** Comma-separated list of GPUs used by the process during
                    the interval.  When the comma-separated list exceeds the
                    width of the column, a hexadecimal value is shown.
    
           **LOCKSZ** The virtual amount of memory being locked (i.e. non-swap‐
                    pable) by this process (or user).
    
           **MAJFLT** The number of page faults issued by this process that
                    have been solved by creating/loading the requested memory
                    page.
    
           **MEM** The occupation percentage of this process related to the
                    available capacity for this resource on system level.
    
           **MEMAVG** Average memory occupation during the interval on all used
                    GPUs.
    
           **MEMBUSY** Busy percentage of memory on all GPUs (one GPU is 100%),
                    i.e.  the time needed for read and write accesses on mem‐
                    ory.
                    When the **pmdanvidia** daemon does not run with root privi‐
                    leges, this value is not available.
    
           **MEMNOW** Memory occupation at the moment of the sample on all used
                    GPUs.
    
           **MINFLT** The number of page faults issued by this process that
                    have been solved by reclaiming the requested memory page
                    from the free list of pages.
    
           **NET** The occupation percentage of this process related to the
                    total load that is produced by all processes (i.e. con‐
                    sumed network bandwidth of all processes during the last
                    interval).
                    This information will only be shown when the optional
                    pmdabpf(1) or pmdabcc(1) module 'netproc' has been in‐
                    stalled.
    
           **NICE** The more or less static priority that can be given to a
                    process on a scale from -20 (high priority) to +19 (low
                    priority).
    
           **NIVCSW** Number of times the process/thread was context-switched
                    involuntarily, in case that the time slice expired.
    
           **NPROCS** The number of active and terminated processes accumulated
                    for this user or program.
    
           **NVCSW** Number of times that the process/thread was context-
                    switched voluntarily in case of a blocking system call,
                    e.g. to wait for an I/O operation to complete.
    
           **PID** Process-id.
    
           **POLI** The policies 'norm' (normal, which is SCHED_OTHER),
                    'btch' (batch) and 'idle' refer to timesharing processes.
                    The policies 'fifo' (SCHED_FIFO) and 'rr' (round robin,
                    which is SCHED_RR) refer to realtime processes.
    
           **PPID** Parent process-id.
    
           **PRI** The process' priority ranges from 0 (highest priority) to
                    139 (lowest priority).  Priority 0 to 99 are used for re‐
                    altime processes (fixed priority independent of their be‐
                    havior) and priority 100 to 139 for timesharing processes
                    (variable priority depending on their recent CPU consump‐
                    tion and the nice value).
    
           **PSIZE** The proportional memory size of this process (or user).
                    Every process shares resident memory with other process‐
                    es.  E.g. when a particular program is started several
                    times, the code pages (text) are only loaded once in mem‐
                    ory and shared by all incarnations.  Also the code of
                    shared libraries is shared by all processes using that
                    shared library, as well as shared memory and memory-
                    mapped files.  For the PSIZE calculation of a process,
                    the resident memory of a process that is shared with oth‐
                    er processes is divided by the number of sharers.  This
                    means, that every process is accounted for a proportional
                    part of that memory.  Accumulating the PSIZE values of
                    all processes in the system gives a reliable impression
                    of the total resident memory consumed by all processes.
                    Since gathering of all values that are needed to calcu‐
                    late the PSIZE is a very time-consuming task, the 'R' key
                    (or '-R' flag) should be active.  Gathering these values
                    also requires superuser privileges (otherwise '?K' is
                    shown in the output).
    
           **RDDSK** The read data transfer issued physically on disk (so
                    reading from the disk cache is not accounted for).
                    Unfortunately, the kernel aggregates the data transfer of
                    a process to the data transfer of its parent process when
                    terminating, so you might see transfers for (parent)
                    processes like cron, bash or init, that are not really
                    issued by them.
    
           **RDELAY** Runqueue delay, i.e. time spent waiting on a runqueue.
    
           **RGID** The real group-id under which the process executes.
    
           **RGROW** The amount of resident memory that the process has grown
                    during the last interval.  A resident growth can be
                    caused by touching memory pages which were not physically
                    created/loaded before (load-on-demand).  Note that a res‐
                    ident growth can also be negative e.g. when part of the
                    process is paged out due to lack of memory or when the
                    process frees dynamically allocated memory.  For a
                    process which started during the last interval, the resi‐
                    dent growth reflects the total resident size of the
                    process at that moment.
    
           **RNET** The number of TCP- and UDP packets received by this
                    process.  This information will only be shown when the
                    optional pmdabpf(1) or pmdabcc(1) netatop module is in‐
                    stalled.
                    If a process has finished during the last interval, no
                    value is shown since network counters are not part of the
                    standard process accounting record.
    
           **RSIZE** The total resident memory usage consumed by this process
                    (or user).  Notice that the RSIZE of a process includes
                    all resident memory used by that process, even if certain
                    memory parts are shared with other processes (see also
                    the explanation of PSIZE).
    
           **RTPR** Realtime priority according the POSIX standard.  Value
                    can be 0 for a timesharing process (policy 'norm', 'btch'
                    or 'idle') or ranges from 1 (lowest) till 99 (highest)
                    for a realtime process (policy 'rr' or 'fifo').
    
           **RUID** The real user-id under which the process executes.
    
           **S** The current state of the (main) thread: 'R' for running
                    (currently processing or in the runqueue), 'S' for sleep‐
                    ing interruptible (wait for an event to occur), 'D' for
                    sleeping non-interruptible, 'Z' for zombie (waiting to be
                    synchronized with its parent process), 'T' for stopped
                    (suspended or traced), 'W' for swapping, and 'E' (exit)
                    for processes which have finished during the last inter‐
                    val.
    
           **SGID** The saved group-id of the process.
    
           **SNET** The number of TCP and UDP packets transmitted by this
                    process.  This information will only be shown when the
                    optional pmdabpf(1) or pmdabcc(1) netatop module is in‐
                    stalled.
    
           **ST** The status of a process.
                    The first position indicates if the process has been
                    started during the last interval (the value _N_ means 'new
                    process').
    
                    The second position indicates if the process has been
                    finished during the last interval.
                    The value _E_ means 'exit' on the process' own initiative;
                    the exit code is displayed in the column 'EXC'.
                    The value _S_ means that the process has been terminated
                    involuntarily by a signal; the signal number is displayed
                    in the in the column 'EXC'.
                    The value _C_ means that the process has been terminated
                    involuntarily by a signal, producing a core dump in its
                    current directory; the signal number is displayed in the
                    column 'EXC'.
    
           **STDATE** The start date of the process.
    
           **STTIME** The start time of the process.
    
           **SUID** The saved user-id of the process.
    
           **SWAPSZ** The swap space consumed by this process (or user).
    
           **SYSCPU** CPU time consumption of this process in system mode (ker‐
                    nel mode), usually due to system call handling.
    
           **TCPRASZ** The average size of a received TCP buffer in bytes.  This
                    information will only be shown when the optional
                    pmdabpf(1) or pmdabcc(1) netproc module is enabled.
    
           **TCPRCV** The number of tcp_recvmsg()/tcp_cleanup_rbuf() calls from
                    this process.  This information will only be shown when
                    the optional pmdabpf(1) or pmdabcc(1) netproc module is
                    enabled.
    
           **TCPSASZ** The average size of a TCP buffer requested to be trans‐
                    mitted in bytes.  This information will only be shown
                    when the optional pmdabpf(1) or pmdabcc(1) netproc module
                    is enabled.
    
           **TCPSND** The number of tcp_sendmsg() calls from this process.
                    This information will only be shown when the optional
                    pmdabpf(1) or pmdabcc(1) netproc module is enabled.
    
           **THR** Total number of threads within this process.  All related
                    threads are contained in a thread group, represented by
                    **pcp-atop** as one line or as a separate line when the 'y'
                    key (or -y flag) is active.
    
           **TID** Thread-id.  All threads within a process run with the
                    same PID but with a different TID.  This value is shown
                    for individual threads in multi-threaded processes (when
                    using the key 'y').
    
           **TIDLE** Number of threads within this process that are in the
                    state 'idle' (I), i.e. uninterruptible sleeping threads
                    that do not count for the load average.
    
           **TRUN** Number of threads within this process that are in the
                    state 'running' (R).
    
           **TSLPI** Number of threads within this process that are in the
                    state 'interruptible sleeping' (S).
    
           **TSLPU** Number of threads within this process that are in the
                    state 'uninterruptible sleeping' (D).
    
           **UDPRASZ** The average size of a received UDP buffer in bytes.  This
                    information will only be shown when the optional
                    pmdabpf(1) or pmdabcc(1) netproc module is enabled.
    
           **UDPRCV** The number of udp_recvmsg()/skb_consume_udp() calls from
                    this process.  This information will only be shown when
                    the optional pmdabpf(1) or pmdabcc(1) netproc module is
                    enabled.
    
           **UDPSASZ** The average size of a UDP buffer requested to be trans‐
                    mitted in bytes.  This information will only be shown
                    when the optional pmdabpf(1) or pmdabcc(1) netproc module
                    is enabled.
    
           **UDPSND** The number of udp_sendmsg() calls from this process.
                    This information will only be shown when the optional
                    pmdabpf(1) or pmdabcc(1) netproc module is enabled.
    
           **USRCPU** CPU time consumption of this process in user mode, due to
                    processing the own program text.
    
           **VDATA** The virtual memory size of the private data used by this
                    process (including heap and shared library data).
    
           **VGROW** The amount of virtual memory that the process has grown
                    during the last interval. A virtual growth can be caused
                    by e.g. issuing a malloc() or attaching a shared memory
                    segment. Note that a virtual growth can also be negative
                    by e.g. issuing a free() or detaching a shared memory
                    segment.  For a process which started during the last in‐
                    terval, the virtual growth reflects the total virtual
                    size of the process at that moment.
    
           **VPID** Virtual process-id (within an OpenVZ container).  If a
                    process has been started and finished during the last in‐
                    terval, a '?' is shown because the virtual process-id is
                    not part of the standard process accounting record.
    
           **VSIZE** The total virtual memory usage consumed by this process
                    (or user).
    
           **VSLIBS** The virtual memory size of the (shared) text of all
                    shared libraries used by this process.
    
           **VSTACK** The virtual memory size of the (private) stack used by
                    this process
    
           **VSTEXT** The virtual memory size of the (shared) text of the exe‐
                    cutable program.
    
           **WCHAN** Wait channel of thread in sleep state, i.e. the name of
                    the kernel function in which the thread has been put
                    asleep.
                    Since determining the name string of the kernel function
                    is a relatively time-consuming task, the 'W' key (or '-W'
                    flag) should be active.
    
           **WRDSK** The write data transfer issued physically on disk (so
                    writing to the disk cache is not accounted for).  This
                    counter is maintained for the application process that
                    writes its data to the cache (assuming that this data is
                    physically transferred to disk later on).  Notice that
                    disk I/O needed for swapping is not taken into account.
                    Unfortunately, the kernel aggregates the data transfer of
                    a process to the data transfer of its parent process when
                    terminating, so you might see transfers for (parent)
                    processes like cron, bash or init, that are not really
                    issued by them.
    
           **WCANCL** The write data transfer previously accounted for this
                    process or another process that has been cancelled.  Sup‐
                    pose that a process writes new data to a file and that
                    data is removed again before the cache buffers have been
                    flushed to disk.  Then the original process shows the
                    written data as WRDSK, while the process that re‐
                    moves/truncates the file shows the unflushed removed data
                    as WCANCL.
    


## PARSABLE OUTPUT



           With the flag **-P** followed by a list of one or more labels (comma-
           separated), parsable output is produced for each sample.  The la‐
           bels that can be specified for system-level statistics correspond
           to the labels (first verb of each line) that can be found in the
           interactive output: "CPU", "cpu", "CPL", "GPU", "MEM", "SWP",
           "PAG", "PSI", "LVM", "MDD", "DSK", "NFM", "NFC", "NFS", "NET",
           "IFB", "LLC", "NUM" and "NUC".
           For process-level statistics special labels are introduced: "PRG"
           (general), "PRC" (CPU), "PRE" (GPU), "PRM" (memory), "PRD" (disk,
           only if "storage accounting" is active) and "PRN" (only if the op‐
           tional pmdabpf(1) or pmdabcc(1) netproc module is installed).
           With the label "ALL", all system and process level statistics are
           shown.
    
           The command and command line in the parsable output might contain
           spaces and are therefore by default surrounded by parenthesis.
           However, since a space is often used as separator between the
           fields by parsing tools, with the additional flag **-Z** it is possi‐
           ble to exchange the spaces in the command (line) by underscores
           and omit the parenthesis.
    
           For every interval all requested lines are shown whereafter **pcp-**
           **atop** shows a line just containing the label "SEP" as a separator
           before the lines for the next sample are generated.
           When a sample contains the values since boot, **pcp-atop** shows a
           line just containing the label "RESET" before the lines for this
           sample are generated.
    
           The first part of each output-line consists of the following six
           fields: **label**(the name of the label), **host**(the name of this ma‐
           chine), **epoch**(the time of this interval as number of seconds
           since 1-1-1970), **date**(date of this interval in format
           YYYY/MM/DD), **time**(time of this interval in format HH:MM:SS), and
           **interval**(number of seconds elapsed for this interval).
    
           The subsequent fields of each output-line depend on the label:
    
           **CPU** Subsequent fields: total number of clock-ticks per second
                    for this machine, number of processors, consumption for
                    all CPUs in system mode (clock-ticks), consumption for
                    all CPUs in user mode (clock-ticks), consumption for all
                    CPUs in user mode for niced processes (clock-ticks), con‐
                    sumption for all CPUs in idle mode (clock-ticks), con‐
                    sumption for all CPUs in wait mode (clock-ticks), con‐
                    sumption for all CPUs in irq mode (clock-ticks), consump‐
                    tion for all CPUs in softirq mode (clock-ticks), consump‐
                    tion for all CPUs in steal mode (clock-ticks), consump‐
                    tion for all CPUs in guest mode (clock-ticks) overlapping
                    user mode, frequency of all CPUs and frequency percentage
                    of all CPUs.
    
           **cpu** Subsequent fields: total number of clock-ticks per second
                    for this machine, processor-number, consumption for this
                    CPU in system mode (clock-ticks), consumption for this
                    CPU in user mode (clock-ticks), consumption for this CPU
                    in user mode for niced processes (clock-ticks), consump‐
                    tion for this CPU in idle mode (clock-ticks), consumption
                    for this CPU in wait mode (clock-ticks), consumption for
                    this CPU in irq mode (clock-ticks), consumption for this
                    CPU in softirq mode (clock-ticks), consumption for this
                    CPU in steal mode (clock-ticks), consumption for this CPU
                    in guest mode (clock-ticks) overlapping user mode, fre‐
                    quency of all CPUs, frequency percentage of all CPUs, in‐
                    structions executed by all CPUs and cycles for all CPUs.
    
           **CPL** Subsequent fields: number of processors, load average for
                    last minute, load average for last five minutes, load av‐
                    erage for last fifteen minutes, number of context-switch‐
                    es, and number of device interrupts.
    
           **GPU** Subsequent fields: GPU number, bus-id string, type of GPU
                    string, GPU busy percentage during last second (-1 if not
                    available), memory busy percentage during last second (-1
                    if not available), total memory size (KiB), used memory
                    (KiB) at this moment, number of samples taken during in‐
                    terval, cumulative GPU busy percentage during the inter‐
                    val (to be divided by the number of samples for the aver‐
                    age busy percentage, -1 if not available), cumulative
                    memory busy percentage during the interval (to be divided
                    by the number of samples for the average busy percentage,
                    -1 if not available), and cumulative memory occupation
                    during the interval (to be divided by the number of sam‐
                    ples for the average occupation).
    
           **MEM** Subsequent fields: page size for this machine (in bytes),
                    size of physical memory (pages), size of free memory
                    (pages), size of page cache (pages), size of buffer cache
                    (pages), size of slab (pages), dirty pages in cache
                    (pages), reclaimable part of slab (pages), total size of
                    vmware's balloon pages (pages), total size of shared mem‐
                    ory (pages), size of resident shared memory (pages), size
                    of swapped shared memory (pages), smaller huge page size
                    (in bytes), total size of smaller huge pages (huge
                    pages), size of free smaller huge pages (huge pages),
                    size of ARC (cache) of ZFSonlinux (pages), size of shar‐
                    ing pages for KSM (pages), size of shared pages for KSM
                    (pages), size of memory used for TCP sockets (pages),
                    size of memory used for UDP sockets (pages), size of
                    pagetables (pages), larger huge page size (in bytes), to‐
                    tal size of larger huge pages (huge pages), size of free
                    larger huge pages (huge pages), size of available memory
                    (pages) for new workloads without swapping, and size of
                    anonymous transparent huge pages ('normal' pages).
    
           **SWP** Subsequent fields: page size for this machine (in bytes),
                    size of swap (pages), size of free swap (pages), size of
                    swap cache (pages), size of committed space (pages), lim‐
                    it for committed space (pages), size of the swap cache
                    (pages), the real (decompressed) size of the pages stored
                    in zswap (pages), and the size of compressed storage used
                    for zswap (pages).
    
           **LLC** Subsequent fields: LLC id, percentage of LLC in use, to‐
                    tal memory bandwidth of this LLC (in bytes), and memory
                    bandwidth on local NUMA node of this LLC (in bytes).
    
           **PAG** Subsequent fields: page size for this machine (in bytes),
                    number of page scans, number of allocstalls, 0 (future
                    use), number of swapins, number of swapouts, number of
                    oomkills (-1 when counter not present), number of process
                    stalls to run memory compaction, number of pages success‐
                    fully migrated in total, number of NUMA pages migrated,
                    number of pages read from block devices, number of pages
                    written to block devices, number of swapins from zswap,
                    and number of swapouts to zswap.
    
           **PSI** Subsequent fields: PSI statistics present on this system
                    (n or y), CPU some avg10, CPU some avg60, CPU some
                    avg300, CPU some accumulated microseconds during inter‐
                    val, memory some avg10, memory some avg60, memory some
                    avg300, memory some accumulated microseconds during in‐
                    terval, memory full avg10, memory full avg60, memory full
                    avg300, memory full accumulated microseconds during in‐
                    terval, I/O some avg10, I/O some avg60, I/O some avg300,
                    I/O some accumulated microseconds during interval, I/O
                    full avg10, I/O full avg60, I/O full avg300, and I/O full
                    accumulated microseconds during interval.
    
           **LVM/MDD/DSK**
                    For every logical volume/multiple device/hard disk one
                    line is shown.
                    Subsequent fields: name, number of milliseconds spent for
                    I/O, number of reads issued, number of sectors trans‐
                    ferred for reads, number of writes issued, number of sec‐
                    tors transferred for write, number of discards issued (-1
                    if not supported), number of sectors transferred for dis‐
                    cards, number of requests currently in flight (not yet
                    completed), and the average queue depth while the disk
                    was busy.
    
           **NFM** Subsequent fields: mounted NFS filesystem, total number
                    of bytes read, total number of bytes written, number of
                    bytes read by normal system calls, number of bytes writ‐
                    ten by normal system calls, number of bytes read by di‐
                    rect I/O, number of bytes written by direct I/O, number
                    of pages read by memory-mapped I/O, and number of pages
                    written by memory-mapped I/O.
    
           **NFC** Subsequent fields: number of transmitted RPCs, number of
                    transmitted read RPCs, number of transmitted write RPCs,
                    number of RPC retransmissions, and number of authoriza‐
                    tion refreshes.
    
           **NFS** Subsequent fields: number of handled RPCs, number of re‐
                    ceived read RPCs, number of received write RPCs, number
                    of bytes read by clients, number of bytes written by
                    clients, number of RPCs with bad format, number of RPCs
                    with bad authorization, number of RPCs from bad client,
                    total number of handled network requests, number of han‐
                    dled network requests via TCP, number of handled network
                    requests via UDP, number of handled TCP connections, num‐
                    ber of hits on reply cache, number of misses on reply
                    cache, and number of uncached requests.
    
           **NET** First, one line is produced for the upper layers of the
                    TCP/IP stack.
                    Subsequent fields: the verb "upper", number of packets
                    received by TCP, number of packets transmitted by TCP,
                    number of packets received by UDP, number of packets
                    transmitted by UDP, number of packets received by IP,
                    number of packets transmitted by IP, number of packets
                    delivered to higher layers by IP, number of packets for‐
                    warded by IP, number of input errors (UDP), number of no‐
                    port errors (UDP), number of active opens (TCP), number
                    of passive opens (TCP), number of passive opens (TCP),
                    number of established connections at this moment (TCP),
                    number of retransmitted segments (TCP), number of input
                    errors (TCP), number of output resets (TCP), and number
                    of checksum errors on received packets (TCP).
    
                    Next, one line is shown for every interface.
                    Subsequent fields: name of the interface, number of pack‐
                    ets received by the interface, number of bytes received
                    by the interface, number of packets transmitted by the
                    interface, number of bytes transmitted by the interface,
                    interface speed, and duplex mode (0=half, 1=full).
    
           **IFB** Subsequent fields: name of the InfiniBand interface, port
                    number, number of lanes, maximum rate (Mbps), number of
                    bytes received, number of bytes transmitted, number of
                    packets received, and number of packets transmitted.
    
           **NUM** Subsequent fields: NUMA node number, page size for this
                    machine (in bytes), the fragmentation percentage of this
                    node, size of physical memory (pages), size of free memo‐
                    ry (pages), recently (active) used memory (pages), less
                    recently (inactive) used memory (pages), size of cached
                    file data (pages), dirty pages in cache (pages), slab
                    memory being used for kernel mallocs (pages), slab memory
                    that is reclaimable (pages), shared memory including
                    tmpfs (pages), total huge pages (huge pages), and free
                    huge pages (huge pages).
    
           **NUC** Subsequent fields: NUMA node number, number of processors
                    for this node, consumption for node CPUs in system mode
                    (clock-ticks), consumption for node CPUs in user mode
                    (clock-ticks), consumption for node CPUs in user mode for
                    niced processes (clock-ticks), consumption for node CPUs
                    in idle mode (clock-ticks), consumption for node CPUs in
                    wait mode (clock-ticks), consumption for node CPUs in irq
                    mode (clock-ticks), consumption for node CPUs in softirq
                    mode (clock-ticks), consumption for node CPUs in steal
                    mode (clock-ticks), and consumption for node CPUs in
                    guest mode (clock-ticks) overlapping user mode.
    
           **PRG** For every process one line is shown.
                    Subsequent fields: PID (unique ID of task), name (between
                    parenthesis or underscores for spaces), state, real uid,
                    real gid, TGID (group number of related tasks/threads),
                    total number of threads, exit code (in case of fatal sig‐
                    nal: signal number + 256), start time (epoch), full com‐
                    mand line (between parenthesis or underscores for
                    spaces), PPID, number of threads in state 'running' (R),
                    number of threads in state 'interruptible sleeping' (S),
                    number of threads in state 'uninterruptible sleeping'
                    (D), effective uid, effective gid, saved uid, saved gid,
                    filesystem uid, filesystem gid, elapsed time of terminat‐
                    ed process (hertz), is_process (y/n), OpenVZ  virtual pid
                    (VPID), OpenVZ container id (CTID), container/pod name
                    (CID/POD), indication if the task is newly started during
                    this interval ('N'), cgroup v2 path name (between paren‐
                    thesis or underscores for spaces), end time (epoch. or 0
                    if still active), and number of threads in state 'idle'
                    (I).
    
           **PRC** For every process one line is shown.
                    Subsequent fields: PID, name (between parenthesis or un‐
                    derscores for spaces), state, total number of clock-ticks
                    per second for this machine, CPU-consumption in user mode
                    (clockticks), CPU-consumption in system mode (clock‐
                    ticks), nice value, priority, realtime priority, schedul‐
                    ing policy, current CPU, sleep average, TGID (group num‐
                    ber of related tasks/threads), is_process (y/n), runqueue
                    delay in nanoseconds for this thread or for all threads
                    (in case of process), wait channel of this thread (be‐
                    tween parenthesis or underscores for spaces), block I/O
                    delay (clockticks), number of voluntary context switches,
                    and number of involuntary context switches.
    
           **PRE** For every process one line is shown.
                    Subsequent fields: PID, name (between parenthesis or un‐
                    derscores for spaces), process state, GPU state (A for
                    active, E for exited, N for no GPU user), number of GPUs
                    used by this process, bitlist reflecting used GPUs, GPU
                    busy percentage during interval, memory busy percentage
                    during interval, memory occupation (KiB) at this moment
                    cumulative memory occupation (KiB) during interval, and
                    number of samples taken during interval.
    
           **PRM** For every process one line is shown.
                    Subsequent fields: PID, name (between parenthesis or un‐
                    derscores for spaces), state, page size for this machine
                    (in bytes), virtual memory size (Kbytes), resident memory
                    size (Kbytes), shared text memory size (Kbytes), virtual
                    memory growth (Kbytes), resident memory growth (Kbytes),
                    number of minor page faults, number of major page faults,
                    virtual library exec size (Kbytes), virtual data size
                    (Kbytes), virtual stack size (Kbytes), swap space used
                    (Kbytes), TGID (group number of related tasks/threads),
                    is_process (y/n), proportional set size (Kbytes) if in
                    'R' option is specified and virtually locked memory space
                    (Kbytes).
    
           **PRD** For every process one line is shown.
                    Subsequent fields: PID, name (between parenthesis or un‐
                    derscores for spaces), state, obsoleted kernel patch in‐
                    stalled ('n'), standard io statistics used ('y' or 'n'),
                    number of reads on disk, cumulative number of sectors
                    read, number of writes on disk, cumulative number of sec‐
                    tors written, cancelled number of written sectors, TGID
                    (group number of related tasks/threads), obsoleted value
                    ('n'), and is_process (y/n).
    
           **PRN** For every process one line is shown.
                    Subsequent fields: PID, name (between parenthesis or un‐
                    derscores for spaces), state, pmdabpf(1) or pmdabcc(1)
                    module 'netproc' loaded ('y' or 'n'), number of
                    tcp_sendmsg() calls, cumulative size of TCP buffers re‐
                    quested to be transmitted, number of
                    tcp_recvmsg()/tcp_cleanup_rbuf() calls, cumulative size
                    of TCP buffers received, number of udp_sendmsg() calls,
                    cumulative size of UDP buffers requested to be transmit‐
                    ted, number of udp_recvmsg()/skb_consume_udp() calls, cu‐
                    mulative size of UDP buffers transmitted, number of raw
                    packets transmitted (obsolete, always 0), number of raw
                    packets received (obsolete, always 0), TGID (group number
                    of related tasks/threads) and is_process (y/n).
    


## SIGNALS



           By sending the SIGUSR1 signal to **pcp-atop** a new sample will be
           forced, even if the current timer interval has not exceeded yet.
           The behavior is similar to pressing the 't' key in an interactive
           session.
    
           By sending the SIGUSR2 signal to **pcp-atop** a final sample will be
           forced after which **pcp-atop** will terminate.
    


## EXAMPLES



           To monitor the current system load interactively with an interval
           of (default) 10 seconds:
    
             **pcp atop**
    
           To monitor the system load as bar graphs with an interval of 5
           seconds:
    
             **pcp atop -B 5**
    
           Store information about the system and process activity in a PCP
           archive folio with an interval of ten minutes during an hour:
    
             **pcp atop -w /tmp/pcp-atop 600 6**
    
           View the contents of this file interactively:
    
             **pcp atop -r /tmp/pcp-atop**
    
           View the processor and disk utilization of this file in parsable
           format:
    
             **pcp atop -PCPU,DSK -r /tmp/pcp-atop.folio**
    
           View the contents of today's standard logfile interactively:
    
             **pcp atop -r**
    
           View the contents of the standard logfile of the day before yes‐
           terday interactively:
    
             **pcp atop -r yy**
    
           View the contents of the standard logfile of 2023, June 7 from
           02:00 PM onwards interactively:
    
             **pcp atop -r 20230607 -b 14:00**
    
           To monitor the system load and write it to a file (in plain ASCII)
           with an interval of one minute during half an hour with active
           processes sorted on memory consumption:
    
             **pcp atop -M 60 30 > /log/pcp-atop.mem**
    


## NOTES



           **pcp-atop** is based on the source code of the **atop**(1) command from
           _https://atoptool.nl _, maintained by Gerlof Langeveld
           (gerlof.langeveld@atoptool.nl), and aims to be command line and
           output compatible with it as much as possible.
    
           Some features of **pcp-atop**(such as reporting on the Apache HTTP
           daemon, Infiniband, NFS client mounts, hardware event counts, GPU
           statistics and per-process TCP and UDP statistics) are only acti‐
           vated if the corresponding PCP metrics are available. Refer to the
           documentation for pmdaapache(1), **pmdainfiniband**(1),
           pmdanfsclient(1), pmdanvidia(1), pmdaperfevent(1) pmdabcc(1) and
           pmdabpf(1) for further details on activating these metrics.
    
           The semantics of the per-process network statistics deviate
           slightly from the **atop**(1) tool: instead of the number of TCP/UDP
           packets sent/received (which may be inaccurate due to TCP segmen‐
           tation offload), **pcp-atop** shows the number of
           tcp_sendmsg()/udp_sendmsg()/etc. kernel calls per process.
    


## FILES



           **/etc/atoprc**
                Configuration file containing system-wide default values.
                For further information about the default values, refer to
                the pcp-atoprc(5) man page).
    
           **~/.atoprc**
                Configuration file containing personal default values.  For
                further information about the default values, refer to the
                pcp-atoprc(5) man page).
    


## PCP ENVIRONMENT



           Environment variables with the prefix **PCP_** are used to parameter‐
           ize the file and directory names used by PCP.  On each installa‐
           tion, the file _/etc/pcp.conf_ contains the local values for these
           variables.  The **$PCP_CONF** variable may be used to specify an al‐
           ternative configuration file, as described in pcp.conf(5).
    
           For environment variables affecting PCP tools, see
           pmGetOptions(3).
    


## DEBUGGING OPTIONS



           The **-D** or **--debug** pcp option enables the output of additional di‐
           agnostics on _stderr_ to help triage problems, although the informa‐
           tion is sometimes cryptic and primarily intended to provide guid‐
           ance for developers rather end-users.  _debug_ is a comma separated
           list of debugging options; use pmdbg(1) with the **-l** option to ob‐
           tain a list of the available debugging options and their meaning.
    
           Debugging options specific to **pcp-atop** are as follows:
           ┌────────┬───────────────────────────────────────┐
           │ **Option** │ **Description** │
           ├────────┼───────────────────────────────────────┤
           │ **appl0** │ general diagnostics                   │
           ├────────┼───────────────────────────────────────┤
           │ **appl1** │ metric and instance domain retrievals │
           └────────┴───────────────────────────────────────┘
    


* * *