5.2. XFS File System Performance Analysis with Performance Co-Pilot This section describes PCP XFS performance metrics and how to use them. Once started, the Performance Metric Collector Daemon (PMCD) begins collecting performance data from the installed Performance Metric Domain Agents (PMDAs). PMDAs can be individually loaded or unloaded on the system and are controlled by the PMCD on the same host. The XFS PMDA, which is part of the default PCP installation, is used to gather performance metric data of XFS file systems in PCP. For a list of system services and tools that are distributed with PCP, see Table A.1, "System Services Distributed with Performance Co-Pilot in Red Hat Enterprise Linux 7" and Table A.2, "Tools Distributed with Performance Co-Pilot in Red Hat Enterprise Linux 7" . 5.2.1. Installing XFS PMDA to Gather XFS Data with PCP The XFS PMDA ships as part of the pcp package and is enabled by default on installation. To install PCP, enter: To enable and start the PMDA service on the host machine after the pcp and pcp-gui packages are installed, use the following commands: To query the PCP environment to verify that the PMCD process is running on the host and that the XFS PMDA is listed as enabled in the configuration, enter: Installing XFS PMDA Manually If the XFS PMDA is not listed in PCP configuration readout, install the PMDA agent manually. The PMDA installation script prompts you to specify the PMDA role: collector, monitor, or both. The collector role allows the collection of performance metrics on the current system The monitor role allows the system to monitor local systems, remote systems, or both. The default option is both collector and monitor, which allows the XFS PMDA to operate correctly in most scenarios. To install XFS PMDA manually, change to the xfs directory: In the xfs directory, enter: 5.2.2. Configuring and Examining XFS Performance Metrics Examining Metrics with pminfo With PCP installed and the XFS PMDA enabled, instructions are available in Section 5.2.1, "Installing XFS PMDA to Gather XFS Data with PCP" , the easiest way to start looking at the performance metrics available for PCP and XFS is to use the pminfo tool, which displays information about available performance metrics. The command displays a list of all available metrics provided by the XFS PMDA. To display a list of all available metrics provided by the XFS PMDA: Use the following options to display information on selected metrics: -t metric Displays one-line help information describing the selected metric. -T metric Displays more verbose help text describing the selected metric. -f metric Displays the current reading of the performance value that corresponds to the metric. You can use the -t , -T , and -f options with a group of metrics or an individual metric. Most metric data is provided for each mounted XFS file system on the system at time of probing. There are different groups of XFS metrics , which are arranged so that each different group is a new leaf node from the root XFS metric, using a dot ( . ) as a separator. The leaf node semantics (dots) applies to all PCP metrics. For an overview of the types of metrics that are available in each of the groups, see Table A.3, "PCP Metric Groups for XFS" . Example 5.1. Using the pminfo Tool to Examine XFS Read and Write Metrics To display one-line help information describing the xfs.write_bytes metric: To display more verbose help text describing the xfs.read_bytes metric: To obtain the current reading of the performance value that corresponds to the xfs.read_bytes metric: Configuring Metrics with pmstore With PCP, you can modify the values of certain metrics, especially if the metric acts as a control variable, for example the xfs.control.reset metric. To modify a metric value, use the pmstore tool. Example 5.2. Using pmstore to Reset the xfs.control.reset Metric This example shows how to use pmstore with the xfs.control.reset metric to reset the recorded counter values for the XFS PMDA back to zero. 5.2.3. Examining XFS Metrics Available per File System Starting with Red Hat Enterprise Linux 7.3, PCP enables XFS PMDA to allow the reporting of certain XFS metrics per each of the mounted XFS file systems. This makes it easier to pinpoint specific mounted file system issues and evaluate performance. For an overview of the types of metrics available per file system in each of the groups, see Table A.4, "PCP Metric Groups for XFS per Device" . Example 5.3. Obtaining per-Device XFS Metrics with pminfo The pminfo command provides per-device XFS metrics that give instance values for each mounted XFS file system. 5.2.4. Logging Performance Data with pmlogger PCP allows you to log performance metric values that can be replayed later and used for a retrospective performance analysis. Use the pmlogger tool to create archived logs of selected metrics on the system. With pmlogger, you can specify which metrics are recorded on the system and how often. The default pmlogger configuration file is /var/lib/pcp/config/pmlogger/config.default . The configuration file specifies which metrics are logged by the primary logging instance. To log metric values on the local machine with pmlogger , start a primary logging instance: When pmlogger is enabled and a default configuration file is set, a pmlogger line is included in the PCP configuration: Modifying the pmlogger Configuration File with pmlogconf When the pmlogger service is running, PCP logs a default set of metrics on the host. You can use the pmlogconf utility to check the default configuration, and enable XFS logging groups as needed. Important XFS groups to enable include the XFS information , XFS data , and log I/O traffic groups. Follow pmlogconf prompts to enable or disable groups of related performance metrics, and to control the logging interval for each enabled group. Group selection is made by pressing y (yes) or n (no) in response to the prompt. To create or modify the generic PCP archive logger configuration file with pmlogconf, enter: Modifying the pmlogger Configuration File Manually You can edit the pmlogger configuration file manually and add specific metrics with given intervals to create a tailored logging configuration. Example 5.4. The pmlogger Configuration File with XFS Metrics The following example shows an extract of the pmlogger config.default file with some specific XFS metrics added. Replaying the PCP Log Archives After recording metric data, you can replay the PCP log archives on the system in the following ways: You can export the logs to text files and import them into spreadsheets by using PCP utilities such as pmdumptext , pmrep , or pmlogsummary . You can replay the data in the PCP Charts application and use graphs to visualize the retrospective data alongside live data of the system. See Section 5.2.5, "Visual Tracing with PCP Charts" . You can use the pmdumptext tool to view the log files. With pmdumptext, you can parse the selected PCP log archive and export the values into an ASCII table. The pmdumptext tool enables you to dump the entire archive log, or only select metric values from the log by specifying individual metrics on the command line. Example 5.5. Displaying a Specific XFS Metric Log Information For example, to show data on the xfs.perdev.log metric collected in an archive at a 5 second interval and display all headers: For more information, see the pmdumptext (1) manual page, which is available from the pcp-doc package. 5.2.5. Visual Tracing with PCP Charts To be able to use the graphical PCP Charts application, install the pcp-gui package: You can use the PCP Charts application to plot performance metric values into graphs. The PCP Charts application allows multiple charts to be displayed simultaneously. The metrics are sourced from one or more live hosts with alternative options to use metric data from PCP log archives as a source of historical data. To launch PCP Charts from the command line, use the pmchart command. After starting PCP Charts, the GUI appears: The PCP Charts application The pmtime server settings are located at the bottom. The start and pause button allows you to control: The interval in which PCP polls the metric data The date and time for the metrics of historical data Go to File New Chart to select metric from both the local machine and remote machines by specifying their host name or address. Then, select performance metrics from the remote hosts. Advanced configuration options include the ability to manually set the axis values for the chart, and to manually choose the color of the plots. There are multiple options to take images or record the views created in PCP Charts: Click File Export to save an image of the current view. Click Record Start to start a recording. Click Record Stop to stop the recording. After stopping the recording, the recorded metrics are archived to be viewed later. You can customize the PCP Charts interface to display the data from performance metrics in multiple ways, including: line plot bar graphs utilization graphs In PCP Charts, the main configuration file, known as the view , allows the metadata associated with one or more charts to be saved. This metadata describes all chart aspects, including the metrics used and the chart columns. You can create a custom view configuration, save it by clicking File Save View , and load the view configuration later. For more information about view configuration files and their syntax, see the pmchart (1) manual page. Example 5.6. Stacking Chart Graph in PCP Charts View Configuration The example PCP Charts view configuration file describes a stacking chart graph showing the total number of bytes read and written to the given XFS file system loop1 .

Commands:
yum install pcp
systemctl enable pmcd.service
systemctl start pmcd.service
pcp Performance Co-Pilot configuration on workstation: platform: Linux workstation 3.10.0-123.20.1.el7.x86_64 #1 SMP Thu Jan 29 18:05:33 UTC 2015 x86_64 hardware: 2 cpus, 2 disks, 1 node, 2048MB RAM timezone: BST-1 services pmcd pmcd: Version 3.10.6-1, 7 agents pmda: root pmcd proc xfs linux mmv jbd2
cd /var/lib/pcp/pmdas/xfs/
xfs]# ./Install You will need to choose an appropriate configuration for install of the "xfs" Performance Metrics Domain Agent (PMDA). collector collect performance statistics on this system monitor allow this system to monitor local and/or remote systems both collector and monitor configuration for this system Please enter c(ollector) or m(onitor) or (both) [b] Updating the Performance Metrics Name Space (PMNS) Terminate PMDA if already installed Updating the PMCD control file, and notifying PMCD Waiting for pmcd to terminate Starting pmcd Check xfs metrics have appeared ... 149 metrics and 149 values
pminfo xfs
pminfo -t xfs.write_bytes xfs.write_bytes [number of bytes written in XFS file system write operations]
pminfo -T xfs.read_bytes xfs.read_bytes Help: This is the number of bytes read via read(2) system calls to files in XFS file systems. It can be used in conjunction with the read_calls count to calculate the average size of the read operations to file in XFS file systems.
pminfo -f xfs.read_bytes xfs.read_bytes value 4891346238
pminfo -f xfs.write xfs.write value 325262
pmstore xfs.control.reset 1 xfs.control.reset old value=0 new value=1
pminfo -f xfs.write xfs.write value 0
pminfo -f -t xfs.perdev.read xfs.perdev.write xfs.perdev.read [number of XFS file system read operations] inst [0 or "loop1"] value 0 inst [0 or "loop2"] value 0 xfs.perdev.write [number of XFS file system write operations] inst [0 or "loop1"] value 86 inst [0 or "loop2"] value 0
systemctl start pmlogger.service
systemctl enable pmlogger.service
pcp Performance Co-Pilot configuration on workstation: platform: Linux workstation 3.10.0-123.20.1.el7.x86_64 #1 SMP Thu Jan [...] pmlogger: primary logger:/var/log/pcp/pmlogger/workstation/20160820.10.15
pmlogconf -r /var/lib/pcp/config/pmlogger/config.default
It is safe to make additions from here on # log mandatory on every 5 seconds { xfs.write xfs.write_bytes xfs.read xfs.read_bytes } log mandatory on every 10 seconds { xfs.allocs xfs.block_map xfs.transactions xfs.log } [access] disallow * : all; allow localhost : enquire;
pmdumptext -t 5seconds -H -a 20170605 xfs.perdev.log.writes Time local::xfs.perdev.log.writes["/dev/mapper/fedora-home"] local::xfs.perdev.log.writes["/dev/mapper/fedora-root"] ? 0.000 0.000 none count / second count / second Mon Jun 5 12:28:45 ? ? Mon Jun 5 12:28:50 0.000 0.000 Mon Jun 5 12:28:55 0.200 0.200 Mon Jun 5 12:29:00 6.800 1.000
yum install pcp-gui
#kmchart version 1 chart title "Filesystem Throughput /loop1" style stacking antialiasing off plot legend "Read rate" metric xfs.read_bytes instance "loop1" plot legend "Write rate" metric xfs.write_bytes instance "loop1"