Chapter 12. Configuring subsystem logs This chapter describes... For an overview on logs, see 2.3.14 Logs in Chapter 2 Architecture Overview in the Planning, Installation and Deployment Guide (Common Criteria Edition) . For log configuration during the installation and additional information, see Chapter 13 Configuring Logs in the Planning, Installation and Deployment Guide (Common Criteria Edition) . 12.1. Managing logs The Certificate System subsystem log files record events related to operations within that specific subsystem instance. For each subsystem, different logs are kept for issues such as installation, access, and web servers. All subsystems have similar log configuration, options, and administrative paths. 12.1.1. Configuring logs Logs are configured through the subsystem's CS.cfg file. Specialized logs, such as signed audit logs and custom logs, can also be created through the Console or configuration file. Audit logs can be configured through the subsystem Console for the CA, OCSP, TKS, and KRA subsystems. TPS logs are only configured through the configuration file. In the navigation tree of the Configuration tab, select Log . The Log Event Listener Management tab lists the currently configured listeners. To create a new log instance, click Add , and select a module plugin from the list in the Select Log Event Listener Plug-in Implementation window. Set or modify the fields in the Log Event Listener Editor window. The different parameters are listed in the below table. Table 12.1. Log event listener fields Field Description Log Event Listener ID Gives the unique name that identifies the listener. The names can have any combination of letters (aA to zZ), digits (0 to 9), an underscore (_), and a hyphen (-), but it cannot contain other characters or spaces. type Gives the type of log file. E.g. signedAudit . enabled Sets whether the log is active. Only enabled logs actually record events. The value is either true or false . level Sets the log level in the text field. The level must be manually entered in the field; there is no selection menu. The choices are Debug , Information , Warning , Failure , Misconfiguration , Catastrophe , and Security . For more information, see 13.1.2 Log Levels (Message Categories) in the Planning, Installation and Deployment Guide (Common Criteria Edition) . fileName Gives the full path, including the file name, to the log file. The subsystem user should have read/write permission to the file. [id=""] bufferSize Sets the buffer size in kilobytes (KB) for the log. Once the buffer reaches this size, the contents of the buffer are flushed out and copied to the log file. The default size is 512 KB. For more information on buffered logging, see 13.1.3 Buffered and unbuffered logging in the Planning, Installation and Deployment Guide (Common Criteria Edition) . flushInterval Sets the amount of time before the contents of the buffer are flushed out and added to the log file. The default interval is 5 seconds. maxFileSize Sets the size, in kilobytes (KB), a log file can become before it is rotated. Once it reaches this size, the file is copied to a rotated file, and the log file is started new. For more information on log file rotation, see 13.1.4 Log file rotation in the Planning, Installation and Deployment Guide (Common Criteria Edition) . The default size is 2000 KB. rolloverInterval Sets the frequency for the server to rotate the active log file. The available options are hourly, daily, weekly, monthly, and yearly. The default value is 2592000 which represents monthly in seconds. For more information, see 13.1.4 Log file rotation in the Planning, Installation and Deployment Guide (Common Criteria Edition) . During post-installation configuration, you have the opportunity to make some configuration changes directly by updating the configuration files prior to deployment. Log configuration is one of the operations. For instructions on how to configure logs by editing the CS.cfg file or running the pki-server command, see Chapter 13 Configuring Logs in the CS.cfg File in the Planning, Installation and Deployment Guide (Common Criteria Edition) . 12.1.2. Managing audit logs The audit log contains records for events that have been set up as recordable events. If the logSigning attribute is set to true , the audit log is signed with a log signing certificate belonging to the server. This certificate can be used by auditors to verify that the log has not been tampered with. By default, regular audit logs are located in the /var/log/pki/instance_name/subsystem_name/ directory with other types of logs, while signed audit logs are written to /var/log/pki/instance_name/subsystem_name/signedAudit/ . The default location for logs can be changed by modifying the configuration. Note Signed audit logs are optional. To enable them, please refer to Section 12.1.2.3, "Configuring a signed audit log in the console" The signed audit log creates a log recording system events, and the events are selected from a list of potential events. When enabled, signed audit logs record a verbose set of messages about the selected event activity. Signed audit logs are configured by default when the instance is first created, but it is possible to configure signed audits logs after installation. (See Section 12.1.2.2, "Enabling signed audit logging after installation" .) It is also possible to edit the configuration or change the signing certificates after configuration, as covered in Section 12.1.2.3, "Configuring a signed audit log in the console" . 12.1.2.1. A List of audit events For a list of audit events in Certificate System, see Appendix E, Audit events . 12.1.2.2. Enabling signed audit logging after installation By default, audit logging is enabled upon installation. However, you need to enable log signing manually after installation. Please see "Enabling signed audit logging" in the Installation Guide. 12.1.2.3. Configuring a signed audit log in the console Signed audit logs are configured by default when the instance is first created, but it is possible to edit the configuration or change the signing certificates after configuration. Note Provide enough space in the file system for the signed audit logs, since they can be large. A log is set to a signed audit log by setting the logSigning parameter to enable and providing the nickname of the certificate used to sign the log. A special log signing certificate is created when the subsystems are first configured. Only a user with auditor privileges can access and view a signed audit log. Auditors can use the AuditVerify tool to verify that signed audit logs have not been tampered with. The signed audit log is created and enabled when the subsystem is configured, but it needs additional configuration to begin creating and signing audit logs. Open the Console. Note To create or configure the audit log by editing the CS.cfg file, see Chapter 13 Configuring Logs in the CS.cfg File in the Planning, Installation and Deployment Guide (Common Criteria Edition) . In the navigation tree of the Configuration tab, select Log . In the Log Event Listener Management tab, select the SignedAudit entry. Click Edit/View . There are three fields which must be reset in the Log Event Listener Editor window. Fill in the signedAuditCertNickname . This is the nickname of the certificate used to sign audit logs. An audit signing certificate is created when the subsystem is configured; it has a nickname like auditSigningCert cert- instance_name__subsystem_name . Note To get the audit signing certificate nickname, list the certificates in the subsystem's certificate database using certutil . For example: Set the logSigning field to true to enable signed logging. Set any events which are logged to the audit log. Appendix E, Audit events lists the loggable events. Log events are separated by commas with no spaces. Set any other settings for the log, such as the file name, the log level, the file size, or the rotation schedule. Note By default, regular audit logs are located in the /var/log/pki/instance_name/subsystem_name/ directory with other types of logs, while signed audit logs are written to /var/log/pki/instance_name/subsystem_name/signedAudit/ . The default location for logs can be changed by modifying the configuration. Save the log configuration. After enabling signed audit logging, assign auditor users by creating the user and assigning that entry to the auditor group. Members of the auditor group are the only users who can view and verify the signed audit log. See Section 11.3.2.1, "Creating role users" for details about setting up auditors. Auditors can verify logs by using the AuditVerify tool. See the AuditVerify(1) man page for details about using this tool. 12.1.2.4. Handling audit logging failures There are events that could cause the audit logging function to fail, so events cannot be written to the log. For example, audit logging can fail when the file system containing the audit log file is full or when the file permissions for the log file are accidentally changed. If audit logging fails, the Certificate System instance shuts down in the following manner. Servlets are disabled and will not process new requests. All pending and new requests are killed. The subsystem is shut down. When this happens, administrators and auditors should work together with the operating system administrator to resolve the disk space or file permission issues. When the IT problem is resolved, the auditor should make sure that the last audit log entries are signed. If not, they should be preserved by manual signing, archived, and removed to prevent audit verification failures in the future. When this is completed, the administrators can restart the Certificate System. 12.2. Using logs 12.2.1. Displaying and verifying signed audit logs This section explains how a user in the Auditor group displays and verifies signed audit logs. 12.2.1.1. Listing audit logs As a user with auditor privileges, use the pki subsystem -audit-file-find command to list existing audit log files on the server. For example, to list the audit log files on the CA hosted on server.example.com:8443 : The command uses the client certificate with the auditor nickname stored in the ~/.dogtag/nssdb/ directory for authenticating to the CA. For further details about the parameters used in the command and alternative authentication methods, see the pki(1) man page. 12.2.1.2. Downloading audit logs As a user with auditor privileges, use the pki subsystem -audit-file-retrieve command to download a specific audit log from the server. For example, to download an audit log file from the CA hosted on server.example.com : Optionally, list the available log files on the CA. See Section 12.2.1.1, "Listing audit logs" . Download the log file. For example, to download the ca_audit file: The command uses the client certificate with the auditor nickname stored in the ~/.dogtag/nssdb/ directory for authenticating to the CA. For further details about the parameters used in the command and alternative authentication methods, see the pki(1) man page. After downloading a log file, you can search for specific log entries, for example, using the grep utility: 12.2.1.3. Verifying signed audit logs If audit log signing is enabled, users with auditor privileges can verify the logs: Initialize the NSS database and import the CA certificate. For details, see Section 2.5.1.1, "Initializing the pki CLI" and 10.5 Importing a certificate into an NSS Database in the Planning, Installation and Deployment Guide (Common Criteria Edition) . If the audit signing certificate does not exist in the PKI client database, import it: Search the audit signing certificate for the subsystem logs you want to verify. For example: Import the audit signing certificate into the PKI client: Download the audit logs. See Section 12.2.1.2, "Downloading audit logs" . Verify the audit logs. Create a text file that contains a list of the audit log files you want to verify in chronological order. For example: Use the AuditVerify utility to verify the signatures. For example: For further details about using AuditVerify , see the AuditVerify(1) man page. 12.2.1.4. Viewing signed audit logs using pkiconsole Log into the administrative console using Auditor user: Select Status in the right navigation menu: Select SignedAudit from the log dropdown: Select the audit event and click on View: 12.2.2. Viewing non-audit logs To troubleshoot the subsystem for a pki administrator, check the error or informational messages that the server has logged. Examining the log files can also monitor many aspects of the server's operation. For logs that are not audit logs, such as debug logs under /var/lib/pki/ <instance>/<subsystem type> /logs , contact your operating system administrators to share the non-audit logs with you. 12.2.3. Displaying OS-level audit logs Note To see operating system-level audit logs using the instructions below, the auditd logging framework must be configured per 13.2.1 Enabling OS-level Audit Logs in the Planning, Installation and Deployment Guide (Common Criteria Edition) . To display operating system-level access logs, use the ausearch utility as root or as a privileged user with the sudo utility. 12.2.3.1. Displaying audit log deletion events Since these events are keyed (with rhcs_audit_deletion ), use the -k parameter to find events matching that key: 12.2.3.2. Displaying access to the NSS database for secret and private keys Since these events are keyed (with rhcs_audit_nssdb ), use the -k parameter to find events matching that key: 12.2.3.3. Displaying time change events Since these events are keyed (with rhcs_audit_time_change ), use the -k parameter to find events matching that key: 12.2.3.4. Displaying package update events Since these events are a typed message (of type SOFTWARE_UPDATE ), use the -m parameter to find events matching that type: 12.2.3.5. Displaying changes to the PKI configuration Since these events are keyed (with rhcs_audit_config ), use the -k parameter to find events matching that key: 12.2.4. Smart card error codes Smart cards can report certain error codes to the TPS; these are recorded in the TPS's debug log file, depending on the cause for the message. Table 12.2. Smart card error codes Return Code Description General Error Codes 6400 No specific diagnosis 6700 Wrong length in Lc 6982 Security status not satisfied 6985 Conditions of use not satisfied 6a86 Incorrect P1 P2 6d00 Invalid instruction 6e00 Invalid class Install Load Errors 6581 Memory Failure 6a80 Incorrect parameters in data field 6a84 Not enough memory space 6a88 Referenced data not found Delete Errors 6200 Application has been logically deleted 6581 Memory failure 6985 Referenced data cannot be deleted 6a88 Referenced data not found 6a82 Application not found 6a80 Incorrect values in command data Get Data Errors 6a88 Referenced data not found Get Status Errors 6310 More data available 6a88 Referenced data not found 6a80 Incorrect values in command data Load Errors 6581 Memory failure 6a84 Not enough memory space 6a86 Incorrect P1/P2 6985 Conditions of use not satisfied

Commands:
certutil -L -d /var/lib/pki-tomcat/alias Certificate Authority - Example Domain CT,c, subsystemCert cert-pki-tomcat u,u,u Server-Cert cert-pki-tomcat u,u,u auditSigningCert cert-pki-tomcat CA u,u,Pu
pki -h server.example.com -p 8443 -n auditor ca-audit-file-find ----------------- 3 entries matched ----------------- File name: ca_audit.20170331225716 Size: 2883 File name: ca_audit.20170401001030 Size: 189 File name: ca_audit Size: 6705 ---------------------------- Number of entries returned 3 ----------------------------
pki -U https://server.example.com:8443 -n auditor ca-audit-file-retrieve ca_audit
grep "\[AuditEvent=ACCESS_SESSION_ESTABLISH\]" log_file
pki ca-cert-find --name "CA Audit Signing Certificate" --------------- 1 entries found --------------- Serial Number: 0x5 Subject DN: CN=CA Audit Signing Certificate,O=EXAMPLE Status: VALID Type: X.509 version 3 Key Algorithm: PKCS #1 RSA with 2048-bit key Not Valid Before: Fri Jul 08 03:56:08 CEST 2016 Not Valid After: Thu Jun 28 03:56:08 CEST 2018 Issued On: Fri Jul 08 03:56:08 CEST 2016 Issued By: system ---------------------------- Number of entries returned 1 ----------------------------
pki client-cert-import "CA Audit Signing Certificate" --serial 0x5 --trust ",,P" --------------------------------------------------- Imported certificate "CA Audit Signing Certificate" ---------------------------------------------------
cat > ~/audit.txt << EOF ca_audit.20170331225716 ca_audit.20170401001030 ca_audit EOF
AuditVerify -d ~/.dogtag/nssdb/ -n "CA Audit Signing Certificate" -a ~/audit.txt Verification process complete. Valid signatures: 10 Invalid signatures: 0
pkiconsole -d /root/.dogtag/pki_ecc_bootstrap/certs_db/ -n ecc_SubCA_AuditV https://rhcs10.example.com:21443/ca
ausearch -k rhcs_audit_deletion
ausearch -k rhcs_audit_nssdb
ausearch -k rhcs_audit_time_change
ausearch -m SOFTWARE_UPDATE
ausearch -k rhcs_audit_config