567 A permission associated with a handle was used. Note: A handle is created with certain granted permissions (such as Read and Write). When the handle is used, up to one audit is generated for each of the permissions that were used. 568 An attempt was made to create a hard link to a file that is being audited. 569 The resource manager in Authorization Manager attempted to create a client context. 570 A client attempted to access an object. Note: An event will be generated for every attempted operation on the object. 571 The client context was deleted by the Authorization Manager application. 572 The Administrator Manager initialized the application. 772 The Certificate Manager denied a pending certificate request. 773 Certificate Services received a resubmitted certificate request. 774 Certificate Services revoked a certificate. 775 Certificate Services received a request to publish the certificate revocation list (CRL). 776 Certificate Services published the CRL. 777 A certificate request extension was made. 778 One or more certificate request attributes changed. 779 Certificate Services received a request to shut down. 780 Certificate Services backup started. 781 Certificate Services backup completed. 782 Certificate Services restore started. 783 Certificate Services restore completed. 784 Certificate Services started. 785 Certificate Services stopped. 786 The security permissions for Certificate Services changed. 787 Certificate Services retrieved an archived key. 788 Certificate Services imported a certificate into its database. 789 The audit filter for Certificate Services changed. 790 Certificate Services received a certificate request. 791 Certificate Services approved a certificate request and issued a certificate. 792 Certificate Services denied a certificate request. 793 Certificate Services set the status of a certificate request to pending. 794 The certificate manager settings for Certificate Services changed. 795 A configuration entry changed in Certificate Services. 796 A property of Certificate Services changed.
Pass and Fail Criteria Before tests were performed, the following criteria were defined to ensure defect prevention and bug resolution: All test cases must pass with expected results as described in the individual test case spreadsheets. A test case is considered to have passed if the actual result matched the expected result that is documented for the case. If the actual result does not match the expected result, it was treated as a failed test case, a bug was created, and a severity score was assigned. If a test case failed, it was not assumed that the solution guidance was necessarily defective. For example, misinterpretation of product documentation, incomplete documentation, or inaccurate documentation could cause failures. Each failure was analyzed to discover its cause based on actual results and the results that were described in project documentation. Failures were also escalated to the appropriate owners of the respective Microsoft products. Release Criteria The primary release criterion for the Windows Server 2003 Security Guide was related to the severity of bugs that were still open. However, other issues that were not being tracked through bugs were also discussed. The criteria for release are: No bugs are open with severity levels 1 and 2. All open bugs are triaged by the leadership team, and their impacts are fully understood. Solution guides are free of comments and revision marks. The solution successfully passes all test cases in the test lab environment. Solution contents have no conflicting statements. Bug Classification The bug severity scale is described in the following table. The scale is from 1 to 4, with 1 as the highest severity and 4 as the lowest severity. Table D.1 Bug Severity Classification Severity Most common types Conditions required 1 – Bug blocked build or further testing. – Bug caused unexpected user accessibility. – Steps defined in the documentation were not clear. – Results or behavior of a function or process contradicts expected results (as documented in functional specification). – Major mismatch between the security template files and the functional specification. – Solution did not work. – User could not begin to use significant parts of the computer or network. – User had access privileges that should not be allowed. – User access was blocked to certain server(s) that should be allowed. – Expected results were not achieved. – Testing cannot proceed without being addressed.
2 – Steps defined in the guide are not clear. – Documented functionality is missing (in this case, test was blocked). – Documentation is missing or inadequate. – Inconsistency between security template files and content in the guide, but security template file is in sync with functional specification. – User had no simple workaround to amend the situation. – User could not easily figure out a workaround. – Primary business requirements could not be met by the computer or network. 3 – Documented format issue. – Minor documentation errors and inaccuracies. – Text misspellings. – User has a simple workaround to mend situation. – User can easily figure out workaround. – Bug does not cause a bad user experience. – Primary business requirements are still functional. 4 – Suggestions. – Future enhancements. – Clearly not related to this version. Summary This appendix enables an organization that uses the Windows Server 2003 Security Guide to understand the procedures and steps that were used to test the implementation of the solution in a test lab environment. The actual experience of the Windows Server 2003 Security Guide test team is captured in this appendix, which includes descriptions of the test environment, types of tests, the release criteria, and bug classification details. All of the test cases that were executed by the test team passed with the expected results. The test team confirmed that the requisite functionality was available after the recommendations from the Windows Server 2003 Security Guide for the defined environments were applied.
Acknowledgments The Microsoft Solutions for Security and Compliance group (MSSC) would like to acknowledge and thank the team that produced the Windows Server 2003 Security Guide. The following people were either directly responsible or made a substantial contribution to the writing, development, and testing of this solution. Authors Mike Danseglio Kurt Dillard José Maldonado Brad Warrender Content Contributors Liam Colvin, 3Sharp, LLC William Dixon, V6 Security Inc. Tony Dowler, 3Sharp, LLC Eric Fitzgerald Devin Ganger, 3Sharp, LLC Goetz Ian Hellen Jesper Johansson Steve Ryan, Content Master Kirk Soluk Program Managers Bomani Siwatu Alison Woolford, Content Master Editors Reid Bannecker Wendy Cleary, S&T Onsite John Cobb, Volt Information Sciences Kelly McMahon, Content Master Lynne Perry, Content Master Jon Tobey Steve Wacker, Wadeware LLC Release Managers Karl Seng, Siemens Agency Services Testers Kenon Bliss, Volt Information Sciences Gaurav Singh Bora, Infosys Technologies Paresh Gujar, Infosys Technologies Vince Humphreys, Volt Information Sciences Ashish Java, Infosys Technologies Mehul Mediwala, Infosys Technologies Rob Pike Varun Rastogi, Infosys Technologies Reviewers Roger Abell, Jose Luis Auricchio Avi Ben-Menahem Rich Benack Shelly Bird Susan Bradley Steve Clark Rob Cooper Duane Crider Karel Dekyvere Christine Duell Eric Fitzgerald Mike Greer Robert Hensing Hilton Andrew Mason Don McGowan James Noyce Joe Porter Joel Scambray Debra Littlejohn Shinder Tom Shinder Steve Smegner Ben Smith Allen Stewart Didier Vandenbroeck Ryan Vatne Jeff Williams Jim Whitney, Configuresoft Shain Wray