(General Information, Eligibility, and Entitlement Manual)

 Maintain traceability between each test case and the requirement that the case was designed to test.   2. The Shared System Maintainers and Single Testing Contractor (STC) shall distinguish each test requirement with a unique Requirement Identifier.  The Requirement Identifier must be a number or qualifier preceded by the CMS CR number and SSM CR number, separated by dashes.  The format of the Requirement Identifier is: [SSM CR No.] -[CMS CR No.] -[Requirement No.], where:    SSM CR No.  is a number that identifies a CMS mandate or user change request under test. Free form text can also be used to identify changes not associated with maintainer CR numbers, e.g., Regression to indicate regression testing.  Avoid spac es and use underscore symbol _ instead.  Dashes not allowed within this number; they are reserved for separation between the individual numbers in the Requirement Identifier.    CMS CR No.  is a minimum 4 -digit number that identifies the CMS CR associated  with the maintainer CR number.  If no CMS CR is associated with a maintainer CR, use 0000.  Dashes are not allowed.  Avoid spaces and use underscore symbol _ instead.  Dashes not allowed within this number; they are reserved for separation between the  individual numbers in the Requirement Identifier.    Requirement No.  is a number that uniquely identifies the requirement with the CR.  For any requirement taken from the Business Requirements section of a CMS CR, use the actual number from the Requirement # column.  Do not repeat the CMS CR number.  Dashes not allowed within this number; they are reserved for separation between the individual numbers in the Requirement Identifier.   Example:  Maintainer CR 22522 corresponds to CMS CR 2634.  Business Requirement 2.8 was taken directly from CMS CR 2634.  The Requirement Identifier would be 22522-2634- 2.8.  3. The Shared System Maintainer and the Single Testing Contractor (STC) shall complete Test Case specifications that include specific input situa tions and the expected results associated with a single test purpose.  Each test case specification must include the following:    A unique Test Case Identifier (which includes a cross -reference to the requirement in which the case is designed to test);    The specific objective or purpose of the case;    Input specifications (i.e., a description of the input situation[s]);    Output specifications (i.e., a description of the expected results); and    Intercase dependencies -  in instances where the test results of on e test case may impact other test cases, the test case specification must identify the other test case(s) and describe the relationship(s).  Refer to section 40.3.10, Test Case Specification Standard, for the specific format required to electronically main tain test cases.   4. All Test Cases must contain a unique Test Case Identifier.  The CMS standard for the Test Case Identifier is the Requirement Identifier, followed by a number that uniquely qualifies the test case specification, separated by a dash.   The format of the Test Case Identifier is:  [Requirement Identifier] -[Test Case Number], where:    Requirement Identifier  is the actual identifier of the requirement being testing by the case.    Test Case Number  is a number that uniquely qualifies the tes t case.  This is generally a sequential number.  This is necessary since more than one test case is often needed to test a single requirement.   Dashes not allowed within this number; they are reserved for separation between the individual numbers in the T est Case Identifier.   Example:  Two test cases were developed to test the implementation of Requirement 22522- 2634- 2.8 (see example above).  The unique  Test Case Identifier for the two test cases would be 22522 -2634- 2.8-01 and 22522- 2634- 2.8-02.  5. The Sh ared System Maintainers and the Single Testing Contractor (STC) shall document and execute both  positive and negative test cases to ensure the requirements of the release are correctly implemented.    Positive test cases are required to ensure that the system is directly fulfilling the requirements as specified.  One or more positive test cases are required for each requirement. As an example, if a program mandate effects a change for services beginning on July 1, a positive test case would include service da tes in July or later and validate that the actual mandate was correctly implemented.    Negative test cases test cases are required to ensure that the system does not perform an incorrect action.  As an example, if a program mandate effects a change for serv ices beginning on July 1, a negative test case would ensure that implementing the mandate did not negatively impact claims with service dates prior to July 1.  Unlike positive test cases, a negative test case may not be applicable to every requirement with in a CR.  Additionally, although due diligence might necessitate a negative test case, the need may be mitigated by an existing case in your regression test set.   6. The Shared System Maintainers and the Single Testing Contractor (STC) shall document all test cases and the actual results for each test case electronically.  Each test case and the associated results must be stored in a test management repository (i.e., Application Lifecycle Management (ALM) tool) and must at a minimum contain the data element s outlined in the CMS Test Case specification standard.  See subsection 40.3.10 for the Test Case specification standard.   7. The Shared System Maintainers and the Single Testing Contractor (STC) shall maintain a test log that provides a record of each tes t execution.  Test Log requirements may be fulfilled by correctly using the ALM run feature (or an equivalent tool or approach) as outlined in the Quarterly Release Test Management User Guide.   8. The Shared System Maintainers and the Single Testing Cont ractor (STC) shall execute a full regression test set on their system for every quarterly release.  Each testing entity shall perform regression testing within their designated testing window as outlined in subsection 40.3.7, Timeframe Requirements.   9. The Shared System Maintainers and the Single Testing Contractor (STC) shall perform interface testing.    The Shared System Maintainers and the Single Testing Contractor (STC) shall validate that all  output files are correctly created by their system.  The S SMs and STC shall validate that their system can accept and correctly process all  input files.    The Shared System Maintainers and the Single Testing Contractor (STC) shall perform interface testing that includes full data exchanges (both ways) between the shared system and any principal claims processing adjudication or financial system (e.g., the CWF and HIGLAS respectively).  The STC is required to perform data exchanges with HIGLAS after HIGLAS is implemented at the STCs data center.    The Shared System Maintainers and the Single Testing Contractor (STC) shall complete an integrated system test coordinating the maintenance of test data baselines, such as beneficiary data, with the CWF Beta tester.   40.3.3  Testing Standards Applicable to all Beta Testers  (Rev. 97, Issued: 01- 15-16, Effective: 09- 21-15, Implementation: 09- 21-15)  1. Each shared system maintainer and each beta t ester shall complete integrated testing with the CWF Beta tester, using coordinated beneficiary data, in the execution of their tes t cases.   All test cases involving CWF functionality (related to claims adjudication) must be executed in an integrated test with the CWF.  This requires full data exchanges between testing entities including:    Satellite files being sent from the shared systems to the CWF Beta tester; and    Response files being sent from the CWF to the shared system Beta testers.   2. Each Beta tester shall:    Utilize the standard CMS Test Management tool and repository to documents all test cases and results.    Follow the pr ocedures outlined in the Quarterly Release Test Management User Guide in order to complete the documentation of test runs and results.  3. Each STC/beta tester shall review all Shared System Maintainer release documentation for completeness, accuracy, and usability.  Any questions, problems, or issues with the documentation shall be forwarded to both the Shared System Maintainer and CMS.   4. Each STC/beta tester shall conduct performance testing to reasonably assure that the system provides acceptable respo nse times, throughput rates, and processing windows and can accommodate production workloads.   5. The CMS testing requirements outlined in section 40.3 may require the STC/beta tester to test a specific type of bill, specialty, or claim situation for which  they do not possess the required level of expertise.  In these instances, the STC/beta tester must partner with a Medicare A/B, DME MAC that possesses both the expertise and capabilities to test the specialty or claim type.  As an example, should the STC/ beta tester not have the operational capability or expertise to process Home Health claims, they are expected to partner with an A/B MAC (HHH)  to complete the required HHA testing.  Ultimately, the STC/beta tester is responsible for ensuring all test cases  are exercised.  Any partnerships that are established to complete the testing requirements shall be arranged and managed by the STC/beta tester.   40.3.4 - Testing Requirements Applicable to the CWF Beta Tester  (Rev. 43; Issued:  03- 30-07; Effective:  07- 01-07; Implementation:  07 -02-07)  The CWF Beta tester shall act as a test host and exchange data with entities testing the FISS, MCS, and VMS shared systems.  The testing entities include all shared system maintainers and  the shared system Beta t esters .  40.3.5  Part A/Part B (A/B) Durable Medical Equipment (DME) Medicare Administrative Contractor (MAC) (User) and the Single Testing Contractor (STC) Testing Requirements  (Rev. 97, Issued: 01- 15-16, Effective: 09- 21-15, Implementation: 09- 21-15)  A/B, DME MACs are not mandated to prepare and execute test cases that cover Medicare business requirements implemented within the base system in standard system and CWF quarterly releases.  Shared System Maintainers and the STC are fully responsible for testing  the base functionality.  The A/B, DME MAC (users) shall test their local/unique components and conduct a limited, end- to-end, operational test.  1. A/B, DME MACs shall fully test their local components and processing rules prior to production implementation of the quarterly release.  This testing is applicable for all local components and processing rules modified since the previous quarterly release.   A. A/B, DME MACs shall test any system components they maintain and implement to support claims processing in a ddition to the base system.  This includes front -end and back -end components such as those for EDI entry and translation, EDI outbound processing, and printing (e.g., MSN generation).   B. A/B, DME MACs shall test changes they make to user control files, facil ities, and tables in order to implement new Medicare policy or business rules.  Examples of these facilities include but are not limited to auto adjudication facilities (e.g., ECPS and MCS SCF) and the MCS SPITAB.   C. A/B, DME MAC shall fully test any shared system functionality that was:   - Developed by the shared system maintainer solely for them, or  - Developed by the shared system maintainer under a special project in which they were the exclusive participant.   An example would be a MAC working with CMS on a special demonstration project.  In this example the MAC shall fully test the shared system functionality that was implemented for the demonstration project.   2. A/B, DME MACs shall complete a limited end -to-end operational test that incorporates the shared sy stem release, integrated with their other claims processing components. These components include the front -end for claims receipt, translators, the CWF, the financials, and back -end EDI and report generation.  The test must ensure that processing is contig uous from claims entry, to claims adjudication, and ultimately remittance and Medicare Summary Notice generation.  Through contiguous processing, the interfaces between all key claims processing components must be exercised.  The banking system interfaces such as National Clearing House (NCH) transfers need not be exercised.  The test is limited in the number of test cases that are required, since shared system maintainers and the STC are testing the base functionality of the shared system.   A. A/B, DME MAC s shall ensure that the integrated systems software can complete cycles without system abends and produce the expected output.  The A/B, DME Medicare Contractor shall ensure their operational test:    Exercises all claims entry points not fully incorporated in the base system such as paper and EMC front -end components;    Includes all allowable standard electronic formats and versions;    Includes a variety of claims types; and    Includes all components that support their claims workload and interfaces to the shared system.