government rejects the test results the contractor shall retest until such time the results are acceptable to the government. The contractor shall perform BSS verification testing according to the accepted BSS Test Plan at a mutually acceptable date with the government. BSS Test Scenarios Testing Prerequisites Prior to initiating BSS testing, the contractor shall: Demonstrate the contractor’s BSS has passed its internal unit testing prior to BSS interface testing with the GSA system. Provide a finalized BSS Test Plan that is verified and accepted by GSA. The purpose of the verification and acceptance testing is to ensure that the contractor’s BSS requirements in Sections G.3 – G.9 and Sections J.2.2 – J.2.10. The contractor shall support BSS security and functional testing as defined in Section G.5.6 BSS Security Requirements and Section G.5.5.1 BSS Testing. Test Scenarios The following table contains a high-level list of BSS Test Scenarios for which the contractor’s BSS must pass the defined acceptance criteria. The contractor shall address the test scenarios based on the functional requirements defined in the relevant portions of Section G and J (see the “RFP References” column for references). The scenarios address relevant data exchange mechanisms and validation of data exchanged. Each Test Scenario is associated with one or more Test Cases defined in Section E.2.1.3. Test Scenario # RFP References Description Acceptance Criteria BSS-TS01 G.5.3.2 J.2.9 Exchange structured data using the defined direct data exchange methods: XML via secure web services Pipe, “|”, delimited table via SFTP The contractor shall demonstrate bidirectional exchange of defined data structure that meets the interface specifications as defined in Section G.5.3.2 and Section J.2.9 and is verified and accepted by GSA. BSS-TS02 G.3 J.2.2 J.2.3 The contractor’s BSS manages the following: Provide TO and Account Management reference data as specified in Section J.2.2 and J.2.3 The contractor shall demonstrate successful TO and Account Management initial setup and updates.
1) Service Order Acknowledgement 2) Service Order Rejection Notice 3) Service Order Confirmation 4) Firm Order Commitment Notice 5) Service Order Completion Notice BSS-TS06 G.3 J.2.4 The contractor’s BSS handles orders for administrative changes to the records for previously provisioned services as described in G.3 The contractor shall demonstrate that an authorized government user can place an administrative change order using the methods specified in Section J.2.4 and the order populates the fields in the contractor’s ordering system in a way that meets the requirements in G.3 and G.5 and Sections and is verified and accepted by GSA. the direct data exchange method defined in Section J.2.4, the contractor shall demonstrate that its ordering system can provide all required CDRLs including: 1) Service Order Acknowledgement 2) Service Order Completion Notice BSS-TS07 G.3.5.6 J.2.4.2.4 The contractor’s BSS manages Self-Service Provisioning and other Rapid Provisioning orders and provides the correct notices. The contractor successfully demonstrate the completion of these orders. Non-self-service orders shall be tested using both correctly placed orders and orders with related errors. Self-service orders shall be tested with correctly placed orders and to ensure that the contractor’s BSS does not permit the placement of incorrect orders. Using the direct data exchange method defined in Section J.2.4, the contractor shall demonstrate that its ordering system can provide all required CDRLs including: 1) Service Order Acknowledgement 2) Service Order Completion Notice BSS-TS08 G.4 J.2.5 J.2.6 J.2.7 The contractor’s BSS properly manages inventory and billing: Generates the inventory of services delivered by the contractor The contractor shall demonstrate that its service inventory management system maintains a complete and accurate inventory of EIS service orders in a way that J.2.10
Produces output that is consistent with order and billing details Generates the detailed billing in accordance with the Billing Invoice (BI) CDRL Properly handles usage-based billing Correctly calculates the AGF due to GSA and produces the required AGF CDRLs Provides accurate calculation of rounding and proration related to Billing, Taxes, Fees and Surcharges meets the requirements in G.7 and G.5 and Section J.2 CDIP and is verified and accepted by GSA. The contractor shall demonstrate that the output of its billing data elements is consistent with the orders entered into its ordering system and that the billing data elements meet the requirements in G.4 and G.5 and Section J.2 CDIP and is verified and accepted by GSA. Using the direct data exchange method defined in Sections the contractor shall demonstrate that its ordering system can provide all required CDRLs including: 1) Billing Invoice 2) Billing Adjustment 3) Tax Detail 4) AGF Detail 5) AGF Electronic Funds Transfer Report 6) Inventory Reconciliation BSS-TS09 G.4.4 J.2.6 The contractor’s BSS properly manages all dispute types with appropriate handling for: Government initiated disputes Contractor initiated disputes Billing disputes Inventory disputes SLA disputes Dispute tracking and reporting The contractor shall demonstrate that its BSS can accept and issue disputes as well as tracking them to resolution. Using the direct data exchange method defined in Sections the contractor shall demonstrate that its ordering system can provide all required CDRLs including: 1) Dispute 2) Dispute Report BSS-TS10 G.8 J.2.8 The contractor’s BSS properly manages SLA Management: SLA Reporting SLA Credit Request handling and response The contractor demonstrate that its BSS successfully tracks SLAs with associated KPIs as well as reporting SLA performance and providing sufficient information to response to SLA Credit Requests. Using the direct data exchange method defined in Sections the contractor shall demonstrate that its ordering system can provide all required CDRLs including:
1) SLA Report 2) SLA Credit Request Response BSS-TS11 F J.2 The contractor’s BSS produces acceptable open-format reports defined in the CDIP: Monthly Billing Information Memorandum Trouble Management Incident Performance Report Trouble Management Performance Summary Report The contractor shall demonstrate, via sample reports, that the reports specified are sufficiently detailed and clear so as to meet the government’s requirements. BSS-TS12 J.2 The contractor’s BSS testing includes regression testing of all key features including ordering, service assurance, and billing. NOTE: Applies only to testing conducted as part of system changes, not initial BSS development. The contractor shall demonstrate that its BSS meets regression testing and is verified and accepted by GSA. The BSS test plan shall include regression testing; however, actual regression testing will not be part of initial test and acceptance. BSS-TS13 G.5.6 The contractor’s BSS has been provided an Authority to Operate (ATO) as defined by the following NIST.gov publications: SP.800-53 REV 4 FIPS Publication 200 The contractor shall demonstrate that the BSS meets FISMA Moderate requirements compliance and verified and accepted by GSA. BSS Test Cases The contractor shall accept, incorporate into the BSS Test Plan, and successfully execute test cases provided for each of the test scenarios above. The contractor shall accept the following test conditions: No testing between the contractor and GSA occur until both the contractor’s BSS and the GSA System have passed unit testing. All testing to be performed on the actual system to be used in delivering service (i.e. special purpose “test systems” shall not be used). Unless otherwise specified, all data transfers are to use the mechanism specified in Section J.2 for that data set. format
BSS-TS01-04: Error Handling: PSV over SFTP Test Scenario BSS-TS01: Direct Data Exchange Test Case ID BSS-TS01-04 Test Case Description Error Handling: PSV over SFTP Requirements Reference(s) G.5.3.2 J.2.9 Prerequisites N/A Government Input(s) Invalid government data set listed as using SFTP as the transfer mechanism in Section J.2 Expected Output(s) Notification to contractor of failed import Acceptance Criteria Evidence of failure notification No partial import Data Set Description Each government-provided test data set shall include: One government data set listed as using SFTP as the transfer mechanism in Section J.2 One or more formatting errors (e.g. missing delimiters) BSS-TS02: Task Order and Account Management Setup BSS-TS02-01: Direct Billing Account Setup Test Scenario BSS-TS02: Task Order and Account Management Setup Test Case ID BSS-TS02-01 Test Case Description Direct Billing Account Setup Requirements Reference(s) G.3 J.2.2 J.2.3 Prerequisites N/A Government Task order setup data as defined in Section J.2.2 Customer Registration data as defined in Section J.2.3 System Reference data as defined in Section J.2.3 Expected Output(s) Contractor POC CDRLs Customer Registration CDRLs as defined in Section J.2.3.3.2 Agency Setup (AS) Permissions Permissions Service Permissions Task Order
BSS-TS03: Role Based Access Control BSS-TS03-01: Authorized User Access Verification Test Scenario BSS-TS03: Role Based Access Control Test Case ID BSS-TS03-01 Test Case Description Authorized User Access Verification Requirements Reference(s) G.5 J.2.3.1.2 Prerequisites TO setup data loaded into contractor BSS Account Management data loaded into contractor BSS Government Input(s) User attempts to: Sign into BSS Access BSS areas to which they are an authorized user Exercise the full range of functionality (read/write) permitted for their role Expected Output(s) User is permitted to: Access to BSS Access BSS areas as authorized Exercise assigned functionality Acceptance Criteria Access is granted No security errors displayed Data Set Description Each government-provided test data set shall include: Role to be tested Functionality to be tested BSS-TS03-02: Unauthorized User Access Denial Verification Test Scenario BSS-TS03: Role Based Access Control Test Case ID BSS-TS03-02 Test Case Description Unauthorized User Access Denial Verification Requirements Reference(s) G.5 J.2.3.1.2 Prerequisites TO setup data loaded into contractor BSS Account Management data loaded into contractor BSS Government Input(s) User attempts to: Sign into BSS Access BSS areas to which they are not an authorized user Exercise functionality (read/write) not permitted for their role
BSS-TS04-02: New Order via Email Test Scenario BSS-TS04: Service Ordering Test Case ID BSS-TS04-02 Test Case Description New via Email Requirements Reference(s) G.3 G.5.3.2 J.2.4 Prerequisites TO setup data loaded into contractor BSS Account Management data loaded into contractor BSS Government Input(s) Service Order (SO) with all required data elements as described in Section J.2.4 Expected Output(s) Service Order notification CDRLs as defined in Section J.2.4: Service Order Acknowledgment (SOA) Service Order Confirmation (SOC) Firm Order Commitment Notice (FOCN) Service Order Completion Notification (SOCN) Acceptance Criteria All required as defined in Section J.2.4 Accurate data based on inputs Correct technical aspects Data Set Description Each government-provided test data set shall include: A complete SO for one or more services listed in Section C.2 of the contract SO submitted via a means listed in J.2.4 other than the contractor’s web interface BSS-TS04-03: Disconnect Order Test Scenario BSS-TS04: Service Ordering Test Case ID BSS-TS04-03 Test Case Description Disconnect Order Requirements Reference(s) G.3 J.2.4 Prerequisites TO setup data loaded into contractor BSS Account Management data loaded into contractor BSS Previously provisioned circuit or service element entered into the contractor BSS Government Input(s) Service Order (SO) with all required data elements as described in Section J.2.4 for the disconnect of A circuit or service element (CLIN) A feature of a circuit or service element
Expected Output(s) Service Order notification CDRLs as defined in Section J.2.4: Service Acknowledgment (SOA) Service Confirmation (SOC) Firm Order Commitment Notice (FOCN) Service Completion Notification (SOCN) UBI for new service has the same service group prefix (see J.2.10.1.1.2) as the provided UBI Acceptance Criteria All required CDRLs as defined in Section J.2.4 Accurate data based on inputs Correct technical aspects Data Set Description Each government-provided test data set shall include: A complete SO for one or more services listed in Section C.2 of the contract BSS-TS04-10: Bulk Orders Test Scenario BSS-TS04: Service Ordering Test Case ID BSS-TS04-10 Test Case Description Bulk Orders Requirements Reference(s) G.3 J.2.4 Prerequisites TO data loaded into contractor BSS Account Management data loaded into contractor BSS Government Input(s) Bulk Service Order (SO) with all required data elements as described in Section J.2.4 Expected Output(s) Service notification CDRLs as defined in Section J.2.4: Service Order Acknowledgment (SOA) Service Order Confirmation (SOC) Firm Order Commitment Notice (FOCN) Service Order Completion Notification (SOCN) Acceptance Criteria All required CDRLs as defined in Section J.2.4 Accurate data based on inputs Correct technical aspects Data Set Description Each government-provided test data set shall include: A SO including at least 20 line items for services listed in Section C.2 of the contract SO data provided via a delimited text file or MS Excel file BSS-TS04-11: Error Checking: Missing Info Test Scenario BSS-TS04: Service Ordering
Requirements Reference(s) G.3 J.2.4 Prerequisites TO setup data loaded into contractor BSS Account Management data loaded into contractor BSS One or more previously provisioned orders Government Input(s) Service Order (SO) that specifies a change to the administrative data associated with a previously provisioned service as described in Section G.3 Expected Output(s) Service Order notification CDRLs as defined in Section J.2.4: Service Order Completion Notification (SOCN) Acceptance Criteria All required CDRLs Accurate data based on inputs Correct technical aspects Data Set Description Each government-provided test data set shall include: A complete SO for a change to the administrative data for a previously provisioned service as described in Section G.3 BSS-TS07: Rapid Provisioning & Self-Provisioning Orders BSS-TS07-01: Rapid Provisioning Orders Test Scenario BSS-TS07: Rapid Provisioning & Self-Provisioning Orders Test Case ID BSS-TS07-01 Test Case Description Rapid Provisioning Orders Requirements Reference(s) G.3 G.5.3.2 J.2.4 Prerequisites TO setup data loaded into contractor BSS Account Management data loaded into contractor BSS Government Input(s) Service Order (SO) for one or more services subject to rapid provisioning as defined in Section G.3 with all required data elements as described in Section J.2.4 Expected Output(s) Service Order notification CDRLs as defined in Section J.2.4: Service Order Acknowledgment (SOA) Service Order Completion Notification (SOCN) Acceptance Criteria All required CDRLs as defined in Section J.2.4 Accurate data based on inputs Correct technical aspects Data Set Description Each government-provided test data set shall include: A complete SO for one or more services subject to provisioning as defined in Section G.3