# An Experimental Evaluation of Path-Based Product Line Integration Testing and Test Coverage Metrics
## 6 Discussion  
This paper revised the initial version of MM-path-based SPL integration testing and validated the effectiveness of it with test coverage metrics through two experimental evaluations. The path-based integration testing method, extended for SPLT, can be executed during platform testing if the driver and stub for the call-return of variability-related methods are implemented, as variability is part of the path. However, with the proposed method and metrics, similar to single software testing, finding paths can be challenging due to the large size of the software. This challenge can be addressed by using the call graph to search for _callee_ methods beginning with the start method. Experimental evaluation of the proposed method and metrics may encounter the following issues:  
* _Worst-case xx-MM-path-based testing_: cc-MM-paths, cv-MM-paths, and vc-MM-paths can be exceedingly rare. In such cases, it is possible to separate the thread of the path into sub-paths, rather than testing end-to-end paths. The sub-path containing variants can then be tested during platform testing, facilitated by driver or stub implementation. This approach allows for an increased number of testable paths during platform testing. While this scheme may reduce the advantages of path-based integration that relies on actual code without the need for driver or stub implementation, it proves valuable when only a small number of variabilities exist in the path due to the characteristics of SPLT. Variability may be implemented during platform development or by introducing drivers and stubs for testing. If variabilities are implemented, testing must encompass every possible combination that can be bound to the variation point. This enables the subdivision of paths containing a variation point into multiple detail paths.
* _Path-explosion and path management_: Because real-world SPLs include hundreds of variabilities, it is challenging to apply path-based testing comprehensively. This challenge also applies to path-based testing in individual software products. In single software testing, MM-path-based integration testing is used for integration-level testing and ASF (atomic system function) testing, which verifies function visibility at the system level [24]. This paper focuses on method and message-level MM-path-based integration testing for SPL. One approach to reduce the number of paths is to base them on input and output port events of the atomic system function, rather than methods or messages. However, ASF graphs become available only after lower-level verification is completed. Path-based testing can adapt path threads to different levels, such as unit, integration, subsystem, or system, depending on the context. While the detailed discussion of this adaptation is beyond the paper's scope, it offers a solution to the challenges of path explosion and path management stemming from numerous path threads.
* _Test data (test inputs) definition_: A significant challenge in path-based testing involves defining test data that covers path threads [29]. Research into automatic test data generation methods, including those for path-based software testing [30], is currently ongoing. This challenge is not unique to SPLT and requires ongoing investigation to develop automatic test generation methods that consider the specific characteristics of SPLT, similar to what is performed in single software testing.
* _Handling variability dependencies and constraints_: The proposed method considers variability dependencies (optional, mandatory, alternative) and variability constraints based on the feature model. This is intended to define the expected results of test cases that cover the paths. However, if the implementation deviates from the specification, including variability dependencies and constraints of the feature model, and message passing occurs on an invalid path, it is evident that MM-paths may include invalid paths.