# An Architecture for a Tri-Programming Model-Based Parallel Hybrid Testing Tool
## 3 Literature Review  
Multiple studies have tested HPC and parallel software for various purposes and scopes. Testing techniques, namely static, dynamic, and hybrids, are used to detect various types of errors. Static testing involves examining the source code prior to system execution [31], while dynamic testing examines a source code while the system is running [24]. Although dynamic techniques have benefits, they also have significant drawbacks, such as the need for multiple test cases and the possibility of undetected errors when the system starts up. Hybrid techniques combine both static and dynamic testing [31; 32]. Many tools, such as Intel(r) Trace Analyzer and Collector [33] and MPI-Checker [34], underutilise static testing to detect run-time errors in MPI. Dynamic testing, however, is widely used by other tools, such as MEMCHECKER [35], Umpire [36], the Marmot Umpire Scalable Tool (MUST) [37; 38; 39], MAD [40], MOPPER [41], and run-time error detection (RTED) [42], to detect run-time errors in MPI. Meanwhile, other tools test different run-time errors. These tools use hybrid techniques [43] to detect deadlocks. The ACC_TEST [44] was specifically designed to swiftly identify and precisely locate any run-time errors, including, but not limited to, deadlocks, race conditions, or mismatches. Various OpenMP testing tools use multiple techniques to detect run-time errors. For instance, GPUverify [45], OmpVerify [46], and DRACO [47] use static techniques, while Helgrind [48], RTED [42], Valgrind [49], ROMP [50], Intel(r) Thread Checker [51; 52], and Sun Studio(r) Thread Analyzer [51; 53] use dynamic techniques. Meanwhile, AddressSanitizer (ASan) and ThreadSanitizer (TSan) [54; 55] can be used to detect run-time errors in systems with OpenMP directives. ARCHER [56], on the other hand, uses a hybrid technique to detect data races in large OpenMP applications. Dynamic techniques are also used to test dual systems that combine MPI + OpenMP, such as Marmot [57; 58]. GPUverify [45], and PolyOMP [59] use static techniques to detect data race errors in systems that contain CUDA. GUARD [60], RaceTM [61], and KUDA [62] are examples of dynamic testing tools, while GMRace [63], GRace [64], Grace [65; 66], and SESA [67] are examples of hybrid testing tools that test run-time errors in systems that contain CUDA. The literature review reveals a significant gap in the market in terms of testing tools for tri-level programming models. As such, this present study creates a highly specialised testing tool that incorporates MPI, OpenMP, and CUDA specifically for this unique programming model. Although debugging tools are a vital part of testing, their testing techniques often require clarification. Some debugging tools are commercial, while others, such as such as AutomaDeD [68], ALLINEA DDT [69; 70], TotalView [71; 72], MPVisualizer [73], Intel(r)Inspector [74], PDT [75], and Arm DDT [76; 77], are not. However, as these non-commercial debugging tools pinpoint the causes of errors instead of testing or detecting errors, they cannot be categorised according to their testing techniques. The literature review also reveals that there is no testing tool with which to detect run-time errors in MPI + OpenMP + CUDA tri-level programming models. As such, this present study develops a hybrid technique to identify many actual and potential errors in the C++ source code beforehand. Although multiple extant studies have developed and proposed software testing tools for parallel applications, the detection of static and dynamic errors in tri-programming models warrants further examination. Furthermore, in the case of heterogeneous systems, tri-programming models still require significant improvement. Therefore, it is imperative to address the lack of testing tools with which to detect run-time errors in applications that have been developed using MPI + OpenMP + CUDA tri-level programming models, and it must be resolved immediately to ensure the efficiency and effectiveness of these applications.