# Quality Assurance Technologies of Big Data Applications: A Systematic Literature Review
## 4 Results
### Identify the Focused Quality Attributes in Big Data Applications (RQ1)  
The goal of this section is to answer RQ1 (which of the quality attributes are focused in QA of big data applications?). Besides identifying the quality attributes focused in the primary studies, we also analyze the influencing factors brought by the big data attributes, so that the factors can be taken into consideration in QA of big data applications. We use the ISO/IEC 25010:2011 to extract the quality attributes of big data applications. Table 8 provides the statistical distribution of different quality attributes. According to statistics, we identify related quality attributes, as shown in Figure 7. For some articles that may involve more than one quality attribute, such as [38; 39], we choose the main quality attribute that they convey. From the 83 primary studies, some quality attributes are discussed, including correctness, performance, availability, scalability, reliability, efficiency, flexibility, robustness, stability, interoperability and consistency. From Figure 7, we can see that the five main quality attributes have been discussed in the 83 primary studies. However, there are fewer articles focused on other attributes, such as stability, consistency, efficiency and so on. * Correctness: Correctness measures the probability that big data applications can \"get things right.\" If the big data application cannot guarantee the correctness, then it will have no value at all. For example, a weather forecast system that always provides the wrong weather is obviously not of any use. Therefore, correctness is the first attribute to be considered in big data applications. The papers [41, 44] provide the fault tolerance mechanism to guarantee the normal operation of applications. If the big data application runs incorrectly, it will cause inconvenience or even loss to the user. The papers [45, 40] provide the testing method to check the fault of big data applications to assure the correctness. * Performance: Performance refers to the ability of big data applications to provide timely services, specifically in three areas, including the average response time, the number of transactions per unit time and the ability to maintain high-speed processing. Due to the large amounts of data, performance is a key topic in big data applications. In Table 8, we show the many papers that refer to the performances of big data applications. The major purpose of focusing on the performance problem is to handle big data with limited resources in big data applications. To be precise, the processing performance of big data applications under massive data scenarios is its major selling point and breakthrough. According to the relevant literature, we can see that common performance optimization technologies for big data applications are generally divided into two parts [44, 55, 56, 71, 72, 94]. The first one consists of hardware and system-level observations to find specific bottlenecks and make hardware or system-level adjustments. The second one is to achieve optimization mainly through adjustments to specific software usage methods. \begin{table}
\begin{tabular}{l l} \hline \hline
**Attributes** & **Studies** \\ \hline Correctness & [40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53] \\ \hline Performance & [54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93] \\ \hline Reliability & [94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113] \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Distribution of the quality attributes. Figure 7: The quantity distribution of the quality attributes. * Availability: Availability refers to the ability of big data applications to perform a required function under stated conditions.