# Quality Assurance Technologies of Big Data Applications: A Systematic Literature Review
## 1 Introduction  
The big data technology market grows at a 27% compound annual growth rate (CAGR), and big data market opportunities will reach over 203 billion dollars in 2020 [1]. Big data application systems [2, 3], abbreviated as big data applications, refer to the software systems that can collect, process, analyze or predict a large amount of data by means of different platforms, tools and mechanisms. Big data applications are now increasing, being used in many areas, such as recommendation systems, monitoring systems and statistical applications [4, 5]. Big data applications are associated with the so-called 4V attributes, e.g., volume, velocity, variety and veracity [6]. Due to the large amount of data generated, the fast velocity of arriving data and the various types of heterogeneous data, the quality of data is far from ideal, which makes the software quality of big data applications far from perfect [7]. For example, due to the volume and velocity attributes [8, 9], the data generated from big data applications are extremely numerous and more so with high speed Internet, which may affect data accuracy and data timeliness [10], and consequently lead to software quality problems, such as performance and availability issues [10, 11]. Due to the huge variety of heterogeneousdata [12; 13], data types and formats are increasingly rich, including structured, semi-structured and unstructured, which may affect data accessibility and data scalability, and hence lead to usability and scalability problems. In general, quality assurance (QA) is a way to detect or prevent mistakes or defects in manufactured software/products and avoid problems when solutions or services are delivered to customers [14]. However, compared with traditional software systems, big data applications raise new challenges for QA technologies due to the four big data attributes (for example, the velocity of arriving data, and the volume of data) [15]. Many scholars have illustrated current QA problems for big data applications [16; 17]. For example, it is a hard task to validate the performance, availability and accuracy of a big data prediction system due to the large-scale data size and the feature of timeliness. Due to the volume and variety attributes, keeping big data recommendation systems scalable is very difficult. Therefore, QA technologies of big data applications are becoming a hot research topic. Compared with traditional applications, big data applications have the following special characteristics: (a) statistical computation based on large-scale, diverse formats, with structured and non-structured data; (b) machine learning and knowledge-based system evolution; (c) intelligent decision-making with uncertainty; and (d) more complex visualization requirements. These new features of big data applications need novel QA technologies to ensure quality. For example, compared with data in traditional applications (graphics, images, sounds, documents, etc. ), there is a substantial amount of unstructured data in big data applications. These data are usually heterogeneous and lack integration. Since the handling of large-scale data is not required in the traditional applications, traditional testing processes lack testing methods for large-scale data, especially in the performance testing. Some novel QA technologies are urgently needed to solve these problems. In the literature, many scholars have investigated the use of different QA technologies to assure the quality of big data applications [15; 18; 19; 20]. Some papers have presented overviews on quality problems of big data applications. Zhou et al. [18] presented the first comprehensive study on the quality of the big data platform. For example, they investigated the common symptoms, causes and mitigation strategies of quality issues, including hardware faults, code defects and so on. Juddoo [19] et al. have systematically studied the challenges of data quality in the context of big data. Gao et al.