## 4 Experiment  
For the experimental evaluation, an Apache Hadoop cluster was used. It has the characteristics described in Table 1.  
For the study, a dataset obtained using the Digital Psychological Platform for Mass Web-Surveys [15] was used. This platform collects data using web interfaces built into the platform itself. Based on the surveys, a population data base is formed. These data become available for analysis by interdisciplinary research teams.  
The data are a set of JSON objects describing the results of the online testing of each individual participant. Figure 1 shows a simplified example of such an object.  
Each JSON object contains the following information about the completed questionnaire:  
* name and surname of the test taker;
* the device platform from which the test was passed (commonly, it is information about the operating system installed on the device)--this information can help in understanding which type of device (PC, mobile, old platforms) is used to improve the support of such operating systems;
* an array of test completion information for each question. The most interesting information for this study from the array is the time the test taker spent on each question.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Configuration Item** & **Characteristics** \\ \hline Nodes count & 3 \\ Replication factor & 2 \\ Hadoop Distributive & Cloudera v. 4.2 \\ \hline \hline \multicolumn{3}{c}{**Nodes Characteristics**} \\ \hline Processor & Intel Core i7 2 Cores \\ RAM & 4 GB \\ OS & Debian 9 \\ Platform & Java Virtual Machine \\ Programming language & Java v.1.8 \\ Framework & Apache Spark v. 2.4 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Experimental stand configuration.  
Figure 1: An example of an object containing information about the passing of testing by one participant.  
As a result, it is necessary to obtain a data mart containing aggregated information for each question and device: the type of device, question number, mean response time, median time, standard deviation, and first and third quartiles. The total volume of all the data prepared for the experiment is 2 GB. These data were obtained during mass psychological testing in different schools of the country.  
An application was developed using the Apache Spark framework [30], which has a large set of functions for working with data in a distributed system. The application implements an algorithm for reading, creating a data mart, and writing to a specific format, depending on the transmitted parameters.  
Figures 2-4 show the implemented algorithms for creating a data mart based on the resulting dataset.  
The experiment was as follows. From the dataset, it is necessary to prepare a mart containing statistical data on all the participants in the mass testing. Test pass data are a set of JSON files containing JSON objects for each participant in a bulk test. All the files are located in the HDFS cluster. The developed application was used to build the showcase.  
Figure 3: Developing data mart.  
Figure 2: Reading and clearing data.  
The experiment is shown in Figure 5. The spark-submit function launches a Spark application and reads data from the HDFS. To build a report, a data frame is generated, consisting of data aggregated by the application. The result is written into one of the data storage formats. Then, a link to the location is automatically created to create a table in Apache Hive [31], which is necessary for further reading of the results by employees without programming skills, as well as by analysts. From the moment the application starts working until its termination, the time of working with the data is measured.