## 2 Materials and Methods
### Crowdsourcing for Dataset Creation, Validation, and User Cooperation  
Crowdsourcing is a methodology for distributed problem-solving that happens online by the collective intelligence of a community in a specific predefined direction set by an organization [49]. It has gained interest thanks to its efficiency and applicability in multiple domains and tasks [57; 58; 59; 60; 61]. In machine learning and automation, it has become very popular for collaborative problem solving and dataset formulation and validation [57; 59]. Users are expected to participate according to intrinsic (fun, personal growth, etc.) and extrinsic (payment, rewards, etc.) motives [59; 60].  
Within the MAthE approach, crowdsourcing is promoted in several ways. First of all, crowdsourcing was promoted for the collaboration on the formulation of a database  
\begin{table}
\begin{tabular}{l l l} \hline \hline
**Layer** & **Type** & **Configuration** \\ \hline  & & 16 filters \\
1 & Convolutional 2D Layer & Kernel size = (3,3) \\  & & Strides = (1,1) \\
2 & Max Pooling 2D Layer & Pool size = (2,2) \\
3 & Dropout & Rate = 0.25 \\  & & 32 filters \\
4 & Convolutional 2D Layer & Kernel size = (3,3) \\  & & Strides = (1,1) \\
5 & Max Pooling 2D Layer & Pool size = (2,2) \\
6 & Dropout & Rate = 0.25 \\  & & 64 filters \\
7 & Convolutional 2D Layer & Kernel size = (3,3) \\  & & Strides = (1,1) \\
8 & Dropout & Rate = 0.25 \\  & & 128 filters \\
9 & Convolutional 2D Layer & Kernel size = (3,3) \\  & & Strides = (1,1) \\  & & 256 filters \\
10 & Convolutional 2D Layer & Kernel size = (3,3) \\  & & Strides = (1,1) \\
11 & Flatten Layer & \\
12 & Dense Neural Network & Activation = ReLU \\  & & L2 regularizer \\
13 & Dense Neural Network & Output weights = 64 \\
14 & Dropout & Activation = ReLU \\
15 & Dense Neural Network & Output weights = 24 \\  & & Activation = Linear \\ \hline \hline \end{tabular}
\end{table}
Table 2: The architecture and hyper-parameters of the CNN model for reverberation level estimation.  
of tampered audiovisual files. The importance of a real-world dataset for the training and evaluation of different machine learning and computer-assisted tampering detection approaches was highlighted in the previous sections. When a user provides a file for analysis in the web application, the file is stored temporarily to perform the analysis. After the results are provided, the user is asked for permission to save the file in our dataset. The file can be stored with or without accompanying metadata concerning the user's final decision. In case of a positive response, the file is stored.  
Besides submitting files, users can also provide validation of existing files in the database. Such files may have been uploaded by other users but have no evaluation concerning their integrity, or may be files that are already annotated. It is common practice to include several annotators to strengthen the reliability and overall quality of the dataset.  
In the case of crowdsourcing, the validation of files that have been uploaded by other users without an evaluation, collective intelligence, and collaboration are integrated into the framework. Users are encouraged to submit their files even if the analysis did not help them determine the authenticity of the file, to get help from other users. In case of a response, the uploader is informed about the other users' suggestions. This facilitates collaborative decision making, and is also a more efficient dataset creation strategy because files are not submitted only when the uploader can decide with confidence.