dataesr/scientific-paragraphs-categorization huggingface dataset:
We have developed data quality tool in R language. Our application name is Package-Data-Quality-Assessment (PackDQA)". It developed following five points: Quality dimension approaches identification, design of quality measures, global coefficient design, development of the quality model, test and deployment model. This model test performed on health data in Burkina Faso show 97. 69% observations is quality. The current version does not include qualitative data. We will have to improve theme to use all types of data.
Introduction The advent of digital technologies has favored the development of data collection tools. In hospitals and research centers in Africa can produce big database in short time. However, data management steps not evaluate data quality. The quality of the data is sidelined when the database is set up. Ensuring the quality of these data is therefore a crucial process for the quality of health information systems. Data analysis is often done on poor quality data and it goes without saying that decisions will be distorted. Specifically, it involves data errors, i. e. , duplicates, inconsistencies, missing, incomplete, uncertain, outdated, weak or outliers, etc. These elements are the sources of non-quality that make decision making difficult and generate financial costs. Data collection or production has been possible for several decades.
Methods Our approach uses R software to develop an application (PackDQA). This application is based on standards dimensions quality. Its design and implementation use tools VIMO (Valid Invalid Missing and Outlier) methodology. Our model evaluates data quality in two parts: Tabulate method and graphic method. In the development part we will be evaluate dimensions quality as follows: x If the combination of invalid, missing and outliers is less than 5%, the data quality problems are considered to be zero or minimal; x The interpretation remains the same when we have a validity rate above 95%. x If the combination is between 5% and 30%, then the data quality problems are considered moderate, as well as if the validity rate is between 70% and 95%. x Data quality problems are considered significant if the combination is less than 30% (>30%), i.
Scores and mains dimensions of data quality We are four main dimensions of data quality: Valid, Invalid, Missing and Outlier values. In tabulate method, we have the dimensions of quality in columns and in rows, we have variables or data parameters (Tab 1). The graphic model shows graphical analysis through the wed interface (Fig  1 ). The implementation of health data, level quality of observations is 97. 69%. The database is cleaning to 97. 69%, so doubtful observations represent 2%.
Research on Android malware detection based on Machine learning has been prolific in recent years. In this paper, we show, through a large-scale evaluation of four state-of-the-art approaches that their achieved performance fluctuates when applied to different datasets. Combining existing approaches appears as an appealing method to stabilise performance. We therefore proceed to empirically investigate the effect of such combinations on the overall detection performance. In our study, we evaluated 22 methods to combine feature sets or predictions from the state-of-the-art approaches. Our results showed that no method has significantly enhanced the detection performance reported by the state-of-the-art malware detectors. Nevertheless, the performance achieved is on par with the best individual classifiers for all settings.
Overall, we conduct extensive experiments on the opportunity to combine state-of-the-art detectors. Our main conclusion is that combining state-of-theart malware detectors leads to a stabilisation of the detection performance, and a research agenda on how they should be combined effectively is required to boost malware detection. All artefacts of our large-scale study (i. e. , the dataset of ∼0. 5 million apks and all extracted features) are made available for replicability.
Unfortunately, this focus on proposing new detectors without first thoroughly assessing and building on existing work impedes the progress of the research domain. This paper Building on large-scale datasets and re-executing existing approaches, we empirically show that state-of-the-art Android malware detectors yield performance results that significantly depend on the evaluation dataset. Indeed, none of the studied approaches has been reported to reach the highest prediction performance on all the evaluation settings. This finding suggests that trusting a single approach in a real world setting is unrealistic. To overcome this limitation, we investigate whether the combination of state-of-theart Android malware detectors can yield better detection performance. We build on a recent study on the Reproducibility/Replicability of ML-based Android malware detectors  (Daoudi et al.
2021b ) which has considered research works from 16 major venues in Machine Learning, Security, and Software Engineering. In this study,  Daoudi et al. (2021b)  were able to successfully reproduce/replicate four state-of-the-art malware detectors. These detectors will be used as the basis for our work. Our work assesses the impact of combining the best approaches from the literature, each of which contributing with a specific way of modelling Android apps. Specifically, DREBIN extracts eight types of string features from the Manifest file and the DEX bytecode, including permissions, intents, hardware components, and suspicious API calls. On the other hand, MAMADROID models the behaviour of the apps using Markov Chains representation of the abstracted API calls. As for REVEALDROID  (Garcia et al.
Our work evaluates the four state-of-the-art approaches on two large datasets of over 197k and 265k apps and studies the impact of either combining their feature sets or combining the detectors themselves using Ensemble Learning. Overall, we make the following contributions: -We conduct a comparative evaluation of four state-of-the-art Android malware detectors (+ variants) using the same experimental setup to identify the best performing approach. The studied detectors are: DREBIN, MAMADROID FAMILY (two variants of the approach), REVEALDROID, MALSCAN (six variants of the approach). -We examine the similarities/difference in the malware detected by state-of-the-art approaches. -We investigate the impact of merging feature sets from state-of-the-art Android malware approaches on the detection performance.
-We investigate the impact of combining predictions from state-of-the-art malware detectors using 16 combination methods. Our work has resulted in the following findings: The performance of state-of-the-art Android malware detectors is highly dependent on the experimental dataset. None of the studied approaches has reported the best detection performance on all the evaluation settings. Some families of malware are detected very accurately by some state-of-the-art approaches, but almost completely escape detection of some other approaches. Combining features and predictions from state-of-the-art malware detectors (i. e. , using Bagging and Ensemble Selection) is promising to leverage the capabilities of the best detectors and maintain a stable detection rate on all the evaluation settings.
Research questions In previous works,  Allix et al. (2016a)  then  Pendlebury et al. (2019)  have presented study results which suggest that literature evaluation of Android malware detection approaches generally suffers from spatial and temporal biases. Most of the times, each approach is assessed only on a specific dataset, with limited comparison to existing work. Thus, there is a missed opportunity to definitively understand the contribution of each approach and eventually build up on existing works for improved detection. Given that each new approach claims to outperform others, a first step towards addressing the biased comparison issues would be to undertake an independent and fair assessment of state-of-the-art approaches in order to compare their results under different settings: -RQ1: Is there a state-of-the-art malware detector that outperforms all others across all datasets? To further investigate the similarities and differences between state-of-the-art malware detectors, a possible direction would be to examine the similarities and differences in the malware detected by these approaches.
Dataset Our study considers two main datasets, which are summarised in Table  1 : Literature dataset This dataset is constructed by collecting app samples used in the literature 1  to validate state-of-the-art malware detection approaches (cf. Section 2. 4). Apps in this dataset span from 2010 to 2018. Overall, the literature dataset includes 43819 malware and 153616 benign apps. In our experiments, we evaluate our classifiers not only on the whole literature dataset, but also on each of its subsets separately (i. e. , the dataset used for DREBIN, for MAMADROID, for REVEALDROID, and for MALSCAN).
AndroZoo dataset This dataset is collected from the AndroZoo  (Allix et al. 2016b)  repository whose maintainers continuously crawl Android apps from different sources (including Google Play, AppChina, etc. ). We consider that an app is labelled as benign if it has not been detected by any Antivirus engine from VirusTotal 2. Following up on previous work  (Arp et al. 2014) , we consider an app to be a malware if it has been detected by at least two Antivirus from VirusTotal. For this dataset we focused on recent apps created 3 in 2019 and 2020. Overall our AndroZoo dataset includes 78002 malware samples and 187797 ** The total is calculated after removing redundant apps benign samples. Similarly, we also conduct our experiments on the whole AndroZoo dataset as well as its subsets (i. e. , 2019 and 2020 subsets).
Experimental setup All experiments (to evaluate the literature approaches, the merged feature sets, and the combinations of predictions) are performed on each of the collected datasets. We consider two evaluation scenarios per experiment and per dataset, following up on previous work  (Pendlebury et al. 2019; Allix et al. 2015)  which highlighted biases in empirical evaluation of machine learning-based malware detection: -Temporally-consistent: The classifiers are trained on old apps, and tested on new apps (i. e. , the dataset is split based on the apps' creation dates). -Temporally-inconsistent: the classification experiment does not take into account the creation time of the apps (i. e. , the dataset is shuffled before the split into training, validation, and test sets) In our experiments, each dataset is split into training (80%), validation (10%), and test (10%).
For the Temporally-inconsistent settings, we repeat each experiment ten times after randomly shuffling and splitting the datasets. As for the Temporally-consistent settings, we also repeat the experiments ten times by randomly selecting 90% of the apps from the training, validation, and test splits (i. e. , the training and the test contain the oldest and the newest apps, respectively). For both settings we report an average detection performance. We rely on Recall, Precision, F1-score, and the Accuracy to measure the classification performance. In our evaluation, we use these metrics to refer to the average detection scores since our experiments are repeated ten times. We present the formulas of these metrics below:.
Study subjects: literature detectors Our work builds on four state-of-the-art Android malware detectors presented at major venues. These approaches have been identified in a recent study  (Daoudi et al. 2021b ) that assessed the reproducibility/replicability of Machine Learning-based Android malware detection approaches in the literature. We have considered these malware detectors for two main reasons: -Indeed, a tremendous number of malware detection papers are published in the literature, but our study focuses on the best approaches with the most significant contributions in the field. Thus, our study subjects are selected among papers published in 16 top venues in Software Engineering, Security, and Machine Learning: EMSE, TIFS, TOSEM, TSE, FSE, ASE, ICSE, NDSS, S&P, Usenix Security, CCS, AsiaCCS, SIGKDD, NIPS, ICML, and IJCAI.
Study results To answer our research questions, we have conducted our experiments using Literature dataset, AndroZoo dataset, and their subsets. 3. 1 RQ1: Is there a state-of-the-art malware detector that outperforms all others across all datasets? Android malware detectors in the literature are usually evaluated using different experimental setups and datasets. In this section, we aim to assess the performance of the state-of-the-art malware detection approaches under consistent experimental conditions. Specifically, we evaluate the effectiveness of the four state-of-the-art malware detectors (and their variants) using the datasets described in Section 2. 2 and under the experimental Since we consider five Literature datasets and three AndroZoo datasets (i. e. , whole datasets and their subsets), the total number of our experimental settings reaches 16.
In the remainder of this paper, we use "dataset" and "setting" interchangeably. We report the average F1 score for the considered malware detection approaches in the upper part of Table  4. We also present the Recall, Precision, and Accuracy scores of our experiments in Tables  7, 8 , 9, and 10 in the Appendix. We observe that the performance of the classifiers varies considerably across datasets. On the whole LITTEMPCONSIST, all the approaches have reported detection scores that are significantly low, with a best F1 score of 0. 44. This result is consistent with the finding of Tesseract  (Pendlebury et al. 2019 ) on a temporally-consistent setting. For the other datasets (i. e. , AndroZoo datasets and Literature subsets), the detection performance on the temporally-consistent experiment is also generally lower than the performance reported in the temporally-inconsistent experiment.
The detection performance on the whole LITTEMPCONSIST is much lower due to the composition of this dataset. We remind that the whole AndroZoo dataset contains apps that span over two years (i. e. , apps from 2019 and 2020). As for the whole Literature dataset, it contains Android apps that are spanning over eight years (i. e. , apps from 2010 to 2018), which makes this dataset considerably difficult for all the classifiers. In the experiments involving Literature datasets, DREBIN yielded the highest F1 score in nine out of ten experiments. DREBIN's feature set seems to be more suitable to detect the apps created before and until 2018, which is demonstrated by the temporally-inconsistent and the temporally-consistent experiments, respectively. Indeed, DREBIN has not outperformed the other detectors only on the whole Literature dataset but also on its subsets created in the sub-years of 2010-2018.
As for the AndroZoo datasets, no approach has reported the highest detection performance in all the experiments. Consequently, no specific feature set from the evaluated state-of-the-art approaches consistently helps to detect the highest number of malware created between 2019 and 2020. We also observe that no approach has reported the highest detection performance on all the datasets. Specifically, DREBIN has achieved the best F1 score in nine out of 16 experiments. MAMADROID PACKAGE is considered the best approach in three experiments. As for MALSCAN CLOSENESS, MALSCAN DEGREE, MALSCAN HARMONIC, and MAMADROID FAMILY, each of them has reported the highest F1 score on one dataset. In the seven experiments where DREBIN has not reported the highest detection performance, the difference in the F1 score between DREBIN and the best approach on each dataset varies from 1 to 8 percentage points.
dataset The entries in bold show the best detector for each dataset and for each RQ based on the F1 score before rounding * We note that we have verified the standard deviation of the F1 scores over the ten runs of the experiments, and our results showed that the F1 scores do not vary sensibly We further conduct a statistical test to compare the F1 scores reported by the state-ofthe-art classifiers in all the datasets. We rely on the non-parametric Friedman test  (Friedman 1937 ) that is designed to compare multiple data groups. Our selection of this test is motivated by the fact that our dataset of F1 scores does not follow a normal distribution. Additionally, previous studies  (Perinetti 2016; Parab and Bhalerao 2010; Sheldon et al. 1996)  have recommended using the Friedman test to statistically compare more than two datasets.
Furthermore,  Demšar (2006)  have examined several statistical tests to compare ML classifiers and advised to use the Friedman test when comparing multiple classifiers on multiple datasets. The null hypothesis states that state-of-the-art malware detectors have statistically equivalent detection performance. The Friedman test has reported a p value of 3. 75 -81 , which means that the null hypothesis can be rejected. This result shows that our classifiers do not have the same detection performance. To conduct a pairwise comparison on our classifiers, we proceed with the Nemenyi  (Nemenyi 1963)  Post-Hoc test. This test aims to identify which classifiers have different detection performances after the null hypothesis of the Friedman test is rejected. We represent the p-values for the different pairs of classifiers in the sub-figure (a) of Fig.
1. As shown in Fig. 1 (a), many state-of-the-art malware detector pairs do not have the same detection performance. For example, DREBIN's performance is not similar to that of five classifiers, including MAMADROID PACKAGE, which has outperformed it on six datasets with a maximum difference of seven percentage points. This result confirms our observations that state-of-the-art approaches do not perform equally in all the settings. Overall, our results show that the performance of state-of-the-art Android malware detectors is highly affected by the dataset used for the evaluation. Also, none of the studied approaches has reported the best detection performance in all the settings. Such finding motivates to further analyse the similarities and differences of state-of-the-art approaches by examining the malware they detect.
RQ2: To what extent do state-of-the-art approaches detect similar/different malware? In this section, we propose to examine the type of malware detected by state-of-theart approaches in order to inspect their similarities/differences and assess whether some classifiers perform better at detecting specific malware families. To this end, we first collect the detection reports for malware samples from VirusTotal 5. Then, we leverage AVCLASS  (Sebastián et al. 2016)  to process the detection reports and assign a unique family label to each malware app. We infer the family label for malware apps in both the whole Literature dataset and the whole AndroZoo dataset. Overall, 642 and 204 unique malware families are present in Literature dataset and AndroZoo dataset respectively. We start our investigation by identifying the family labels present in the test sets.
Specifically, since we repeat the experiments ten times, we gather the family labels from the ten test subsets. Then, we merge these family labels to identify the top families in the test sets on average. For each top family, we investigate how many samples belonging to that family are correctly detected by our approaches in the ten test splits on average. We conduct our s on the whole Literature dataset and the whole AndroZoo dataset in both Temporally consistent and Temporally inconsistent settings. We select four top families from each setting and we present them in Table  5. We also report the results for the top 20 families in each setting in Tables  11  and 12  in Appendix.
RQ3: Does merging the feature sets from state-of-the-art approaches lead to a high-performing malware detector in all the settings? In this section, we investigate whether merging features from Android malware detectors has an added value on the detection performance. Specifically, we aim to assess whether such a method can lead to high detection scores independently of the dataset. In Section 3. 1, we have considered an approach as a whole (i. e. , feature set + algorithm). In this experiment, we consider only the set of features proposed by each detector, and we merge them to create a single set of features. This latter is then used to train an ML algorithm to construct a new malware detector. Since the studied state-of-the-art approaches rely on different ML algorithms, we train the merged feature set using the same algorithms.
Thus, we construct three malware detectors using LinearSVC, Random Forest, and K-Nearest Neighbour, and we train them with the merged feature set. Moreover, we also assess the detection performance of three additional ML algorithms: -AdaBoost (Freund and Schapire 1997), which fits a series of base classifiers (e. g. , Decision Tree) on the dataset such that each classifier focuses more on the incorrect predictions made by the previous classifier. This method assigns higher weights to the incorrectly predicted samples in order to enhance their prediction by the subsequent classifiers. -Bagging  (Breiman 1996) , which trains a series of base classifiers on random subsets of the dataset and aggregates their predictions. -GradientBoosting  (Friedman 2001) , which fits a series of base classifiers on the dataset in order to improve the prediction performance.
Each classifier is trained to minimise the prediction errors of the previous classifier using the Gradient descent algorithm. We again conduct our experiments under various experimental setups similarly to RQ1 (cf. Section 3. 1). We report the F1 scores of our experiments in the middle part of Table  4. We also present the Recall, Precision, and Accuracy values of our evaluation in Tables  7, 8 , 9, and 10 in the Appendix. We observe that the six malware detectors that are trained with the merged feature set also report detection performance that vary across datasets. Specifically, the whole LITTEM-PCONSIST is still considered as the most difficult dataset since the highest F1 score reported by these classifiers is 0. 49. On the 2020 ANDTEMPINCONSIST, the highest F1 score has reached a value of 0.
99. Compared to the other algorithms trained on the merged feature set, Bagging reports the highest F1 score in 11 out of 16 experiments. For GradientBoosting and AdaBoost, they achieve the best detection score in four and one experiment respectively. In the five experiments where Bagging has not reported the highest detection scores, the difference in the F1 score between this detector and the best approaches reaches a maximum value of six percentage points. We also compare the detection performance of these classifiers using the Friedman test. The p value of the test is 8. 85 -111 , which indicates that the classifiers trained on the merged feature set do not have the same detection performance. We conduct the Nemenyi test and report its results in the sub-figure (b) of Fig. 1. We observe that Bagging has a different detection performance than that of the other classifiers including those that have outperformed it on five datasets.
RQ4: Does combining predictions from state-of-the-art approaches lead to a high-performing malware detector in all the settings? Following up on the findings of RQ3, we hypothesise that the performance of state-ofthe-art approaches is brought by the right association between feature sets and learning algorithms. Therefore, we investigate the possibility to exploit the combined value of detectors via combining their independent predictions. To that end, we consider Ensemble Learning and study its impact on the detection performance. In our experiment, we consider the detectors trained in RQ1 (cf. Section 3. 1) as base learners for Ensemble Learning, and we examine whether the combination of their predictions produces a high-performing malware detector on all the datasets. Among the many ways of combining model predictions, which are commonly referred to as Ensemble Learning in the literature  (Sagi and Rokach 2018; Dong et al.