dataesr/scientific-paragraphs-categorization huggingface dataset:
Missing Data Insertion To evaluate our imputation method, we varied the missing data percentage in the dataset as 5%, 10%, 25% and 50%. For instance, if the dataset has 100 records, 10% of missing data means that each attribute has ten missing values. As the original dataset already has missing data, we first verify how much data are missing for each attribute. Then, we remove the necessary amount to achieve the desired missing data percentage. After defining the number of values to be removed, we randomly chose them using a uniform distribution. For all missing data percentages, we performed this process 50 times. We then divided the dataset into training and test datasets and standardized the datasets to have zero mean and unit variance. The training dataset has data from 2011 to 2014, and the testing dataset has data from 2015 to 2016.
Missing data can occur as isolated or in bursts. The burst size corresponds to how many consecutive missing values happen for one attribute. Hence, an isolated loss is a burst of size one. Figure  3a  shows the cumulative probability for each burst size of the original test dataset. The majority of missing data burst sizes are small. However, there are bursts with more than 100 consecutive missing values. Figure  3b  shows the cumulative probability of the burst size of one sample of the test dataset after removing values to reach 50% of missing data. The burst sizes of the attributes are very close to each other. The probability of having an isolated loss is approximately 76%, greater than the original test dataset probability( 0. 56). This behavior was expected since we selected data to remove using a uniform random distribution.
Results Our experiments used MonitorAr data from 2011 to 2014 to train the neural network models. We evaluated all imputation methods using data from 2015 and 2016. Each method predicted a value for the missing data as soon as the record arrived at the gateway, respecting timestamp ordering. Therefore, the imputation only uses records received before the current timestamp. All results were expressed with the mean of 50 samples and a confidence interval of 95%, although imperceptive in some figures.
The plotted values refer to the missing data randomly inserted to reach 50% of missing data on the dataset. Furthermore, they are in their original magnitude. Namely, they are not transformed by the standard normal distribution. It is possible to see that, with NN1 and NN2 methods, the points are more concentrated in the ideal line when compared to the other methods. We present a sample of the humidity attribute since its results are worse than the others' attributes. However, for all samples and attributes, the neural networks perform better than the other methods.
Execution Time and Memory Usage Analysis Our mechanism aims to work on IoT gateways, which have scarce computational resources. Therefore, we also need to analyze the implementation's processing and memory requirements beyond the prediction quality. The training phase usually runs in a Cloud server with more resources. However, verifying whether an IoT gateway can train the models is essential to give more autonomy on the edge for applications that need continuous training. Therefore, the execution time and the memory usage are measured in both training and testing phases. We used the time and tracemalloc Python libraries for the execution time and the allocated memory, respectively. We presented a mean of 50 samples for each missing data percentage with a confidence interval of 95%. The experiments employed a Raspberry Pi 4 Model B with a 1.
5 GHz 64-bit quad-core Arm Cortex-A72 CPU and 4 GB of RAM. We chose the Raspberry Pi due to its wide use in the literature of IoT gateways  [37, 38]. Only the neural network models need training. Thus, we measure the execution time and memory usage to train all attributes in the training phase. In the testing phase, we measure each method's execution time and memory usage to impute all missing data from the test dataset. In other words, this analysis imputes missing data in a batch. It is important to note that it is a worst-case analysis since only one record is imputed at a time in an online environment. Hence, the execution time and memory usage to impute a single record are much shorter than the following results. Figure  7  shows the execution time and total memory allocated during the training phase for NN1 and NN2.
However, we imputed missing data in a batch rather than online as a worst-case analysis. Hence, the execution time increase is not prohibitive for the major part of IoT applications. In an IoT scenario, gateways might have different hardware configurations. Therefore, we take the execution time of the simplest methods (i. e. , LastValue) as a reference and compare it to the others to verify how slow they are in the testing phase. This comparison allows a more generalized analysis, showing relative values. Figure  9  shows the relative execution time, that is, a method's execution time over LastValue's execution time. All methods are slower than the LastValue for all percentages, but their worst result occurs when the dataset contains 5% of missing data. Both NN1 and NN2 are approximately 2.
Neural Network Training Analysis In Sections 5. 1 and 5. 2, both the training and testing datasets have the same missing data percentage. However, in reality, it is not possible to know a priori the missing data percentage that an IoT gateway receives. It means that, after training a model with a certain missing data percentage, one cannot guarantee that the received data have the expected missing data percentage. In this section, using the same methodology as that described in Section 4. 2, we trained the neural network models with a fixed missing data percentage, and we tested them in datasets with varying missing data percentages. We aimed to analyze the difference between the missing data percentages in training and testing datasets. Figures  10  and 11  show, respectively, the R 2 scores and RMSE when we train the neural networks considering training datasets with 5% and 50% of missing data for the temperature attribute.
We only show this attribute's results for conciseness since the behavior for the other attributes leads to the same conclusions. The LastValue method does not need any training, but we show its results for comparison purposes. The Average_3v and Average_4v also do not need training. However, we omit them since their results were worse than the LastValue in Section 5. 1. The results of Figure  10  show that the neural network models have close R 2 scores, regardless of whether the training dataset has 5% or 50% of missing data. Differently from NN1, it is possible to note that NN2 trained with 5% missing data performed better than when it was trained with 50% missing data. This happened because NN2 needs one more input value than NN1 (i. e. , the measure of the previous day at the same hour).
Thus, it has a higher sensibility to missing data on the training dataset. In Figure  11 , the RMSE results for the neural networks are smaller than the LastValue, and they remain low even when the dataset has 50% of missing data. In a nutshell, these results show that, despite training the neural networks with a fixed missing data percentage, their performance is still better than the other methods, regardless of the missing data percentage in the test dataset. The occurrence of isolated losses can explain this low sensibility to losses after missing data insertion. Since they are isolated, the training phase is not severely affected. It is an interesting result since a real scenario has small loss bursts, as shown in Section 4. 2. Hence, for a real scenario, we expected a low sensibility to losses in our models.
Case Study-Clustering Application Although widely used in different fields, some machine learning algorithms did not perform well when they were missing data in the dataset, requiring only complete records. Eliminating the incomplete records may reduce the analysis reliability, which justifies imputing missing data. This subsection presents a case study of an application that uses the DBSCAN  [39]  clustering algorithm, analyzing how each imputation method affects the algorithm's outlier detection. DBSCAN requires two parameters: eps, which specifies the maximum distance that two points must have to be considered neighbors; and min_points, which corresponds to the minimum number of neighbors a point may have to be considered a core point. Hence, all neighbors that are a radius eps apart from the core point belong to the same cluster as that core point  [39, 40].
The min_points parameter is usually twice the number of the dataset's attributes  [40]. However, for large datasets, with much noise, higher values for min_points may improve the clustering results. In this experiment, we used two base datasets. For the first set, named DropTest, all records that have at least one missing value were eliminated from the set. Hence, all records are complete and have only data measured by sensors. For the second set, named PosInf, instead of eliminating the incomplete records, we used the imputation methods discussed before. Therefore, we create five new sets: PosIn f -NN1, PosIn f -NN2, PosIn f -Average_3v, PosIn f -Average_4v, and PosIn f -LastValue. They used, respectively, the imputation methods NN1, NN2, Average_3v, Average_4v and LastValue. We ran the DBSCAN algorithm over all six datasets, with different min_points values (i.
e. , 24, 72, 120, 240, and 720). The eps parameter was calculated using the distance from the record's k-nearest neighbors  [39]. As the Droptest dataset does not have records with predicted values, and it has less data than the other sets, we use it as baseline to choose the eps parameter. We considered k = min_points. Therefore, using the DropTest dataset, an eps value was computed for each k = min_point and used in the DBSCAN executions for all datasets with such min_point value. In this analysis, we aimed to verify whether records with predicted values harm the DBSCAN clustering results and outlier detection. We sought to verify whether: Table  3  shows, in percentage, how many outliers the PosInf datasets have in common with the DropTest dataset, for all tested min_points. For min_points over 24, the PosInf-NN2 dataset is the set with more outliers in common, tying with PosInf-NN1 and PosInf-Average_4v at some min_points.
For min_points 24 and 720, the PosInf-Average_4v dataset has less outliers than DropTest, and that is the reason for which it shows a low intersection with it. The same occurs in the PosInf-Average_3v dataset, when min_points = 720. In these cases, some outlier records are no longer considered outliers when the records with imputed data are inserted. Therefore, depending on the imputation method used, records previously identified as outliers might become valid in DBSCAN. It happens since the inclusion of records with imputed data can decrease the distance between a record and a core point. Table  4  shows the DBSCAN results for all datasets when min_points = 72. This shows the number of records in each dataset and the number of outliers detected by DBSCAN. In addition, the "new outliers" column shows how many records are not considered outliers in the DropTest, but are outliers in the PosIn f sets.
Manual analysis indicates that all "New outliers" are records with predicted data. Hence, they do not exist in the DropTest dataset. Therefore, imputing missing values does not make a valid record into an outlier. In addition, some methods insert more outliers than others. The PosInf datasets have 767 records with imputed values. Our subsequent analysis is to verify how many of those are identified as outliers by DBSCAN, as we vary the min_points. Table  5  shows the outliers' percentage in each PosInf dataset. It is possible to notice that the percentage of records identified as outliers is less than 2. 2% for all cases. In addition, the larger the min_points, the smaller the percentage of outliers inserted by the methods. It happens because some records can become core points and thus bring previous outliers closer together.
As defined before, the Average_3v and Average_4v methods compute the average of 3 or 4 values of the dataset, respectively. Therefore, unless these input values are already outliers, the predicted value is probably in the range of valid values. This behavior can be noticed in the PosInf-Average_4v, which is the set with fewer outliers inserted after the imputation of missing data. However, considering the PosInf-Average_3v dataset, we can note that the Average_3v method inserted more outliers than PosInf-Average_4v. This behavior is in line with the R 2 and RMSE results, in which this method performed worse than Average_4v. In this section, the DBSCAN algorithm is an example of an application that requires that all records are complete. The results confirm that the missing data imputation methods provide good results without distorting the DBSCAN results.
Conclusions Collected data from IoT systems might have missing data due to various reasons such as network problems, damaged sensors, or security attacks. To provide a more efficient analysis and thus reliable services, we must address the missing data problem. Traditionally, a Cloud server performs this task. However, some applications require real-time response, which can be challenging to achieve using a cloud server due to the amount of data, the networking traffic, and the delay. Furthermore, we propose a simple mechanism to run in IoT gateways that imputes missing data using regression models based on neural networks. Our goal is to verify whether these simple models perform well in IoT gateways, which usually have scarce resources. We used actual weather data from Rio de Janeiro to validate our models.
This work also presented a case study that used a clustering algorithm to analyze the neural network models as imputation methods. The algorithm's results show that only a small percentage of the records with imputed values are considered outliers. Therefore, the imputation methods can deal with missing data without distorting the application's results. Finally, using simple imputation methods, we confirm that an application can use all data received by sensors without eliminating the incomplete records. In this work, we inserted the missing data in the dataset using uniform distribution to study the impact of the isolated losses. In future work, other patterns of missing data insertion may be used. For example, we could analyze how the neural network models performed when the missing data occur in large-sized bursts.
Although the molecular composition of asphalt is extremely complex, many scholars are still committed to this research to deeply understand the performance of asphalt from a microscopic perspective. In the early research, Jennings et al. selected eight asphalt binders as a standard sample in the Strategic Highway Research Program (SHRP). The Nuclear Magnetic Resonance (NMR) spectroscopic was used to analyze the molecular parameters such as percentage of aromatic carbon, elementary composition. Besides, the average molecular structures were proposed in the research  9. Asphaltenes are polyarene compounds consisting of aromatic cores and aliphatic side chains, and usually containing heteroatoms such as sulfur, nitrogen, and oxygen  10. Artok et al. discussed the structure of asphaltene molecular based on the data of NMR and built several models of asphaltene molecular  11.
With the development of computing science and computer technology, many programs, and software for molecular simulation have been created. The most representative is the Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS) for MD simulation and Towhee for Monte Carlo simulation. Many researchers studied the molecular composition and MD simulation of asphalt. Zhang and Greenfield purposed a three components model of asphalt and then calculated the properties of the model by molecular simulation. Two different asphaltene molecular models were selected from the literature, both of which had medium-sized aromatic cores. The aliphatic side chain of asphaltene1 is longer than asphaltene2. In addition, the saturate and the aromatic compositions of this model are represented by n-docosane (n-C 22 H 46 ) and 1,7-dimethylnaphthalene, respectively  7.
Finally, the simulation data was analyzed to predict the physical properties of the model. NPT ensemble and NVT ensemble were adapted in different simulation procedures of this study, respectively. In addition, another MD simulation method named geometry optimization (also named energy minimization) has been used in this research to optimize the initial geometry structure and minimize the primary energy of the molecular system. The DREIDING Force Field and AMBER Cornell Extension Force Field were implemented to predict system density and verify the asphalt model in this study. And the AMBER Cornell Extension Force Field was applied to calculate further properties of the asphalt molecular model. DREIDING Force Field is a simple but universal force field, that has been proved to be useful for predicting the structure and dynamics of organic, biological, and main-group inorganic molecules.
Acknowledgements The authors appreciate the financial support of the  Beijing Key Laboratory of Traffic Engineering (Beijing University of Technology, China)  under grant number:  2018BJUT-JTJD007  and  Beijing Municipal Commission of Transport of China  under Grant Number:  40038003201805. The computational studies were performed by the Scienomics MAPS software suite (MAPS, Version 4. 3, Scienomics, Paris, France, 2020) and the MD simulation part is calculated by  LAMMPS  ( Large-scale Atomic/Molecular Massively Parallel Simulator ). Any opinion, finding, and conclusion expressed in this paper are those of the authors and do not necessarily represent the view of any organization.
Mass In addition to the large distance of NGC 3109, we also need to explain its large mass for a backsplasher in the ΛCDM framework. Its virial mass can be estimated using rotation curve fits that add various halo profiles to the observed baryonic components. Several such fits were conducted by  Li et al. (2020)  based on the Spitzer Photometry and Accurate Rotation Curves dataset (SPARC;  Lelli et al. 2016). The results are summarized in Figure  1 , which shows only those halo profiles with a reduced χ 2 below 9. Based on these results, we conservatively assume that the mass of NGC 3109 is at least 10 10. 6 M = 4. 0 × 10 10 M , since no acceptable fits were obtained with a lower mass. Several studies give larger values, with  Valenzuela et al. (2007)  estimating a virial mass of 8. 1×10 10 M (see their table 4).
Since Illustris is a hydrodynamical simulation, we may instead compare the baryonic mass of each backsplasher to that of NGC 3109. This was estimated at 2. 1 × 10 9 M by applying rotation curve fitting techniques to high-resolution N -body models (table 4 of  Valenzuela et al. 2007). Their estimated neutral hydrogen mass of (6 -8)×10 8 M is similar to the (4. 6 ± 0. 5) × 10 8 M reported in table 8 of  Carignan et al. (2013). Using stellar population synthesis modelling,  Valenzuela et al. (2007)  estimated that ≈ 5 × 10 8 M is contributed by stars, with the rest coming from gas. The more recent SPARC database gives a 3. 6 µm luminosity for NGC 3109 of (1. 94 ± 0. 02) × 10 8 L , which suggests a stellar mass of only 1. 0 × 10 8 M for a mass to light ratio of 0. 5  (Schombert & McGaugh 2014).
ACKNOWLEDGEMENTS IB is supported by an  Alexander von Humboldt Foundation  postdoctoral research fellowship. MSP and BF thank the  Deutscher Akademischer Austauschdienst for PPP grant   57512596  funded by the  Bundesministerium für Bildung und Forschung , and the  Partenariat Hubert Curien (PHC)  for  PROCOPE  project  44677UE. MSP thanks the  Klaus Tschira Stiftung and German Scholars Organization e. V. for support via a  Klaus Tschira Boost Fund. BF  acknowledges funding from the  Agence Nationale de la Recherche  (projects  ANR-18-CE31-0006  and  ANR-19-CE31-0017 ) and from the  European Research Council (ERC)  under the European Union's  Horizon 2020 Framework Programme  (grant agreement number  834148 ). The authors are grateful to the referee for detailed constructive comments on the article.
Consequently, it remains crucial to speed up the simulation of the electrolyte infiltration process to pave the way towards the computational screening of the impact of electrolyte and electrode properties on the electrolyte infiltration dynamics and therefore envisage autonomous algorithms able to optimize the electrolyte infiltration for low required times. Meanwhile, Artificial Intelligence (AI) has seen a tremendous rise in the last decade. In LIBs, machine learning (ML) techniques have enabled tools that significantly reduce the slow time frames related to trial-anderror approaches or physics-based simulations for faster and more efficient data assessment  [17] [18] [19] [20] [21] [22] [23]. Such techniques have also been applied to datasets produced from LBM calculations in the geology domain through images-based prediction to obtain fluid flow properties in porous media  [15, 24, 25].
Our aim in this study is to report, for the first time to our knowledge, a ML model based on a multi-layers perceptron (MLP) approach that can describe the dynamics of the electrolyte infiltration process in 3D, given a particular mesostructure of a LIB electrode and its associated porenetwork, while accounting for different external infiltration pressure conditions. Extraction of REVs: From the full tomography dataset of the NMC 94 % -CBD 6 % electrode, eleven 100×100×75 μm 3  [3]  sub-volumes of similar porosities were extracted with a maximal relative error of 5%. The carbon binder domain (CBD) location in the REVs was resolved using an in house stochastic algorithm  [21]. Individual pore identification: An accurate reconstruction of the threedimensional pore spaces and the subsequent identification of individual pores was done by the PoroDict library within the GeoDict® software using the watershed algorithm.
The Python libraries, Tensorflow and Keras, are used in the backend to complete the architecture within a 3. 7 version of Python. After training the modeling function, a sensitivity analysis is applied to analyze the effect of the input parameters on the uncertainty of the outputs. The architecture of the MLP comprises five hidden layers, with 80 nodes each, and one output layer containing 7 nodes. While no specific rule exists for selecting DNN hyperparameters  [33] , this architecture was sufficient to fit the training data correctly and to obtain trustable predictions as shown in the results section. Nodes in green represent the bias that is applied to each hidden layer. Scheme 1 illustrates the workflow that was used in this work. The details of the data extraction procedure of pore-resolved saturation curves and the ML model are given in the supporting information.
Particularly, this compromise is met by selecting S\S 3-7 = {S 1 , S 2 , S 8 , S 9 , S 10 } as the outputs of the MLP, along with T f0 and T f1 (as shown in Scheme 1E-F). Fig. 1C  and D  display regression plots comparing the predicted values from the MLP model and the actual values from the LBM simulations for the testing dataset, for the time at which the pore-filling starts (T f0 ) and stops (T f1 ), respectively. Comparison of LBM simulation and prediction based on ML: As mentioned above, our trained MLP can accurately predict the saturation, initial electrolyte entering and fully wetting time at an individual pore in the structure for the test dataset. In order to further compare and contrast our model, an additional REV was used, whose pore-network was not part of neither the training nor testing datasets.