dataesr/scientific-paragraphs-categorization huggingface dataset:
19, 37, 38  One of the difficulties in the development of polymer database resources is their stochastic and hierarchical structures with multiple length scales. This is a consequence of the statistical nature of the polymerization reactions that yield structures with distributions in molecular mass, composition, and topology. To overcome these barriers, a very recent initiative is impulsing the development of a Community Resource for Innovation in Polymer Technology (CRIPT), in particular with the objective of defining a scalable polymer material data structure. 39  Driven by FAIR (findable, accessible, interoperable, and reusable)  40, 41  and open-source principles, CRIPT has the potential to accelerate the democratization and the adoption of polymer informatics. An alternative strategy, which permits to avoid the heavy workload of dealing with huge amounts of experimental polymer data, is possible, as evidenced by the release of RadonPy, an open-source Python library that automates polymer properties calculations using DFT and all-atom classical MD simulations.
42  In this work, Hayashi et al. successfully validated simulation protocols by systematically comparing the 15 calculated properties, which include thermal conductivity, bulk modulus and refractive index, amongst others, with experimental results for more than 1000 amorphous polymers from PoLyInfo. Furthermore, the recent software developments within the atomistic modeling community permit to harness the power of graphical processing units (GPU) hardware, which provides a tremendous acceleration of both classical and quantum-mechanical simulations. Therefore, RadonPy positions high-throughput (HT) simulations as an efficient and reliable data source. Polymer simulations are now capable of quantitatively predicting experiments, enabling the deployment of additional polymer informatics approaches and their rapid expansion to new regions of the polymer design space.
50  Indeed, as they illustrated for the polyhydroxyalkanoate polymers family, models designed to target a wide and diverse chemical space usually show higher predictive errors and model uncertainties than models with a narrower applicability domain. Throughout the work reported in this paper, MD simulations have been harnessed to acquire accurate and consistent data with the objective of generating tailor-made ML models targeting a specific region of the chemical space. We will show that this mixed approach, in comparison to generic models trained on experimental databases, permits to reach promising predictive accuracy with 10 to 20 times less data. More specifically, the goal of this work has been to train an ML model capable of predicting the T g of biopolymers. High-throughput MD simulations of polymers have permitted us to obtain the T g of 546 polymers, which constitutes the target property of our supervised ML model.
Dataset collection and combinatorial generation of copolymers To assemble a dataset of monomers with a focus on biopolymers we have reviewed the literature on biopolymers and the freely accessible polymer databases. 4, 5, 9, 32, 93  As discussed in the introduction, this manual data collection is a cumbersome process. Besides, as the list of gathered known biopolymers, which includes polyhydroxylkanoates and polysaccharides, could be considered limited, we have expanded our dataset with a few common monomers such as polyvinyls, polyacrylates, and polyesters to reach 58 monomers. This number looks very small in comparison to the more than 12,000 monomers of PoLyInfo, but as will be discussed in section 2. 2. 3, they cover a smaller part of the polymer chemical space than PoLyInfo. To permit the exploration and to densify the sampling of this region of interest for our application, namely the ML prediction of the T g of biopolymers, a set of 14,877 combinatorial copolymers were generated from the binary combinations of the 58 collected monomers with monomer ratios in the range [0.
Encoding polymer structures for machine learning For training an ML model to predict a polymer property, it is first necessary to encode the polymer structures into a set of features from which the algorithm will learn to make predictions. In our dataset, homopolymers are represented by the polymer SMILES (Simplified Molecular Input Line Entry System) code of their repeat unit,  23  in which the symbol "*" indicates the polymerization point. For binary copolymers, the SMILES codes of the two repeat units involved and their respective weight ratios are used. One might wonder if relying merely on repeat units SMILES would not limit the encoding of the complex polymer hierarchical structures, since alternative strategies have been tested or are currently under development. Nevertheless, according to the state-of-the-art review of the alternative strategies reported in the Supporting Information and the very complete work from Tao et al.
Selection of polymers to simulate As previously discussed, the existence of a trade-off between generality and predictive accuracy for ML models developed to predict the T g of polymers has been observed in several occasions. 50  Therefore, to select the polymers worth simulating as efficiently as possible, it seemed of interest to first position the homopolymers and hypothetical copolymers in the broader context of polymer chemical diversity. Despite the limitations to access the broad information of polymer databases presented in the Introduction, the work of Ma et al. 28  released a useful benchmark database for polymer informatics. Training a generative model on 12,000 homopolymers from PoLyInfo, they built a database of about 1 million hypothetical polymers named PI1M. Moreover, they showed that PI1M covers a similar chemical space as PoLyInfo, while populating more densely regions where PoLyInfo data are sparse.
Therefore, we have used PI1M as a proxy for the chemical diversity of synthetic polymers and implemented a selection strategy, detailed in the Supporting Information, which relies on dimensionality reduction and clustering approaches to sample easily and broadly the chemical space of interest for our application. Overall, we simulated all 58 homopolymers built from the monomers collected and we identified 1,000 copolymer combinations via the approach detailed in the Supporting Information as upper dataset limit. Then, from this pool of 1,000 copolymer combinations, we randomly selected structures to simulate and to fill our dataset until the subsequent ML models reached a satisfying performance level. All polymers simulated are displayed in Figure  3  after a UMAP (Uniform Manifold Approximation and Projection)  95  dimension reduction; in green for the 58 homopolymers and in black for the 488 copolymers, while Figure  S4  also shows all combinatorial copolymers.
The only precaution that we took was to ensure that we did not over-sample combinations representative of little populated clusters positioned in the sparser and outer regions. Indeed, despite the non-linearity of the UMAP transformation, they would likely carry a higher probability of sampling sub-areas ultimately assessed as outside of the applicability domain of the ML model (to be defined at a later stage). Figure  3 : Representation of the homopolymers (green) and copolymers (black) constituting our dataset within the UMAP transformation trained on PI1M. Figure  S4  offers an alternative orientation also displaying all generated copolymers.
In Figure  4 , we show the T g of 47 homopolymers obtained via the MD simulations against their experimental reference values. All these data, with the sources of the experimental references, which are either collected from the literature or from databases such as PoLyInfo  26  or the Polymer Database, 32 are provided in the Supporting Information. For the remaining 11 homopolymers we could not find any reliable experimental T g reference, therefore they have been discarded from this comparison. In the case of multiple references being available in databases, we followed the prescription of Jha et al. 97  and took the median value of the entry. The MD-calculated T g values are typically higher compared to their experimental counterparts, which is expected, as discussed in the introduction, due to the difference between the cooling rates used in the simulations compared to the ones employed in experiments.
Although some of our points Figure  4  seem to visually indicate a similar deviation, a linear fit of our data does not capture it, as will be discussed in section 3. 4, and such effect seems very dependent of the dataset studied. Besides, the standard deviations on T g values from our MD simulations seems smaller (see Figure  S5  for histogram). Furthermore, it is worth mentioning that the homopolymers with the largest deviations from the experimental trend, namely pectine (357 K in experiment vs 612 K in MD), hyaluronic acid (409 K vs 618 K), and cyamopsis-tetragonobola (393 K vs 571 K), are polysaccharide polymers for which the experimental assessment of T g is usually challenging and for which only limited references could be found. Overall, the good agreement between simulations and experiments validates our approach to use MD simulations to sample efficiently and accurately this region of the polymer chemical space for the training of ML models.
Training machine learning models The learning curves represented in Figure  7  for the best KNN model and in Figure  S9  for the best models obtained with each learning algorithm permit to evaluate the models average performances on training datasets and on hold out validation datasets, separated via cross validation (CV) procedure, for different sizes of training datasets. That task was performed using a 10-fold ShuffleSplit CV. On the one hand, the three models reach similar levels of performances when the entire training dataset is used, with MAE values around 20 K and the MAE curves obtained on the validation datasets keep decreasing (they do not plateau) at the maximum training set size used, indicating that the performances of the models could be improved by further addition of samples to the training dataset.
This is a positive sign for model health and it further illustrates the great potential of the entire strategy implemented in this work. On the other hand, the RF and GB models exhibit a significantly higher variance than the KNN model, i. e. , there is a gap between their learning curves on the training and on the validation datasets. The superior performances of RF and GB on the training datasets compared to the validation datasets indicates a slight overfitting, as also observed by Tao et al. 27  who reported similar MAE differences of the order of 10 K between training and testing dataset. Nevertheless, as the performances on the validation datasets are similar to the KNN model, we continued to characterize the performance of all three models and evaluated their generalization abilities both in 10-fold ShuffleSplit CV and in LOOCV, as reported in Table  S2.
The final performances of the best models trained with each learning algorithm are assessed by applying them to the testing set isolated before the start of the training procedure. Table  1  summarizes the results and shows that the KNN model is performing best with a MAE (the optimized scoring metric) of 19. 34 K on the testing set, which corresponds to a very good coefficient of determination R 2 = 0. 89. The good performances of the final model, illustrated in the bottom panel of Figure  7  that shows the T g predicted by the KNN model as function of the reference MD-calculated T g and permits to visually compare the performance of the model between training and testing datasets, are consistent with the evaluations of the cross-validations reported in Table  S2. Furthermore, it is interesting to note that the good fit of the KNN model, which predicts the T g of polymers based on the values of the neighboring structures in the chemical space, is consistent with the generally relatively smooth transitions over domains of different T g ranges visible in Figure  5.
Indeed, the ML models trained from these data show good performances and can thus drastically accelerate the exploration of the design space identified as relevant to the specific targeted application. In that process, we have compiled a dataset constituted of 58 homopolymers, with a focus on biopolymers, and implemented a polymer building and simulation procedure permitting to perform high-throughput simulations relying either on GLYCAM06 for polysaccharides or on GAFF for all remaining polymers. The comparison of the glass transition temperature values obtained with the simulation protocols implemented in this work and the data collected from a variety of experimental references reported in databases and the literature has evidenced that the experimental trends are well reproduced. Indeed, accounting for a well known constant offset, with a value of 105.
02 K in our case, attributed to the mismatch between experimental and computational cooling rates which are orders of magnitude different, such comparison yields an R 2 score of 0. 83. Then, a set of 14,877 copolymers was generated via binary combination of the collected 58 monomers with ratios in the range [0. 1; 0. 9] and a machine learning driven selection approach was implemented to sample the targeted design space by calculating the glass transition temperature of 546 polymers (58 homopolymers and 488 copolymers) via molecular dynamics simulations. From this dataset, a k-nearest neighbors model has been trained to predict the MD-calculated glass transition of polymers, within the design space defined by its applicability domain, with a mean absolute error of 19. 34 K and an R 2 score of 0.
89 when evaluated on the testing dataset isolated before the start of the training procedure. Finally, a comparison with 14 polymers from an external dataset within the applicability domain of our ML model has confirmed that both the MD simulation protocols and the ML model are performing as good as expected. This comparison also shows the differences between two series of MD simulations with different polymer building and T g simulation protocols, and thus highlights the difficulties of merging data from different sources. This challenge exists both when working with experimental data and with computational data and thus remains an important bottleneck for the implementation of large scale data-driven approaches in science. Therefore, it further emphasizes the great potential of mixed strategies combining data-driven and molecular dynamics approaches for targeted applications as implemented in this work.
Importantly, such approaches are not limited to polymers. They can be applied to many classes of molecules or materials. Besides, the work undertaken here could be further refined and improved by future efforts in generating additional data or in improving data quality. On the one hand, the learning curves of the ML model have shown that adding more data would permit to reduce its mean absolute error. On the other hand, even though it would require an important human effort, specifically looking at the outliers found in the predictions of both training and testing datasets could permit to increase data quality. Such outliers may be indicating some inconsistencies in a specific set of MD simulations, e. g. , difficulties for the force field to handle some specific polymer chemistry, or highlighting the presence of pairs of molecules with similar structures but large difference in T g , the so-called activity cliffs, 109 requiring a specific treatment.
After the usual separation of the data collected from MD simulations into a training dataset with 491 samples and a testing dataset with 55 samples via a 90/10 train/test split and the preprocessing of the training dataset, a series of regression algorithms (see Supporting Information for all technical details), including logistic regression, k-nearest neighbors, support vector machines, and ensemble learning algorithms, were compared for the prediction of the MD-calculated T g of polymers. The mean absolute error (MAE) is used as scoring function throughout this work. The results obtained across the range of number of tested features (between 5 and 40) are displayed in Figure S7, and highlight the good performances of out-of-the-box random forest (RF), gradient boosting (GB), and k-nearest neighbors (KNN).
103,104  We have adopted a probability density distribution-based approach, namely the clustering-based local outlier factor (CBLOF),105  which builds on the local outlier factor 106 at the cluster level. Therefore, using the implementation of the Python package PyOD,107  we trained a CBLOF model to detect 15% of outliers within the dataset of 491 polymers used to train the ML model, from a Kmeans clustering model set to identify 20 clusters. In this procedure, only the 29 features kept as relevant for the final ML model were used as features for the outliers detection and the other parameters were kept at their default values. Using the web application hosted by Nextmol (see Supporting Information for access details), the reader can seamlessly apply the final KNN model trained in this work to obtain a prediction of the MD-calculated T g for homopolymers and binary copolymers while obtaining an AD evaluation (is the polymer of interest within the AD of the ML model?) by simply providing the relevant SMILES codes and monomers ratios.
Furthermore, to illustrate the acceleration provided by the approach implemented in this work for the exploration of the chemical space covered by the assembled dataset, we have applied the KNN model to predict the MD-calcualted T g of the 9,029 combinatorial copolymers within its applicability domain. With the HPC architecture used in this work (the compute node contains 4 Nvidia A100 GPUs and 64 CPU cores), we are able to simulate 1936. 0 ns per day. Therefore, as 4 replicas are needed, obtaining the T g of these 9,029 combinatorial copolymers would require 3. 34 node -years, whereas the entire series of ML predictions is obtained within a few seconds. These predictions are displayed with the 496 MD simulations results in FigureS12and FigureS13.
Leveraging the SIMD capability of modern CPU architectures is mandatory to take full advantage of their increased performance. To exploit this capability, binary executables must be vectorized, either manually by developers or automatically by a tool. For this reason, the compilation research community has developed several strategies for transforming scalar code into a vectorized implementation. However, most existing automatic vectorization techniques in modern compilers are designed for regular codes, leaving irregular applications with non-contiguous data access patterns at a disadvantage. In this paper, we present a new tool, Autovesk, that automatically generates vectorized code from scalar code, specifically targeting irregular data access patterns. We describe how our method transforms a graph of scalar instructions into a vectorized one, using different heuristics to reduce the number or cost of instructions.
As a counterpart, there is the SLP algorithm, originally introduced in  [18]. It performs instruction-oriented vectorization. It is often commonly used to convert linear code to vector code, complementing traditional loop-based vectorizers. The SLP algorithm analyzes the input code by scanning the intermediate representation (IR) generated by the compiler looking for instruction groups that can be combined into vectors. It replaces these selected instruction groups with the corresponding vector instructions. Rosen et al. [29]  proposed the Bottom-Up SLP algorithm, which is implemented in the well-known GCC and LLVM compilers. The main contribution of the algorithm is its ability to handle sequential code anywhere in the program, not just in loops. It can also vectorize code inside loops if the loop vectorizer fails.
Our proposed tool has a similar objective, but is currently limited to working with code that uses C++ templates, as it has not yet been integrated into a compiler at the IR level. While the evaluation focuses on vectorization opportunities within loops, which are generally more common, it still provides valuable insight into the potential of our techniques. The SLP approach has some limitations. First, it operates locally or even atomically on individual SLP graphs which can be limiting in the case of consecutive graphs sharing data. In such cases, vectorization costs may be overestimated, leading to missed opportunities even in simple code. The SLP vectorizer may miss several vectorization opportunities in loops, as shown in the results of our work (Section 5). Rocha et al. [28]  presented Vectorization Aware Loop Unrolling (VALU), a new heuristic accompanying the compiler in searching for loop unrolling opportunities to enable automatic vectorization of sequential code.
VALU checks whether loop unrolling is cost effective for the SLP vectorizer and identifies the loop unrolling factor that can maximize the use of vector units in the target architecture. In Autovesk, we fully unroll the kernels as we create the scalar instruction graph, allowing for more instruction grouping possibilities, i. e. , better vectorization of the code. Porpodas et al. [26]  presented Look-Ahead SLP (LSLP), an improved vectorization algorithm based on SLP that focuses mainly on commutative operations. It is implemented in the LLVM compiler. The main contributions of LSLP are, first, the ability to extend the SLP graph data structure to form multi-nodes of chains of commutative operations of the same type. Second, it introduces a more powerful operand reordering scheme that makes informed decisions based on instructions deeper in the code.
Our method enables chain reduction for improved vectorization efficiency by transforming the scalar instruction graph and grouping and annotating commutative operation nodes. Porpodas et al. [25]  go further and present Super-Node SLP (SN-SLP), a new algorithm similar to LSLP with multi-nodes, including both commutative operations and their corresponding inverse elements, for example, addition and its inverse subtraction. SN-SLP leverages the algebraic properties of these operators to allow additional transformations, increasing the possibilities of vectorization. It is implemented in LLVM. Several works have proposed heuristics to optimize instruction ordering and code regions for vectorization in SLP-based algorithms. Nevertheless, our new approach is competitive and offers additional flexibility by allowing vectorization of kernels with irregular data access patterns through instruction grouping and reordering transformations and heuristics.
A conflict detection instruction (vpconflict) has been added to the Intel AVX-512 instruction set to facilitate conflict masking. The vectorization performance of conflict-masking depends on the input distribution. If conflicts arise frequently in the input, conflict-masking will result in low SIMD utilization and thus poor vectorization performance. Jiang and Agrawal  [15]  propose a code transformation called in-vector reduction that can efficiently vectorize a class of associative irregular applications. Further research aims to support aggressive SIMD vectorization of important loops, even when their bodies contain complex control flow and the entire computation cannot be fully parallelized, such as speculative execution  [33, 34]. Beyond this, Chen et al. [7]  propose VeGen, an extensible framework that targets non-SIMD vector instructions by introducing a new model of vector concurrency called lane-level concurrency (LLP).