dataesr/scientific-paragraphs-categorization huggingface dataset:
Parallel Speedups FINN supports parallelisation on a layer-by-layer basis. The amount of parallel hardware resources used to implement each layer of a neural network is user definable. Parallelism is controlled with two settings: (1) the number of hardware processing elements (PE) to process each output channel, and (2) the number of input channels processed within one clock cycle (SIMD)  [24]. Using more parallel hardware for a layer shortens the layer's clock cycle latency, at the cost of increased hardware requirements. If the layer is on the critical path, i. e. , is has the highest latency cost, then parallelisation of that layer should shorten overall latency thereby increasing throughput. Our throughput evaluation used a multi-layer perceptron with three fully connected layers with the FASHION-MNIST dataset.
Quantisation Results Discussion The sweet spot in the quantisation design space for the MNIST and FASHION-MNIST datasets is about 3 bit weights and 3 bit activation functions. Beyond 3 bit quantisation and with enough training, there is no significant improvement to accuracy performance. This confirms results in  [25]. Our methodology for evaluating the trade-off between accuracy, throughput and hardware efficiency is similar to  [32]. We extend that work by also measuring the impact of varying training of quantised models, and a more fine grained benchmark suite measuring weight precision independently of activation function precision. In summary, our quantisation experiments show: • LUT and FF resources increase with increased activation function precision, because increasing arithmetic calculation complexity increases the number of required processing units.
3. The Resnet32x4 model distilled into a Resnet8x4 student model with CIFAR100. For this experiment, the number of layers remained fixed. The experiments used 30 epochs for knowledge distillation and the number of NSGAII generations varies for each experiment, ranging from 14 to 27. For our pruning benchmarks we used Platypus  [34]  for multi-objective optimisation, RepDistiller  [35]  for knowledge distillation, and OpenVino's Python API to execute trained exported PyTorch models on the NSC2 device. Figure  9b  shows that some combinations of the knowledge distillation parameters had a negative effect on the accuracy of the baseline model. We observed that this method produced better accuracy than could be obtained by distilling knowledge into the baseline model, once again at the expense of latency.
The MobileNetV2/FlexStudent experiment in Figure  9a  is similar to Figure  9b , but rather than CIFAR100 it used the simpler CI-FAR10 dataset. In this case, every combination of knowledge distillation hyper parameters provided significant improvement over baseline. In Figure  9c , the majority of combinations of knowledge distillation hyper-parameters had a negative impact on the baseline model accuracy, though certain combinations did provide improvements as shown in Figure  9c. In this case, no major trends were observed with respect to the individual hyper-parameters. We observed that this method, once again, produced better accuracy than could be obtained by distilling knowledge into the baseline model with 30 epochs of training.
Efficacy of NEMOKD Evolution Figure  10  shows the latency and accuracy performance of student models after 30 epochs for student models. It shows the baseline model trained with just knowledge distillation (green diamonds). The red and blue points show performance of the models at intermediate and final generations of student models. As with the quantisation experiments, accuracy was measured using a testing dataset to asses how the models generalised to new unseen data. Figure  10b  shows FlexStudent student performance with the larger Resnet32x4 teacher model, for the CIFAR100 dataset. Student models evolved from the same baseline FlexStudent as the experiment in Figure  10a. Figure  10b  illustrates the population at two distinct generations of the evolutionary process, in addition to the baseline architecture from which all the students evolved.
Interestingly, the combination of knowledge distillation hyperparameters we chose for this experiment had a negative impact on the accuracy of the baseline model. However, the evolved students appeared to adapt their architecture to accommodate these parameters, resulting in student models with significant accuracy improvements for the same inference latency. The best accuracy produced by the NEMOKD algorithm was obtained by an architecture with a higher latency. In contrast to Figure  10a , every student model in the final generation evolved to have the same layer structure as the baseline model. Figure  10c  shows Resnet8x4 student performance student performance with the Resnet32x4 teacher, for the CIFAR100 dataset. It differs from Figures  10a, b   Our quantisation experiments (Section 4.
This is a throughput of between 100 and 590 FPS. More work is required to scale accuracy-preserving compression methods to real world computer vision applications e. g. , from 28 × 28 MNIST and FASHION-MNIST images, and 32 × 32 CIFAR10 and CIFAR100 images, to much higher dimensions such as 400 × 150 road lane detection images for autonomous driving  [43]. Scaling compressing experiments to (1) deeper models with tens/hundreds of hidden layers, and (2) datasets with thousands of classes e. g. , ImageNet, would be an intermediate step in that direction.
This opens up the opportunity for automating profile guided layer-by-layer quantisation methodologies in simulation i. e. , without having to run models on hardware, to find the optimal trade-off between throughput and accuracy for each combination of model and dataset. When using evolutionary algorithms with knowledge distillation for larger datasets and models, enabling more parameters to be the subject of mutation throughout the evolutionary process could prove beneficial in automating search for optimal compressed models. Recent teacher-student methods  [35]  outperform knowledge distillation in a wide range of problems. Designing a flexible student model that accommodates both evolution and more complex distillation methods would be considerably more challenging, but given the positive results we report for NEMOKD we believe this would be important future work.
A post-processing algorithm is used to label the identified pores (using the multi-dimensional image processing python package scipy. ndimage) (Weaver, 1985)  and evaluate their volumes (using the label toolkit of the python package SPAM (Software for the Practical Analysis of Materials)). The pore equivalent diameter, determined from these analyses, corresponds to the diameter of a sphere having the same volume. Noise limits the ability to reliably identify pores   24 ), ((  25 ), (  21 ) and (  29 )). In this figure, the influence of strain-rate on the inception stresses of pores in UHPC concrete is not taken into account (see next sections). smaller than a threshold size. For the SiC ceramic and the UHPC, the lower limit of neighbouring voxels required to adequately represent a single pore was fixed at 5 pixels in diameter and smaller pores were eliminated from the analysis.
Methodology Discrete numerical simulations have been developed using the programming language Python. The routine is composed of the following steps. Each pore is identified from the tomographic scan using the Python package SPAM. A cut-off value was considered to distinguish an effective pore from potential noise of the tomographic scan. This size limit was set to 3 voxels in diameter for the porous ice and 5 voxels for the SiC ceramic and UHPC concrete materials. The coordinates of each pore are assimilated to the centre of mass position of ideal ellipsoids fitting the pores  (Ikeda et al. , 2000). The considered finite volume was obtained by combining the paralepidid crops of two scanned specimens for the UHPC. The cubic scanned volume is cropped from a large cylindrical sample for the ice, and from beam of millimetric size for the ceramic.
Credit score is the basis for financial institutions to make credit decisions. With the development of science and technology, big data technology has penetrated into the financial field, and personal credit investigation has entered a new era. Personal credit evaluation based on big data is one of the hot research topics. is paper mainly completes three works. Firstly, according to the application scenario of credit evaluation of personal credit data, the experimental dataset is cleaned, the discrete data is one-HOT coded, and the data are standardized. Due to the high dimension of personal credit data, the pdC-RF algorithm is adopted in this paper to optimize the correlation of data features and reduce the 145-dimensional data to 22-dimensional data. On this basis, WOE coding was carried out on the dataset, which was applied to random forest, support vector machine, and logistic regression models, and the performance was compared.
e importance of credit evaluation score is becoming more and more prominent, and any institution needs to avoid some risks through a high-performance credit scoring model  [3]. Put aside the technical research on credit scoring technology; adopt alternative data sources to improve the performance of statistical and economic models. In this way, the big data source calls the network, and then, the feature selection increases the profit value and uses the dataset to create the score of users applying for credit cards. Big data have been applied and recognized in various fields. On the premise of protecting privacy, people hope that the big data-driven model can be applied more efficiently in financial credit scoring. A projected gradient-based scorecard learning method (FL-LRBC) can make the positive coefficients of the model form  [4] , and it has abundant data, especially excellent performance in statistics, and it has been applied in some financial and credit businesses.
In the financial retail credit scoring, we need to prove the performance of machine learning (ML) model in the face of complex ML algorithm  [5]. Besides reanalyzing the defects of traditional models, we must also put forward new viewpoints on structure selection and sensitivity, thus proposing a model criticism based on reverse generated counterfactual examples. Credit score directly affects the profitability of the industry, and it is also a key research topic. e credit score classification of Spiking Extreme Learning Machine (SELM) has been verified in many European countries  [6]. It is a new spike energy function. Compared with some traditional spike neuron models, such as classification accuracy and area under curve (AUC), the accuracy and execution speed of SELM have been greatly improved, which has statistical significance for credit score datasets, indicating that the combination of biological spike function and ELM can better classify.
Peerto-peer lending has gradually emerged in recent years, and researchers have discussed and analyzed this phenomenon. Research is carried out through open source data and the applicability and effectiveness of credit scorecards is checked. Scorecard is based on P2P loan open dataset, so it contains repayment records and loan functions. In order to improve credit management and the stability of credit score, a method is used. First, the user's data are obtained, and the data are input into the neural network configuration module for processing  [9] , so as to obtain a credit probability value and the score. In actual cases, this method is obviously stable, which greatly improves the user experience. It is very necessary for financial institutions to predict customers' default. At present, a conservative credit scoring model is established by using Kruskal-Wallis nonparametric statistics  [10].
Brief Introduction and Preprocessing of Experimental Dataset e personal credit evaluation model in this paper uses experimental data from Lending Club. Lending Club dataset is adopted in this paper because the dataset is public and can be evaluated. e dataset can reflect the evaluation standard of credit data. rough the Lending Club dataset algorithm, the design can show the advantages and better performance of the proposed algorithm. e credit data provided by the Company in the first quarter of 2017 have a total of 96,781 samples and 145 characteristics, which mainly include the financial data of Lending Club Company, but do not include the credit rating. e dataset feature loan_status contains seven types of debit and credit states, as shown in Table  1. For the convenience of follow-up research, we take loan_status feature as the target vector.
Data Cleaning. Data cleaning is the first step in the process of data preprocessing; the main purpose of which is to fill the missing values and deal with outliers. For missing samples in datasets, the missing value processing method in this paper follows the following principles: (1) If more than half of the data in a sample is null, the set of data is discarded. In the environment of big data, there are deficiencies in datasets due to differences in dataset collection. It is reasonable for a large-scale dataset to have NULL values. If there are multiple missing data, it can be deleted directly. If there is a large amount of data, deleting some will not affect the evaluation of the whole dataset. (2) Usually, the missing value of the sample is filled by the mode of the feature. Mode filling principle is aimed at the value with the highest number of occurrences in the data, which is based on the maximum probability filling method and can improve the efficiency of dataset integration.
erefore, the preference of the whole dataset is better, which accords with the effect of data filling. (3) If the missing feature of a sample exceeds 95%, the feature will be discarded. In order to ensure the integrity of data as much as possible, this paper converts the format of discrete data before filling in missing data, and the missing data are shown in Table  2. Among them, the missing ratio is the ratio of the number of missing data of this feature to the number of rows of the dataset, which is 32469. rough data format conversion and mode filling, the dataset is complete. e processing of outliers is simpler. It only needs to judge whether the data are within the reasonable data range, and then, delete the sample data with outliers. e Lending Club dataset used in this article found no outliers in the dataset.
z-Score Standardization. e idea of z-score standardization is to scale the data based on the arithmetic mean and standard deviation of the original data  [12]. After zscore standardization, the data show standard normal distribution, that is, the mean value is 0, the standard deviation is 1, and the conversion function is x * � x -μ σ. ( 2 ) In view of the continuous characteristics of the dataset, this paper uses the min-max standardization method to deal with it. Compared with the standardization of continuous variables, the standardization of discrete variables is more complicated. In order to measure the distance between samples, it is necessary to introduce dummy variables by means of coding. Common encoding methods are one-hot encoding and dummy encoding.
Dimension Reduction of Experimental Dataset Based on Pdc-RF Algorithm. On the basis of data preprocessing in the previous section, this section is based on Python programming, and Jupyter Notebook data analysis platform has reduced the dimension of 32469 samples and 74 experimental datasets with characteristics. In Random Forest algorithm, test_size � 0. 2 is chosen as the partition of the training set and the test set; the number of trees in the tree is set to n_estimators � 500, the depth of the tree is set to max_depth � 10, and other parameters are set as default. e features are sorted according to their importance, and then, the features whose importance is less than 0. 01 are taken out as nonimportant features and eliminated. Finally, the first 41 features are selected as feature subsets.
According to the formula, monthly arrears � (loan amount ÷ number of loan periods) + (loan amount-repaid amount) × interest rate; it is not difficult to conclude that the monthly amount of arrears is directly related to the loan amount, which is consistent with the correlation coefficient of 0. 95. Based on the dimensionality reduction idea of preventing overfitting, loan_amnt can be eliminated. Finally, 21 features selected from Pdc-RF algorithm features are shown in Tables  5  and 6. In this section, min-max standardization and one-hot coding are used to standardize the continuous variables and discrete variables in the dataset, respectively, and the first step of the data preprocessing stage is completed at this time. en, the improved pdc-RF algorithm is applied to standardized data, and the experimental data are extracted and reduced by features.
Construction of Personal Credit Rating Evaluation Model In the previous section, based on the Pdc-RF feature selection algorithm, the dimensionality reduction of data is completed, and a dataset with 21 features and 32469 samples is obtained. On this basis, this section further builds a supervised personal credit scoring model. A complete personal credit evaluation model should include data processing, model building, model evaluation, and credit score card conversion. Based on the data preprocessing and dimensionality reduction in the previous section, the personal credit evaluation model of this paper starts with data partition, carries out box processing on the data, and completes the discretization of the data. en, based on the related algorithms of machine learning, the samples are trained, and then, the performance of each machine learning algorithm is compared to obtain the best performance training model.
How to find a W which makes the probability of all predictions of G(Z) correct is maximized, and the cost function can be optimized by the gradient descent method or the quasi-Newton method, so as to estimate. e cost function of the logistic regression model during training is shown in J(θ) � - 1 m 􏽘 m 1 Cost f θ T x 􏼐 􏼑, Y 􏼐 􏼑. ( 10 ) Among them, Cost f θ T x 􏼐 􏼑, Y 􏼐 􏼑 � -log f θ T x 􏼐 􏼑 􏼐 􏼑, Y � 1, -log 1 -f θ T x 􏼐 􏼑 􏼐 􏼑, Y � 0. ⎧ ⎪ ⎨ ⎪ ⎩ (11) Logistic regression algorithm has obvious advantages in dealing with large-scale data, and the calculation time is effectively saved by the gradient descent method. After data training, the expression with real parameters can be obtained, and the model is intuitive and reliable. e machine learning experiments in this paper are all based on Jupyter Notebook platform, using python programming language.
e best features are used as the watershed of the left and right subtrees and cycle back and forth until all the best features are selected. e idea of randomly selecting nodes in random forest improves the generalization ability of the model. e steps of binary classification in random forest algorithm are as follows: Step 1: input: dataset X and sampling times k; output: the decision result of dataset X after K times sampling and training Step 2: bootstrap is used to carry out k-round sampling, and m features are extracted from dataset X as training samples, and finally, a sample set S is formed Step 3: classifying and training different samples in the sample set S Step 4: output: for discrete variables, Gini coefficient is used as the evaluation standard for random forest, and the calculation is carried out according to Gini � 1 -􏽘(P(i) * P(i)),  (12)  where P(i) represents the proportion of class I samples in the total samples of the current node of the decision tree, and the more average the class, the greater the Gini coefficient.
Support Vector Machine Algorithm. It belongs to a classical binary classification model. e basic model of SVM is a linear classifier with the largest interval in feature space. e basic idea of SVM algorithm  [16, 17]  is to solve a hyperplane which can divide the training dataset correctly and maximize the geometric interval between the data. For the nonlinear classification problem, the input can be mapped into a linear classification problem in a certain feature space by nonlinear transformation, and the linearly separable support vector machine model can be solved in the high-dimensional linear space. Considering the huge scale of credit data and the nonlinear relationship of the whole data, this section uses nonlinear support vector machine algorithm. ere are two very important concepts in the computation of nonlinear support vector machines: kernel function and penalty coefficient C.
e K-S value represents the difference between positive and negative samples of the model. e larger the K-S value, the better the prediction accuracy of the model. Generally speaking, KS > 0. 2 means that the model has good prediction accuracy. e judgment threshold of K-S is shown in Table  10. In this section, the dataset is divided into the training set and the test set according to the ratio of 7 : 3. Training is carried out on the training set, and then, the prediction results are given on the test set. rough grid search and   11  are the experimental data obtained after superparameter selection through random grid search. AUC and K-S indexes are within a reasonable range, but the discrimination of the model needs to be improved. Considering the imbalance of positive and negative samples in the confusion matrix, good customers are 7 times more than bad customers, which may cause prediction deviation.
Credit Scorecard Based on Logistic Regression. Before the evaluation, we screened the features with IV value greater than 0. 02, finally built a logistic regression model based on 22728 samples in the training set, and called statsmodels library in python library, when the significance level is 0. 05; all the significance tests are passed, in which coef represents the coefficient of logistic regression, all of which are negative and significant. Based on the above inspection, the scorecard can be set next. e score scale of the scorecard can be defined by the linear expression of the logarithm of default and nondefault probability ratio (log (odds)):   odds � 􏽢 p 1 -􏽢 p , ( (14) ) 15 where the offset A � offset, the scale factor B � factor, and 􏽢 p represents the default probability; the conversion process from log (odds) to score includes three commonly used parameters base: bp: (basepoint), od: odds corresponding to the reference score, Po: the score that the credit score increases when the odds doubles.
Introduction Big-Bang nucleosynthesis (BBN) is the primordial era of the History of the Universe in which nucleons gathered into nuclei. This period occurred during the dark times before the emission of the Cosmic Microwave Background, and no direct observation of the Universe during BBN is possible. However, the current abundance of the elements can be observed, and in particular the observed helium and deuterium abundances can be directly compared to the abundances predicted by BBN models to set constraints on the properties of the Universe at the time of BBN. Several public codes exist to make predictions for the abundances from BBN, such as PRIMAT  [1] , PArthENoPE  [2, 3]  and NUC123  [4]. We present here AlterBBN  [5, 6] , which is a public code for computing the abundance of the elements in the standard cosmological models as well as in alternative scenarios.
The AlterBBN package AlterBBN is a public code written in C (C99 standard), which does not rely on any external package. It can be downloaded from the website: https://alterbbn. hepforge. org/ To use it, after downloading the compressed archive alterbbn_v2. X. tgz, it has to be uncompressed with tar xzvf alterbbn_v2. X. tgz and compiled with cd alterbbn_v2. X make make program_name where program_name is the name of one of the program files contained in the main directory: • stand_cosmo. c Computation of the abundances in the standard cosmological model The exact parameters for each of the main programs are described in the manual  [6] , and they can be run with. /program_name. x parameters In absence parameters, running. /program_name. x displays a short description of the possible parameters.