dataesr/scientific-paragraphs-categorization huggingface dataset:
The classification problem tackled in this study is a multi-class problem with three classes. Regarding the casualty severity from all the datasets used, 1 denotes a fatal crash; 2 represents if the crash caused any serious injury to the driver, passenger(s), or any bystander; and 3 implies slight injuries to any of these subjects. Table  3  shows the summary of the attributes in the training and test dataset with exploratory data analysis. In terms of temporal distribution, the monthly and yearly distribution of the crashes with their severity count is given in Fig. 3. It can be seen that the data is highly imbalanced between the three classes.
For instance, if 95 instances belong to class 1 and 5 belong to class 2, but a classifier assigns a single label to all elements, it will still result in an accuracy value of 95%. In such a case, the NMI score will be quite low even though accuracy is rather high indicating that the classifier did a poor job in the classification task. In addition, this study also employs specificity and sensitivity to evaluate the performance of ML models. Specificity is the ability of a model to detect the True Negative (TN) of each output class whereas sensitivity is the detection of the True Positive (TP) by the classifier. Consequently, specificity is called the true Negative Rate (TNR), and sensitivity is referred to as True Positive Rate (TPR). If FP is the number of samples that were negative but predicted as positive, FN denotes the 2 Leeds City Council dataset (https://www.
data. gov. uk/dataset/6efe5505-9 41f-45bf-b576-4c1e09b579a1/road-traffic-accidents). 3 The cycling casualty dataset in Leeds (https://datamillnorth. org/dataset /cycling-casualties-in-leeds). 4 The Manchester road casualty accidents dataset (https://www. data. gov. uk/dataset/25170a92-0736-4090-baea-bf6add82d118/gm-road-casualty-a ccidents-full-stats19-data). number of samples that were positive but forecasted as negative, TP denotes the number of samples that were positive and predicted as negative and TN represents the number of instances that were negative and are also predicted as negative and j represents the respective labels. Then sensitivity and specificity for jth class is given by: Sensitivity j = TP j TP j + FN j (9) Specificity j = TN j TN j + FP j (10).
Selecting the features The first contribution of this study is to use an automated feature selection technique to select only the distinguishing features rather than the entire feature set. The DMI has the added effect of selecting features that only help to distinguish instances of one class from all other features as opposed to the standard MI technique which may select features that have a higher average or total MI score. To see which features are the most helpful, and to determine how many features should be selected, we compute the DMI score of each feature. The test dataset must not be used in any computation where prior knowledge of the class labels is necessary, hence we only compute these scores from the training dataset. The result is shown in Fig. 4. Not all features in the dataset have equal importance in the classification process.
Usually, the question of how many features to select is a balance between the size of the data (to reduce) and the information content to be retained. In our case, the type of vehicle, the casualty class (i. e. , whether a pedestrian, driver, or passenger), the number of vehicles involved, the gender of the casualty, and the road class show the highest scores. Together, these five (05) features retain over 91% of the information content. Therefore, we select these five features when using feature selection. The rest of the features are left out in both the training and test dataset. The distribution of the selected features concerning the crash severity is given in Fig. 5 , where crash count denotes the occurrence values in Table  3. As observed in Fig. 4 , the type of vehicle had the highest information score.
Effect of feature selection We use the full version of the dataset X train and X test that employ the complete feature set and the reduced version of the dataset X S train and X S test with only the selected features. Each of the classifiers discussed in Section 3 is used to classify the two datasets along with the proposed OvRCL algorithm. Additionally, commonly used ML models like Ada-Boost and ANN have also been compared with the OvRCL framework. As the data is linearly non-separable so the ANN with three hidden layers was implemented. This number was selected by trying a few other models with different layers and the model with the better result is selected. Furthermore, an Adaboost with 30 learners is used to predict the crash severity. The accuracy of the ML models is shown in Fig. 6  whereas the results for NMI are compared in Table  4.
Validation of OvRCL algorithm To acquire the most relevant features for crash severity prediction MI-based feature selection (Eq. (  4 ) is applied to all the datasets. Fig. 7  and Fig. 8  illustrate the MI score of features and the selected subset in the dataset of cycling casualties in Leeds, and roads accident in Manchester and the UK, respectively. Feature subset is selected as they are more relevant and contain 90% of the information regarding the accident severity (as specified in section 5. 1). Furthermore, OvRCL is better able to detect the hard-to-detect classes irrespective of the size of the data as illustrated in Fig. 9  and Fig. 10. The proposed scheme also surpassed the conventional ML models in terms of NMI score as shown in Table  10  and Table 11 , for sensitivity and specificity in Table  12  and Table  13  for the bicycle crashes and Manchester accidents data, respectively.
Comparison with other state-of-the-art methods Finally, we compare the proposed method with several other stateof-the-art algorithms used in the literature on the Leeds and other datasets. The results are shown in Table  14  using the accuracy measure and different datasets. Since we do not have the code for the other methods, we simply cite the results as mentioned and explain the difference in the datasets used. For the k-NN, SVM, Bagging, and Random Forest, the datasets are the same as the OvRCL algorithm. In which k-NN represents the baseline algorithm, SVM is a binary classifier and utilizes the OvR strategy for multi-class problems, while BG and RF represent ensemble techniques. Hence, these methods provide a good comparison to see the effect of using existing techniques of OvR and Ensemble in the existing literature.
For  (Kabeer, 2016)  and  (Kannojiya et al. , 2020)  the dataset is the same but differs in the years used. Whereas  (Chandra et al. , 2019; Ma et al. , 2021) , and  (Sattar et al. , 2022 ) make use of a different but comparable dataset for UK road traffic accidents. The results are evident from Table  8  in the previous section. On the same dataset, the existing machine learning classifiers of k-NN, SVM, Bagging and RF have comparable results of around 81-82% accuracy. From  (Kannojiya et al. , 2020)  and  (Chandra et al. , 2019) , Logistic regression and RF are more suitable for predicting crash severity. This may be because the crash data is usually highly imbalanced with distinctive features showing correlations with multiple classes under different conditions. In the literature, many authors have tried to overcome this in two ways: firstly, by combining severity levels and transforming the problem into a binary classification problem and predicting only severe and nonsevere crashes irrespective of whether the severe crashes were fatal or not  (Sattar et al.
, 2022). Secondly, some techniques employ machine learning techniques to cluster the data using spatial dimensions  (Iranitalab & Khattak, 2017; Ma et al. , 2021). This can be done using a clustering method, such as the k-means or other clustering techniques  (Hussain, 2011) , to group accidents geographically. Their results show that it is easier to predict the severity of the crash after grouping them and predicting the severity of each group separately. However, in all the cases and using all the methods, the crash severity prediction remains below 90% on all the datasets.
Discussion In this paper, we present a new machine learning approach for the classification of crash severity prediction by using a combination of feature selection and One-vs-Rest Consensus Learning (OvRCL). The previous sections show the performance of the proposed strategy on the Leeds accident dataset in comparison to established state-of-the-art techniques. In this section, we further analyze the features concerning the crash severity along with the advantages and limitations of the proposed model, and policy implications from this study.
Effect of classifiers on OvRCL In discussed previously and shown in Fig. 2 , any choice of classifiers could be used for the consensus learning. In the previous section, we chose SVM, K-NN, RF, and BG as classifiers because they have been extensively used in the literature for crash severity prediction on a variety of datasets and show comparatively good results  (Haleem et al. , 2015; Iranitalab & Khattak, 2017; Kabeer, 2016; Kannojiya et al. , 2020; Yan et al. , 2021; Yang et al. , 2022). Here we show that OvRCL improves on the classification accuracy for all the accident datasets compared to the individual classifiers and traditional ensemble approaches. To show this, we replace SVM and RF with Artificial Neural Network (ANN) and AdaBoost (AB) in the proposed OvRCL framework. It can be seen from Table  15  that both versions of OvRCL outperforms the individual models, including traditional ensemble methods like AdaBoost or Bagging.
Feature analysis In this section, we analyze the selected features by using Shapley values (SHAP). (Lundberg & Lee, 2017)  proposed to use the Shapley value to interpret the output of the model. Fig. 11  illustrates a SHAP summary plot that represents the range and distribution of the selected features to injury severity level. SHAP plot is an agnostic framework that orders features based on their influence to detect crashes since it qualitatively represents the relation of attributes to the output class (i. e. , crash severity). The scatter here represents the different Shapley values of the features, and each point is colored by the value of the feature from low (blue) to high (red). The density of points demonstrates the distribution in the dataset. For instance, the type of vehicle has a greater negative value which means that changing the type of vehicle would have a greater influence on crashes with slight injuries.
Advantages and limitations of the proposed model The key advantage of the proposed model is its ability to identify crash severity in a multi-class problem even in the presence of highly imbalanced data. While several machine learning algorithms have been used in the past, they are not able to differentiate between hard-toclassify classes, such as between slight and serious severity and between fatal and serious severity. The proposed model employs a feature selection technique that reduces the feature set, only retaining the discriminatory features. This has the added advantage of giving feature importance ranking which can be used to select pertinent features for the crash severity modeling. The proposed framework predicted indistinguishable classes in the imbalanced dataset by implementing classwise OvR along with the ensemble of binary learning models for classification.
Policy implications The proposed algorithm developed in this paper has been evaluated on a real dataset. Firstly, using the proposed method provides better performance, in terms of evaluation metrics, than existing models in the literature. Secondly, the analysis of the features used, and their rankings can also provide helpful insight to safety planners and policymakers. SHAPley values are also utilized for the analysis of selected features regarding the crash severity. Based on the above analysis, the severity of a casualty in an accident involving a single vehicle would be minimal, and as the number of vehicles involved increased, the severity level also increased. This is in line with earlier research  (Ma et al. , 2021)  that found that the number of vehicles increases the likelihood of accidents and the severity of injuries.
The binary classification in the OvR strategy helps in predicting instances of a single class particularly when the features have been selected to help identify one class from the others. The overall result is the higher performance with low complexity since we do not employ any deep networks or any extra steps of data clustering. This study also analyzes the features that are mostly correlated to the severity of the crash. This can also help in further focused research to help avoid crashes that may lead to severe injuries or fatalities. In the future, we plan to extend this work and test the proposed method on other detailed datasets to analyze the cause of crashes, for instance by associating the selected features to the time and location of the accident so that the number of crashes resulting in severe injuries can be reduced.
Ensemble city council, UK. technique. (Iranitalab Random Forest, Crash datasets from Predicts crash & SVM, k-NN, Nebraska, United severity level and Khattak, Multinomial Logit, States. proposed cost- 2017) k-Means, latent based approach for class clustering. accidents. (Lee et al. , Structural Equation Traffic accident data Examine the 2018) Modeling (SEM). from Traffic influence of water Accident Analysis level and rainfall on System (TAAS), the crash severity. Seoul, Korea. (Chandra Random Forest, Data of traffic Examine crash et al. , Logistic regression. accidents and severity prediction 2019) vehicles, UK. with and without feature extraction. (Kannojiya SVM, k-NN, RF, Road traffic accident Analogize weather et al. , Logistic Regression, datasets from Leeds conditions that may 2020) Naïve Bayes.
e. serious or non- serious). (Rahim & t-SNE, Convex hull Louisiana Comparison of Deep Hassan, algorithm, Department of learning model with 2021) EfficientNet-B7, Transportation and SVM on the feature SMOTE, CART, Development selected and MARS (LDOTD). balanced dataset. (Yan et al. , DT, RF, Adaboost, Dataset of the single Comparison of ML 2021) Gradient Boosting vehicle crashed by algorithms for crash Decision Tree, University of North severity prediction Extreme Gradient Carolina Highway and analysis of Boosting, Quadratic Safety Research factors responsible Discriminant Center, US. for crashes. Analysis, SVM, k-.
Education, China. (Hou et al. , Fixed parameter Police-reported Predicts out-of- 2022) multinomial logit crash dataset from sample injury model, random Heilongjiang severities, along parameters logic Province, China. with the model, random investigation of parameters logic marginal effects model with and temporal heterogeneity in instability of crash means, random severity. parameters logic model with heterogeneity in means and variances. (Sattar Vanilla-MLP, MLP Data of traffic et al. , with embedded accident, UK. 2022) layers, TabNet.
OvR Consensus Learning (OvRCL) Inputs: Classifiers CL i i = 1. P, train and test data (i. e. , Xtrain, Xtest), class labels y. Set of selected features S = {ɸ}, Set of non-selected features S ' = {ɸ}, Initialize ŷi = 0 ∀ i = 1. N test. Begin 1. for each fi in F 2. for each cj in C 3. compute I(f i ,c j ) 4. endfor 5. compute DMI(fi) using Eq. (3) 6. endfor 7. select top |S| features using Eq. (4) 8. 9. select the dataset X S train and X S test for each classifier i = 1. P 10. train CLi where i∈{RF, SVM, k-NN, BG} 11. endfor 12. 13. 14. for each i = 1. N test ŷRF i CLRF ( X S train , X S test 15. 16. 17. 18. 19. 20. End Outputs: Consensus output label, ŷ. Initialization:.
That dataset used for this study was obtained from the open data source of Leeds city council 2. We refer to this as the Leeds accidents dataset. It comprises information on accidents from different years across Leeds. In our study, the crash data from 2015-2018 is used as training data while the data from 2019 is used as the test dataset. In total, the dataset contains roughly 11,300 instances divided as follows -9400 instances are used in the training data, whereas the test data contains approximately 1900 instances. The data records multiple features for each crash including the Reference Number, Grid Ref: Easting, Grid Ref: Northing, Number of Vehicles, crash Date, Time of the crash, 1st Road Class, Road Surface, Lighting Conditions, Weather Conditions, Type of Vehicle, Casualty Class, Casualty Severity, Sex of Casualty (referred to as Gender of casualty) and Age of Casualty.
2. The second part of the analysis is done to substantiate the performance of the OvRCL framework proposed in this study and its suitability for such as analysis. To do this, we leverage two more publicly available datasets with variations for crash severity prediction. The first is a cycling casualty in Leeds 3 data from 2009 to 2015 (referred to as bicycle crashes). This is a relatively small dataset with approximately two thousand instances and 16 features in total. The second relates to road casualty accidents from 2010 to 2021 in the city of Manchester 4 (referred to as Manchester accidents), having over 45 thousand records with 27 attributes. The two datasets are chosen to observe the performance on a small and medium sized dataset.
(Chandra et al. , UK 86-86. 5% RF shows better performance with 2019) dataset and without feature selection in comparison with the logistic regression. (Ma et al. , 2021) UK 75-80% Computed on 4 spatial classes with dataset only injury classes, i. e. , serious and non-serious, using Auto-encoders and Deep Neural Networks. (Sattar et al. , UK 73-75% Compared various DL models in 2022) dataset which Vanilla-MLP has the fastest training time whereas MLP with embedded layers has the highest accuracy while predicting binary classes (i. e. serious and non- serious). OvRCL (proposed) Leeds 97. 64% Applying discriminative feature dataset selection along with class-wise OvR with ensemble classifier.
Decoding speech from brain activity is a long-awaited goal in both healthcare and neuroscience. Invasive devices have recently led to major milestones in this regard: deep-learning algorithms trained on intracranial recordings can now start to decode elementary linguistic features such as letters, words and audio-spectrograms. However, extending this approach to natural speech and non-invasive brain recordings remains a major challenge. Here we introduce a model trained with contrastive learning to decode self-supervised representations of perceived speech from the non-invasive recordings of a large cohort of healthy individuals. To evaluate this approach, we curate and integrate four public datasets, encompassing 175 volunteers recorded with magneto-encephalography or electro-encephalography while they listened to short stories and isolated sentences.
Article https://doi. org/10. 1038/s42256-023-00714-5 performances can reach high top-1 accuracy in the best performing participants: for example, above 80% top-1 accuracy in the best participant of the Gwilliams 2022 dataset  30  (Fig. 2a ). For comparison, a model that predicts a uniform distribution over the vocabulary ('random model') only achieves less than 1% top-10 accuracy on the same MEG datasets. Decoding performance for EEG datasets is substantially lower: our model reaches 17. 7% and 25. 7% top-10 accuracy for the two EEG datasets currently analysed. While modest, these scores are much higher than the random baseline.
Is MEG really much better than EEG? To investigate whether these performances depend on the total recording duration and/or the number of recording sensors, we train our model on a subset of the data that homogenizes recording time, the number of sensors and the number of participants. For this, we discard the dataset of Brennan and Hale  31  , to avoid over-limiting the analysis dataset. Consequently, we match all datasets to the smallest number of channels of the three remaining datasets by keeping a random but fixed subset of channels (for example, 128). We keep only 19 participants per dataset, again aligning on the smallest for all three datasets. Finally, we keep the same average duration per participant for all three datasets, by dropping out some training segments (that is, the same segments are dropped for all participants or repetitions within one participant).
All test segments are kept to maximize reliability. Overall, this subsampling diminishes decoding performance (for example, top 10: 30. 3% for the Schoffelen dataset  32  and 31. 7% for the Gwilliams dataset  33  ), but MEG decoding remains much better than the EEG (Mann-Whitney across MEG and EEG participants: all P < 10 -6 ). Although these results should be confirmed by presenting the same stimuli to participants recorded with both EEG and MEG, they suggest that the difference in decoding performance observed between studies is mainly driven by the type of device. spanning over 22 hours. Finally, Metzger et al. 18  recently showed that a patient with severe limb and vocal-tract paralysis and a BCI implanted in the sensori-motor cortex could efficiently spell words using a code word that represented each English letter (for example, 'alpha' for 'a'): this approach leads to a character error rate of 6.
In sum, decoding language from brain activity is, so far, limited either to invasive recordings or to impractical tasks. Interestingly, both of these approaches tend to follow a similar method: that is, (1) training a model on a single participant and (2) aiming to decode a limited set of interpretable features (Mel spectrogram, letters, phonemes, small set of words). Instead, here we propose to decode speech from non-invasive brain recordings by using (1) a single architecture trained across a large cohort of participants and (2) deep representations of speech learned with self-supervised learning on a large quantity of speech data. We focus the present work on speech perception in healthy volunteers rather than speech production in patients to design a deep-learning architecture that effectively addresses two core challenges: (1) the fact that non-invasive brain recording can be extremely noisy and variable across trials and across participants and (2) the fact that the nature and format of language representations in the brain remain largely unknown.
For this, we introduce a 'brain module' and train it with contrastive learning to align its representations to those of a pretrained 'speech module', namely, wav2vec 2. 0 (ref. 29) (Fig. 1 ). We train a single model for all participants, sharing most of the weights except for one participant-specific layer. Figure  1  provides a broad overview of our approach. To validate our approach, we curate and integrate four public MEG and EEG datasets, encompassing the brain activity of 175 participants passively listening to sentences of short stories (see Table  1  for details). For each MEG and EEG recording, we evaluate our model on its ability to accurately identify the corresponding audio segment from a large set of more than 1,500 segments (that is, 'zero shot' decoding). This study provides three main contributions for the development of a non-invasive method to decode speech from brain activity.