dataesr/scientific-paragraphs-categorization huggingface dataset:
However, we aim here to characterize suites of filters, so we chose to base our analysis on simulated data reflective of the high standards of current asteroid spectroscopic surveys. -7520 In Table  1  we report the final number of cloned spectra per taxonomic type used in this study. We thus used a sample of 7520 spectra to generate asteroid reflectances. The breakdown of the input spectra into Bus-DeMeo taxonomic types is presented in Table  1. We have divided the data into additional, more general classes (first column in Table  1 ). That is, all the Bus-DeMeo S subtypes have been merged into a general S class, and all the X subtypes into an X class. Types from the Bus-DeMeo C complex have been split into C and Ch general classes. The later contains the Cgh and Ch types which contain water absorption bands and thus were treated as a separate class.
Therefore, the computed predictability rates represent the expected top performance of the surveys. The resulting normalized (to one at 0. 55 µm) reflectances were used as input for the classification algorithms. An example of the mean values and standard deviations of cloned spectra for a single asteroid -(5) Astraea -for the LSST and VISTA filters are presented in Table  2,  and Fig   Table  2. Mean values and standard deviations of LSST and VISTA filters for (5) Astraea. Filter Mean and standard deviation u 1. 041 ± 0. 016 g 1. 056 ± 0. 009 r 1. 113 ± 0. 010 i 1. 152 ± 0. 009 z 1. 067 ± 0. 006 y 1. 088 ± 0. 005 Y 1. 123 ± 0. 005 J 1. 303 ± 0. 021 H 1. 406 ± 0. 028 Ks 1. 361 ± 0. 022 We note that for a single original spectrum, these values are representative of an object, and higher variability is present in the collection of input data for the entire type containing spectra of a larger number of objects (see  Fig.
2000). The Gaia mission. It is an ongoing ESA space mission launched in 2013 and is scanning the entire sky  (Prusti et al. 2016). The main goal of the mission is to create a threedimensional map of the Milky Way; however, it also has dedicated pipelines to study various objects, including asteroids  (Tanga & Mignard 2012). The mission will obtain low-resolution spectra for about 100 000 asteroids  (Delbo et al. 2012 ). A10, page 4 of 15  1-meter telescope at La Silla, k Chile  (Epchtein et al. 1994 ). The observations were taken at I, J, and K passbands to study extragalactic sources. Baudrand et al. (2004)  recovered the color data for 1233 asteroids. Most of them were measured once and some were measured twice in each filter, resulting in 1385 measurements all together. The AAVSO Photometric All-Sky Survey (APASS) is an all-sky photometric survey providing measurements in eight filters: Johnson B and V, and Sloan u, g, r, i, z, and Z  (Henden et al.
Methods and evaluation metrics We made use of several machine learning classification methods implemented in the Python scikit-learn package  (Pedregosa et al. 2011 ). Those methods have been described in more detail in our recent work  (Klimczak et al. 2021) , thus they are only briefly summarized below. The descriptions below also list parameter variations, which were used to parameterize each method, with up to tensets of parameters preselected per method. Multinomial logistic regression  (Hastie et al. 2009)  estimates the probability of a sample belonging to each class using the softmax function of a linear combination of input parameters. The weights for each parameter are trained during the learning stage by minimizing the cross-entropy between the empirical class label distribution and the model distribution.
01 to 0. 1. MultiLayer Perceptron (MLP)  (Goodfellow et al. 2016 ) -we used a feed-forward artificial neural network with either two or three layers of perceptrons, each with 32 or 64 neurons, followed by a softmax function to obtain the probabilities. Cross-entropy was minimized during the learning phase. The learning rate was selected in between 0. 01 and 0. 1, and the choice of optimizer was either stochastic gradient descent or Adam. We performed five runs of five-fold cross-validation for each method to compensate for the randomness of data splitting. In each fold, the parameters of each algorithm were optimized based on the random selection of four-fifths of the data and then evaluated for the remaining fifth of the dataset. Four-fifths of the data were further split into train and validation sets to select the best parameter combination.
The optimization of parameters was performed by training up to ten models with different parameters on the subset of data and assessing the performance on the validation split. The parameter combination with the best score (balanced accuracy) was selected and used on the test split (the aforementioned fifth of the data). The result on the test split were recorded. Finally, we report average scores and standard deviations for each algorithm performed by the model with the best parameter combination on the test split. We note that for each method and each survey, we trained up to 500 models (two tasks × tfive runs × five-fold cross-validation × ten parameter combinations). Altogether, this results in 27 500 models (500 models × 11 surveys × five methods) trained in this work. This required significant computing time and processing power.
As a result of the size of the dataset, as well as the amount of models that needed to be trained due to the experimental setup, the computing time reached up to 12 h per survey performed on a CPU. Following  Klimczak et al. (2021) , we report several evaluation metrics: prediction accuracy (Acc), balanced prediction accuracy (BAcc), F1 measure, and the Matthews correlation coefficient (MCC): Acc = k i=1 T P i N , BAcc = 1 k k i=1 T P i T P i + FN i F 1 = 1 k k i=1 T P i T P i + 1 2 (FP i + FN i ) , MCC = N × k i=1 T P i -k i (T P i + FP i ) × (T P i + FN i ) (N 2 -k i (T P i + FP i ) 2 ) × (N 2 -k i (T P i + FN i ) 2 ) , where k is the number of classes, N is the total number of all samples, T P i ("true positives") is the number of correctly classified objects from class i, FP i ("false positives") is the number of objects incorrectly classified as class i, and FN i ("false negatives") is the number of incorrectly classified objects from class i.
This suggest that there might be an over-abundance of parameters, and there is not a need for dense spectra to achieve a similar prediction rate. Indeed  Klimczak et al. (2021)  show that the top five and top six features A10, page 7 of 15 A&A 667, A10 (2022) Fig. 7. Confusion matrix for the prediction of taxonomic types on LSST+VISTA data using a MLP. contribute to 93 and 81% of balanced accuracy of complex and type classification, respectively. The high success ratios in our study could also be due to the conservative bounds in cloning of the original spectra as compared to a larger variation of the simulated spectra in  Penttilä et al. (2021). Therefore, our success rates, despite being internally consistent, are not directly comparable with  Penttilä et al. (2021). To assess the usefulness of the clones even further, we performed an additional experiment, where for each object we only selected one clone.
This reduced our dataset to 579 samples. Spread across 12 classes and 21 types, the quantity of samples per taxonomic type ranged from four to 169. In this way, we wanted to quantify what scores the models could achieve if we did not use the clones. This experiment was only performed for combined surveys LSST and VISTA, a dataset that yielded the best results in our experiments. The results for the prediction of classes (Table  C. 1) remained at a relatively high level (74-78% balanced accuracy), falling 20% behind on balanced accuracy compared to the full dataset. What is worth noting is that there is not such a big difference in scores between simpler and more complex models. That is most likely due to the small size of the training data, which was not sufficient enough to train a MLP or gradient boosting.
Those models normally operate on much larger volumes of data. These results are consistent with  Klimczak et al. (2021)  who reported an ∼74-80% balanced accuracy for predicting asteroid types based on the full spectral range for the best performing methods. The training dataset for that study contained spectra for 504 objects split into 12 taxonomic types. Thus the models in  Klimczak et al. (2021)  were trained on a size-comparable spectral dataset, separated into similar number of types, but on a larger number of features. Interestingly, the top five spectral features (comparable to the number of filters per survey in this study) contributed to the majority of balanced accuracy. Slight differences in the balanced accuracy as compared to  Klimczak et al. (2021)  might be due to a nonidentical division into types, a larger number of features, and sample size.
For the prediction of taxonomic types (Table  C. 2), the results are 35% less than for the full dataset. The lower dataset size and higher number of types significantly impacted the predictability power of the models. Again, the results across models do not differ as drastically as they do in the case of the full data. We conclude that the cloning technique may have impacted the predictability scores, while the sample size and number of predicted classes also play an important role. The number of features beyond the top few has a smaller impact on the resulting balanced accuracy. We highlight that despite the possible estimation of the overall predictability scores, our results are internally consistent. In the taxonomic class prediction task, three methods (SVM, gradient boosting, and MPL) turned out to be the most efficient, independent of the survey.
A higher balanced accuracy was only achieved for combined surveys (e. g. ,a 90 and 96% balanced accuracy for a combination of LSST and VISTA data in type and in complex prediction, respectively). This is not surprising as the Bus-DeMeo taxonomy is based on combined VIS and NIR spectra, thus classifications made using only a subset of the wavelengths should perform poorer. Though the gradient boosting was the most efficient method for Euclid and VISTA surveys, these surveys outperformed all the others independent of the method and metric used. One exception is naive Bayes, for which the balanced accuracy of Euclid is on the level of other surveys and the VISTA and 2MASS surveys performed the best in complex prediction. The broad VISTA and Euclid surveys' filters covered the 1 and 2 µm olivine and pyroxene absorption bands well, which are the most pronounced spectral features.
Owing to the well-matching wavelength coverage, VISTA and Euclid performed the best in our study out of single survey. Having complementary data in wavelengths, for example for LSST (VIS) and VISTA (NIR), can further boost the prediction score. For the combination of LSST and VISTA data, we obtain about a 96% balanced accuracy for predicting the taxonomic class and around 90% for predicting taxonomic types for the most efficient methods. In Figs. 6  and 7 , we show confusion matrices for the prediction of taxonomic classes and types, respectively. Values on the diagonal represent correct classifications, whereas the values off the diagonal are misclassifications. As expected, some taxonomic types are easier to predict than others. The unique and distinct V-type objects are never misclassified, whereas for example types within the S, C, and X complexes are sometimes misclassified.
In the first quarter of 2020, the World Health Organization (WHO) declared COVID-19 a public health emergency around the globe. Different users from all over the world shared their opinions about COVID-19 on social media platforms such as Twitter and Facebook. At the beginning of the pandemic, it became relevant to assess public opinions regarding COVID-19 using data available on social media. We used a recently proposed hierarchy-based measure for tweet analysis (H-TFIDF) for feature extraction over sentiment classification of tweets. We assessed how H-TFIDF and concatenation of H-TFIDF with bidirectional encoder representations from transformers (BH-TFIDF) perform over state-of-the-art bag-of-words (BOW) and term frequency-inverse document frequency (TF-IDF) features for sentiment classification of COVID-19 tweets.
, 2019). The main objective of our work is to perform sentiment classification of COVID-19 tweets by taking into account both spatial and semantic aspects with H-TFIDF and concatenation of BERT and H-TFIDF (BH-TFIDF) features. The objective is achieved by using a supervised learning approach. Moreover, machine learning models, i. e. , linear and nonlinear, are chosen to perform the sentiment classification task. These machine learning models are trained using a publicly labeled dataset  (KazAnova, 2016). Moreover, the best model is chosen among them for sentiment classification. The model predicts results using different sets of features, i. e. , Bag-of-words (BOW), TF-IDF, H-TFIDF, BH-TFIDF, and BOW+BERT. Finally, the purpose of the proposed work is to evaluate how H-TFIDF features and BH-TFIDF perform over BOW features and TF-IDF features for sentiment classification of COVID-19 tweet data.
(Wang and Lin, 2020)  proposed a new method when selecting a suitable number of features by using the chi-square feature selection algorithm to employ feature selection using a preset score threshold. Another study  (Ansari et al. , 2019)  proposed recursive feature elimination to select the optimal feature set and an evolutionary method based on binary particle swarm optimization of the final feature subset. These approaches were validated for sentiment analysis in five different domain balanced datasets including movie reviews and Amazon product reviews. Further work  (Rustam et al. , 2021)  proposed a comparison of sentiment classification using different features, i. e. , Bag-of-words (BOW), TF-IDF, and concatenation of BOW and TF-IDF to boost the performance. In this paper, the concatenation of BOW and TF-IDF outperformed other features in sentiment classification of COVID-19 tweets.
However, the issues with features were the computational cost of model learning and overfitting of the model. To address this research gap,  (Decoupes et al. , 2021)  proposed a set of features that are extracted from a COVID-19 tweet dataset by considering the spatial and temporal aspects of COVID-19 data. In this work, the main focus was on the hierarchical characteristics of spatial and temporal dimensions for extracting a more relevant set of features in the context. These important features, i. e. , hierarchical term frequency inverse document frequency (H-TFIDF) in the tweets for different regions and time, help determine the local situation, crisis management, and opinions of inhabitants. Moreover, these reduced sets of features (H-TFIDF) may be important for sentiment classification of COVID-19 tweets.
PROPOSED METHODOLOGY In this work, we performed sentiment analysis of COVID-19 tweets for sentiment classification using different features, i. e. , H-TFIDF, BH-TFIDF, BOW, and TF-IDF. The flow of our experiments (training and prediction steps) is shown in Figure 1. There are two major types of learning techniques: supervised learning and unsupervised learning. In supervised learning, the models are trained and tested with labeled data. However, unsupervised learning learns using features and predicts unlabeled data. The dataset for sentiment analysis of COVID-19 tweets is unlabeled and needs to be classified. For sentiment classification, prediction of sentiment of these tweets is performed using machine-learning-trained models. The process of our proposed work has two phases:.
Training Dataset The training dataset is the well-known kaggle Sentiment140 dataset for sentiment analysis of tweets in English only. The dataset is available at https://www. kaggle. com/kazanova/ sentiment140 (KazAnova, 2016). It has labeled data for supervised learning for the classification of tweets. The dataset contains 1. 6 million tweets. Tweets are annotated as (0 = negative) and (4 = positive). Later, the trained model will be used to detect sentiments for COVID-19 tweet data. The training dataset for learning models will be used for the binary classification of tweets. virus causes mental stress deaths D1 1 1 1 1 0 D2 1 1 0 0 1 Table  1 : Document-term matrix.
Model Selection It is better to evaluate the performance of each model by calculating train-test chunks of data with a cross-validation strategy  (Raschka, 2018). Cross-validation is a data resampling method to assess the generalization ability of predictive models and to prevent overfitting  (Berrar, 2019). For the experimental setup, a train-test validation scheme of 90% and 10% is used with 10-fold cross validation. The dataset of 1. 6 million is divided into 10 splits such that the first split has test data and the remaining nine splits are used for training in the first iteration. Similarly, in the second iteration, the first and last eight are used for training, the second iteration has test data, and a similar pattern is shown in Figure  2. The performance of each model is calculated after each iteration.
COVID-19 Dataset In the second phase, we first selected the dataset of COVID-19 tweets that were extracted from E. Chen dataset  (Chen et al. , 2020). For the experiments, we extracted the COVID-19 tweets for the month of January 2020. The tweet IDs of COVID-19 were extracted using the Twitter Streaming API by using COVID-related keywords. The analysis dataset contains 165,537 tweets. Each tweet contains the information ID, UserID, text, location, country, and its creation date. Furthermore, data preprocessing was performed with the same strategy as discussed in section 3. 1. 2. Finally, sentiment analysis was performed using different sets of features, i. e. , BOW, TF-IDF, H-TFIDF, and BH-TFIDF. These features are discussed in Section 3. 3. 1.
Data-Preprocessing Similar to the training phase, the tweets were preprocessed through the Python library tweetpreprocessor  ( Özcan, 2016). Some examples of preprocessed tweets are as follows: <Tweet1>:"@pearlylondon Don't worry, if she does contract a fatal dose of coronavirus at least she will have a dignified burial \n#Blackadder https://t. co/8KdpMIItki". <Tweet2>:"5 confirmed cases of #coronavirus in Brighton. In the meantime, local news. #Brighton https://t. co/KTXkQCOApg" <Preprocessed Tweet1>:"do not worry, if she does contract a fatal dose of coronavirus at least she will have a dignified burial" <Preprocessed Tweet2>:"confirmed cases of in brighton in the meantime, local news".
TF-IDF: The second experiment was performed using term frequency-inverse document frequency (TF-IDF) features using the LR model. TF-IDF is defined as in two parts. The term frequency (TF) indicates the frequency of each of the words present in the document or dataset. The second part is inverse document frequency (IDF), which actually tells us how important the word is to the document  (Qaiser and Ali, 2018; Yahav et al. , 2018). The basic purpose of this is to enable us to determine how each word is relevant in the document and the corpus (see equations below): tf -idf (t, d) = tf (t, d) * idf (t) (1) tf (t) = f t f tot (2) idf (t) = log(N df t ) (3) 3. H-TFIDF: In the third experiment, a hierarchy-based measure for tweet analysis known as H-TDIDF features is used to perform sentiment analysis of the COVID-19 dataset.
These trends for the positive classification and negative classification are the same in both datasets. This clearly shows that BH-TFIDF features are more enriched toward positive classification of tweet data. On the other hand, BOW and BOW+BERT are more tilted toward negative classification of tweet data. In future work, we will focus on terminology We selected best machine learning model i. e. , LR, among other models, i. e. , SVM and RF, by applying cross-validation to evaluate model performance. Furthermore, experiments were performed with LR using different features (BOW, TF-IDF, H-TFIDF, and BH-TFIDF) to predict the sentiments of the tweets. Furthermore, the analysis of the results showed that BOW features performed better for predicting negative tweets. However, BH-TFIDF features were useful in predicting positive tweets in the COVID-19 dataset.
11, 15 ] [16] [17] [18] [19] [20] [21] [22]  People who present with any of these indicators may be referred to higher facilities for comprehensive clinical evaluation and inpatient care. The potential impact of this strategy in preventing early deaths (within 2 months) in routine program settings has not been explored. During October-November 2020, in 16 districts in Karnataka (a state in South India), paramedical TB programs and general health staff at PHIs screened people with TB for "high risk of severe illness" at diagnosis/notification (Box). 11, 15 he findings of the screening process have already been published. 23 Due to the prevailing COVID-19 pandemic, the coverage of screening was low (51%); however, the time interval from notification to screening was short (only a few days) and data was collected for all indicators in most instances.
Data Collection, Variables, and Sources of Data The laboratory technician or the TB health visitor collected the screening data (primary data) at the first opportunity in a paper-based form (Supplement 2) with the support of a staff nurse or medical doctor. The STS transcribed the details into the Epicollect5 mobile application (an openaccess application that allows offline mobile or tablet-based data capture and synchronizes data in the cloud). Before the study period, we piloted this data collection process for 10 days to address unforeseen issues. On December 15, 2020, we extracted the screening data from Epicollect5 and baseline characteristics from NIKSHAY (Supplement 3). We tracked the program-reported treatment outcome (in treatment, death, loss to follow-up, and transfer to drug-resistant TB care) for each person for up to 2 months.
Data Analysis and Statistics We analyzed the data using EpiData Analysis (v 2. 2. 2. 183 EpiData Association) and STATA version 12. 1 (StataCorp) software. We summarized the distribution of early deaths since beginning treatment (since diagnosis if treatment was not initiated), overall and stratified by exposure status, using incidence proportion, incidence rate (per 100 person-months), and Kaplan-Meier failure curves. We compared the baseline characteristics with a high risk of severe illness (exposure) as well as early deaths (outcome) using chi square, ANOVA, and Kruskal-Wallis tests, as appropriate. We identified the potential confounders for the association between exposure and outcome: baseline characteristics associated with exposure (unadjusted P<. 05) as well as outcome (unadjusted P<.
Acknowledgments: We thank the  State TB Centre, Bengaluru Karnataka (India)  for administrative support during the study implementation. The  District TB program  staff (district TB officers, DEOs, STSs, laboratory technicians, and TB health visitors) of  Belgaum, Bellary, Bengaluru City, Bengaluru Rural, Bengaluru Urban, Chikballapur, Chikkamagaluru, Dakshina Kannada, Dharwad, Gulbarga, Hassan, Kolar, Ramanagara, Shivamogga, Tumakuru , and  Udupi  deserve a special mention for their commitment and support in successfully completing the study. We acknowledge  Dr. Vasna Joshua , Scientist C, and  Dr. Jeyashree K ,  Scientist D ,  ICMR-National Institute of Epidemiology, Chennai, India , for reviewing the manuscript and providing critical input.
5, 244, and 246 GHz. A fourth spectral window with a reduced bandwidth of 1. 875 GHz split into 3840 channels was centered on the 12 CO J = 2-1 line at 230. 538 GHz, although no gas emission was detected. The same calibration sources were used for all eight SBs. Bandpass, flux, and atmospheric calibration along with pointing were performed using the bright blazar J0538-4405 (21°. 0 away from the target). Phase calibration made use of J0700-6610. All data processing, calibration, and imaging commands were executed in CASA  (version 5. 4. 0; McMullin et al. 2007). Calibrated visibilities produced by the ALMA pipeline were then time-averaged in 30 s intervals to reduce data volume and improve modeling speed.