dataesr/scientific-paragraphs-categorization huggingface dataset:
The third module ranks the filtered hash-codes depending on their relevancy, while the fourth component filters the irrelevant codes. The goal of this research is not only to normalise the text but also to see how it affects text classification. Recently, a novel term-weighting technique was introduced known as discriminative feature spamming method (DFST), which is a unique word weighting technique for Roman Urdu sentiment analysis that selects distinctive terms based on a term utility criteria (TUC) and then spams them to improve their discriminative strength  [28]. The experiments were conducted on 11,000 Roman Urdu reviews and a custom tokeniser was built to improve the accuracy. In an earlier study Ayesha et al. [51]  collected Roman Urdu product reviews for sentiment analysis. They experimented with three machine learning schemes: Naive Bayes, SVM, and Logistic Regression.
Naive Bayes beats the other classifiers, according to the findings. Chandio et al. [55]  has recently experimented with a bunch of machine learning algorithms for the sentiment analysis of Roman Urdu and with their proposed SVM. Some researchers have also utilised deep neural networks for the sentiment classification of Roman Urdu. Mehmood et al. [12]  created a new Roman Urdu corpus RUSA-19 containing 10,012 reviews. The reviews belongs to six different domains: (i) drama, movies, and talk shows; (ii) food and recipes; (iii) politics; (iv) apps, blogs, forums, and gadgets; and (v) sports gathered from numerous websites on social media. They proposed a deep network RCNN for deep sentiment detection and experimented with newly created dataset. Their proposed deep model was based on recurrent layers and CNN layers.
However, it is observed by several researchers that CNN is better at handling non-sequential data, whereas for sequential data, CNN has not shown satisfactory results. Ghulam et al. [56]  conducted a deep-learning-based comparison on Roman Urdu for the sentiment analysis problem. Different machine learning classifiers were empirically examined in conjunction with the suggested deep learning model LSTM. It was discovered that the deep learning model outperforms the machine learning models. In a recent study, a multi-channel hybrid approach that proposes a method based on BiGRU and word embeddings was utilised  [7]. However, their dataset was too small, containing just 3241 sentiment reviews, and is not yet publicly available. Rizwan et al. [57]  has conducted a research study on hate speech detection on Roman Urdu and employed static CNN with different word embedding schemes.
Methodology Roman Urdu is perceived as a symbolic representation of a linguistically-rich but morphologically complex language: Urdu  [7]. The nastaliq format is basically a cursive style of writing Urdu that is based on 36 alphabets. In contrast, Roman Urdu relies on English/Latin alphabets and phonetically sounds like Urdu. Owing to less morphological support from English/Latin alphabets, Roman Urdu confronts many challenges. People in South-Asia mostly prefer to communicate on social platforms via Roman Urdu, which can easily be typed using a QWERTY keyboard. The increased use of Roman Urdu on the Internet by a wider Urdu-speaking community has made processing it difficult. The smart processing of Roman Urdu text could be a step forward in developing intelligent systems such as cyber bullying and hate speech detection, recommendation systems, neural dialog generation, and text summarisation and generation.
Experimental Setup We evaluated our newly created Roman Urdu E-commerce Dataset (RUECD) annotated for three classes: positive, negative, and neutral. Moreover, the RUSA-19  [12]  dataset is also experimented on using our proposed model. The sentiment analysis task is performed on the model for both binary and ternary classification so as to investigate the potential of the proposed model. The binary classification aimed at classifying the positive and negative reviews, whereas the ternary classification will target three classes, i. e. , positive, negative, and neutral. We performed a series of deep learning experiments for both classification tasks and will be discussed in subsequent sections. In our experimental setting, we used the tensorflow keras deep learning library in python. Moreover, sckit-learn was used to split the training and testing data by random sampling.
Dataset Properties The Roman Urdu E-commerce dataset (RUECD) was annotated for the sentiment analysis task and was categorised into three classes, i. e. , positive, negative, and neutral. The RUECD dataset includes 26,824 user reviews with 9833 positive, 8252 negative, and 8739 neutral reviews, as illustrated in Table  3. In the dataset, the shortest review has a length of 1 word and the longest review has a length of 296 words. The average length of reviews was computed as 74 words, with standard deviation of 54. The median estimated was 66, which is almost at par with the mean. This distribution of reviews, as illustrated in Figure  6 , depicts that most reviews are between 1 to 12 words, with long reviews being very few and easy to classify but with short reviews being a little tricky when deciding the sentiment class.
Annotation Process The annotation process was initiated by designating three annotators-A, B, and C-to manually annotate user reviews separately. Each annotator was assigned a user review to label to one of the three classes (positive, negative, or neutral) according to the furnished annotation guidelines. In order to address the efficacy of annotation process, a sample of 300 reviews (annotated by A and B) was evaluated for conflicting pairs. In case of any disagreement, the third annotator C was assigned to label the conflicting review. Moreover, an inter-annotator agreement (IIA) was established for the whole corpus. For the whole corpus, we achieved an Inter-Annotator Agreement (IAA) of 72. 45% and a Cohen Kappa value of 0. 58. The Roman Urdu RUECD dataset, which consists of 26,824 text reviews, has the following class distribution: 9833 positive, 8252 negative, and 8739 neutral reviews.
Results and Discussion In this section, we deliberate on the results of our set of experiments conducted on two Roman Urdu datasets as well as the IMDB Movie reviews dataset for sentiment analysis. The results as tabulated in Table  4  determines the validation accuracy of the RUECD dataset. It can be clearly observed that binary classification outperforms the ternary classification task, as depicted in Figure  5. Our proposed model clearly outperforms the baseline deep learning models under three different neural word embedding schemes.
In order to determine whether the proposed model is generalisable, we evaluated our model on the famous IMDB Movie Reviews dataset  [63]. The classification results of our proposed model as reported in Table  7  are promising on this dataset with an accuracy of 0. 88. On the other hand, the classic BiLSTM has also shown superior performance. The empirical evaluation reveals that the proposed attention-based model is not language-specific. However, it aimed at mitigating text that is informal in nature and needs to assign more weight to features with greater contribution. In a separate experiment on machine learning algorithms, we employed the TFIDF model with Support Vector Machine (SVM)  [13]  and experienced an accuracy not more than 61%.
The problem is known as the gradient vanishing problem, which LSTM can better deal with. However, the classic LSTM performance is not found satisfactory, with an accuracy of 0. 62. On the other hand, our proposed model RU-BiLSTM, as discussed above, can better preserve the contextual information in the Roman Urdu case too. Since our dataset contains both short and long reviews, they are evaluated on our proposed model and attained superior performance compared with baseline models. An improvement of almost 8% is achieved on the RU-BiLSTM model. Hence, our proposed model outperforms the baseline models. Beside some astounding features of SVM coupled with the TFIDF model, the sentiment analysis results for Roman Urdu in multi-class environment was not very satisfactory. Even if the SVM has the advantage of tuning fewer parameters than deep networks, it is still quite very tricky to choose the right kernel and other parameters such as the width of the hyperplane.
Conclusions In E-commerce, a sentiment analysis task on text, which has a colloquial nature, has several challenges. Among these challenges, feature extraction and design of classifiers are the significant ones. Another ability that semantically empowers the deep learning model is pattern recognition. Most of the baseline deep neural networks lack this capability. The classic LSTM has solved this problem to some extent, but it still needs improvement. This research study proposed an attention-based bidirectional LSTM network, RU-BiLSTM, that has shown superior performance on our newly generated Roman Urdu dataset, RUECD. The dataset is one of the major contributions of this paper, and it is openly available (https://github. com/bilalbaloch1/RU-BiSLTM/blob/main/RUECD. csv) (accesed on 15 February 2022) along with all the model code.
The code and dataset have been made public so that future researchers may improve upon the accuracy of the dataset. Roman Urdu being a resource-starved language has always needed a benchmark dataset. This study presented the largest ever Roman Urdu dataset containing 26K+ user reviews carefully annotated by three annotators with excellent knowledge of Roman Urdu. The dataset and API for Roman Urdu sentiment analysis is publicly available for future research in this area. We conducted a series of experiments in the presence of three different neural word embedding for Roman Urdu: word2vec, Glove, and FastText. Among three word embedding schemes, word2vec embeddings has shown better performance. The neural word embedding and attention layer are used to semantically empower the model and to enhance the pattern extraction capability.
The experimental results reveals that the classic LSTM, RNN, RCNN, and CNN have shown an average level performance. On the other hand, our proposed model RU-BiLSTM has shown significant improvement in all four evolutionary metrics i. e. , precision, recall, F1-measure, and accuracy. The experimental results shows that our proposed model outperforms the baseline models, with enhanced semantic capability and a significant improvement of 8% in terms of accuracy. Future work regarding Roman Urdu focuses on the creation of a benchmark dataset for fine-grained sentiment analysis with research in transformer learning and on designing a new self-attention mechanism to further improve the performance. Future research is based on the following agendas: 1. The development of a benchmark Roman Urdu dataset to support fine-grained sentiment analysis and question classification.
Experiments were made on multi-core machines. For example, on one node of a multi-core machine (Troll from Grid'5000), we showed that when combining existing heuristic with graph compression, with Pagerank being executing on Live Journal dataset, we can reduce with cn-order: cache-references from 29. 94% (without compression) to 39. 56% (with compression), cache-misses from 37. 87% to 51. 90% and hence time from 18. 93% to 28. 66%.
Effect of compression on Cache References Reduction Figures 5, 6, 7, 8 (reported with Orkut dataset) and figures 9, 10, 11, 12 (reported with Live Journal dataset) represent the cache references gotten with pagerank. At the left of every figure, we have log of mean number of cache references per thread (from 1 to 32 threads). At the right, we have the gain in percentage (per number of thread) compared to the original (without using any heuristic). At each figure, we add another curve that represents the cache references gotten when using compression (press 1).
Orkut Dataset. With graph ordering heuristics in figures 5 and 6, we have observed as in our previous work that: Cn-order is the best with almost 30% of reduced cache references. It is close to Goder with almost 29%. NumBaCo is the third with almost 13% and Rabbit is the last with almost 10%. The gained percentage is nearly the same from one thread to 32 threads. The new observation here is that the compress version of the graph (thanks to logarithmization, global approach) allows to reduced references by a percentage more than 36% (from one to 32 thread). We have quite the same observations with. We have quite the same observations with degree-aware scheduling heuristics in figures 7 and 8, Comm-deg-scheduling is the best with 30% of reduced cache references. But using compression improve cache references reduction to more than 36%.
Live Journal Dataset. In figures 9 and 10, with graph ordering heuristics, Cn-order reduces cache references by almost 30%. Compressing the graph contributes to reduce by around 40%. We have the same observations with degree-aware scheduling heuristics in figures 11 and 12, where Comm-deg-scheduling is the best with 30% but this result is outperformed with compression. Interpretation. Considering the observations made with Orkut and Live Journal datasets on cache references reduction, we are able to say that: • As already explained in our previous work, Cn-order is the best even with many threads (compared to Gorder, NumBaCo and Rabbit). • Compressing the graph allow to improve the percentage of the reduction. Does the compression improve the reduction of cache misses as it did with cache compression when using graph ordering heuristics and degree-aware scheduling heuristics? In the next section, we study cache misses reduction on the same datasets with Pagerank and will provide an answer to this question.
Effect of compression on Cache Misses Reduction In this section, we discuss about cache misses reported with Orkut dataset present in figures 13, 14, 15, 16 and with Live Journal dataset in figures  17, 18, 19, 20. As with cache references reduction (section 5. 2. 1, at the left of every figure, we have log of mean number of cache misses per thread (from 1 to 32 threads) and at the right, we have the gain in percentage (per number of thread) compared to the original (without using any heuristic). We add another curve that represents the cache misses gotten when combining heuristic with compression. Orkut Dataset. In figures 13 and 14, we can see that, with Cn-order, we have a reduction between 28% and 35% of cache misses, using compression allows to have a reduction between 40% and 49% of cache misses.
Live Journal Dataset. In figures 17 and 18, Cn-order and NumBaCo have the best cache misses reduction with almost 30% to 38%. When using compression, cache misses are now reduced from 45% to 52%. In figures 19 and 20, as for Orkut dataset, degree-aware scheduling heuristics have also the same behavior as their homologous of graph ordering heuristics in cache misses reduction. And the using of compression increases the performances. Interpretation. According to all the observations, we can say that: • Combining graph order with compression improves cache misses reduction • degree-aware scheduling heuristics with compression improves cache misses reduction. In the next section, we discuss about the consequences of cache misses reduction and cache references reduction. Live Journal Dataset. Remember that with one thread, Cn-order was the best in time reduction (compared to the other graph ordering heuristics).
Figures 21, 22, 23 , 24 (gotten with Live Journal dataset) and figures 25, 26, 27, 28 (gotten with Orkut dataset) represent time reduction due to graph ordering heuristics (figures 21, 22, 25, 26) and degree-aware scheduling heuristics (figures 23, 24, 27, 28). We add at each figure a curve that shows the reduced time gotten with compression.
Figures 21, 22 22 Figures 21, 22, 23 , 24 (gotten with Live Journal dataset) and figures 25, 26, 27, 28 (gotten with Orkut dataset) represent time reduction due to graph ordering heuristics (figures 21, 22, 25, 26) and degree-aware scheduling heuristics (figures 23, 24, 27, 28). We add at each figure a curve that shows the reduced time gotten with compression.
INTRODUCTION The Research Topic on "Knowledge Discovery and Data Exploitation" aims at promoting interdisciplinary research in computational linguistics and in Natural Language Processing (NLP) applied to the fields of Bibliometrics, Scientometrics, and Information Retrieval. It is a follow-up of our previous Research Topic: "Mining Scientific Papers: NLP-enhanced Bibliometrics"  (Atanassova et al. , 2019). The processing of scientific texts, which includes the analysis of citation contexts but also the task of information extraction from scientific papers for various applications, has been the object of intensive research during the last decade. This has become possible thanks to two factors. The first one is the growing availability of scientific papers in full text and in machine-readable formats together with the rise of the Open Access publishing of papers on online platforms such as ArXiv, Semantic Scholar, CiteSeer, or PLOS.
The second factor is the relative maturity of open source tools and libraries for natural language processing that facilitate text processing (e. g. , Spacy, NLTK, Mallet, OpenNLP, CoreNLP, Gate, CiteSpace). As a result, a large number of experiments have been conducted by processing the full text of papers for citation context analysis, but also summarization and recommendation of scientific papers. This Research Topic aims to discuss novel approaches that focus on the processing and exploitation of data extracted from scientific literature. In particular, the possibility to enrich metadata by the full-text processing of papers offers new fields of investigation that are related to the representation of data and the production of knowledge by the aggregation of data from multiple documents.
continues the series of two papers that were published on the NLP4NLP corpus in our previous Research Topic. This new paper uses similar methods, but adds to the dataset 5 more years of publications, between 2016 and 2020. Research in the field of Speech and Language Processing during these years has been intense and some significant evolution in the Research Topics can be observed. The analysis of the dataset shows that large communities have shifted their research to novel topics such as Neural Networks and Word Embeddings. This, together with the acceleration of the publication process and the growth in the use of language resources, account for some important transformations in this field of research. The authors provide a thorough analysis of the dataset that shows these phenomena.
CONCLUSION The topic of mining scientific papers, and more broadly text mining methods used in the fields of NLP-enhanced Bibliometrics and knowledge discovery, generate much interest from the community. At the moment of publication of this editorial, the two Research Topics on Mining Scientific Papers Vol 1 NLP-enhanced Bibliometrics 1 and Vol 2 Knowledge Discovery and Data Exploitation 2 have attracted more than 99,000 and 24,000 views respectively. 1 https://www. frontiersin. org/research-topics/7043/mining-scientific-papers-nlpenhanced-bibliometrics 2 https://www. frontiersin. org/research-topics/13388/mining-scientific-papersvolume-ii-knowledge-discovery-and-data-exploitation The set of papers that were published in the two Research Topics show various methods that were applied to the full text of articles, or their metadata, references and abstracts.
In this paper, we propose a neoteric and high-efficiency single image dehazing algorithm via contrast enhancement which is called STRASS (Spatio-Temporal Retinex-Inspired by an Averaging of Stochastic Samples) dehazing, it is realized by constructing an efficient high-pass filter to process haze images and taking the influence of human vision system into account in image dehazing principles. The novel high-pass filter works by getting each pixel using RSR and computes the average of the samples. Then the low-pass filter resulting from the minimum envelope in STRESS framework has been replaced by the average of the samples. The final dehazed image is yielded after iterations of the high-pass filter. STRASS can be run directly without any machine learning. Extensive experimental results on datasets prove that STRASS surpass the state-of-the-arts.
The algorithm takes one or more haze images as input, and a clear image can be obtained after a series of high-pass filtering operations. Experiments on the datasets of NTIRE 2019 and NTIRE 2020 show that STRASS outperforms the state-of-the-art, especially in practical applications and is easy to use. The contributions of our work are summed up as follows: 1) We implement a novel contrast enhancement algorithm called STRASS, which is realized by constructing an efficient high-pass filter to process haze images and taking the influence of human vision system into account in image dehazing principles. STRASS can be run directly without any machine learning, so our algorithm is more superior for practical applications. The advantage of STRASS is that it does not require complex training processes and high-cost computer equipment.
2) We design novel high-pass filters to recover image structures. The purpose of filter design is to enhance some useful image details while reducing most of the noises in image. The innovation of the proposed high-pass filter is that it improves the original low-pass filter and replaces the minimum envelope value in STRESS framework with a novel mean filter. 3) We compare the effectiveness and practical application of STRASS on both NTIRE 2019 and NTIRE 2020 datasets. The experiment results demonstrate that our proposed algorithm is superior to the state-of-the-arts in these two aspects.
Experiment and Results In this section, we verify the effect of the STRASS algorithm in dehazing tasks, and test its generalization ability on different datasets. The datasets we use mainly come from the NTIRE challenge. Then we show some details of the network training process. Finally, we compare the STRASS algorithm with the state-of-art methods, including Berman et al. [5] , Cho et al. [6] , He et al. [1] , Meng et al. [10] , Tarel et al. [8].
Implementation Details In our work, we run the code on a Linux machine with Ubuntu OS 18. 04 LTS, with 8GB of memory, we conduct dehazing experiments on C++, and only one CPU is used in the training process (Intel Core i7, 2. 40 GHZ x 4). We use O(MNn) to represent the complexity of the STRASS, where M represents the number of samples, N denotes the number of iterations, and the total number of pixels contained in the image is n. All the parameters in this article first set the initial threshold. In the experiment, we tested a series of images to find out the approximate range of the optimal values of these parameters through subjective evaluation and objective indicators.
Qualitative Evaluation on Dataset We firstly show the results of the qualitative experiment on NTIRE2020 dataset in Fig. 2. The images from left to right represent the haze image and the results of the five state-of-the-art dehazing algorithms: GT, Berman et al. [5] , Cho et al. [6] , He et al. [1] , Meng et al. [10] , Tarel et al. [8]  results and ours. Some dehazing methods  [1, 8, 10]  rely on prior knowledge that cannot accurately estimate the density of haze. If some areas of the image are over-processed, the dehazing results will be darker than original image, or get a brighter one. [6]  showed color distortion in some areas, and there is still some remaining haze. In the experiment, we noticed that STRASS will amplify white noise when the haze is inhomogeneous and dense. By observing the image as shown in Fig.