Reviewerly/exHarmony huggingface dataset:
Why Do Big Data and Machine Learning Entail the Fractional Dynamics?

Chapter 2 explores the fundamental question of why big data and machine learning inherently involve fractional dynamics. The investigation unfolds through an exploration of fractional calculus (FC) and fractional-order thinking (FOT), shedding light on their relevance in understanding the intricate dynamics of complex systems. The chapter delves into the concept of complexity and inverse power laws (IPLs), establishing a connection between heavy-tailed distributions and fractional dynamics. Various heavy-tailed distributions are examined in the context of their implications for machine learning in diverse applications. The discussion extends to the interplay between big data, variability, and fractional calculus, incorporating topics such as the Hurst parameter, fractional Gaussian noise (fGn), etc. The chapter progresses to elucidate the concept of optimal machine learning and optimal randomness, distinguishing between derivative-free methods and gradient-based methods. Additionally, it explores the contributions of the control community to machine learning. A detailed case study on optimal randomness for Stochastic Configuration Network (SCN) with heavy-tailed distributions is presented, encompassing an introduction, SCN with heavy-tailed probability density functions (PDFs), a regression model, parameter tuning, and a practical application involving MNIST handwritten digit classification.
Multi-stage Legal Instrument Grammatical Error Correction via Seq2Edit and Data Augmentation

Legal instruments serve as the carrier for judicial organs and citizens to exercise legal rights and enjoy legal benefits. Therefore, the accuracy of the textual content is highly demanded. In order to use machine-intelligent text proofreading technology to assist judicial personnel in automatically detecting and correcting errors in legal instruments, we propose a multi-stage Chinese Grammatical Error Correction model (MsCGEC). The model consists of two components: a grammatical error correction model and a spelling check model. The grammatical error correction model is based on the neural sequence-to-edit model–GECToR and enhanced by pinyin information which focuses on correcting grammatical errors, while the spelling check model based on the MacBERT4CSC model focuses on correcting spelling errors. In addition, we propose a rule-based data augmentation strategy to generate a large amount of training data that is as similar as possible to legal instrument error texts, alleviating the data shortage problem. Our model achieves 67.281 correct F1 and 77.028 dectect F1 on CAIL2022 legal instruments proofreading track stage2 testing set.
An Application of Co-plot Analysis: A Multidimensional Scaling Data Visualization

Exploratory graphs are a crucial element of statistical analysis and models that help us identify and interpret data patterns. As one example of exploratory graphs, multidimensional scaling (MDS) allows the visualization of large and complex datasets in a reduced dimensional space, while preserving pairwise differences and similarities between data points. Co-plot analysis extends the capabilities of traditional MDS and enables a multifaceted view of relationships between variables from different datasets. To develop a co-plot graph, the analyst starts out by standardizing the dataset and by generating a special coefficient (MADCC – median absolute deviation correlation coefficient) that subjects the dataset to MDS-embedding and helps to determine how the dataset fits the surface. In the current paper, we demonstrate how co-plot analysis can be applied to describe and compare the behavioral patterns of ecological communities and species across multiple sites and environmental contexts. We demonstrate the use of non-metric multidimensional scaling (NMDS) and robust multidimensional scaling (RMDS) techniques to visualize dissimilarities and distances of the observations in a reduced dimensional space. We also explain the use of Shepard graphs as a second technique to visualize categorical variables in a reduced-dimensional space, with points closer together indicating higher similarity. Shepard graphs also allow us to assess the goodness of fit, which is indicated by the strength of a positive linear relationship. For our application, the co-plots graphs are useful to visualize a complete set of co-relations between the species of different communities. It also allows us to plot how every factor affects the number of species against the mean and against the median and to determine the best approach: A co-plot graph against the mean may be influenced by the shape of the distribution, especially if the data is skewed, whereas a co-plot graph against the median is typically less affected by the shape of the distribution. The demonstration and discussion in the current paper highlight the benefits of co-plot graphs in general, as well as three different techniques in particular: Shepard analysis, NMDS, and RMDS techniques.
Robust Sentiment Classification Based on the Backdoor Adjustment

Deep NLP models are correlation-based learning, which has a critical limitation of over-fitting over spurious features and shows poor generalization capability in the out-of-distribution (OOD) setting. Existing methods encourage the model to exploit causal features and exclude spurious correlations through Counterfactually Augmented Data (CAD) and feature regularization. However, those methods still face challenges due to the low quality of counterfactual generation and the high cost of annotation. This paper proposes an improved method for OOD generalization motivated by causal inference tools. Specifically, taking the topic of the text as the confounder of the input and the label, the model fits the causal correlation between the representations and the label through the backdoor adjustment to alleviate the exploitation of the spurious correlations. The proposed method is evaluated on the counterfactual adversarial test set (movie review text) and the challenging test set with synonym perturbation (financial news text) provided in the previous work. The experimental results show that this method improves the OOD generalization capability of the sentiment classification models in these two attacks while preserving the predictive ability, especially in the case of long text.
R-TES: Regularized Template Style for Generative Joint Relational Triple Extraction

Joint Relational Triple Extraction (RTE) is an important task in the field of information extraction. With the development of the pre-trained language models, the sequence-to-sequence (seq2seq) approaches have become one of the promising methods for this task, utilizing a predefined template to convert the relational triples into a structure target sequence, which can be easily decoded as relational triples. However, most existing seq2seq studies focus on the improvement of methods but ignore that template styles also have impacts on performance. Inspired by this idea, we first explore the effects of different template styles on performance and find that some template styles can help generate models to achieve better performance. Based on the above findings, we argue that different template styles lead to various understandings of the relation triple. Therefore, we propose Regularized template style (R-TES) to improve the performance of a main template by reducing the gap between it and other selected templates. Specifically, R-TES uses the pre-trained language model to select the templates with kullback-leibler (KL) divergence. Then, we further reduce the gap between the main template and these selected templates by minimizing KL divergence. Experimental results show that our method outperforms state-of-the-art methods on the publicly available dataset.
Watch Your English Language: Text Normalization from Written Expressions to Spoken Forms

Based on the existing research results for speech-to-text and methods based on statistical modelling, machine learning, etc., this paper applied the feature engineering method based on machine learning to speech-totext, converting English text from written expressions into spoken forms. First, I preprocessed the data, including calculating the file structure and contenthad, missing values as well as reformating features. Next I had a look at normalized sentences and overview visualizations. And then I researched the impact of the normalization. Moreover, I researched token classes and their text normalization, which compose of the "PLAIN" (modifications), the "PUNCT" (punctuation), numerical classes, the "LETTERS" (acronyms anhd initials), the "VERBATIM" (special symbols) as well as the "ELECTRONIC" (websites) class, with the most frequent transformations of each class. Meanwhile, I did a next-neighbour analysis to prove that context matters a lot, inlcuding token class overview - previous vs next, treemap overviews relative percentages, exploration of all neighbour relations. Then, I analysed the transformation statistics, such as sentence length and classes, as well as normalized tokens and where to found them. On the basis of these foundings, I did the feature engineering composing of string length, class-specific token distance, numeric tokens and what’s in a name. I also did a case study - a tale of four twos, including the class "CARDINAL", "ORDINAL", "LETTERS", and "PLAIN". Ultimately, I evaluated the results with 11.11 MB size test data, getting a extremely high accuracy score of 99.776Additional Key Words and Phrases: Text normalization, Text-to-speechFeature engineering, Tok
Sentiment and Interest Detection in Social Media using GPT-based Large Language Models

In the ever-expanding realm of social media, deciphering sentiments and identifying relevant topics from various textual content is challenging. This paper comprehensively investigates sentiment analysis and interest/topic detection using cutting-edge language models, ChatGPT3.5 and gpt4All. The methodology encompasses data collection, meticulous text pre-processing, innovative prompt design, and the exploration of zero-shot, one-shot, and few-shot learning techniques. We unveil the nuances of model performance in sentiment analysis and interest/topic detection through a detailed comparative analysis. Our findings highlight the power of ChatGPT3.5 in achieving a substantial accuracy enhancement in sentiment analysis compared to gpt4All. Moreover, we delve into the intricacies of interest detection, demonstrating the complexities of linguistic structures and model biases. We offer a holistic view of social entities’ preferences by categorizing topics into distinct domains. A web portal developed using Google’s Flutter SDK facilitates the visualization of user-friendly sentiment and interest outcomes. This research contributes to the understanding of sentiment analysis and interest detection and underscores the evolving capabilities of AI and NLP in navigating the dynamic landscape of social media content.
Entity relation extraction based on Biaffine model with embedded context features

The extraction of entities and relationships from unstructured text is not only a critical issue in information extraction, but also an essential component of constructing knowledge graphs. Most existing models for relation extraction first extract all subjects, and then proceed to extract objects and relations based on the identified subjects. This method is highly dependent on the extraction of subjects, and the quality of subject extraction significantly impacts the extraction of objects and relations. During relation processing, most models only use the simplest classifier, with the difference lying in the input for relation processing, which sometimes leads to the mutual relationship between the subject and the object being overlooked.To avoid overlooking the inherent features of input sentences, we generate three distinct token representation sequences for subject, object, and relation at the encoding layer. Additionally, combining explicit injection of context features from the encoding layer, entity extraction in both directions can mutually promote each other, aiming to obtain as many entity pairs as possible. This approach aims to address the limitations imposed by the relationship being subject to the extracted subject. While most existing models use a simple classifier for handling relationships, our model employs the Biaffine model combined with an attention mechanism to assign all potential relationships for each entity pair. Its advantage over other simple classifiers is maintaining a matrix for each relation, accurately modeling the features of relations, and its probability calculation mechanism can precisely explore the interaction between subjects and objects.We evaluated the proposed model on the NYT and WebNLG datasets. Extensive experimental results demonstrate the high effectiveness of the proposed model, achieving significant performance across all datasets.
Research on Tibetan-Chinese Neural Machine Translation Integrating Statistical Method

In recent years, with the emergence of deep learning methods, Neural Machine Translation has become a new research direction of machine translation. Due to the scarcity of digital resources in Tibetan, there is only a small-scale Tibetan-Chinese bilingual parallel corpus to train translation models. In the case of insufficient resources, the performance of Neural Machine Translation decreases significantly. In this regard, considering that the word alignment results in Tibetan-Chinese Statistical Machine Translation are better, and the alignment information in Tibetan-Chinese Neural Machine Translation model is significantly different from that in Tibetan-Chinese Statistical Machine Translation, this paper proposes a Tibetan-Chinese Neural Machine Translation method that integrates statistical methods. We using Statistical Machine Translation method to generate word alignment information of Tibetan-Chinese parallel corpus to supervise the training process of Tibetan-Chinese Neural Machine Translation model to make the model achieve more accurate translation and alignment effect. Experimental results show that this method can effectively improve translation quality in low resource environment.
GPT Rotational Position Embedding for Length Extrapolation

Since the introduction of the GPT model as the mainstream model for dialog generation, a hot issue is how to enable the model to extend the prediction length and generate longer dialog texts while the training context length remains constant. Rotary incorporates absolute position information in the form of relative position encoding, an approach that has been shown to be advantageous for extrapolating on transformer models. However, in the actual extrapolation process, when the prediction length is too long, its perplexity climbs sharply. In order to solve the above problems, this paper proposes a method of scaling frequency basis. In contrast to the previous manual test for selecting the optimal parameter ɑ, this method allows the model to dynamically train the parameters based on the input sequence length to obtain the optimal ɑ. Using this method, we demonstrate that the GPT model can effectively extrapolate to contexts that are longer than its original pre-training length. In addition, in order to verify the effectiveness of this method in practical applications, this paper applies it to the field of medical dialog generation, which is a more complex scenario, and the experimental results demonstrate that the extrapolation length is several times the length of the training context, which greatly enhances the extrapolation performance compared to the original absolute position coding.
A Lexicon Enhanced Chinese Long Named Entity Recognition Using Word-Aware Attention

In recent years, due to the rapid growth of space science and utilization research of China Manned Space Engineering, a considerable amount of technical documents and web data have been produced. Named entity recognition (NER) plays a vital role in extracting valuable information from these resources. However, the presence of numerous long named entities, which consist of complex, specialized terms and diverse phrase combinations, poses significant challenges for existing methods in identifying them correctly. To resolve this issue, we introduced two modules, SkipWord-Lattice and Word-Aware Attention, to improve the widely used lexical enhancement model in Chinese NER. SkipWord-Lattice reduces the overlap, redundancy, and confusion of word tokens, while Word-Aware Attention enhances the semantic interaction between character tokens and word tokens, improving the model’s comprehension of word tokens. Collectively, these modules significantly increase the model’s ability to identify long named entities. Moreover, using relevant corpus data publicly available in the field of space science and utilization of China Manned Space Engineering, we built a Chinese NER dataset, named SSUIE-NER, comprising rich entity types and a substantial number of long named entities. Experimental results indicate that, our method significantly improves the recognition of long named entities, outperforming other state-of-the-art (SOTA) approaches on SSUIE-NER and other three benchmarks in Chinese NER.
A Pedestrian Attribute Recognition Method Based On Cross-modal Fusion

Pedestrian Attribute Recognition (PAR), propelled by advancements in computer vision and deep learning, involves predicting attributes to describe pedestrians, including gender, age, clothing, and posture. Recent research has unveiled the oversight of valuable textual information within attribute annotations by many traditional methods. To address this issue, we proposed a cross-modal fusion model integrating image-textual features to enhance recognition capabilities. The model primarily consists of three parts: an image feature extraction module, a textual processing module, and a cross-modal fusion module, each serving its specific function. The dataset utilized in this paper is PA-100K which is the largest PAR dataset to date. Extensive experiments were conducted to demonstrate the performance disparity between the cross-modal approach and the only-visual method, confirming the necessity of integrating cross-modal fusion in pedestrian attribute recognition.
Reinforcement Learning in Natural Language Processing: A Survey

Reinforcement learning (RL) is a powerful technique for learning from data and feedback, but its effective application to natural language processing (NLP) tasks remains an open question. Consequently, this paper first introduces the general concepts of RL and the common approaches. Subsequently, we review the task construction settings and the application of RL for various NLP problems, such as machine translation, dialogue system, and text generation. Finally, we discuss some promising research directions and challenges of RL in NLP. We hope that our work can provide a comprehensive overview and inspire more research on this promising yet challenging topic.
Admission Graph based Healthcare Insurance Patient Fraudster Detection

Healthcare insurance fraud has been a pressing issue in countries all over the world, resulting in large financial losses and adverse effects on human health. However, patient fraudsters are trying to mislead auditors by imitating normal patients, and thus can avoid being detected by the available fraudster detection techniques. In this paper, we utilize a heterogeneous admission graph to capture the complex relationships among patients, admissions and hospitals that the patients has visited. We explore the interactions between different types of nodes and develop an iterative algorithm to identify suspicious patient fraudsters. Meanwhile, we propose an effective method to compute the credibility of patients, the reality of admission and the authority of hospitals. Different from available fraduste detection approaches, we consider not only details of admissions. Our model also discuss the interactions between different types of entities and able to find more subtle patient fraudsters.Then we evaluate the framework with real healthcare insurance dataset and the results show the effeciency of our method.
Experimental design of emotion recognition based on machine learning

In this paper, a wrist bracelet was used to measure the subjects' heart rate signals and skin electrical signals, and three types of emotions (positive, neutral and negative) were induced by designing an experimental environment. The heart rate and skin electrical signals corresponding to emotions were collected through the bracelet, and the recognition accuracy of the three types of emotions was finally obtained through data preprocessing and the support vector machine model in machine learning. At present, how to build an experimental environment that can fully induce subjects' emotions is a major difficulty in emotion recognition, so this paper provides a specific emotion recognition environment design and detailed steps. Meanwhile, through a large number of experiments, the kernel function in the support vector machine model, namely Gaussian kernel function, is determined, and grid search is introduced to search for hyperparameter C. To get the optimal parameters and results. A total of 20 people were measured in this experiment. The experimental results obtained by SVM model were positive: the recognition accuracy of skin electrical signal was 0.8422, the recognition accuracy of heart rate signal was 0.8345, and the recognition accuracy of skin electrical signal and heart rate signal feature fusion was 0.8832. Negative: the recognition accuracy of skin electrical signal is 0.9812, the recognition accuracy of heart rate signal is 0.9385, and the recognition accuracy of skin electrical signal and heart rate signal feature fusion is 0.9902. Neutral: the recognition accuracy of skin electrical signal was 0.6403, the recognition accuracy of heart rate signal was 0.5308, and the recognition accuracy of skin electrical signal and heart rate signal feature fusion was 0.5438
An improved under-sampling algorithm based on density Peak clustering

Aiming at the noise in unbalanced data sets and the imbalance between classes, an undersampling algorithm based on density peak clustering is proposed. First of all, the density peak clustering is used for all the majority of samples to eliminate the noise points; then the sampling weight is assigned according to the different sparsity of each cluster after clustering, and the sampling number is obtained; finally, undersampling is carried out in each cluster. The proposed undersampling algorithm is compared with five commonly used sampling algorithms on six unbalanced data sets. The experimental results show that this method can effectively avoid the generation of noise in unbalanced data sets, overcome the imbalance between classes, and achieve better classification performance on the whole.