Reviewerly/exHarmony huggingface dataset:
Offline Imitation Learning Using Reward-free Exploratory Data

Offline imitative learning(OIL) is often used to solve complex continuous decision-making tasks. For these tasks such as robot control, automatic driving and etc., it is either difficult to design an effective reward for learning or very expensive and time-consuming for agents to collect data interactively with the environment. However, the data used in previous OIL methods are all gathered by reinforcement learning algorithms guided by task-specific rewards, which is not a true reward-free premise and still suffers from the problem of designing an effective reward function in real tasks. To this end, we propose the reward-free exploratory data driven offline imitation learning (ExDOIL) framework. ExDOIL first trains an unsupervised reinforcement learning agent by interacting with the environment, and collects enough unsupervised exploration data during training; Then, a task independent yet simple and efficient reward function is used to relabel the collected data; Finally, an agent is trained to imitate the expert to complete the task through a conventional RL algorithm such as TD3. Extensive experiments on continuous control tasks demonstrate that the proposed framework can achieve better imitation performance(28% higher episode returns on average) comparing with previous SOTA method(ORIL) without any task-specific rewards.
Enhanced Human-Machine Interactive Learning for Multimodal Emotion Recognition in Dialogue System

Emotion recognition has been well researched in mono-modality in the past decade. However, people express their emotion or feelings naturally via more than one modalities like voice, facial expressions, text, and behaviors. In this paper, we propose a new method to model deep interactive learning and dual modalities (e.g., speech and text) to conduct multimodal emotion recognition. An unsupervised triplet-loss objective function is constructed to learn representation of emotional information from speech audio. We extract text emotional feature representation by transfer learning of text-to-text embedding from T5 pre-trained model. Human-machine interaction like user feedback plays a vital role in improve multimodal emotion recognition in dialogue system. Deep interactive learning model is constructed by explicit and implicit feedback. Human-machine interactive learning enhanced transformer model can achieve higher levels of accuracy and precision than their non-interactive counterparts.
Self-attention mechanism-based SAR for YOLO-v3 maritime ships image target detection

In recent years, China's maritime construction has been gradually strengthened, and the security of our territorial waters has become a top priority. In this paper, we propose a self-attentive mechanism-based target detection model for YOLO-v3SAR images, and through experiments, we add a self-attentive mechanism before and after the feature fusion part for target detection, and compare the accuracy, we conclude that adding a self-attentive mechanism before each predicted feature layer can effectively improve the detection accuracy. After adding the self-attention mechanism, the detection accuracy of SSDD dataset increases by 10%, Increased from 84.7 to 94.3%, and that of Ship-dataset dataset increases by 9%, from 79% to 88%. The experiments prove that the improved algorithm model is adapted to SAR image target detection and reaches the advanced level, which provides a new idea for SAR image target detection of maritime ships.
A New Handwritten Essay Dataset for Automatic Essay Scoring with A New Benchmark

The study of algorithms for Automatic Essay Scoring (AES) currently is motivated by textual essay-scoring datasets constructed by anonymous teachers from schools. We propose VisEssay, the first essay-scoring dataset containing handwriting images. VisEssay consists of over 13,000 visual essays originating from 25+ professional in-service teachers whose personal scoring accuracy are recorded by his/her scoring history, together with crowdsourced OCR result per handwriting image. VisEssay differs from the many existing AES datasets because 1) handwriting images are captured from non-native speakers with complementary essay types for existing datasets, 2) teachers scoring these essays are with personal profiles and score accuracy, and 3) corresponding text is checked to keep the consistency. Evaluation of modern algorithms for AES and text classification reveals that the proposed VisEssay is a challenging dataset. In the cause of encouraging a larger community to develop more generalized educational algorithms, we introduce three novel AES systems together with VisEssay and analysis the result as a new benchmark.
Dynamic Graph Representation Based on Temporal and Contextual Contrasting

Dynamic graph representation learning is critical for graph-based downstream tasks such as link prediction, node classification, and graph reconstruction. Many graph-neural-network-based methods have emerged recently, but most are incapable of tracing graph evolution patterns over time. To solve this problem, we propose a continuous-time dynamic graph framework: dynamic graph temporal contextual contrasting (DGTCC) model, which integrates temporal and topology information to capture the latent evolution trend of graph representation. In this model, the node representation is first generated by a self-attention–based temporal encoder, which measures the importance weights of neighbor nodes in temporal sub-graphs and stores them in the contextual memory module. After sampling the node representation from the memory module, the model maximizes the mutual information of the same node that occurred in two nearby temporal views by the contrastive learning mechanism, which helps track the evolutional trend of nodes. In inductive learning settings, the results on four real datasets demonstrate the advantages of the proposed DGTCC model.
Gap Sentences Generation with TextRank for Chinese Text Summarization

Recently Transformer-based pre-trained models have become a new paradigm for text summarization, and one of the milestones is PEGASUS. It suggests the pre-training objective Gap Sentences Generation designed for summarization. However, the ROUGE1-F1-based strategy in Gap Sentences Generation is unfavorable to Chinese text summarization, considering that unigram is not the basic semantic unit of Chinese in most cases. Furthermore, ROUGE1-F1 is based upon the co-occurrences of the unigrams other than distributional semantics such as word or sentence representations. Accordingly, the gap sentences are evaluated on a biased basis and thus the performance of GSG objective pre-training is hampered. In this work, we propose an improved GSG objective that upgrades the ROUGE1-F1-based strategy with TextRank. In TextRank, the texts are first segmented to extract semantically meaningful words and form distributional representations of sentences. The significance of a sentence is assessed in light of its cosine similarity with representations of the neighboring sentences in a graph. GSG with TextRank exhibits remarkable performance gains on the LCSTS dataset, and employment of R-Drop further boosts the improvements.
Hybrid Feature Measurement based on Linear and Nonlinear Nonnegative Matrix Factorization

The nonnegative matrix factorization algorithm is an effective data dimensionality reduction method. The principle is to convert the image into a nonnegative linear combination of low dimensional basis images. Nonnegative matrix factorization can be divided into linear algorithm and nonlinear algorithm. Because of different decomposition theory, linear NMF algorithms mainly extract first-order features of data, while nonlinear NMF algorithms mainly extract high-order features. Most of the current studies only focus on one of the models without combining the two together, which leads to the lack of data features. Therefore it is necessary to integrate the two types of algorithms for research. The paper proposes hybrid feature measurement based on linear and nonlinear nonnegative matrix factorization. The algorithm utilizes the idea of feature fusion. The basis image features of the two algorithms are mixed in the model. Finally a feature similarity measurement is obtained as the measure method. The proposed algorithm has good performance on the public datasets and effectively improves the recognition.
Trust Region Method Using K-FAC in Multi-Agent Reinforcement Learning

A challenging problem in multi-agent reinforcement learning (MARL) is to ensure that the policy converges quickly and is effective with limited computing resources. This paper extends the second-order optimization to MARL using Kronecker-factored approximate curvature (K-FAC) to approximate the natural gradient update. And it solves the challenge of training policy networks in MARL which requires a lot of time and computing costs. We propose a Heterogeneous-agent Trust Region algorithm using K-FAC (HAKTR). Further more, we endow HAKTR with monotonic performance improvement based on the multi-agent advantage decomposition theorem. Our algorithm is evaluated on continuous tasks in the MuJoCo environment. The experimental results show that HAKTR can achieve higher rewards with less computing costs compared to the baselines such as HATRPO and HAPPO. Moreover, HAKTR has good scalability regarding the number of agents and can be applied to large-scale networks.
Regularization Strength Impact on Neural Network Ensembles

In the last decade, several approaches have been proposed for regularizing deeper and wider neural networks (NNs), which is of importance in areas like image classification. It is now common practice to incorporate several regularization approaches in the training procedure of NNs. However, the impact of regularization strength on the properties of an ensemble of NNs remains unclear. For this reason, the study empirically compared the impact of NNs built based on two different regularization strengths (weak regularization (WR) and strong regularization (SR)) on the properties of an ensemble, such as the magnitude of logits, classification accuracy, calibration error, and ability to separate true predictions (TPs) and false predictions (FPs). The comparison was based on results from different experiments conducted on three different models, datasets, and architectures. Experimental results show that the increase in regularization strength 1) reduces the magnitude of logits; 2) can increase or decrease the classification accuracy depending on the dataset and/or architecture; 3) increases the calibration error; and 4) can improve or harm the separability between TPs and FPs depending on the dataset, architecture, model type and/or FP type.
Part-GCNet: Partitioning Graph Convolutional Network for Multi-Label Recognition

During the rapid development of deep learning, the multi-label recognition task has achieved pretty performance. Recently, the emergence of graph convolution network (GCN) has further improved the accuracy of multi-label recognition. However, in the learning process, how to better represent the feature information of labels and innovatively design structures to obtain good recognition performance is still unclear. To solve these problems, we propose a partitioning graph convolutional network framework for multi-label recognition. First, we segregate the computational graph into multiple sub-graphs. Then, we perform batch normalization operation on each output layer, which can further improve the recognition performance of the network. Finally, extensive experiments are carried out on a multi-label PPT dataset, showing that our proposed solution can greatly improve the feature information utilization of labels and improve the recognition performance.
Survey on measures of image similarity applied in image inpainting base on deep learning

The research of image similarity measures is an important content of the computer image processing. It have been widely studied and applied in image inpainting, image super resolution, image matching and other applications. In these application, the choice of image similarity measure has a direct impact on the final results. In order to enable more researchers to explore the image similarity theory and its specific role in different applications, this paper summarizes the research status in this field. This paper first classifies the existing image similarity evaluation methods from two different perspectives. Next, since the performance of image similarity measure is closely related to its application field, so we select the image inpainting field based on depth learning, and illustrate the application of image similarity measurement methods in this field. Then we roughly compare the advantage and disadvantage of these methods. Finally, we gave our conclusions and prospects.
Idea Generation using Transformer Decoder Models

Our work aims to generate new ideas to explore in a specific domain using generative language models. For example, doctors can write about known symptoms as cues to the system, and then the system will generate ideas based on the cues. Similar scenarios can be thought of for other scientific domains. We used transformer-based decoders, especially GPT3-based transformer decoders, as the language models and generators. As the data, we used COVID-19 open research dataset [18]. We finetuned GPT-NEO-125M and GPT-NEO-1.3B models with 125 million and 1.3 billion parameters, respectively. The later model generated more coherent text and could link ideas relevant to the same problem better. We report here our findings with examples generated from our finetuned models.
Learning a Model with the Most Generality for Small-Sample Problems

In machine learning, when the number of available samples is very limited, it constitutes a small-sample problem. Small-sample problem is extremely challenging in learning a good model. As the first study on building the most reliable and generalizable model and providing the most reliable measure of a model's generalization performance, a large number of random sampling sets without or with noise added are created from observation dataset to simulate unseen training sets or unseen test sets respectively, where the former makes it available to consider constituting a model of the most generalization ability, and the latter makes it available to measure generalization performance of a model in a most reliable manner. In modeling process, each training set is used to train a model, and the models are then combined to form global/final model. In evaluation process, a performance metric (such as mean square error) is calculated on each test set, and the model's generalization ability is the most reliably represented by the average value of these metrics. The method is especially suitable to small or even ultra-small sample problems due to its no need of data splitting in cross-validation approaches. Comparative experiments on different datasets show the effectiveness of the proposed method in the case of ultra-small-sample problems: for simulated data, the generalization error is approximately 80% lower than those of the conventional method, and for a typical ultra-small-sample problem, the real data of glomerular filtration rate prediction, it is 46.36%∼83.02% lower than those of many popular models.
θ is all you need: Revisiting SVD in caputuring changes in matrices

Given a series of matrices (e.g., frames in video) vary over time, figuring out the spatially changing regions (e.g., moving objects) is a critical issue both in theory and applications. In this article, we propose a change detection scheme based on E-SVD, a theory uses Givens transformation, that only determined by the rotation angle θ, to reduce the number of parameters representing a matrix after singular value decomposition (SVD) compression. Inspired by the close relationship between SVD and principal component analysis (PCA), we firstly provide the analytical dependence between θ and matrix elements when changes happen, which guarantees the theoretical rationality of our selection of target θ to efficiently capture these changes. Secondly, we present our detection scheme which is implemented to accurately locate the changing regions of a matrix spatially. The proposed methodology is verified using both simulation and empirical data, results of which show its efficiency and effectiveness. In order to clarify the realistic application of this scheme in object detection without supervision, an additional experiment is also conducted using surveillance video to further demonstrate its potential. Our findings in both theory and application give a new perspective of figuring out spatial variation in a matrix, leading to a wider usage of matrix factorization methods in the domain of unsupervised object detection.
The Soul Safety

This Sexual assault has been one of the serious problem across the globe. A lot of people, especially women go through this but unfortunately not many will be able to open up to the public about such incidents. The main goal of this research is to suggest a strategy for integrating machine learning with various algorithms to improve the classification of the many forms of assault actions and construct harassment detection systems. This study offers a view for the better understanding of sexual assault and to detect it with text mining, qualitative, and temporal trend analysis methods that are efficient.
A hybrid model using 2D and 3D Convolutional Neural Networks for violence detection in a video dataset

In the era of Internet of Things(IoT), the data from edge devices, wearables and other sensors have become available easily and can be efficiently used for knowledge extraction. One such form of streaming data is in the form of videos which maybe captured from CCTV devices installed in various locations. While deep learning algorithms such as Convolutional Neural Networks(CNN) are not only effective for image classification but their application can also be extended for classifying videos. Two dimensional(2D) CNN classify images by extracting the spatial features from an image. However, when they are applied to videos, the temporal features of a video may get suppressed leading to a poor accuracy. In this work, we implement a hybrid CNN model, by combining a three dimensional convolutional neural network(3D CNN) and 2D CNN for identifying if the video footage contains violence related activities. Results prove that the hybrid CNN model attains a maximum training accuracy upto 99.93% and validation accuracy of 84.5% for violence recognition from video footages.
Anti Fraud Detection Model Using Deep Learning Approach

Recently, Internet finance has become more and more popular. However, bad debts becamea serious threat to online finance corporations. A commonly used fraud detection models by the traditional monetary companies is logistic regression. In this paper we use dataset consisting large publicloans data of a financial company i.e., Lending Club to check the potential of deep neural networks in fraud detection. Once this dataset is loaded we dealtwith the missing values and data pre-processing. With this pre-processed data, we extracted important features using the XGBoost algorithm and developed a CNN deep neural network to detect loan fraud on the Internet. Extensive experiments were conducted to prove that deep neural networks are superior tocommonly used models. This easy and effective model can give enlightenment for the utilization of deep learning to combat online loan fraud, which can profit the financial engineers of tiny and medium-sized financial corporations.
Audio to Sign Language conversion using Natural Language Processing

People who are deaf never get to enjoy things that hearing folks do, whether it's conversation, playing the game, going to seminars, or participating in virtual meetings, for example. The major challenge they have while interacting among regular people is connection, because every regular person has no concept what sign language is or how it works.. Our project's objective is to help those who have hearing loss develop a system for communication. The spoken message is converted into signs. It converts the spoken word into signs. This system receives voice as inputs, converts the soundtrack into a text, and displays the key symbols used in Indian language using well before graphics or GIFs. The use of this technology makes communication between those who have normal hearing and those who are difficult to hear easier This takes into account how your arms, body, and appearance are combined. Around the universe, there really are 135 different varieties of hand signals. There are numerous different sign languages, including American Sign Language (ASL), Indian Sign Language (ISL), British Sign Language (BSL), Australian Sign Language (Auslan), and many more In this initiative, Indian Sign Language is being used. The deaf community may take advantage of all the activities that hearing people engage in, from social engagement to accessing information, thanks to this technology. NLP is used to pre-process the text (Natural Language Processing). With the help of this technology, hearing-impaired people may participate in a wide range of activities, from socialising to accessing information NLP is used to post the text (Natural Language Processing). This programme converts audio into text, displays Indian sign language, and then accepts speech as input. Accuracy rate for the planned project work is 99%, and Validation accuracy is 96%.
Quantum Vs Classical Computing: a Comparative Analysis

Despite the significant technological advancements, the amount of data to be processed remains enormous, and certain problems remain unsolved or take millions of years to be solved. The advent of quantum computing, with its unique properties, represents a pivotal point in the technological world. Although quantum computers are still under development in the laboratories, research is progressing in parallel with the aim of creating a good basis for any contribution in order to be ready for the commercialization of the first computer. This article clarifies the essential basics required before entering the quantum realm. We present the basic properties of quantum computing based on four fondations and their differences from classical computing.
Resource Allocation and Scheduling of Real-Time Workflow Applications in an IoT-Fog-Cloud Environment

The explosive growth of the Internet of Things (IoT) has led to the emergence of the IoT-fog-cloud continuum, in an attempt to facilitate the real-time processing of IoT data. In such multi-tier environments, it is crucial to adopt an efficient resource allocation and scheduling scheme, in order to provide effective load balancing and timeliness for the real-time workload. A load balancing approach that has been proven to be efficient and effective in traditional distributed environments, is the power of two choices – or <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$d$</tex> choices, in its general form. Only recently has this technique been examined in multi-tier environments, without considering, however, important aspects of such frameworks. To this end, in this paper we propose and investigate three resource allocation and scheduling heuristics for real-time workflow jobs in an IoT-fog-cloud environment. The first strategy, performs exhaustive search at each scheduling step in order to find the most suitable resource in the fog and cloud layers for the workload assignment. On the other hand, the two other policies adopt the power of two choices approach. The simulation results shed light on interesting insights regarding the performance and applicability of each method.