Reviewerly/exHarmony huggingface dataset:
SubCrime: Counterfactual Data Augmentation for Target Sentiment Analysis

The goal of Target Sentiment Analysis (TSA) is to predict the users’ sentiment towards specific targets from review sentences. However, the predicting results may not perform well due to the sparsity of training data. Data augmentation is a fruitful technology to alleviate the influence of imperfect training data, which obtains additional data by transforming the original samples. Unfortunately, there is hardly a particular data augmentation approach for TSA. To address this problem, in this paper, we propose a low-cost and effective data augmentation method called SubCrime, which constructs auxiliary sentences in two steps: Substitute and disCriminate. The former aims to substitute reasonable targets for the observed sentences through the masked language model, while the latter discriminates the restructured sentences via the constrained objective. SubCrime does not require extra knowledge and tedious manual annotation. We design SubCrime to answer the key counterfactual question: “If the review target in the sentence changed, would its sentiment be different ?”. Experiments show SubCrime improves on average 2 to 4 points in F1 scores on four datasets compared to methods without enhancement. Moreover, SubCrime also outperforms other data augmentation methods widely used in other Natural Language Processing (NLP) tasks.
MTHGAT: A Neural Multi-task Model for Aspect Category Detection and Aspect Term Sentiment Analysis on Restaurant Reviews

Aspect-based sentiment analysis (ABSA) is one of the fundamental task in text sentiment analysis which aims to analyze the sentiment polarity of a specific aspect (terms or categories) in a given sentence. Aspect category detection (ACD) and aspect term sentiment analysis (ATSA) are both sub-task of ABSA. However, most of the previous methods regard them as two separate tasks, and ignore the potential relationship between them. In this paper, we propose a multi-task hierarchical graph attention network (MTHGAT) which contains two levels of graph attention networks, a sentence level and a document level. The former is built based on the reshaped dependency parse tree, the latter is built to gather information between sentences in a document. Extensive experiments are conducted on the two restaurant datasets in SemEval-2015 and SemeEval-2016. The results show that our proposed model performs better than most baseline methods.
SSMFRP: Semantic Similarity Model for Relation Prediction in KBQA Based on Pre-trained Models

Pre-trained Relation Extraction (RE) models are widely employed in relation prediction in Knowledge Base Question Answering (KBQA). However, pre-trained models are usually optimized and evaluated on datasets (e.g. GLUE) which contain various Natural Language Processing (NLP) tasks except a RE task. As a result, it is difficult to select a best pre-trained model for relation prediction unless we evaluate all available pre-trained models on a relation prediction dataset. As the Semantic Similarity (SS) task in GLUE is similar to a RE task, a Semantic Similarity Model for Relation Prediction (SSMFRP) is proposed in this paper to convert a RE task in relation prediction to a SS task. In our model, a relation candidate in a RE model is converted into the corresponding question which contains a relation candidate. Then a modified SS model is employed to find the best-matched relation. Experimental results show that the effectiveness of our proposed model in relation prediction is related to the effectiveness of the original pre-trained model in SS and GLUE. Our model achieves an average accuracy of 91.2% with various pre-trained models and outperforms original models by an average margin of 1.8% with the similar training cost. In addition, further experiments show that our model is robust to abnormal input and outperforms original models by an average margin of 1.0% on datasets of abnormal input.
A Unified View on Self-Organizing Maps (SOMs) and Stochastic Neighbor Embedding (SNE)

We propose a unified view on two widely used data visualization techniques: Self-Organizing Maps (SOMs) and Stochastic Neighbor Embedding (SNE). We show that they can both be derived from a common mathematical framework. Leveraging this formulation, we propose to compare SOM and SNE quantitatively on two datasets, and discuss possible avenues for future work to take advantage of both approaches.
Temporal Graph Transformer for Dynamic Network

Graph neural networks (GNN) have received great attention in recent years due to their unique role in mining graph-based data. Although most work focuses on learning low-dimensional node representation in static graphs, the dynamic nature of real-world networks makes temporal graphs more practical and significant. Continuous-time dynamic graph (CTDG) is a general approach to express temporal networks in fine granularity. Owing to the high time consumption in training and inference, existing CTDG-based algorithms capture information from 1-hop neighbors, ignoring the messages from high-order neighbors, which inevitably leads to model degradation. To overcome the challenge, we propose Temporal Graph Transformer (TGT) to efficiently capture the evolving and semantic information from high-order neighborhoods in dynamic graphs. The proposed TGT consists of three modules, i.e., update module, aggregation module, and propagation module. Different from previous works that aggregate messages layer by layer, the model captures messages from 1-hop and 2-hop neighbors in a single layer. In particular, (1) the update module learns from messages derived from interactions; (2) the aggregation module aggregates 1-hop temporal neighbors to compute node embedding; (3) the propagation module re-updates the hidden state of temporal neighbors to introduce 2-hop information. Experimental results on three real-world networks demonstrate the superiority of TGT in efficacy and efficiency.
Parallel Relationship Graph to Improve Multi-Document Summarization

Multi-document summarization (MDS) is an important branch of information aggregation. Compared with the single-document summary (SDS), MDS faces the problem of large search space, redundancy and complex cross-document relation. In this paper, we propose an abstractive MDS model based on Transformer, which considers the parallel information of documents with the graph attention network. During decoding, our model can utilize graph information to guide the summary generation. In addition, combined with the pre-trained language model, our model can further improve the summarization performance. Empirical results on the Multi-News and WikiSum datasets show that our model brings substantial improvements over several strong baselines, and ablation studies verify the effectiveness of our key mechanisms.
Continual Learning by Task-Wise Shared Hidden Representation Alignment

One of the goals of machine learning is learning a sequence of tasks more naturally. Continual learning imitates the real learning mode of human beings and continually learns new knowledge without forgetting old knowledge at the same time. In the past decades, considerable attention has been paid to this learning method. However, avoiding forgetting old knowledge in the process of learning new knowledge remains an ongoing challenge due to catastrophic forgetting. Therefore, it is desirable to exploit a new method to improve the stability in a continual learning scenario. In this paper, we specifically focus on the shared feature extraction between two consecutive tasks. To this end, we explore a continual learning paradigm by using the task-wise shared hidden representation alignment module, which contrasts shared representations from the current task and shared representations from reconstruction pseudo samples of previous tasks. Our proposed TSHRA model grasps similarity features provided by the alignment module to learn shared representations more consummately when the model is learning the current task. To verify our proposed model, we conduct experiments on Split-MNIST and Fashion-MNIST. The experimental results show that our proposed TSHRA’s performance is outstanding which justify that the alignment module has a positive effect on learning shared representations among different tasks for the continual learning scenario.
AMMUNIT: An Attention-Based Multimodal Multi-domain UNsupervised Image-to-Image Translation Framework

We address the open problem of unsupervised multimodal multi-domain image-to-image (I2I) translation using a generative adversarial network with attention mechanism. Previous works, such as CycleGAN, MUNIT, and StarGAN2 are able to translate images among multiple domains and generate diverse images, but they often introduce unwanted changes to the background. In this paper, we propose a simple yet effective attention-based framework for unsupervised I2I translation. Our framework not only translates solely objects of interests and leave the background unaltered, but also generates images for multiple domains simultaneously. Unlike recent studies on unsupervised I2I with attention mechanism that require ground truth for learning attention maps, our approach learns attention maps in an unsupervised manner. Extensive experiments show that our framework is superior than the state-of-the-art baselines.
Multiscale Spatial and Temporal Learning for Human Motion Prediction

Human motion prediction is a classic computer vision task that has numerous applications in the field of autonomous driving, human-computer interaction and motion synthesis. Recently, a large number of motion prediction methods employ graph convolutional networks to encode the spatial relationships among joints within a pose, which ignore the impact of adjacent poses in the encoding process, resulting in unsatisfactory extraction of spatial information. Moreover, the recurrent neural network is difficult to capture long-term temporal dependencies. To solve these challenges, we design a prediction model that consists of two crucial components. Firstly, A novel spatio-temporal graph convolution module utilizes multi-graph convolution layers to capture the spatial correlation information of the current frame and adjacent frames simultaneously. Secondly, a multiscale temporal encoding module consists of a series of temporal convolutions with different dilation rates to establish various temporal relationships between poses at both short-term and long-term temporal distance. We evaluate the proposed method on the largest 3D human motion capture dataset (i.e., Human 3.6 Million). Extensive experimental results show that our model achieves state-of-the-art performance in both short and long-term predictions compared with existing methods.
A Long and Short Term Preference Model for Next Point of Interest Recommendation

The goal of next point of interest (POI) recommendation is to predict users’ next location by analyzing their check-in sequence. The next POI recommendation has attracted more research attention as an important application in location based social networks. Some next POI recommendation algorithms learn spatial and time difference for modelling user check-in sequence based on recurrent neural network (RNN) and its variants. However, these methods do not take into account the difference between the users’ long-term and short-term preferences, and cannot analyze the users’ historical preferences accurately without considering the properties of POI about category and comment comprehensively. In order to improve the precision of recommendation result, a long and short term preference model combined with contextual information (LSPMC) is proposed for next POI recommendation in this paper. Specifically, we take geographical information, temporal influence, the category and rating comment of POI into account to construct the input embedding vector. Three long and short term memory (LSTM) networks and attention mechanism are designed for modeling user long term preference. RNN model is used to implement user short term preference. The probability of POIs that user will visit next is obtained based on the long term preference and short term preference. The experimental results on two real datasets demonstrate that the proposed method performs better than other baseline methods in terms of four commonly evaluation indicators.
Adaptive Knowledge Distillation for Efficient Relation Classification

Knowledge Distillation (KD) methods are widely adopted to reduce the high computational and memory costs incurred by large-scale pre-trained models. However, there are currently no researchers focusing on KD’s application for relation classification. Although directly leveraging traditional KD methods for relation classification is the easiest way, it should not be neglected that the concept of “relation” is highly ambiguous so machine learning models are likely to give uncertain predictions of relations. Moreover, the label smoothing progress in KD would result in further uncertainty in supervision, leading to bad student model performances. In this work, we propose a confusion-based KD method through which the uncertainty in supervision can be adaptively adjusted based on how confused teacher models are in relation classification. In addition, we propose a new knowledge adjustment method called logit replacement, which can adaptively fix teachers’ mistakes to avoid genetic errors. We conducted comprehensive experiments on the basis of the SemEval-2010 Task 8 relation classification benchmark. Test results demonstrate the effectiveness of the proposed methods.
Long-Horizon Route-Constrained Policy for Learning Continuous Control Without Exploration

Imitation Learning and Offline Reinforcement Learning that learn from demonstration data are the current solutions for intelligent agents to reduce the high cost and high risk of online Reinforcement Learning. However, these solutions have struggled with the distribution shift issue with the lack of exploration of the environment. Distribution shift makes offline learning prone to making wrong decisions and leads to error accumulation in the goal-reaching continuous control tasks. Moreover, Offline Reinforcement Learning generates additional bias while learning from human demonstration data that does not satisfy the Markov process assumptions. To alleviate these two dilemmas, we present a Long-horizon Route-constrained (LHRC) policy for the continuous control tasks of goal-reaching. At a state, our method generates subgoals by long-horizon route planning and outputs actions based on the subgoal constraints. It can constrain the state space and action space of the agent. And it can correct trajectories with temporal information. Experiments on the D4RL benchmark show that our approach achieves higher scores with state-of-the-art methods and enhances performance on complex tasks.
Multi-task Alignment Scheme for Span-level Aspect Sentiment Triplet Extraction

Aspect Sentiment Triplet Extraction (ASTE) aims to extract aspect terms, opinion terms, and the corresponding sentiments from the target sentence, which is a universal subtask in the field of aspect-based sentiment analysis. Recent models achieve considerable performance in a joint learning manner but still suffer from significant limitations, such as the inability to capture span representations accurately and redundant triples are extracted due to ignoring syntactic dependencies. To tackle these issues, we propose a novel multi-task alignment scheme (MAS) for the ASTE task to tackle these issues. Particularly, we explore the aspect/opinion semantic composition module to obtain the aspect candidate set and the opinion candidate set at span-level. Moreover, the pointer-specific tagging strategy is designed to characterize internal associations between aspects and opinions by incorporating syntactic dependencies. Furthermore, a triplet alignment scheme is designed to generate triplets by aligning aspects and corresponding opinions with the position of specific pointers. Experimental results on four benchmark datasets demonstrate that our proposed model outperforms other representative ones in terms of Precision, Recall and Macro-Averaged F1.
Integrating Label Semantic Similarity Scores into Multi-label Text Classification

The target of multi-label text classification (MLTC) is to annotate texts with the most relevant labels from a candidate label set. In MLTC models, representing a true label as a one-hot vector is a common practice. However, the inadequate one-hot representation may ignore the similar predicted scores between semantically relevant labels that share the same text information in the same document. To overcome this challenge and improve the performance of multi-label text classification tasks, we propose a model for learning label embedding to calculate the semantic Similarity Scores between label embeddings and document dense vector. Then adaptively integrate them with the One-hot Vector predicted score (SSOV). SSOV has two parts, one using a transformer model to extract the text information to build a one-hot vector to improve the performance of each label predict score. The other part feeds the document word embeddings to a deep learning network and generates a dense vector to learn label embeddings, which can explicitly measure the similarity score with the document’s dense vector by cosine similarity. Meanwhile, we construct a new loss function that can Adaptively Weighted above two parts’ score to calculate loss value (AWLoss). Furthermore, AWLoss can also assign weights to labels to eliminate the limitation of BCELoss. SSOV outperforms the state-of-the-art methods on all three benchmark datasets.
Progressive Image Restoration with Multi-stage Optimization

With the recent development of deep learning technology, researchers have achieved significant results in small-scale image inpainting. However, when the missing area is large, undesirable artifacts and noise are introduced into the inpainting area. Hence, we present a multi-stage progressive image inpainting framework based on the well-known generative adversarial network(GAN) to solve this problem. In our MOPR-GAN method, generator uses a progressive inpainting module(PIM) and an image optimization module(IOM), while discriminator combines a patchGAN with an attention mechanism and a globalGAN. The PIM can gradually repair the image loss area and generate an attention map simultaneously. The IOM optimizes the details of the generated image based on the information provided by the attention map. The discriminator can capture the local continuity and universal global features of the image better. When comparing the test results with the latest research, the model showed a significant effect in both qualitative and quantitative analyses.
SNNet: Specific Node Network of Human Parsing

Recent works have made significant progress in human parsing by exploiting the hierarchical human structure. However, their modeling of human body is incomplete and how to generate adaptive contextual features for various shapes and sizes of human parts is still a challenge. In order to model more complete and specific human body, this paper proposes a novel Specific Node Network (SNNet), comprised of three modules: the node extract module, the low-high module and multi-set union module. The node extract module aims at extracting the features that represent the characteristics of each human part from backbone as a node for each human part. The low-high module aims at modeling the complex relations (including the inner and inter relations) between the human parts from the low aspect, normal aspect and high aspect, where the normal aspect means the direct modeling of connected human parts, the low aspect means the hidden modeling of human parts via less human parts than the normal aspect, and the high aspect means the hidden modeling of human parts via more human parts than the normal aspect, and we model the relations from two aspects: inner and inter. Multi-set union module aims at modeling the human parts more specific by considering the human parts as different sets such as the part set, half-body set and full-body set. Our experiments show that the node extract module can generate the node with better representations of human parts. The low-high module and multi-set union module can generate adaptive contextual features for various sizes and shapes of human parts.
Learning Hierarchical Graph Convolutional Neural Network for Object Navigation

The goal of object navigation is to navigate an agent to a target object using visual input. Without GPS and the map, one challenge of this task is how to locate the target object in the unseen environment, especially when the target object is not in the field of view. Previous works use relation graphs to encode the concurrence relationships among all the object categories, but these relation graphs are usually too flat for the agent to locate the target object efficiently. In this paper, a Hierarchical Graph Convolutional Neural Network (HGCNN) is proposed to encode the object relationships in a hierarchical manner. Specifically, the HGCNN consists of two graph convolution blocks and a graph pooling block, which constructs the hierarchical relation graph by learning an area-level graph from the object-level graph. Consequently, the HGCNN based framework enables the agent to locate the target object efficiently in the unseen environment. The proposed model is evaluated in the AI2-iTHOR environment, and the performance of object navigation shows a significant improvement.
Contrast and Aggregation Network for Generalized Zero-shot Learning

Generalized zero-shot learning (GZSL) aims to classify both seen classes and unseen classes by training seen classes and the semantic information shared by seen and unseen classes. Some methods try to synthesize unseen samples by unseen semantic information; however, the performance of the GZSL is limited as the distribution of synthesized samples and of real samples is misalignment. To tackle this problem, we propose a Contrast and Aggregation Network (CAN). CAN employs a feature and contrast module to align the real and synthesized feature, which maps the real and synthesized visual feature to a new space and aggregates both real and synthesized features to the centroid of real visual features. Since the semantic information may be lost during the generating and mapping stage, we designed a semantic feature alignment module to project the mapped feature into semantic space. We evaluate our proposed CAN on four benchmark datasets to demonstrate that our method is against the state-of-the-art GZSL methods. The source code is available at https://github.com/fesfa/CAN .
A Multi-stack Denoising Autoencoder for QoS Prediction

In the era of network information overload, personalized service recommendation is paid more and more attention by researchers, and Quality of Service (QoS) is a key criterion for service selection and recommendation. QoS is described as a non-functional attribute of Web services, so the prediction of QoS with high precision is an important means to realize personalized recommendation. In this paper, a Multi-stack Denoising Autoencoder (MSDAE) model is proposed to predict QoS. Firstly, the location information and the improved Jaccard similarity coefficient are used to obtain the trusted similar neighbors of users and services. Partially missing values of sparse user-service QoS matrix are pre-populated and users’ preference informations are filled. Then, MSDAE is used to learn and train the processed QoS matrix to predict the missing QoS. Finally, the experimental results on WSDream-dataset1 show that the proposed method predicts QoS with higher accuracy than other prediction methods.