Reviewerly/exHarmony huggingface dataset:
SumBART - An Improved BART Model for Abstractive Text Summarization

In this project we introduce SumBART - an improved version of BART with better performance in abstractive text summarization task. BART is a denoising autoencoder model used for language modelling tasks. The existing BART model produces summaries with good grammatical accuracy but it does have certain amount of factual inconsistency. This issue of factual inconsistency is what makes text summarization models unfit to use in many real world applications. We are introducing 3 modifications on the existing model that improves rouge scores as well as factual consistency.
AFFSRN: Attention-Based Feature Fusion Super-Resolution Network

Recent, the single image super-resolution (SISR) methods are primarily based on building more profound and more complex convolutional neural networks (CNN), which leads to colossal computation overhead. At the same time, some people introduce Transformer to low-level visual tasks, which achieves high performance but also with a high computational cost. To address this problem, we propose an attention-based feature fusion super-resolution network (AFFSRN) to alleviate the network complexity and achieve higher performance. The detail capture capability of CNN makes its global modeling capability weak, we propose the Swin Transformer block (STB) instead of convolution operation for global feature modeling. Based on STB, we further propose the self-attention feature distillation block (SFDB) for efficient feature extraction. Furthermore, to increase the depth of the network with a small computational cost and thus improve the network’s performance, we propose the novel deep feature fusion group (DFFG) for feature fusion. Experimental results show that this method achieves a better peak signal-to-noise ratio (PSNR) and computation overhead than the existing super-resolution algorithms.
HYCEDIS: HYbrid Confidence Engine for Deep Document Intelligence System

Measuring the confidence of AI models is critical for safely deploying AI in real-world industrial systems. One important application of confidence measurement is information extraction from scanned documents. However, there exists no solution to provide reliable confidence score for current state-of-the-art deep-learning-based information extractors. In this paper, we propose a complete and novel architecture to measure confidence of current deep learning models in document information extraction task. Our architecture consists of a Multi-modal Conformal Predictor and a Variational Cluster-oriented Anomaly Detector, trained to faithfully estimate its confidence on its outputs without the need of host models modification. We evaluate our architecture on real-wold datasets, not only outperforming competing confidence estimators by a huge margin but also demonstrating generalization ability to out-of-distribution data.
MCHPT: A Weakly Supervise Based Merchant Pre-trained Model

In the last few years, pre-trained models (PTMS) have become the foundation of the downstream natural language processing tasks. The large scale corpus with abundant latent semantical knowledge in the pre-training tasks makes the model learn the semantics of language. However, the general mask language model is not suitable for corpus with a lot of irrelevant and noisy semantics such as merchant information. In our merchant system, we have collected millions of merchants information, including merchant names and address. To deal with these kind of short and noisy corpus and incorporate multi-source external information into the model, in this paper, we propose a weakly supervise based merchant pre-trained model called MCHPT model to learn representations of merchant-language. The model is pre-trained by our designed pre-training tasks on a large scale weakly supervised real-world merchant dataset. The experiment results present that our model outperforms the state-of-the-art pre-trained language models in four downstream merchant related tasks.
Data Representation and Clustering with Double Low-Rank Constraints

High-dimensional data are usually drawn from an union of multiple low-dimensional subspaces. Low-rank representation (LRR), as a multi-subspace structure learning method, uses low rank constraints to extract the low-rank subspace structure of high-dimensional data. However, LRR is highly dependent on the multi-subspace property of the data itself, which is easily disturbed by some higher intensity global noise. Thus, a data representation learning method (UV-LRR) capable of handling both sparse global noise and locally structured sparse noise with dual low-rank constraints on the input data and the representation coefficients is proposed in this paper. The sparse global noise and the local structured noise are constrained by using $$l_1$$ and $$l_{2,1}$$ norm, respectively, to separate a large amount of noise latent in the data. The UV decomposition and nuclear norm minimization is also used to compress the rank of input data and representation coefficients to extract the multi-subspace structure underlying the observed data. The experimental results show that the proposed method has superior performance in the clustering experiments for each dataset.
A Multi-label Feature Selection Method Based on Feature Graph with Ridge Regression and Eigenvector Centrality

In multi-label learning, instances with multiple semantic labels suffer from the impact of high feature dimensionality. The goal of multi-label feature selection is to process multi-dimensional and multi-label data and keep relevant information in the original data. However, varieties of existing feature ranking-based multi-label feature selection methods do not take into account the relationship between features. Methods based on feature graph can present and utilize the association between features but still have imperfections. Among them, the exploration of the correlation between features and labels is not sufficient, and there is no efficient use of the association between features to evaluate features. In this paper, a multi-label feature selection method based on feature graph with ridge regression and eigenvector centrality is proposed. Ridge regression is used to learn a valid representation of feature label correlation. The learned correlation representation is mapped to a graph to efficiently display and use feature relationships. Eigenvector centrality is used to evaluate nodes in the graph to obtain scores for features. The effectiveness of the proposed method is testified according to three evaluation metrics (Ranking loss, Average precision, and Micro-F1) on four datasets by comparison with seven state-of-the-art multi-label feature selection methods.
Sketch Image Style Transfer Based on Sketch Density Controlling

Sketch Image Style Transfer aims to stylize the sketch images from given art images to make them look like the same artistic styles while still persevering the original sketch contents. Previous methods generally disentangle the content and style of reference image and transfer the style to sketch image. However, the textures or the painting strokes of the reference art image could be a part of content as well as style. It is difficult to decide how much it should be involved for sketch image style transferring. In this paper, we propose a novel sketch image style transfer method to use the sketch density to control it, reaching the common sense that simpler sketch images own richer textures of reference style images after stylization and otherwise the opposite. The proposed model is built upon the general content and style encoding and fused decoding architecture, but adds sketch density extraction to obtain the density level of input sketch image, and uses it to control texture information during transferring. With these special designing, our model is more flexible for inputting multi-density sketch images and reasonable for deciding the degree to transfer the texture styles of reference image. Experimental results on different datasets demonstrate the effectiveness and superiority of our method for sketch image style transfer.
Sequential Three-Way Rules Class-Overlap Under-Sampling Based on Fuzzy Hierarchical Subspace for Imbalanced Data

The imbalanced data classification is one of the most critical challenges in the field of data mining. The state-of-the-art class-overlap under-sampling algorithm considers that the majority nearest neighbors of minority class instances are more prone to class-overlap. When the number of minority instances is small, the instances removed by such methods are not thorough. Therefore, a Sequential Three-way Rules class-overlap undersampling method based on fuzzy hierarchical subspace is proposed, which is inspired by sequential three-way decision. First, the fuzzy hierarchical subspace (FHS) concept is proposed to construct the fuzzy hierarchical subspace structure of the dataset. Then, a sequential three-way rules is constructed to find the equivalent majority instances of the minority instances from the fuzzy hierarchical subspace. We assume that the equivalent majority instances are overlapping instances of the minority class. Finally, in order to preserve the information of the majority instances in the equivalence class, we keep the majority instances with the largest Mahalanobis distance from the center of the equivalence class. Experimental results on 18 real datasets show that S3RCU outperforms or partially outperforms state-of-the-art class-overlap under-sampling methods on two evaluation metrics, F-measure and KAPPA.
Evolving Temporal Knowledge Graphs by Iterative Spatio-Temporal Walks

Predicting facts that occur in the future is a challenging task in temporal knowledge graphs (TKGs). TKGs represent temporal facts about entities and their relations, where each fact is associated with a timestamp. Inspired from the human inference process that predictions are usually made by analyzing relevant historical clues, in this paper, we propose a model based on temporal evolution and temporal graph attention mechanism to infer future facts. Specifically, we construct a node pool to keep the importance of all nodes encountered in the historical search. We learn temporal evolution features and sub-graph structures based on temporal random walks and graph attention networks. Moreover, these sub-graphs are sets of objects with the same subjects and relations as the query. Experiments on five temporal datasets demonstrate the effectiveness of the model compared with the state-of-the-art methods. Codes are available at https://github.com/lendie/SWGAT .
MAR2MIX: A Novel Model for Dynamic Problem in Multi-agent Reinforcement Learning

As a challenging problem in the Multi-Agent Reinforcement Learning (MARL) community, the cooperative task has received extensive attention in recent years. Most current MARL algorithms use the centralized training distributed execution approach, which cannot effectively handle the relationship between local and global information during training. Meanwhile, many algorithms mainly focus on the collaborative tasks with a fixed number of agents without considering how to cooperate with the existing agents when the new agents enter in the environment. To address the above problems, we propose a Multi-agent Recurrent Residual Mix model (MAR2MIX). Firstly, we utilize the dynamic masking techniques to ensure that different multi-agent algorithms can operate in dynamic environments. Secondly, through the cyclic residual mixture network, we can efficiently extract features in the dynamic environment and achieve task collaboration while ensuring effective information transfer between global and local agents. We evaluate the MAR2MIX model in both non-dynamic and dynamic environments. The results show that our model can learn faster than other benchmark models. The training model is more stable and generalized, which can deal with the problem of agents joining in dynamic environments well.
Infinite Label Selection Method for Mutil-label Classification

Mutil-label classification is a machine learning task on a large number of labels where an instance may be associated with multiple class labels simultaneously. Although significant progress achieved, multi-label classification is still challenging due to the high-dimensional label space resulting from the emergence of multiple applications. To this end, dimensionality reduction originally for feature space is also applied to label space via exploiting label correlation information, deriving two kinds of techniques: label embedding and label selection. There have been many successful theories in the field of label embedding, but less attention has been paid to label selection. Infinite feature selection algorithm (Inf-FS) finds the most discriminative features subset. It treats feature subsets as paths in a graph, and by algebraic theory, the values of paths of arbitrary lengths can be evaluated. Letting the lengths of paths go to infinite allows to simplify the computational complexity of the selection process and considers the values of any path (subset) that contains a specific feature. Sorting features provides the feature subset to keep. We can apply this algorithm to label selection by treating label subsets as paths. After executing label selection, we need to design an operator to recover the original label space from the selected one. An effective classifier can be trained on the label subset. Then, we can propagate the predicted value for the selected label subset to the full label set in order to recover the original label space. We apply our model to five benchmark data sets with more than 100 labels. Experimental results show that our method achieves superior classification performance over other state-of-the-art methods, in terms of two performance evaluation metrics (precision and discounted gain@n) for high-dimensional label space.
Aggregating Intra-class and Inter-class Information for Multi-label Text Classification

This paper is concerned with the multi-label text classification (MLTC) task, whose goal is to assign one or more categorical labels to a document. The two critical characteristics of this task are the intra-class and inter-class information. The former means the distribution of samples belonging to the same category, and the latter models the relationships between labels such as label co-occurrence and label hierarchy. However, previous methods focus on either of them instead of combining both. This paper proposes a novel two-branch architecture to capture both intra-class and inter-class information. Experimental results show that considering both information improves the performance of the model. Besides, our model achieves competitive results on two widely used datasets.
Knowledge Transfer from Situation Evaluation to Multi-agent Reinforcement Learning

Recently, multi-agent reinforcement learning (MARL) has achieved amazing performance on complex tasks. However, it still suffers from challenges of sparse rewards and contradiction between consistent cognition and policy diversity. In this paper, we propose novel methods for transferring knowledge from situation evaluation task to MARL task. Specifically, we utilize offline data from a single-agent scenario to train two situation evaluation models for: (1) constructing guiding dense rewards (GDR) in multi-agent scenarios to help agents explore real sparse rewards faster and jump out of locally optimal policies without changing the global optimal policy; (2) transferring a situation comprehension network (SCN) to multi-agent scenarios that balances the contradiction between consistent cognition and policy diversity among agents. Our methods can be easily combined with existing MARL methods. Empirical results show that our methods achieve state-of-the-art performance on Google Research Football which brings together above challenges.
A Temporal Consistency Enhancement Algorithm Based on Pixel Flicker Correction

When the image algorithm is directly applied to the video scene and the video is processed frame by frame, an obvious pixel flickering phenomenon is happened, that is the problem of temporal inconsistency. In this paper, a temporal consistency enhancement algorithm based on pixel flicker correction is proposed to enhance video temporal consistency. The algorithm consists of temporal stabilization module TSM-Net, optical flow constraint module and loss calculation module. The innovation of TSM-Net is that the ConvGRU network is embedded layer by layer with dual-channel parallel structure in the decoder, which effectively enhances the information extraction ability of the neural network in the time domain space through feature fusion. This paper also proposes a hybrid loss based on optical flow, which sums the temporal loss and the spatial loss to better balance the dominant role of the two during training. It improves temporal consistency while ensuring better perceptual similarity. Since the algorithm does not require optical flow during testing, it achieves real-time performance. This paper conducts experiments based on public datasets to verify the effectiveness of the pixel flicker correction algorithm.
Improving Document Image Understanding with Reinforcement Finetuning

Successful Artificial Intelligence systems often require numerous labeled data to extract information from document images. In this paper, we investigate the problem of improving the performance of Artificial Intelligence systems in understanding document images, especially in cases where training data is limited. We address the problem by proposing a novel finetuning method using reinforcement learning. Our approach treats the Information Extraction model as a policy network and uses policy gradient training to update the model to maximize combined reward functions that complement the traditional cross-entropy losses. Our experiments on four datasets using labels and expert feedback demonstrate that our finetuning mechanism consistently improves the performance of a state-of-the-art information extractor, especially in the small training data regime.
MnRec: A News Recommendation Fusion Model Combining Multi-granularity Information

Personalized news recommendations can alleviate information overload. Most current representation matching-based news recommendation methods learn user interest representations from users’ behavior to match candidate news to perform recommendations. These methods do not consider candidate news during user modeling. The learned user interests are matched with candidate news in the last step, weakening the fine-grained matching signals (word-level relationships) between users and candidate news. Recent research has attempted to address this issue by modeling fine-grained interaction matching between candidate news and each news article viewed by the user. Although interaction-based news recommendation methods can better grasp the semantic focus in the news and focus on word-level behavioral interactions, they may not be able to abstract high-level user interest representations from the news users browse. Therefore, it is a worthwhile problem to make full use of the above two architectures effectively so that the model can discover richly detailed cues of user interests from fine-grained behavioral interactions and the abstraction of high-level user interests representations from the news that users browse. To address this issue, we propose MnRec, a framework for fusing multigranularity information for news recommendation. The model integrates the two matching methods via interactive attention and representation attention. In addition, we design a granularity network module to extract news multigranularity information. We also design an RTCN module to implement multilevel interest modeling of users. Extensive experiments on the real news dataset MIND verify the method’s validity.
Variance Reduction for Deep Q-Learning Using Stochastic Recursive Gradient

Deep Q-learning often suffers from poor gradient estimations with an excessive variance, resulting in unstable training and poor sampling efficiency. Stochastic variance-reduced gradient methods such as SVRG have been applied to reduce the estimation variance. However, due to the online instance generation nature of reinforcement learning, directly applying SVRG to deep Q-learning is facing the problem of the inaccurate estimation of the anchor points, which dramatically limits the potentials of SVRG. To address this issue and inspired by the recursive gradient variance reduction algorithm SARAH, this paper proposes to introduce the recursive framework for updating the stochastic gradient estimates in deep Q-learning, achieving a novel algorithm called SRG-DQN. Unlike the SVRG-based algorithms, SRG-DQN designs a recursive update of the stochastic gradient estimate. The parameter update is along an accumulated direction using the past stochastic gradient information, and therefore can get rid of the estimation of the full gradients as the anchors. Additionally, SRG-DQN involves the Adam process for further accelerating the training process. Theoretical analysis and the experimental results on well-known reinforcement learning tasks demonstrate the efficiency and effectiveness of the proposed SRG-DQN algorithm.
Generative Generalized Zero-Shot Learning Based on Auxiliary-Features

Although generalized zero-shot learning (GZSL) has achieved success in recognizing images of unseen classes, most previous studies focused on feature projection from one domain to another, neglecting the importance of semantic descriptions. In this paper, we propose auxiliary-features via GAN(Af-GAN) to deal with the semantic loss problem. Auxiliary-features contain both real features of seen classes and instructive-features mapped by attributes. For the seen classes, we deploy the auxiliary-features to train the generator and regularize the synthesized samples to be close to auxiliary-features. For the unseen classes, we take the instructive-feature mapped by attributes to synthesize unseen class samples for training the final classifier. We construct the constraint between real visual features and instructive-features, and reduce the dependence on the class attributes. Considering that features synthesized from a set of similar attributes overlap each other in visual space, we combine Cosine similarity and Euclidean distance to constrain the distribution of synthesized features. Our method outperforms state-of-the-art methods on four benchmark datasets and also surpasses prior work by a large margin in generalized zero-shot learning.
Argument Classification with BERT Plus Contextual, Structural and Syntactic Features as Text

In Argument Mining (AM), the integral sub-task of argument component classification refers to the classification of argument components as claims or premises. In this context, the content of the component alone does not actually suffice to accurately predict its corresponding class. In fact, additional lexical, contextual, and structural features are needed. Here, we propose a unified model for argument component classification based on BERT and inspired by the new prompting NLP paradigm. Our model incorporates the component itself together with contextual, structural and syntactic features – given as text – instead of the usual numerical form. This new technique enables BERT to build a customized and enriched representation of the component. We evaluate our model on three datasets that reflect a diversity of written and spoken discourses. We achieve state-of-art results on two datasets and 95% of the best results on the third. Our approach shows that BERT is capable of exploiting non-textual information given in a textual form.
Classification by Components Including Chow’s Reject Option

In this paper, we present an approach how reject options are integrated into Classification-by-Component networks. Classification-by-Component networks are relatively accurate classifiers that offer a fair interpretability. Yet, their performance can be increased by allowing rejection of an uncertain classification. We will modify the original Classification-by-Component model so that the adaptive parameters adapt, taking reject options into account.