Reviewerly/exHarmony huggingface dataset:
Please don't answer out of context: Personalized Dialogue Generation Fusing Persona and Context

In realistic conversations, “responses” are closely related to persona and context. However, current personalized dialogue generation methods only focus on the consistency of the persona of responses and yet ignore context coherence, as may produce low-quality responses. To address the issue, we propose a novel model, named PCF (Persona and Context Fusion), which builds two decoders for understanding personality consistency and context coherence respectively on a common encoder-decoder architecture. In this model, an inter-layer attention fusion mechanism is designed for the two decoders to effectively fuse persona and context, and a decoupled way of training is conducted on an additional large-scale non-dialogue inference dataset to enhance the consistent understanding ability of the two decoders. Furthermore, considering that generating responses may be tedious, the ScaleGrad loss function is applied to enhance the diversity of responses. Experimental results on two publicly available datasets show that the dialogues generated by our PCF are significantly higher in quality than strong baselines.
Learning Cooperative Policies with Graph Networks in Distributed Swarm Systems

Deriving efficient cooperative policies in uncertain dynamic environments poses huge challenges for a distributed swarm system due to the limited capability of the agents and the complex dynamics of the environment. In this paper, a novel distributed method based on deep reinforcement learning using observation-level and communication-level graph networks is proposed to learn cooperative policies for the distributed swarm system. Specifically, a relational directed graph attention neural network is designed to model observation-level graphs composed of heterogeneous relational graphs among each agent and each type of entities (e.g., obstacles, other teammates, opponents), for extracting different relational representations. Moreover, a relevant directed graph attention network is presented to cut off the ineffective communication among irrelevant agents, and model a relevant communication topology between each agent and relevant homogeneous neighbor agents as an communication-level graph, for promoting efficient inter-agent interactions. Furthermore, a distributed actor-critic algorithm with full parameter sharing is implemented to learn cooperative swarm policies by using distributed critics, which avoids the curse of dimensionality under a centralized critic. Various simulation results validate the effectiveness and generalization of the proposed method, and demonstrate that the proposed method outperforms existing state-of-the-art methods on coverage and pursuit tasks.
Accelerating Self-Imitation Learning from Demonstrations via Policy Constraints and Q-Ensemble

Deep reinforcement learning (DRL) provides a new way to generate robot control policy. However, the process of training control policy requires lengthy exploration, resulting in a low sample efficiency of reinforcement learning (RL) in real-world tasks. Both imitation learning (IL) and learning from demonstrations (LfD) improve the training process by using expert demonstrations, but imperfect expert demonstrations can mislead policy improvement. Offline to Online reinforcement learning requires a lot of offline data to initialize the policy, and distribution shift can easily lead to performance degradation during online fine-tuning. To solve the above problems, we propose a learning from demonstrations method named Accelerating Self-Imitation Learning from Demonstrations (A-SILfD), which treats expert demonstrations as the agent's successful experiences and uses experiences to constrain policy improvement. Furthermore, we prevent performance degradation due to large estimation errors in the Q-function by the ensemble Q-functions. Our experiments show that A-SILfD can significantly improve sample efficiency using a small number of different quality expert demonstrations. In four Mujoco continuous control tasks, A-SILfD can significantly outperform baseline methods after 150,000 steps of online training and is not misled by imperfect expert demonstrations during training. In addition, our ablation experiments demonstrate the effectiveness of each part of the method.
A Three-Stage Pipeline for Conditional Entity and Relation Extraction

Traditional entity and relation extraction (ERE) task aims at extracting (subject, object, predicate) triplets from natural language texts. However, this ignores the prerequisites for some knowledge. CHIP2022 TASK2 first uses condition spans to support causal triplets. We expand it to the conditional entity and relation extraction task (Cond-ERE), the goal of which is to extract condition spans for conditional triplets on the basis of traditional ERE tasks. Currently, the one-step joint extraction model is almost unable to extract condition spans. Therefore, based on the existing models, we present a three-stage pipeline: first use one model for ERE to jointly extract triplets with all unconditional relations, then for each conditional one use a separate model to extract triplets, and finally use the Machine Reading Comprehension (MRC) model to judge whether these triplets with conditional relations are actually conditional or not and extract condition spans at the same time. Experiment results show that our three-stage pipeline can better extract triplets for multiple relations, and has significant advantages to extract condition spans. Besides, The MRC query template we designed further improves the performance of condition span extraction.
An Improved Trust-Region Method for Off-Policy Deep Reinforcement Learning

Reinforcement learning (RL) is a powerful tool for training agents to interact with complex environments. In particular, trust-region methods are widely used for policy optimization in model-free RL. However, these methods suffer from high sample complexity due to their on-policy nature, which requires interactions with the environment for each update. To address this issue, off-policy trust-region methods have been proposed, but they have shown limited success in highdimensional continuous control problems compared to other off-policy DRL methods. To improve the performance and sample efficiency of trust-region policy optimization, we propose an off-policy trust-region RL algorithm. Our algorithm is based on a theoretical result on a closed-form solution to trust-region policy optimization and is effective in optimizing complex nonlinear policies. We demonstrate the superiority of our algorithm over prior trust-region DRL methods and show that it achieves excellent performance on a range of continuous control tasks in the Multi-Joint dynamics with Contact (MuJoCo) environment, comparable to state-of-the-art off-policy algorithms.
Boosting Aerial Object Detection Performance via Virtual Reality Data and Multi-Object Training

Deep neural network (DNN) architectures, such as R-CNN and YOLO, have demonstrated impressive performance in object detection tasks with respect to both time and accuracy. However, detecting small aerial objects remains challenging from both a data and algorithmic perspective. Collecting and annotating videos to detect small aerial objects is a time-consuming task and can quickly become a burden when new classes of objects are added to a database. In addition, the current objective functions for DNNs are not specifically designed for smaller objects. To address these challenges, we propose a virtual reality (VR) dataset for aerial object detection, which can generate large volumes of small-object aerial data. By combining VR data with real-world data, we are able to improve the performance of aerial object detection. We also introduce a cost function derived from the normalized Wasserstein distance to replace the Intersection-over-Union loss for YOLO. Experimental results demonstrate that the VR dataset and normalized Wasserstein distance improve the performance of state-of-the-art object detection methods in detecting small aerial objects. Our source code is publicly available at https://github.com/naddeok96/yolov7_mavrc
Topic-Aware Modeling for Unsupervised Extractive Summarization

The recent success of extractive summarization depends on the availability of large-scale annotated datasets. Existing unsupervised approaches are mostly directed graph based by combining location information with centrality computing. These methods tend to generate summaries with two problems, one is low topic coverage of the source document called the facet bias problem, and the other is continuous position distribution of extracted sentences called the position bias problem. To solve these problems, we propose the topic-aware centrality-based sum-marization method (TACSUM). Specifically, we employ clustering techniques to explicitly model the topics of the document and define the metrics for topic consistency and topic coverage to improve the performance of summarization. The metric topic consistency is used to guide the calculation of centrality, which solves the position bias problem and achieves a more general effect in different scenarios. We combine the metric topic coverage with the centrality to enhance the topic awareness of the model, which ensures the selected sentences are important and diverse. Numerical experimental results on four datasets show that our method outperforms previous unsupervised methods, especially in long document domains. Extensive analyses confirm that our method can generate high-quality summaries by eliminating position bias and facet bias problems.
Improving Math Word Problems Solver with Logical Semantic Similarity

Math word problems (MWPs) solving has achieved promising results recently. However, most existing methods focus only on learning the mapping function between problem text and the target equation, ignoring the logical semantic similarity among problem texts with same target prototype equation and different topic description. Under the condition of maintaining the logical semantics, modifying only the topic words of a question text will let these models generate completely different equations and answers. In this paper, we propose a novel approach called Logical Semantic Aggregator (LSA) which solves the math word problem efficiently and effectively by extracting logical semantics from problem texts for giving a guidance to equation generation. In addition, a Implicit Constants Predictor (ICP) mechanism is used to predict the corresponding numerical label, which is used as the information prompts to improve the semantic representations of MWPs. Experimental results on the Math23K dataset revealed that our proposed methods can achieve better performance than baselines and higher equation accuracy with the help of implicit numerical label prompts.
Inductive Dummy-based Homogeneous Neighborhood Augmentation for Graph Collaborative Filtering

In the era of information explosion, we urgently need recommendation systems to filter massive amounts of information. Recent advancements in graph neural networks have led to the widespread adoption of graph collaborative filtering algorithms for recommendation systems. Despite their effectiveness, graph collaborative filtering algorithms have several limitations, such as data sparsity and long-tailed distribution. This sparse data with numerous long-tailed nodes can be viewed as an inductive scenario in which models require a robust inductive ability to learn quality representations from sparse data. Inductive graph collaborative filtering methods, such as Pin-SAGE, improve the generalization ability via random neighbor sampling. However, these inductive methods are time-consuming or ineffective in transductive scenarios because of the complicated operations and information loss in random neighbor sampling methods. We propose IDHA, an inductive dummy-based homogeneous neighborhood augmentation method for graph collaborative filtering, to address the issues above. Our method employs dummy nodes connected to all nodes to take advantage of low-degree nodes in graph structure learning. To improve the model's generalization in inductive scenarios, we adopt one-hop random-walk sampling. We propose homogeneous neighborhood augmentation via contrastive learning for inductive graph collaborative filtering. This method exploits contrastive learning in sparse data to its full potential. In addition, we employ a lightweight model design to enhance performance and practicality while reducing model complexity. Extensive experiments on three datasets demonstrate that our method outperforms the state-of-the-art transductive and inductive graph collaborative filtering recommendation methods.
Sparse Gate for Differentiable Architecture Search

Differentiable architecture search is now one of mainstream methods to design the structure of neural networks. It makes the neural architecture search efficient through parameter sharing and differentiable search. However, there are still many significant challenges for designing the optimal architecture, including the phenomenon of skip connection aggregation, excessive memory usage, and large discretization errors. To address these issues, we propose a novel approach called Gate-DARTS where a sparse gating network is introduced to route each input sample to the best <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$k$</tex> operators. This can reduce the memory requirements of the algorithm and break the residual structure. For the issue of discretization error, we then propose a auxiliary loss to enlarge the difference between different operators. We conduct comprehensive experiments on DARTS-like search space, and Gate-DARTS achieves 97.45% test accuracy on CIFAR10 with 0.23 GPU-days, 83.82% on CIFAR100 with 0.28 GPU-days. Our code has been made available at https://github.com/HandingWangXDGroup/Gate-DARTS.
Discriminative Feature Mining and Alignment for Unsupervised Domain Adaptation

Unsupervised domain adaptation (UDA) aims to transfer knowledge from a labeled source domain to an unlabeled target domain by reducing the cross-domain distribution discrepancy. Existing UDA methods mainly learn domain-invariant features by directly aligning the marginal distribution of source and target domains. However, they ignore mining the discriminative information of target data and aligning the cross-domain discriminative features, which may lead to performance degradation. To tackle these two issues simultaneously, we propose a Discriminative Feature Mining and Alignment (DFMA) algorithm for UDA. Specifically, DFMA advances a three-stage Instance-Class-Pseudo (ICP) strategy consisting of instance-level contrastive learning, class-level contrastive learning and pseudo-labeling methods to mine the discriminative structure of target data. Then we conduct the cross-domain discriminative feature alignment by integrating the adversarial-based method which designs a novel conditional domain discriminator, and the discrepancy-based method which leverages the first-order and second-order statistical information of features. Furthermore, we build a reconstruction network to enhance the class-level feature alignment. Extensive experiments on several standard UDA benchmark datasets validate the superiority of our proposed DFMA.
Strip-Cutmix for Person Re-Identification

Person re-identification is a very challenging image retrieval task that aims to match the specific person images from different camera views. Person re-identification model requires a large amount of training data to improve its generalization ability, however the current datasets of person re-identification are not enough that tend to make the model overfit. Therefore, some data augmentation methods are used to increase the amount of training data to improve the generalization ability of the model. Cutmix is a common data augmentation method in the field of deep learning, but it is rarely used in person re-identification task because the triple loss cannot handle the decimal similarity label generated by cutmix. In order to put the cutmix method for data augmentation in person re-identification, we extend the triplet loss that is commonly used in person re-identification to a form which can handle decimal similarity label from the perspective of optimizing image similarity. In addition, we propose Strip-Cutmix data augmentation method, which is more suitable for person re-identification, and discuss the strategies about using Strip-Cutmix in the field of person re-identification. Extensive experiments show that our approach can prevent model overfit and achieve impressive performance on DukeMTMC-ReID, Market-1501 and MSMT17 benchmark datasets.
Transformed-*: A domain-incremental lifelong learning scenario generation framework

Lifelong learning is becoming a popular trend in modern machine learning research. Domain-incremental scenarios are particularly relevant since they closely reflects real-world characteristics. However, one open challenge is the ability to devise scenarios that entail the inherent unpredictability and complexities of domains still unexplored in lifelong learning. To tackle this issue, we propose a framework for domain-incremental scenario generation. The framework enables users to create lifelong learning scenarios using any image dataset, and leveraging a fully customizable pool of transformation functions. We devise an algorithm and criteria that iteratively guide users in evaluating the inclusion of candidate transformation functions to the scenario and in making this decision based on desired outcomes. Experimental results with common lifelong learning strategies and benchmark datasets show that our framework is highly flexible since it allows tweaking complexities and challenges incorporated in generated scenarios. Furthermore, experimental results show that there is a gap between state-of-the-art learning strategies and a proposed upper bound to be exploited in the design of future learning strategies.
Private Data Synthesis from Decentralized Non-IID Data

Privacy-preserving data sharing enables a wide range of exploratory and secondary data analyses while protecting the privacy of individuals in the datasets. Recent advancements in machine learning, specifically generative adversarial networks (GANs), have shown great promise for synthesizing realistic datasets. In this work, we investigate the feasibility of training GAN models privately in practical settings, where the input data is distributed across multiple parties, and local data may be highly skewed, i.e., non-IID. We examine centralized private GAN solutions applied at each local party and propose a federated solution that provides strong privacy and is suitable for non-IID data. We conduct extensive empirical analysis with a wide range of non-IID settings and data from different domains. We provide in-depth discussions about the utility of the synthetic data, the privacy risks in terms of membership inference attacks, as well as the privacy-utility trade-off for private solutions.
Least Redundant Gated Recurrent Neural Network

Recurrent neural networks are important tools for sequential data processing. However, they are notorious for problems regarding their training. Challenges include capturing complex relations between consecutive states and stability and efficiency of training. In this paper, we introduce a recurrent neural architecture called Deep Memory Update (DMU). It is based on updating the previous memory state with a deep transformation of the lagged state and the network input. The architecture is able to learn to transform its internal state using any nonlinear function. Its training is stable and fast due to relating its learning rate to the size of the module. Even though DMU is based on standard components, experimental results presented here confirm that it can compete with and often outperform state-of-the-art architectures such as Long Short-Term Memory, Gated Recurrent Units, and Recurrent Highway Networks.
Pseudo Value Network Distillation for High-Performance Exploration

Solving hard exploration tasks with sparse rewards is notoriously challenging in reinforcement learning (RL), which needs to address two key issues simultaneously: exploiting past successful experiences and exploring the unknown environment. Many prior works take expert demonstrations as successful experiences and learn to imitate them directly. However, these demonstrations are often not available in practice. Recently, curiosity-driven RL methods provide intrinsic rewards, encouraging the agent to explore states with high novelty. Nonetheless, they lack a mechanism for leveraging past good experiences effectively. This work presents a Pseudo Value Network Distillation (PVND) framework to balance the RL agent's exploitative and exploratory behaviors effectively and automatically. In particular, PVND learns to set high exploitation bonuses to the critical states in rewarded trajectories from past experiences and high exploration bonuses to the novel states that agents rarely visit during exploration. We theoretically demonstrate that PVND gives larger positive intrinsic rewards to more critical states. Furthermore, PVND automatically finds meaningful and critical hierarchical sub-tasks for agents to accomplish the final goal progressively. Competitive results in several hard exploration sparse reward problems have verified its effectiveness and efficiency.
Curiosity-driven Exploration for Cooperative Multi-Agent Reinforcement Learning

In multi-agent reinforcement learning, exploration is more challenging because of the large state-action space and the requirement of fine cooperation among multiple agents. We extend ICM, a curiosity-driven exploration method for single-agent environments, to the multi-agent setting and propose multi-agent curiosity-driven exploration (MACDE). We define our intrinsic reward with respect to the curiosity for a team of agents as the summation of individual agents' curiosity given by the prediction error in the next state considering other agents' actions. We evaluate MACDE in the Predator-Prey and StarCraft Multi-Agent Challenge. The results show that MACDE worked effectively and learned better policies in both environments.
Dual-Encoder Attention Fusion Model for Aspect Sentiment Triplet Extraction

Aspect sentiment triplet extraction (ASTE) is a crucial sub-task of aspect-based sentiment analysis, which aims to extract each aspect term along with its opinion term and sentiment polarity. Prior works accomplish ASTE by jointly modeling its two sub-tasks, i.e., term extraction and sentiment classification. However, they ignore that different features have different importance to the two sub-tasks, resulting in feature confusion and insufficient feature fusion. To address this, we propose a dual-encoder attention fusion model (DuaIAF) for ASTE, consisting of a term extraction module and a sentiment classification module. First, we adopt a grid tagging scheme to model word-to-word interactions within word pairs. Second, we employ a dual-encoder framework to obtain BERT-style grid multi-features for term extraction and contextualized features for sentiment classification, thus alleviating feature confusion. Third, deep fusion networks are applied to refine word-level and span-level features. A convolution neural network (CNN)-based self-attention network deeply fuses word-level grid multi-features to explore the 2D structure information and long-distance dependency information. Moreover, attention pooling aggregates contextualized features into span-level features, which helps capture span-to-span interactions between aspect term spans and opinion term spans. The experimental results show that our model outperforms previous state-of-the-art methods over 4 English and 2 Chinese datasets in various domains.