Reviewerly/exHarmony huggingface dataset:
BT Expansion: a Sound and Complete Algorithm for Behavior Planning of Intelligent Robots with Behavior Trees

Behavior Trees (BTs) have attracted much attention in the robotics field in recent years, which generalize existing control architectures and bring unique advantages for building robot systems. Automated synthesis of BTs can reduce human workload and build behavior models for complex tasks beyond the ability of human design, but theoretical studies are almost missing in existing methods because it is difficult to conduct formal analysis with the classic BT representations. As a result, they may fail in tasks that are actually solvable. This paper proposes BT expansion, an automated planning approach to building intelligent robot behaviors with BTs, and proves the soundness and completeness through the state-space formulation of BTs. The advantages of blended reactive planning and acting are formally discussed through the region of attraction of BTs, by which robots with BT expansion are robust to any resolvable external disturbances. Experiments with a mobile manipulator and test sets are simulated to validate the effectiveness and efficiency, where the proposed algorithm surpasses the baseline by virtue of its soundness and completeness. To the best of our knowledge, it is the first time to leverage the state-space formulation to synthesize BTs with a complete theoretical basis.
Code Generation from Natural Language with Less Prior Knowledge and More Monolingual Data

Sajad Norouzi, Keyi Tang, Yanshuai Cao. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers). 2021.
Story Ending Generation with Multi-Level Graph Convolutional Networks over Dependency Trees

As an interesting and challenging task, story ending generation aims at generating a reasonable and coherent ending for a given story context. The key challenge of the task is to comprehend the context sufficiently and capture the hidden logic information effectively, which has not been well explored by most existing generative models. To tackle this issue, we propose a context-aware Multi-level Graph Convolutional Networks over Dependency Parse (MGCN-DP) trees to capture dependency relations and context clues more effectively. We utilize dependency parse trees to facilitate capturing relations and events in the context implicitly, and Multi-level Graph Convolutional Networks to update and deliver the representation crossing levels to obtain richer contextual information. Both automatic and manual evaluations show that our MGCN-DP can achieve comparable performance with state-of-the-art models. Our source code is available at https://github.com/VISLANG-Lab/MLGCN-DP.
Rule-Aware Reinforcement Learning for Knowledge Graph Reasoning

Multi-hop reasoning is an effective and explainable approach to predicting missing facts in Knowledge Graphs (KGs).It usually adopts the Reinforcement Learning (RL) framework and searches over the KG to find an evidential path.However, due to the large exploration space, the RL-based model struggles with the serious sparse reward problem and needs to make a lot of trials.Moreover, its exploration can be biased towards spurious paths that coincidentally lead to correct answers.To solve both problems, we propose a simple but effective RL-based method called RARL (Rule-Aware RL).It injects high quality symbolic rules into the model's reasoning process and employs partially random beam search, which can not only increase the probability of paths getting rewards, but also alleviate the impact of spurious paths.Experimental results show that it outperforms existing multi-hop methods in terms of Hit@1 and MRR.
The Authors Matter: Understanding and Mitigating Implicit Bias in Deep Text Classification

It is evident that deep text classification models trained on human data could be biased.In particular, they produce biased outcomes for texts that explicitly include identity terms of certain demographic groups.We refer to this type of bias as explicit bias, which has been extensively studied.However, deep text classification models can also produce biased outcomes for texts written by authors of certain demographic groups.We refer to such bias as implicit bias, of which we still have a rather limited understanding.In this paper, we first demonstrate that implicit bias exists in different text classification tasks for different demographic groups.Then, we build a learningbased interpretation method to deepen our knowledge of implicit bias.Specifically, we verify that classifiers learn to make predictions based on language features that are related to the demographic attributes of the authors.Next, we propose a framework Debiased-TC to train deep text classifiers to make predictions on the right features and consequently mitigate implicit bias.We conduct extensive experiments on three real-world datasets.The results show that the text classification models trained under our proposed framework outperform traditional models significantly in terms of fairness, and also slightly in terms of classification performance.
The statistical advantage of automatic NLG metrics at the system level

Estimating the expected output quality of generation systems is central to NLG.This paper qualifies the notion that automatic metrics are not as good as humans in estimating systemlevel quality.Statistically, humans are unbiased, high variance estimators, while metrics are biased, low variance estimators.We compare these estimators by their error in pairwise prediction (which generation system is better?) using the bootstrap.Measuring this error is complicated: predictions are evaluated against noisy, human predicted labels instead of the ground truth, and metric predictions fluctuate based on the test sets they were calculated on.By applying a bias-variance-noise decomposition, we adjust this error to a noise-free, infinite test set setting.Our analysis compares the adjusted error of metrics to humans and a derived, perfect segment-level annotator, both of which are unbiased estimators dependent on the number of judgments collected.In MT, we identify two settings where metrics outperform humans due to a statistical advantage in variance: when the number of human judgments used is small, and when the quality difference between compared systems is small. 1
Multi-Domain Multi-Task Rehearsal for Lifelong Learning

Rehearsal, seeking to remind the model by storing old knowledge in lifelong learning, is one of the most effective ways to mitigate catastrophic forgetting, i.e., biased forgetting of previous knowledge when moving to new tasks. However, the old tasks of the most previous rehearsal-based methods suffer from the unpredictable domain shift when training the new task. This is because these methods always ignore two significant factors. First, the Data Imbalance between the new task and old tasks that makes the domain of old tasks prone to shift. Second, the Task Isolation among all tasks will make the domain shift toward unpredictable directions; To address the unpredictable domain shift, in this paper, we propose Multi-Domain Multi-Task (MDMT) rehearsal to train the old tasks and new task parallelly and equally to break the isolation among tasks. Specifically, a two-level angular margin loss is proposed to encourage the intra-class/task compactness and inter-class/task discrepancy, which keeps the model from domain chaos. In addition, to further address domain shift of the old tasks, we propose an optional episodic distillation loss on the memory to anchor the knowledge for each old task. Experiments on benchmark datasets validate the proposed approach can effectively mitigate the unpredictable domain shift.
Communication modulation signal recognition based on the deep multi-hop neural network

Automatic modulation classification (AMC) is one of the core technologies in non-cooperative communication. In the complex wireless environment, it is not easy to quickly and accurately recognize the modulation styles of signals by conventional methods. The deep learning method (DLM) can deal with the problem and achieve good effects. In conventional DLMs, the length of input data is fixed. However, the signal length in communication is changing, which may not make full use of the DLMs’ input signal information to improve the recognition accuracy. In this paper, the deep multi-hop convolutional neural network (CNN) is employed to learn the time-domain signal features with different lengths. The proposed network includes the multi-hop connection rate and the receptive field extension scope to dispose of the limitation. The experiment shows that the proposed network can achieve better recognition results under the sparse multi-hop network structure. The reception field extension scope is also conducive to further improve the recognition effects. Finally, the proposed network has shorter training time and smaller parameters, which is more convenient for training the network and deploying in the existing communication system.
Anticipating Future Relations via Graph Growing for Action Prediction

Predicting actions from partially observed videos is challenging as the partial videos containing incomplete action executions have insufficient discriminative information for classification. Recent progress has been made through enriching the features of the observed video part or generating the features for the unobserved video part, but without explicitly modeling the fine-grained evolution of visual object relations over both space and time. In this paper, we investigate how the interaction and correlation between visual objects evolve and propose a graph growing method to anticipate future object relations from limited video observations for reliable action prediction. There are two tasks in our method. First, we work with spatial-temporal graph neural networks to reason object relations in the observed video part. Then, we synthesize the spatial-temporal relation representation for the unobserved video part via graph node generation and aggregation. These two tasks are jointly learned to enable the anticipated future relation representation informative to action prediction. Experimental results on two action video datasets demonstrate the effectiveness of our method.
New technologies in education

This article contains a study on the topic “Digitalization and education: the current state and prospects.” It has analyzed such aspects of digital education as digital literacy, mobile digital education, and the ethics of artificial intelligence in education, VR and AR technologies in digital education, robotics training in educational institutions. Electronic textbooks, digital education software are very useful in development of foreign language learning right now. The purpose of this article is to familiarize with the results of the study on the introduction of digitalization in the educational process. The most attention is paid to its growing role attention in the condition of pandemic, when the processes of learning and teaching have to be changed in form of new material giving and finding. As it turned out the students have obtained better knowledge thanks to new digital approaches in educational process. As for teachers, they were more effective in monitoring student knowledge allowing them more independence. Conclusions have been drawn on the effectiveness of each aspect and digital education in general. Statistics on access to digital technologies, frequency of software use, percentage of digital technology ownership has also identified
Beating Attackers At Their Own Games: Adversarial Example Detection Using Adversarial Gradient Directions

Adversarial examples are input examples that are specifically crafted to deceive machine learning classifiers. State-of-the-art adversarial example detection methods characterize an input example as adversarial either by quantifying the magnitude of feature variations under multiple perturbations or by measuring its distance from estimated benign example distribution. Instead of using such metrics, the proposed method is based on the observation that the directions of adversarial gradients when crafting (new) adversarial examples play a key role in characterizing the adversarial space. Compared to detection methods that use multiple perturbations, the proposed method is efficient as it only applies a single random perturbation on the input example. Experiments conducted on two different databases, CIFAR-10 and ImageNet, show that the proposed detection method achieves, respectively, 97.9% and 98.6% AUC-ROC (on average) on five different adversarial attacks, and outperforms multiple state-of-the-art detection methods. Results demonstrate the effectiveness of using adversarial gradient directions for adversarial example detection.
NCFT: Automatic Matching of Multimodal Image Based on Nonlinear Consistent Feature Transform

Automatic matching of multimodal images remains a critical challenging task in many remote sensing and computer vision applications. Due to significant nonlinear radiation distortions (NRDs) between multimodal images, it is difficult for many traditional feature matching methods which are sensitive to NRD to achieve satisfactory matching performance. To cope with this problem, this letter proposes a novel feature matching method named nonlinear consistent feature transform (NCFT) that is robust to large NRD. There are three main contributions to NCFT. First, we propose a new consistent feature map instead of image intensity for feature point detection and description, the consistent feature map encodes the structure information and provides a rich and robust feature. Second, we propose a mean-residual maximum index map (MR-MIM) for feature description and the MR-MIM is constructed from the Log-Gabor convolution sequence on the consistent feature map. Finally, the structure descriptors are built according to the MR-MIM, and multimodal image matching is achieved by computing the correspondence. The extensive experimental results demonstrate that NCFT can effectively overcome the problem of NRD, NCFT outperforms other state-of-the-art methods and improves the matching accuracy and robustness on different multimodal image datasets.
Entity Guided Question Generation with Contextual Structure and Sequence Information Capturing

Question generation is a challenging task and has attracted widespread attention in recent years. Although previous studies have made great progress, there are still two main shortcomings: First, previous work did not simultaneously capture the sequence information and structure information hidden in the context, which results in poor results of the generated questions. Second, the generated questions cannot be answered by the given context. To tackle these issues, we propose an entity guided question generation model with contextual structure information and sequence information capturing. We use a Graph Convolutional Network and a Bidirectional Long Short Term Memory Network to capture the structure information and sequence information of the context, simultaneously. In addition, to improve the answerability of the generated questions, we use an entity-guided approach to obtain question type from the answer, and jointly encode the answer and question type. Both automatic and manual metrics show that our model can generate comparable questions with state-of-the-art models. Our code is available at https://github.com/VISLANG-Lab/EGSS.
Boosting Image-based Mutual Gaze Detection using Pseudo 3D Gaze

Mutual gaze detection, i.e., predicting whether or not two people are looking at each other, plays an important role in understanding human interactions. In this work, we focus on the task of image-based mutual gaze detection, and propose a simple and effective approach to boost the performance by using an auxiliary 3D gaze estimation task during the training phase. We achieve the performance boost without additional labeling cost by training the 3D gaze estimation branch using pseudo 3D gaze labels deduced from mutual gaze labels. By sharing the head image encoder between the 3D gaze estimation and the mutual gaze detection branches, we achieve better head features than learned by training the mutual gaze detection branch alone. Experimental results on three image datasets show that the proposed approach improves the detection performance significantly without additional annotations. This work also introduces a new image dataset that consists of 33.1K pairs of humans annotated with mutual gaze labels in 29.2K images.
Global Optimisation through Hyper-Heuristics: Unfolding Population-Based Metaheuristics

Optimisation has been with us since before the first humans opened their eyes to natural phenomena that inspire technological progress. Nowadays, it is quite hard to find a solver from the overpopulation of metaheuristics that properly deals with a given problem. This is even considered an additional problem. In this work, we propose a heuristic-based solver model for continuous optimisation problems by extending the existing concepts present in the literature. We name such solvers ‘unfolded’ metaheuristics (uMHs) since they comprise a heterogeneous sequence of simple heuristics obtained from delegating the control operator in the standard metaheuristic scheme to a high-level strategy. Therefore, we tackle the Metaheuristic Composition Optimisation Problem by tailoring a particular uMH that deals with a specific application. We prove the feasibility of this model via a two-fold experiment employing several continuous optimisation problems and a collection of diverse population-based operators with fixed dimensions from ten well-known metaheuristics in the literature. As a high-level strategy, we utilised a hyper-heuristic based on Simulated Annealing. Results demonstrate that our proposed approach represents a very reliable alternative with a low computational cost for tackling continuous optimisation problems with a tailored metaheuristic using a set of agents. We also study the implication of several parameters involved in the uMH model and their influence over the solver performance.
Distinct Label Representations for Few-Shot Text Classification

Few-shot text classification aims to classify inputs whose label has only a few examples.Previous studies overlooked the semantic relevance between label representations.Therefore, they are easily confused by labels that are semantically relevant.To address this problem, we propose a method that generates distinct label representations that embed information specific to each label.Our method is widely applicable to conventional few-shot classification models.Experimental results show that our method significantly improved the performance of few-shot text classification across models and datasets.
How well do you know your summarization datasets?

State-of-the-art summarization systems are trained and evaluated on massive datasets scraped from the web.Despite their prevalence, we know very little about the underlying characteristics (data noise, summarization complexity, etc.) of these datasets, and how these affect system performance and the reliability of automatic metrics like ROUGE.In this study, we manually analyse 600 samples from three popular summarization datasets.Our study is driven by a six-class typology which captures different noise types (missing facts, entities) and degrees of summarization difficulty (extractive, abstractive).We follow with a thorough analysis of 27 state-of-the-art summarization models and 5 popular metrics, and report our key insights: (1) Datasets have distinct data quality and complexity distributions, which can be traced back to their collection process.(2) The performance of models and reliability of metrics is dependent on sample complexity.(3) Faithful summaries often receive low scores because of the poor diversity of references.We release the code, annotated data and model outputs.
Interpreting Neural Networks as Quantitative Argumentation Frameworks

We show that an interesting class of feed-forward neural networks can be understood as quantitative argumentation frameworks. This connection creates a bridge between research in Formal Argumentation and Machine Learning. We generalize the semantics of feed-forward neural networks to acyclic graphs and study the resulting computational and semantical properties in argumentation graphs. As it turns out, the semantics gives stronger guarantees than existing semantics that have been tailor-made for the argumentation setting. From a machine-learning perspective, the connection does not seem immediately helpful. While it gives intuitive meaning to some feed-forward-neural networks, they remain difficult to understand due to their size and density. However, the connection seems helpful for combining background knowledge in form of sparse argumentation networks with dense neural networks that have been trained for complementary purposes and for learning the parameters of quantitative argumentation frameworks in an end-to-end fashion from data.
AugNLG: Few-shot Natural Language Generation using Self-trained Data Augmentation

Natural Language Generation (NLG) is a key component in a task-oriented dialogue system, which converts the structured meaning representation (MR) to the natural language.For large-scale conversational systems, where it is common to have over hundreds of intents and thousands of slots, neither template-based approaches nor model-based approaches are scalable.Recently, neural NLGs started leveraging transfer learning and showed promising results in few-shot settings.This paper proposes AUGNLG, a novel data augmentation approach that combines a self-trained neural retrieval model with a few-shot learned NLU model, to automatically create MR-to-Text data from open-domain texts.The proposed system mostly outperforms the state-ofthe-art methods on the FEWSHOTWOZ data in both BLEU and Slot Error Rate.We further confirm improved results on the FEW-SHOTSGD data and provide comprehensive analysis results on key components of our system.Our code and data are available at https: //github.com/XinnuoXu/AugNLG. In-domain MR-to-Text pairs in the Restaurant domain (Texts are )MR: inform_no_match (kidsallowed = yes) TEXT: I cannot find restaurants with kids allowed.
Probabilistic, Structure-Aware Algorithms for Improved Variety, Accuracy, and Coverage of AMR Alignments

We present algorithms for aligning components of Abstract Meaning Representation (AMR) graphs to spans in English sentences.We leverage unsupervised learning in combination with heuristics, taking the best of both worlds from previous AMR aligners.Our unsupervised models, however, are more sensitive to graph substructures, without requiring a separate syntactic parse.Our approach covers a wider variety of AMR substructures than previously considered, achieves higher coverage of nodes and edges, and does so with higher accuracy.We will release our LEAMR datasets and aligner for use in research on AMR parsing, generation, and evaluation.