Reviewerly/exHarmony huggingface dataset:
Knowledge-aware Leap-LSTM: Integrating Prior Knowledge into Leap-LSTM towards Faster Long Text Classification

While widely used in industry, recurrent neural networks (RNNs) are known to have deficiencies in dealing with long sequences (e.g. slow inference, vanishing gradients etc.). Recent research has attempted to accelerate RNN models by developing mechanisms to skip irrelevant words in input. Due to the lack of labelled data, it remains as a challenge to decide which words to skip, especially for low-resource classification tasks. In this paper, we propose Knowledge-AwareLeap-LSTM (KALL), a novel architecture which integrates prior human knowledge (created either manually or automatically) like in-domain keywords, terminologies or lexicons into Leap-LSTM to partially supervise the skipping process. More specifically, we propose a knowledge-oriented cost function for KALL; furthermore, we propose two strategies to integrate the knowledge: (1) the Factored KALL approach involves a keyword indicator as a soft constraint for the skip-ping process, and (2) the Gated KALL enforces the inclusion of keywords while maintaining a differentiable network in training. Experiments on different public datasets show that our approaches are1.1x~2.6x faster than LSTM with better accuracy and 23.6x faster than XLNet in a resource-limited CPU-only environment.
OKGIT: Open Knowledge Graph Link Prediction with Implicit Types

Open Knowledge Graphs (OpenKG) refer to a set of (head noun phrase, relation phrase, tail noun phrase) triples such as (tesla, return to, new york) extracted from a corpus using Ope-nIE tools.While OpenKGs are easy to bootstrap for a domain, they are very sparse and far from being directly usable in an end task.Therefore, the task of predicting new facts, i.e., link prediction, becomes an important step while using these graphs in downstream tasks such as text comprehension, question answering, and web search query recommendation.Learning embeddings for OpenKGs is one approach for link prediction that has received some attention lately.However, on careful examination, we found that current OpenKG link prediction algorithms often predict noun phrases (NPs) with incompatible types for given noun and relation phrases.We address this problem in this work and propose OKGIT that improves OpenKG link prediction using novel type compatibility score and type regularization.With extensive experiments on multiple datasets, we show that the proposed method achieves state-of-the-art performance while producing type compatible NPs in the link prediction task.
Class-Incremental Instance Segmentation via Multi-Teacher Networks

Although deep neural networks have achieved amazing results on instance segmentation, they are still ill-equipped when they are required to learn new tasks incrementally. Concretely, they suffer from “catastrophic forgetting”, an abrupt degradation of performance on old classes with the initial training data missing. Moreover, they are subjected to a negative transfer problem on new classes, which renders the model unable to update its knowledge while preserving the previous knowledge. To address these problems, we propose an incremental instance segmentation method that consists of three networks: Former Teacher Network (FTN), Current Student Network (CSN) and Current Teacher Network (CTN). Specifically, FTN supervises CSN to preserve the previous knowledge, and CTN supervises CSN to adapt to new classes. The supervision of two teacher networks is achieved by a distillation loss function for instances, bounding boxes, and classes. In addition, we adjust the supervision weights of different teacher networks to balance between the knowledge preservation for former classes and the adaption to new classes. Extensive experimental results on PASCAL 2012 SBD and COCO datasets show the effectiveness of the proposed method.
OntoGUM: Evaluating Contextualized SOTA Coreference Resolution on 12 More Genres

SOTA coreference resolution produces increasingly impressive scores on the OntoNotes benchmark.However lack of comparable data following the same scheme for more genres makes it difficult to evaluate generalizability to open domain data.This paper provides a dataset and comprehensive evaluation showing that the latest neural LM based end-to-end systems degrade very substantially out of domain.We make an OntoNotes-like coreference dataset called OntoGUM publicly available, converted from GUM, an English corpus covering 12 genres, using deterministic rules, which we evaluate.Thanks to the rich syntactic and discourse annotations in GUM, we are able to create the largest human-annotated coreference corpus following the OntoNotes guidelines, and the first to be evaluated for consistency with the OntoNotes scheme.Outof-domain evaluation across 12 genres shows nearly 15-20% degradation for both deterministic and deep learning systems, indicating a lack of generalizability or covert overfitting in existing coreference resolution models.
Psychological and pedagogical aspects of simulation technology at English lessons for future engineers of the agro-industrial complex

During the period of COVID-19 pandemic when mostly all the universities have transferred to distant learning, the educational effect of many interactive methods partially reduced. However, simulation is underestimated by university professors and scarsely used in the educational process. Despite a profound interest in interactive methods, theoretical and practical issues of simulation still need examination, especially in the context of competence-based education. In present research pedagogical, psychological and linguistic issues of simulation are studied. Though, simulation is often referred to as a method or a game, we consider it to be an educational technology. Contrary to entertaining simulation, educational simulation used as technology usually is more than a case, because it has a structured scenario with a system of rules and prohibitions, which are created with a certain idea of forming specific competences. The practical part of the article contains a description of the empirical study of the simulation technology results used at English lessons at the technical university, which prove a high educational efficiency of simulation technology in the professional development of future engineers of agro-complex.
Differential Privacy for Text Analytics via Natural Text Sanitization

Texts convey sophisticated knowledge.However, texts also convey sensitive information.Despite the success of general-purpose language models and domain-specific mechanisms with differential privacy (DP), existing text sanitization mechanisms still provide low utility, as cursed by the high-dimensional text representation.The companion issue of utilizing sanitized texts for downstream analytics is also under-explored.This paper takes a direct approach to text sanitization.Our insight is to consider both sensitivity and similarity via our new local DP notion.The sanitized texts also contribute to our sanitization-aware pretraining and fine-tuning, enabling privacypreserving natural language processing over the BERT language model with promising utility.Surprisingly, the high utility does not boost up the success rate of inference attacks.
Russian digital education landscape during the current pandemic: is the impact felt?

The paper aims at tracing rapid changes in Russian digital education landscape having been shaken by the pandemic of 2020. To this end the study deals with the bulk of publications contributed by the Russian language authors to the Russian Science Citation Index (RSCI) Database as the fullest national research representation source, which includes the most influential research journals in the educational domain. The main research question that potentiated this study was weather the “shock digitisation” faced by many teaching practitioners has been problematized in recent Russian educational discourse. We rely upon the proposition that quick and dramatic shifts can impede adequate scientific reconceptualization of digitisation phenomenon as practical needs stand first in the priority. To test the proposition, the method of content analysis targeting the titles and abstracts of the papers included into Russian Science Citation Index database has been used. Findings demonstrated insignificant increase in the number of pandemic related publications on education digitisation. The data obtained suggests that the influence of the pandemic is little observed in the issues addressed by the latest publications in the Russian educational journals. The study can be of certain prognostic value for international comparative educationalists and policy makers.
C2C-GenDA: Cluster-to-Cluster Generation for Data Augmentation of Slot Filling

Slot filling, a fundamental module of spoken language understanding, often suffers from insufficient quantity and diversity of training data. To remedy this, we propose a novel Cluster-to-Cluster generation framework for Data Augmentation (DA), named C2C-GenDA. It enlarges the training set by reconstructing existing utterances into alternative expressions while keeping semantic. Different from previous DA works that reconstruct utterances one by one independently, C2C-GenDA jointly encodes multiple existing utterances of the same semantics and simultaneously decodes multiple unseen expressions. Jointly generating multiple new utterances allows to consider the relations between generated instances and encourages diversity. Besides, encoding multiple existing utterances endows C2C with a wider view of existing expressions, helping to reduce generation that duplicates existing data. Experiments on ATIS and Snips datasets show that instances augmented by C2C-GenDA improve slot filling by 7.99 (11.9%↑) and 5.76 (13.6%↑) F-scores respectively, when there are only hundreds of training utterances. Code: https://github.com/Sanyuan-Chen/C2C-DA.
GWLAN: General Word-Level AutocompletioN for Computer-Aided Translation

Computer-aided translation (CAT), the use of software to assist a human translator in the translation process, has been proven to be useful in enhancing the productivity of human translators.Autocompletion, which suggests translation results according to the text pieces provided by human translators, is a core function of CAT.There are two limitations in previous research in this line.First, most research works on this topic focus on sentence-level autocompletion (i.e., generating the whole translation as a sentence based on human input), but word-level autocompletion is under-explored so far.Second, almost no public benchmarks are available for the autocompletion task of CAT.This might be among the reasons why research progress in CAT is much slower compared to automatic MT.In this paper, we propose the task of general word-level autocompletion (GWLAN) from a real-world CAT scenario, and construct the first public benchmark 1 to facilitate research in this topic.In addition, we propose an effective method for GWLAN and compare it with several strong baselines.Experiments demonstrate that our proposed method can give significantly more accurate predictions than the baseline methods on our benchmark datasets.
Domain-Adaptive Pretraining Methods for Dialogue Understanding

Language models like BERT and SpanBERT pretrained on open-domain data have obtained impressive gains on various NLP tasks.In this paper, we probe the effectiveness of domainadaptive pretraining objectives on downstream tasks.In particular, three objectives, including a novel objective focusing on modeling predicate-argument relations, are evaluated on two challenging dialogue understanding tasks.Experimental results demonstrate that domainadaptive pretraining with proper objectives can significantly improve the performance of a strong baseline on these tasks, achieving the new state-of-the-art performances.
Adaptive Pattern-Parameter Matching for Robust Pedestrian Detection

Pedestrians with challenging patterns, e.g. small scale or heavy occlusion, appear frequently in practical applications like autonomous driving, which remains tremendous obstacle to higher robustness of detectors. Although plenty of previous works have been dedicated to these problems, properly matching patterns of pedestrian and parameters of detector, i.e., constructing a detector with proper parameter sizes for certain pedestrian patterns of different complexity, has been seldom investigated intensively. Pedestrian instances are usually handled equally with the same amount of parameters, which in our opinion is inadequate for those with more difficult patterns and leads to unsatisfactory performance. Thus, we propose in this paper a novel detection approach via adaptive pattern-parameter matching. The input pedestrian patterns, especially the complex ones, are first disentangled into simpler patterns for detection head by Pattern Disentangling Module (PDM) with various receptive fields. Then, Gating Feature Filtering Module (GFFM) dynamically decides the spatial positions where the patterns are still not simple enough and need further disentanglement by the next-level PDM. Cooperating with these two key components, our approach can adaptively select the best matched parameter size for the input patterns according to their complexity. Moreover, to further explore the relationship between parameter sizes and their performance on the corresponding patterns, two parameter selection policies are designed: 1) extending parameter size to maximum, aiming at more difficult patterns for different occlusion types; 2) specializing parameter size by group division, aiming at complex patterns for scale variations. Extensive experiments on two popular benchmarks, Caltech and CityPersons, show that our proposed method achieves superior performance compared with other state-of-the-art methods on subsets of different scales and occlusion types.
A Study on the Effect of Parameters for ROS Motion Planer and Navigation System for Indoor Robot

In the past decades, the service robot industry had risen rapidly. The office assistant robot is one type of service robot used to assist officers in an office environment. For the robot to navigate autonomously in the office, navigation algorithms and motion planners were implemented on these robots. Robot Operating System (ROS) is one of the common platforms to develop these robots. The parameters applied to the motion planners will affect the performance of the Robot. In this study, the global planners, A* and Dijkstra algorithm and local planners, Dynamic Window Approach (DWA) and Time Elastic Band (TEB) algorithms were implemented and tested on a robot in simulation and a real environment. Results from the experiments were used to evaluate and compare the performance of the robot with different planners and parameters. Based on the results obtained, the global planners, A* and Dijkstra algorithm both can achieve the required performance for this application whereas TEB outperforms DWA as the local planner due to its feasibility in avoiding dynamic obstacles in the experiments conducted.
Question Generation for Adaptive Education

Intelligent and adaptive online education systems aim to make high-quality education available for a diverse range of students.However, existing systems usually depend on a pool of hand-made questions, limiting how finegrained and open-ended they can be in adapting to individual students.We explore targeted question generation as a controllable sequence generation task.We first show how to fine-tune pre-trained language models for deep knowledge tracing (LM-KT).This model accurately predicts the probability of a student answering a question correctly, and generalizes to questions not seen in training.We then use LM-KT to specify the objective and data for training a model to generate questions conditioned on the student and target difficulty.Our results show we succeed at generating novel, wellcalibrated language translation questions for second language learners from a real online education platform.
Expected Eligibility Traces

The question of how to determine which states and actions are responsible for a certain outcome is known as the credit assignment problem and remains a central research question in reinforcement learning and artificial intelligence. Eligibility traces enable efficient credit assignment to the recent sequence of states and actions experienced by the agent, but not to counterfactual sequences that could also have led to the current state. In this work, we introduce expected eligibility traces. Expected traces allow, with a single update, to update states and actions that could have preceded the current state, even if they did not do so on this occasion. We discuss when expected traces provide benefits over classic (instantaneous) traces in temporal-difference learning, and show that some- times substantial improvements can be attained. We provide a way to smoothly interpolate between instantaneous and expected traces by a mechanism similar to bootstrapping, which ensures that the resulting algorithm is a strict generalisation of TD(λ). Finally, we discuss possible extensions and connections to related ideas, such as successor features.
Demoting the Lead Bias in News Summarization via Alternating Adversarial Learning

In news articles the lead bias is a common phenomenon that usually dominates the learning signals for neural extractive summarizers, severely limiting their performance on data with different or even no bias.In this paper, we introduce a novel technique 1 to demote lead bias and make the summarizer focus more on the content semantics.Experiments on two news corpora with different degrees of lead bias show that our method can effectively demote the model's learned lead bias and improve its generality on out-ofdistribution data, with little to no performance loss on in-distribution data.
Video Paragraph Captioning as a Text Summarization Task

Video paragraph captioning aims to generate a set of coherent sentences to describe a video that contains several events.Most previous methods simplify this task by using groundtruth event segments.In this work, we propose a novel framework by taking this task as a text summarization task.We first generate lots of sentence-level captions focusing on different video clips and then summarize these captions to obtain the final paragraph caption.Our method does not depend on ground-truth event segments.Experiments on two popular datasets ActivityNet Captions and YouCookII demonstrate the advantages of our new framework.On the ActivityNet dataset, our method even outperforms some previous methods using ground-truth event segment labels.
Modeling Event-Pair Relations in External Knowledge Graphs for Script Reasoning

Script reasoning infers subsequent events from a given event chain, which involves the ability to understand relations between events.A human-labeled script reasoning dataset is usually of small size with limited event relations, which highlights the necessity to leverage external eventuality knowledge graphs (KG) consisting of numerous triple facts to describe the inferential relation between events.Existing methods adopt a retrieval and integration paradigm to focus merely on the graph triples that have event overlap with a script, but ignore much more supportive triples in the KG with similar inferential patterns, leading to underexploiting.To fully exploit the KG, we propose a knowledge model to learn the inferential relations between events from the whole eventuality KG and then support downstream models by directly capturing the relation between events in a script.We further present a neural script adapter to extend the knowledge model for inferring the associated relations between an event chain and a subsequent event candidate.We evaluate the proposed approach on a popular multi-choice narrative cloze task for script reasoning and achieve new state-ofthe-art accuracy, compared with baselines either incorporating external KG or not.
Unsupervised Domain Adaptation for Event Detection using Domain-specific Adapters

Due to the multi-dimensional variation of textual data, detection of event triggers from new domains can become a lot more challenging.This prompts a need to research on domain adaptation methods for event detection task, especially for the most practical unsupervised setting.Recently, large transformer-based language models, e.g.BERT, have become essential to achieve top performance for event detection.However, their unwieldy nature also prevents effective adaptation across domains.To this end, this work proposes a Domain-specific Adapter-based Adaptation (DAA) framework to improve the adaptability of BERT-based models for event detection across domains.By explicitly representing data from different domains with separate adapter modules in each layer of BERT, DAA introduces a novel joint representation learning mechanism and a Wasserstein distance-based technique for data selection in adversarial learning to substantially boost the performance on target domains.Extensive experiments and analysis over different datasets (i.e., LitBank, TimeBank, and ACE-05) demonstrate the effectiveness of our approach.
Uncertainty Quantification in CNN Through the Bootstrap of Convex Neural Networks

Despite the popularity of Convolutional Neural Networks (CNN), the problem of uncertainty quantification (UQ) of CNN has been largely overlooked. Lack of efficient UQ tools severely limits the application of CNN in certain areas, such as medicine, where prediction uncertainty is critically important. Among the few existing UQ approaches that have been proposed for deep learning, none of them has theoretical consistency that can guarantee the uncertainty quality. To address this issue, we propose a novel bootstrap based framework for the estimation of prediction uncertainty. The inference procedure we use relies on convexified neural networks to establish the theoretical consistency of bootstrap. Our approach has a significantly less computational load than its competitors, as it relies on warm-starts at each bootstrap that avoids refitting the model from scratch. We further explore a novel transfer learning method so our framework can work on arbitrary neural networks. We experimentally demonstrate our approach has a much better performance compared to other baseline CNNs and state-of-the-art methods on various image datasets.
Neural Bi-Lexicalized PCFG Induction

Neural lexicalized PCFGs (L-PCFGs) (Zhu et al., 2020) have been shown effective in grammar induction.However, to reduce computational complexity, they make a strong independence assumption on the generation of the child word and thus bilexical dependencies are ignored.In this paper, we propose an approach to parameterize L-PCFGs without making implausible independence assumptions.Our approach directly models bilexical dependencies and meanwhile reduces both learning and representation complexities of L-PCFGs.Experimental results on the English WSJ dataset confirm the effectiveness of our approach in improving both running speed and unsupervised parsing performance.