Reviewerly/exHarmony huggingface dataset:
Encoding Human Domain Knowledge to Warm Start Reinforcement Learning

Deep reinforcement learning has been successful in a variety of tasks, such as game playing and robotic manipulation. However, attempting to learn tabula rasa disregards the logical structure of many domains as well as the wealth of readily available knowledge from domain experts that could help "warm start" the learning process. We present a novel reinforcement learning technique that allows for intelligent initialization of a neural network weights and architecture. Our approach permits the encoding domain knowledge directly into a neural decision tree, and improves upon that knowledge with policy gradient updates. We empirically validate our approach on two OpenAI Gym tasks and two modified StarCraft 2 tasks, showing that our novel architecture outperforms multilayer-perceptron and recurrent architectures. Our knowledge-based framework finds superior policies compared to imitation learning-based and prior knowledge-based approaches. Importantly, we demonstrate that our approach can be used by untrained humans to initially provide &gt;80% increase in expected reward relative to baselines prior to training (p &lt; 0.001), which results in a &gt;60% increase in expected reward after policy optimization (p = 0.011).
Exploring Pairwise Relationships Adaptively From Linguistic Context in Image Captioning

For image captioning, recent works start to focus on exploring visual relationships for generating high-quality interactive words (i.e. verbs and prepositions). However, many existing works only focus on semantic level by analysing the feature similarity between objects in the visual domain but ignore the linguistic context included in the caption decoder. When captioning is being carried out, the entity words can be inferred based on visual information of objects. The interactive words representing the relationships between entity words can only be inferred based on high-level language meaning generated in the process of captioning decoding. Such high-level language meaning is called linguistic context, which refers to the relational context between words or phrases in the caption sentences. The linguistic context can be used as strong guidance to explore related visual relationships between different objects effectively. To achieve this, we propose a novel context-adaptive attention module that is strongly driven by the linguistic context from the caption decoder. In this module, a novel design of visual relationship attention is proposed based on a bilinear self-attention model to explore related visual relationships and encode more discriminative features under the linguistic context. To achieve the adaptive process of attending to related visual relationships for generating interactive words or related visual objects for entity words, an attention modulator is integrated as an attention channel controller responding to the changing linguistic context of the caption decoder dynamically. Experimented on MSCOCO dataset, our model achieves promising performances compared with all counterpart models that explore visual relationships.
Dynamic Hybrid Relation Exploration Network for Cross-Domain Context-Dependent Semantic Parsing

Semantic parsing has long been a fundamental problem in natural language processing. Recently, cross-domain context-dependent semantic parsing has become a new focus of research. Central to the problem is the challenge of leveraging contextual information of both natural language queries and database schemas in the interaction history. In this paper, we present a dynamic graph framework that is capable of effectively modelling contextual utterances, tokens, database schemas, and their complicated interaction as the conversation proceeds. The framework employs a dynamic memory decay mechanism that incorporates inductive bias to integrate enriched contextual relation representation, which is further enhanced with a powerful reranking model. At the time of writing, we demonstrate that the proposed framework outperforms all existing models by large margins, achieving new state-of-the-art performance on two large-scale benchmarks, the SParC and CoSQL datasets. Specifically, the model attains a 55.8% question-match and 30.8% interaction-match accuracy on SParC, and a 46.8% question-match and 17.0% interaction-match accuracy on CoSQL.
AutoDropout: Learning Dropout Patterns to Regularize Deep Networks

Neural networks are often over-parameterized and hence benefit from aggressive regularization. Conventional regularization methods, such as dropout or weight decay, do not leverage the structures of the network's inputs and hidden states. As a result, these conventional methods are less effective than methods that leverage the structures, such as SpatialDropout and DropBlock, which randomly drop the values at certain contiguous areas in the hidden states and setting them to zero. Although the locations of dropping areas random, the patterns of SpatialDropout and DropBlock are manually designed and fixed. Here we propose to learn the dropping patterns. In our method, a controller learns to generate a dropping pattern at every channel and layer of a target network, such as a ConvNet or a Transformer. The target network is then trained with the dropping pattern, and its resulting validation performance is used as a signal for the controller to learn from. We show that this method works well for both image recognition on CIFAR-10 and ImageNet, as well as language modeling on Penn Treebank and WikiText-2. The learned dropping patterns also transfers to different tasks and datasets, such as from language model on Penn Treebank to Engligh-French translation on WMT 2014. Our code will be available at: https://github.com/googleresearch/google-research/tree/master/auto_dropout.
Exploring Listwise Evidence Reasoning with T5 for Fact Verification

This work explores a framework for fact verification that leverages pretrained sequence-tosequence transformer models for sentence selection and label prediction, two key sub-tasks in fact verification.Most notably, improving on previous pointwise aggregation approaches for label prediction, we take advantage of T5 using a listwise approach coupled with data augmentation.With this enhancement, we observe that our label prediction stage is more robust to noise and capable of verifying complex claims by jointly reasoning over multiple pieces of evidence.Experimental results on the FEVER task show that our system attains a FEVER score of 75.87% on the blind test set.This puts our approach atop the competitive FEVER leaderboard at the time of our work, scoring higher than the second place submission by almost two points in label accuracy and over one point in FEVER score.
Interp-SUM: Unsupervised Video Summarization with Piecewise Linear Interpolation

This paper addresses the problem of unsupervised video summarization. Video summarization helps people browse large-scale videos easily with a summary from the selected frames of the video. In this paper, we propose an unsupervised video summarization method with piecewise linear interpolation (Interp-SUM). Our method aims to improve summarization performance and generate a natural sequence of keyframes with predicting importance scores of each frame utilizing the interpolation method. To train the video summarization network, we exploit a reinforcement learning-based framework with an explicit reward function. We employ the objective function of the exploring under-appreciated reward method for training efficiently. In addition, we present a modified reconstruction loss to promote the representativeness of the summary. We evaluate the proposed method on two datasets, SumMe and TVSum. The experimental result showed that Interp-SUM generates the most natural sequence of summary frames than any other the state-of-the-art methods. In addition, Interp-SUM still showed comparable performance with the state-of-art research on unsupervised video summarization methods, which is shown and analyzed in the experiments of this paper.
EmailSum: Abstractive Email Thread Summarization

Recent years have brought about an interest in the challenging task of summarizing conversation threads (meetings, online discussions, etc.).Such summaries help analysis of the long text to quickly catch up with the decisions made and thus improve our work or communication efficiency.To spur research in thread summarization, we have developed an abstractive Email Thread Summarization (EMAILSUM) dataset, which contains humanannotated short (<30 words) and long (<100 words) summaries of 2,549 email threads (each containing 3 to 10 emails) over a wide variety of topics.We perform a comprehensive empirical study to explore different summarization techniques (including extractive and abstractive methods, single-document and hierarchical models, as well as transfer and semisupervised learning) and conduct human evaluations on both short and long summary generation tasks.Our results reveal the key challenges of current abstractive summarization models in this task, such as understanding the sender's intent and identifying the roles of sender and receiver.Furthermore, we find that widely used automatic evaluation metrics (ROUGE, BERTScore) are weakly correlated with human judgments on this email thread summarization task.Hence, we emphasize the importance of human evaluation and the development of better metrics by the community. 1Table 8: Examples of high-quality summaries generated by model.Emails are separated by '|||' and some content are omit by '...'.(salience=xx, faithfulness=xx) gives the average human rating for that summary.
Instantaneous Grammatical Error Correction with Shallow Aggressive Decoding

In this paper, we propose Shallow Aggressive Decoding (SAD) to improve the online inference efficiency of the Transformer for instantaneous Grammatical Error Correction (GEC).SAD optimizes the online inference efficiency for GEC by two innovations: 1) it aggressively decodes as many tokens as possible in parallel instead of always decoding only one token in each step to improve computational parallelism; 2) it uses a shallow decoder instead of the conventional Transformer architecture with balanced encoder-decoder depth to reduce the computational cost during inference.Experiments in both English and Chinese GEC benchmarks show that aggressive decoding could yield the same predictions as greedy decoding but with a significant speedup for online inference.Its combination with the shallow decoder could offer an even higher online inference speedup over the powerful Transformer baseline without quality loss.Not only does our approach allow a single model to achieve the state-of-the-art results in English GEC benchmarks: 66.4 F 0.5 in the CoNLL-14 and 72.9 F 0.5 in the BEA-19 test set with an almost 10× online inference speedup over the Transformer-big model, but also it is easily adapted to other languages.Our code is available at https://github.com/AutoTemp/Shallow-Aggressive-Decoding.
Enhancing Label Correlation Feedback in Multi-Label Text Classification via Multi-Task Learning

In multi-label text classification (MLTC), each given document is associated with a set of correlated labels.To capture label correlations, previous classifier-chain and sequenceto-sequence models transform MLTC to a sequence prediction task.However, they tend to suffer from label order dependency, label combination over-fitting and error propagation problems.To address these problems, we introduce a novel approach with multi-task learning to enhance label correlation feedback.We first utilize a joint embedding (JE) mechanism to obtain the text and label representation simultaneously.In MLTC task, a document-label cross attention (CA) mechanism is adopted to generate a more discriminative document representation.Furthermore, we propose two auxiliary label co-occurrence prediction tasks to enhance label correlation learning: 1) Pairwise Label Co-occurrence Prediction (PLCP), and 2) Conditional Label Co-occurrence Prediction (CLCP).Experimental results on AAPD and RCV1-V2 datasets show that our method outperforms competitive baselines by a large margin.We analyze low-frequency label performance, label dependency, label combination diversity and coverage speed to show the effectiveness of our proposed method on label correlation learning.
Virtual Reality (VR) Simulation and Augmented Reality (AR) Navigation in Orthognathic Surgery: A Case Report

VR and AR technology have gradually developed to the extent that they could help operators in the surgical field. In this study, we present a case of VR simulation for preoperative planning and AR navigation applied to orthognathic surgery. The average difference between the preplanned data and the post-operative results was 3.00 mm, on average, and the standard deviation was 1.44 mm. VR simulation could provide great advantages for 3D medical simulations, with accurate manipulation and immersiveness. AR navigation has great potential in medical application; its advantages include displaying real time augmented 3D models of patients. Moreover, it is easily applied in the surgical field, without complicated 3D simulations or 3D-printed surgical guides.
The Development of an Evaluation Model to Assess the Effect of Online English Teaching Based on Fuzzy Mathematics

The evaluation of online English teaching effect involves multiple fuzzy factors. This paper attempts to develop a model that fully considers these factors in evaluating the effect of online English teaching. Firstly, the constraints and weaknesses of online English teaching were enumerated, revealing the key points of improving online English teaching effect. On this basis, several evaluation index systems (EIS) were established for this effect from the perspectives of teachers, students, and administrators. Moreover, an improved multi-attribute fuzzy evaluation model was created for online English teaching effect based on fuzzy mathematics. The proposed model can effectively quantify the effect of online English teaching, and enjoy a good prospect in engineering application.
Semantic Relation-aware Difference Representation Learning for Change Captioning

Change captioning is to describe the difference in a pair of images with a natural language sentence.In this task, the distractors, such as the illumination or viewpoint change, bring the huge challenges about learning the difference representation.In this paper, we propose a semantic relation-aware difference representation learning network to explicitly learn the difference representation in the existence of distractors.Specifically, we introduce a selfsemantic relation embedding block to explore the underlying changed objects and design a cross-semantic relation measuring block to localize the real change and learn the discriminative difference representation.Besides, relying on the POS of words, we devise an attentionbased visual switch to dynamically use visual information for caption generation.Extensive experiments show that our method achieves the state-of-the-art performances on CLEVR-Change and Spot-the-Diff datasets 1 .
Does Head Label Help for Long-Tailed Multi-Label Text Classification

Multi-label text classification (MLTC) aims to annotate documents with the most relevant labels from a number of candidate labels. In real applications, the distribution of label frequency often exhibits a long tail, i.e., a few labels are associated with a large number of documents (a.k.a. head labels), while a large fraction of labels are associated with a small number of documents (a.k.a. tail labels). To address the challenge of insufficient training data on tail label classification, we propose a Head-to-Tail Network (HTTN) to transfer the meta-knowledge from the data-rich head labels to data-poor tail labels. The meta-knowledge is the mapping from few-shot network parameters to many-shot network parameters, which aims to promote the generalizability of tail classifiers. Extensive experimental results on three benchmark datasets demonstrate that HTTN consistently outperforms the state-of-the-art methods. The code and hyper-parameter settings are released for reproducibility.
Domain Adaptation In Reinforcement Learning Via Latent Unified State Representation

Despite the recent success of deep reinforcement learning (RL), domain adaptation remains an open problem. Although the generalization ability of RL agents is critical for the real-world applicability of Deep RL, zero-shot policy transfer is still a challenging problem since even minor visual changes could make the trained agent completely fail in the new task. To address this issue, we propose a two-stage RL agent that first learns a latent unified state representation (LUSR) which is consistent across multiple domains in the first stage, and then do RL training in one source domain based on LUSR in the second stage. The cross-domain consistency of LUSR allows the policy acquired from the source domain to generalize to other target domains without extra training. We first demonstrate our approach in variants of CarRacing games with customized manipulations, and then verify it in CARLA, an autonomous driving simulator with more complex and realistic visual observations. Our results show that this approach can achieve state-of-the-art domain adaptation performance in related RL tasks and outperforms prior approaches based on latent-representation based RL and image-to-image translation.
Robustness to Spurious Correlations in Text Classification via Automatically Generated Counterfactuals

Spurious correlations threaten the validity of statistical classifiers. While model accuracy may appear high when the test data is from the same distribution as the training data, it can quickly degrade when the test distribution changes. For example, it has been shown that classifiers perform poorly when humans make minor modifications to change the label of an example. One solution to increase model reliability and generalizability is to identify causal associations between features and classes. In this paper, we propose to train a robust text classifier by augmenting the training data with automatically generated counterfactual data. We first identify likely causal features using a statistical matching approach. Next, we generate counterfactual samples for the original training data by substituting causal features with their antonyms and then assigning opposite labels to the counterfactual samples. Finally, we combine the original data and counterfactual data to train a robust classifier. Experiments on two classification tasks show that a traditional classifier trained on the original data does very poorly on human-generated counterfactual samples (e.g., 10%-37% drop in accuracy). However, the classifier trained on the combined data is more robust and performs well on both the original test data and the counterfactual test data (e.g., 12%-25% increase in accuracy compared with the traditional classifier). Detailed analysis shows that the robust classifier makes meaningful and trustworthy predictions by emphasizing causal features and de-emphasizing non-causal features.
Understanding and Countering Stereotypes: A Computational Approach to the Stereotype Content Model

Stereotypical language expresses widely-held beliefs about different social categories.Many stereotypes are overtly negative, while others may appear positive on the surface, but still lead to negative consequences.In this work, we present a computational approach to interpreting stereotypes in text through the Stereotype Content Model (SCM), a comprehensive causal theory from social psychology.The SCM proposes that stereotypes can be understood along two primary dimensions: warmth and competence.We present a method for defining warmth and competence axes in semantic embedding space, and show that the four quadrants defined by this subspace accurately represent the warmth and competence concepts, according to annotated lexicons.We then apply our computational SCM model to textual stereotype data and show that it compares favourably with survey-based studies in the psychological literature.Furthermore, we explore various strategies to counter stereotypical beliefs with anti-stereotypes.It is known that countering stereotypes with antistereotypical examples is one of the most effective ways to reduce biased thinking, yet the problem of generating anti-stereotypes has not been previously studied.Thus, a better understanding of how to generate realistic and effective anti-stereotypes can contribute to addressing pressing societal concerns of stereotyping, prejudice, and discrimination.
An efficient authentication and key agreement protocol for IoT-enabled devices in distributed cloud computing architecture

Abstract With the widespread use of Internet of Things and cloud computing in smart cities, various security and privacy challenges may be encountered.The most basic problem is authentication between each application, such as participating users, IoT devices, distributed servers, authentication centers, etc. In 2020, Kang et al. improved an authentication protocol for IoT-Enabled devices in a distributed cloud computing environment and its main purpose was in order to prevent counterfeiting attacks in Amin et al.’ protocol, which was published in 2018. However, We found that the Kang et al.’s protocol still has a fatal vulnerability, that is, it is attacked by offline password guessing, and malicious users can easily obtain the master key of the control server. In this article, we extend their work to design a lightweight pseudonym identity based authentication and key agreement protocol using smart card. For illustrating the security of our protocol, we used the security protocol analysis tools of AVISPA and Scyther to prove that the protocol can defend against various existing attacks. We will further analyze the interaction between participants authentication path to ensure security protection from simulated attacks detailedly. In addition, based on the comparison of security functions and computing performance, our protocol is superior to the other two related protocols. As a result, the enhanced protocol will be efficient and secure in distributed cloud computing architecture for smart city.
A Multifeature Learning and Fusion Network for Facial Age Estimation

Age estimation from face images has attracted much attention due to its favorable and many real-world applications such as video surveillance and social networking. However, most existing studies usually learn a single kind of age feature and ignore other appearance features such as gender and race, which have a great influence on the age pattern. In this paper, we proposed a compact multifeature learning and fusion method for age estimation. Specifically, we first used three subnetworks to learn gender, race, and age information. Then, we fused these complementary features to further form more robust features for age estimation. Finally, we engineered a regression-ranking age-feature estimator to convert the fusion features into the exact age numbers. Experimental results on three benchmark databases demonstrated the effectiveness and efficiency of the proposed method on facial age estimation in comparison to previous state-of-the-art methods. Moreover, compared with previous state-of-the-art methods, our model was more compact with only a 20 MB memory overhead and is suitable for deployment on mobile or embedded devices for age estimation.