pwc-archive/papers-with-abstracts huggingface dataset:
Text Anchor Based Metric Learning for Small-footprint Keyword Spotting
Authors: Li Wang, Rongzhi Gu, Nuo Chen, Yuexian Zou
Date: 2021-08-12 00:00:00
Tasks: Keyword Spotting, Metric Learning, Small-Footprint Keyword Spotting

Keyword Spotting (KWS) remains challenging to achieve the trade-off between small footprint and high accuracy. Recently proposed metric learning approaches improved the generalizability of models for the KWS task, and 1D-CNN based KWS models have achieved the state-of-the-arts (SOTA) in terms of model size. However, for metric learning, due to data limitations, the speech anchor is highly susceptible to the acoustic environment and speakers. Also, we note that the 1D-CNN models have limited capability to capture long-term temporal acoustic features. To address the above problems, we propose to utilize text anchors to improve the stability of anchors. Furthermore, a new type of model (LG-Net) is exquisitely designed to promote long-short term acoustic feature modeling based on 1D-CNN and self-attention. Experiments are conducted on Google Speech Commands Dataset version 1 (GSCDv1) and 2 (GSCDv2). The results demonstrate that the proposed text anchor based metric learning method shows consistent improvements over speech anchor on representative CNN-based models. Moreover, our LG-Net model achieves SOTA accuracy of 97.67% and 96.79% on two datasets, respectively. It is encouraged to see that our lighter LG-Net with only 74k parameters obtains 96.82% KWS accuracy on the GSCDv1 and 95.77% KWS accuracy on the GSCDv2.
Put the Bear on the Chair! Intelligent Robot Interaction with Previously Unseen Chairs via Robot Imagination
Authors: Hongtao Wu, Xin Meng, Sipu Ruan, Gregory Chirikjian
Date: 2021-08-12 00:00:00
Tasks: Motion Planning

In this paper, we study the problem of autonomously seating a teddy bear on a previously unseen chair. To achieve this goal, we present a novel method for robots to imagine the sitting pose of the bear by physically simulating a virtual humanoid agent sitting on the chair. We also develop a robotic system which leverages motion planning to plan SE(2) motions for a humanoid robot to walk to the chair and whole-body motions to put the bear on it. Furthermore, to cope with cases where the chair is not in an accessible pose for placing the bear, a human assistance module is introduced for a human to follow language instructions given by the robot to rotate the chair and help make the chair accessible. We implement our method with a robot arm and a humanoid robot. We calibrate the proposed system with 3 chairs and test on 12 previously unseen chairs in both accessible and inaccessible poses extensively. Results show that our method enables the robot to autonomously seat the teddy bear on the 12 previously unseen chairs with a very high success rate. The human assistance module is also shown to be very effective in changing the accessibility of the chair. Video demos and more details are available at https://chirikjianlab.github.io/putbearonchair/.
Graph Trend Filtering Networks for Recommendations
Authors: Wenqi Fan, Xiaorui Liu, Wei Jin, Xiangyu Zhao, Jiliang Tang, Qing Li
Date: 2021-08-12 00:00:00
Tasks: Collaborative Filtering, Graph Representation Learning, Recommendation Systems, Representation Learning

Recommender systems aim to provide personalized services to users and are playing an increasingly important role in our daily lives. The key of recommender systems is to predict how likely users will interact with items based on their historical online behaviors, e.g., clicks, add-to-cart, purchases, etc. To exploit these user-item interactions, there are increasing efforts on considering the user-item interactions as a user-item bipartite graph and then performing information propagation in the graph via Graph Neural Networks (GNNs). Given the power of GNNs in graph representation learning, these GNNs-based recommendation methods have remarkably boosted the recommendation performance. Despite their success, most existing GNNs-based recommender systems overlook the existence of interactions caused by unreliable behaviors (e.g., random/bait clicks) and uniformly treat all the interactions, which can lead to sub-optimal and unstable performance. In this paper, we investigate the drawbacks (e.g., non-adaptive propagation and non-robustness) of existing GNN-based recommendation methods. To address these drawbacks, we introduce a principled graph trend collaborative filtering method and propose the Graph Trend Filtering Networks for recommendations (GTN) that can capture the adaptive reliability of the interactions. Comprehensive experiments and ablation studies are presented to verify and understand the effectiveness of the proposed framework. Our implementation based on PyTorch is available at https://github.com/wenqifan03/GTN-SIGIR2022.
SR-HetGNN:Session-based Recommendation with Heterogeneous Graph Neural Network
Authors: Jinpeng Chen, Haiyang Li, Xudong Zhang, Fan Zhang, Senzhang Wang, Kaimin Wei, Jiaqi Ji
Date: 2021-08-12 00:00:00
Tasks: Graph Neural Network, Session-Based Recommendations

The Session-Based Recommendation System aims to predict the user's next click based on their previous session sequence. The current studies generally learn user preferences according to the transitions of items in the user's session sequence. However, other effective information in the session sequence, such as user profiles, are largely ignored which may lead to the model unable to learn the user's specific preferences. In this paper, we propose SR-HetGNN, a novel session recommendation method that uses a heterogeneous graph neural network (HetGNN) to learn session embeddings and capture the specific preferences of anonymous users. Specifically, SR-HetGNN first constructs heterogeneous graphs containing various types of nodes according to the session sequence, which can capture the dependencies among items, users, and sessions. Second, HetGNN captures the complex transitions between items and learns the item embeddings containing user information. Finally, local and global session embeddings are combined with the attentional network to obtain the final session embedding, considering the influence of users' long and short-term preferences. SR-HetGNN is shown to be superior to the existing state-of-the-art session-based recommendation methods through extensive experiments over two real large datasets Diginetica and Tmall.
Communication Optimization in Large Scale Federated Learning using Autoencoder Compressed Weight Updates
Authors: Srikanth Chandar, Pravin Chandran, Raghavendra Bhat, Avinash Chakravarthi
Date: 2021-08-12 00:00:00
Tasks: Decoder, Federated Learning

Federated Learning (FL) solves many of this decade's concerns regarding data privacy and computation challenges. FL ensures no data leaves its source as the model is trained at where the data resides. However, FL comes with its own set of challenges. The communication of model weight updates in this distributed environment comes with significant network bandwidth costs. In this context, we propose a mechanism of compressing the weight updates using Autoencoders (AE), which learn the data features of the weight updates and subsequently perform compression. The encoder is set up on each of the nodes where the training is performed while the decoder is set up on the node where the weights are aggregated. This setup achieves compression through the encoder and recreates the weights at the end of every communication round using the decoder. This paper shows that the dynamic and orthogonal AE based weight compression technique could serve as an advantageous alternative (or an add-on) in a large scale FL, as it not only achieves compression ratios ranging from 500x to 1720x and beyond, but can also be modified based on the accuracy requirements, computational capacity, and other requirements of the given FL setup.
How Nonconformity Functions and Difficulty of Datasets Impact the Efficiency of Conformal Classifiers
Authors: Marharyta Aleksandrova, Oleg Chertov
Date: 2021-08-12 00:00:00

The property of conformal predictors to guarantee the required accuracy rate makes this framework attractive in various practical applications. However, this property is achieved at a price of reduction in precision. In the case of conformal classification, the systems can output multiple class labels instead of one. It is also known from the literature, that the choice of nonconformity function has a major impact on the efficiency of conformal classifiers. Recently, it was shown that different model-agnostic nonconformity functions result in conformal classifiers with different characteristics. For a Neural Network-based conformal classifier, the inverse probability (or hinge loss) allows minimizing the average number of predicted labels, and margin results in a larger fraction of singleton predictions. In this work, we aim to further extend this study. We perform an experimental evaluation using 8 different classification algorithms and discuss when the previously observed relationship holds or not. Additionally, we propose a successful method to combine the properties of these two nonconformity functions. The experimental evaluation is done using 11 real and 5 synthetic datasets.
Lutz's Spoiler Technique Revisited: A Unified Approach to Worst-Case Optimal Entailment of Unions of Conjunctive Queries in Locally-Forward Description Logics
Authors: Bartosz Bednarczyk
Date: 2021-08-12 00:00:00

We present a unified approach to (both finite and unrestricted) worst-case optimal entailment of (unions of) conjunctive queries (U)CQs in the wide class of "locally-forward" description logics. The main technique that we employ is a generalisation of Lutz's spoiler technique, originally developed for CQ entailment in ALCHQ. Our result closes numerous gaps present in the literature, most notably implying ExpTime-completeness of (U)CQ-querying for any superlogic of ALC contained in ALCHbregQ, and, as we believe, is abstract enough to be employed as a black-box in many new scenarios.
Memristive Charge-Flux Interaction Still Makes It Possible To Find An Ideal Memristor
Authors: Frank Zhigang Wang
Date: 2021-08-10 00:00:00

In 1971, Chua defined an ideal memristor that links charge q and flux phi. In this work, we demonstrated that the direct interaction between physical charge q and physical flux phi is memristive by nature in terms of a time-invariant phi-q curve being nonlinear, continuously differentiable and strictly monotonically increasing. Nevertheless, this structure still suffers from two serious limitations: 1, a parasitic inductor effect, and 2. bistability and dynamic sweep of a continuous resistance range. Then we discussed how to make a fully-functioning ideal memristor with multiple or an infinite number of stable states and no parasitic inductance, and gave a number of suggestions, such as open structure, nanoscale size, magnetic materials with cubic anisotropy (or even isotropy), and sequential switching of the magnetic domains. At last, we responded to a recent challenge from arXiv.org that claims that the structure reported in our retracted JAP paper is simply an inductor with memory since it did not pass their designed capacitor-memristor circuit test. Contrary to their conjecture that an ideal memristor may not exist or may be a purely mathematical concept, we remain optimistic that researchers will discover an ideal memristor in nature or make one in the laboratory.
Scalable pragmatic communication via self-supervision
Authors: Jennifer Hu, Roger Levy, Noga Zaslavsky
Date: 2021-08-12 00:00:00

Models of context-sensitive communication often use the Rational Speech Act framework (RSA; Frank & Goodman, 2012), which formulates listeners and speakers in a cooperative reasoning process. However, the standard RSA formulation can only be applied to small domains, and large-scale applications have relied on imitating human behavior. Here, we propose a new approach to scalable pragmatics, building upon recent theoretical results (Zaslavsky et al., 2020) that characterize pragmatic reasoning in terms of general information-theoretic principles. Specifically, we propose an architecture and learning process in which agents acquire pragmatic policies via self-supervision instead of imitating human data. This work suggests a new principled approach for equipping artificial agents with pragmatic skills via self-supervision, which is grounded both in pragmatic theory and in information theory.
Reimagining an autonomous vehicle
Authors: Jeffrey Hawke, Haibo E, Vijay Badrinarayanan, Alex Kendall
Date: 2021-08-12 00:00:00
Tasks: Autonomous Driving

The self driving challenge in 2021 is this century's technological equivalent of the space race, and is now entering the second major decade of development. Solving the technology will create social change which parallels the invention of the automobile itself. Today's autonomous driving technology is laudable, though rooted in decisions made a decade ago. We argue that a rethink is required, reconsidering the autonomous vehicle (AV) problem in the light of the body of knowledge that has been gained since the DARPA challenges which seeded the industry. What does AV2.0 look like? We present an alternative vision: a recipe for driving with machine learning, and grand challenges for research in driving.
A general class of surrogate functions for stable and efficient reinforcement learning
Authors: Sharan Vaswani, Olivier Bachem, Simone Totaro, Robert Mueller, Shivam Garg, Matthieu Geist, Marlos C. Machado, Pablo Samuel Castro, Nicolas Le Roux
Date: 2021-08-12 00:00:00
Tasks: MuJoCo, Policy Gradient Methods, reinforcement-learning, Reinforcement Learning (RL)

Common policy gradient methods rely on the maximization of a sequence of surrogate functions. In recent years, many such surrogate functions have been proposed, most without strong theoretical guarantees, leading to algorithms such as TRPO, PPO or MPO. Rather than design yet another surrogate function, we instead propose a general framework (FMA-PG) based on functional mirror ascent that gives rise to an entire family of surrogate functions. We construct surrogate functions that enable policy improvement guarantees, a property not shared by most existing surrogate functions. Crucially, these guarantees hold regardless of the choice of policy parameterization. Moreover, a particular instantiation of FMA-PG recovers important implementation heuristics (e.g., using forward vs reverse KL divergence) resulting in a variant of TRPO with additional desirable properties. Via experiments on simple bandit problems, we evaluate the algorithms instantiated by FMA-PG. The proposed framework also suggests an improved variant of PPO, whose robustness and efficiency we empirically demonstrate on the MuJoCo suite.
The paradox of the compositionality of natural language: a neural machine translation case study
Authors: Verna Dankers, Elia Bruni, Dieuwke Hupkes
Date: 2021-08-12 00:00:00
Proceeding: ACL 2022 5
Tasks: Machine Translation, Math, NMT, Translation

Obtaining human-like performance in NLP is often argued to require compositional generalisation. Whether neural networks exhibit this ability is usually studied by training models on highly compositional synthetic data. However, compositionality in natural language is much more complex than the rigid, arithmetic-like version such data adheres to, and artificial compositionality tests thus do not allow us to determine how neural models deal with more realistic forms of compositionality. In this work, we re-instantiate three compositionality tests from the literature and reformulate them for neural machine translation (NMT). Our results highlight that: i) unfavourably, models trained on more data are more compositional; ii) models are sometimes less compositional than expected, but sometimes more, exemplifying that different levels of compositionality are required, and models are not always able to modulate between them correctly; iii) some of the non-compositional behaviours are mistakes, whereas others reflect the natural variation in data. Apart from an empirical study, our work is a call to action: we should rethink the evaluation of compositionality in neural networks and develop benchmarks using real data to evaluate compositionality on natural language, where composing meaning is not as straightforward as doing the math.
NoFake at CheckThat! 2021: Fake News Detection Using BERT
Authors: Sushma Kumari
Date: 2021-08-11 00:00:00
Tasks: Articles, Claim Verification, Fake News Detection

Much research has been done for debunking and analysing fake news. Many researchers study fake news detection in the last year, but many are limited to social media data. Currently, multiples fact-checkers are publishing their results in various formats. Also, multiple fact-checkers use different labels for the fake news, making it difficult to make a generalisable classifier. With the merge classes, the performance of the machine model can be enhanced. This domain categorisation will help group the article, which will help save the manual effort in assigning the claim verification. In this paper, we have presented BERT based classification model to predict the domain and classification. We have also used additional data from fact-checked articles. We have achieved a macro F1 score of 83.76 % for Task 3Aand 85.55 % for Task 3B using the additional training data.
Attacks against Ranking Algorithms with Text Embeddings: a Case Study on Recruitment Algorithms
Authors: Anahita Samadi, Debapriya Banerjee, Shirin Nilizadeh
Date: 2021-08-12 00:00:00
Proceeding: EMNLP (BlackboxNLP) 2021 11
Tasks: Decision Making, Sentence, text-classification, Text Classification

Recently, some studies have shown that text classification tasks are vulnerable to poisoning and evasion attacks. However, little work has investigated attacks against decision making algorithms that use text embeddings, and their output is a ranking. In this paper, we focus on ranking algorithms for recruitment process, that employ text embeddings for ranking applicants resumes when compared to a job description. We demonstrate both white box and black box attacks that identify text items, that based on their location in embedding space, have significant contribution in increasing the similarity score between a resume and a job description. The adversary then uses these text items to improve the ranking of their resume among others. We tested recruitment algorithms that use the similarity scores obtained from Universal Sentence Encoder (USE) and Term Frequency Inverse Document Frequency (TF IDF) vectors. Our results show that in both adversarial settings, on average the attacker is successful. We also found that attacks against TF IDF is more successful compared to USE.
Ethereum Data Structures
Authors: Kamil Jezek
Date: 2021-08-12 00:00:00

Ethereum platform operates with rich spectrum of data structures and hashing and coding functions. The main source describing them is the Yellow paper, complemented by a lot of informal blogs. These sources are somehow limited. In particular, the Yellow paper does not ideally balance brevity and detail, in some parts it is very detail, while too shallow elsewhere. The blogs on the other hand are often too vague and in certain cases contain incorrect information. As a solution, we provide this document, which summarises data structures used in Ethereum. The goal is to provide sufficient detail while keeping brevity. Sufficiently detailed formal view is enriched with examples to extend on clarity.
AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processing
Authors: Katikapalli Subramanyam Kalyan, Ajit Rajasekharan, Sivanesan Sangeetha
Date: 2021-08-12 00:00:00
Tasks: Self-Supervised Learning, Survey, Transfer Learning

Transformer-based pretrained language models (T-PTLMs) have achieved great success in almost every NLP task. The evolution of these models started with GPT and BERT. These models are built on the top of transformers, self-supervised learning and transfer learning. Transformed-based PTLMs learn universal language representations from large volumes of text data using self-supervised learning and transfer this knowledge to downstream tasks. These models provide good background knowledge to downstream tasks which avoids training of downstream models from scratch. In this comprehensive survey paper, we initially give a brief overview of self-supervised learning. Next, we explain various core concepts like pretraining, pretraining methods, pretraining tasks, embeddings and downstream adaptation methods. Next, we present a new taxonomy of T-PTLMs and then give brief overview of various benchmarks including both intrinsic and extrinsic. We present a summary of various useful libraries to work with T-PTLMs. Finally, we highlight some of the future research directions which will further improve these models. We strongly believe that this comprehensive survey paper will serve as a good reference to learn the core concepts as well as to stay updated with the recent happenings in T-PTLMs.
Kicktionary-LOME: A Domain-Specific Multilingual Frame Semantic Parsing Model for Football Language
Authors: Gosse Minnema
Date: 2021-08-12 00:00:00
Tasks: Prediction, Semantic Parsing

This technical report introduces an adapted version of the LOME frame semantic parsing model (Xia et al., EACL 2021) which is capable of automatically annotating texts according to the "Kicktionary" domain-specific framenet resource. Several methods for training a model even with limited available training data are proposed. While there are some challenges for evaluation related to the nature of the available annotations, preliminary results are very promising, with the best model reaching F1-scores of 0.83 (frame prediction) and 0.81 (semantic role prediction).
Generation Challenges: Results of the Accuracy Evaluation Shared Task
Authors: Craig Thomson, Ehud Reiter
Date: 2021-08-12 00:00:00
Proceeding: INLG (ACL) 2021 8

The Shared Task on Evaluating Accuracy focused on techniques (both manual and automatic) for evaluating the factual accuracy of texts produced by neural NLG systems, in a sports-reporting domain. Four teams submitted evaluation techniques for this task, using very different approaches and techniques. The best-performing submissions did encouragingly well at this difficult task. However, all automatic submissions struggled to detect factual errors which are semantically or pragmatically complex (for example, based on incorrect computation or inference).
Generating Diverse Descriptions from Semantic Graphs
Authors: Jiuzhou Han, Daniel Beck, Trevor Cohn
Date: 2021-08-12 00:00:00
Proceeding: INLG (ACL) 2021 8
Tasks: Decoder, Diversity, Text Generation

Text generation from semantic graphs is traditionally performed with deterministic methods, which generate a unique description given an input graph. However, the generation problem admits a range of acceptable textual outputs, exhibiting lexical, syntactic and semantic variation. To address this disconnect, we present two main contributions. First, we propose a stochastic graph-to-text model, incorporating a latent variable in an encoder-decoder model, and its use in an ensemble. Second, to assess the diversity of the generated sentences, we propose a new automatic evaluation metric which jointly evaluates output diversity and quality in a multi-reference setting. We evaluate the models on WebNLG datasets in English and Russian, and show an ensemble of stochastic models produces diverse sets of generated sentences, while retaining similar quality to state-of-the-art models.