pwc-archive/papers-with-abstracts huggingface dataset:
A Derivative-free Method for Quantum Perceptron Training in Multi-layered Neural Networks
Authors: Tariq M. Khan, Antonio Robles-Kelly
Date: 2020-09-23 00:00:00
Tasks: Computational Efficiency

In this paper, we present a gradient-free approach for training multi-layered neural networks based upon quantum perceptrons. Here, we depart from the classical perceptron and the elemental operations on quantum bits, i.e. qubits, so as to formulate the problem in terms of quantum perceptrons. We then make use of measurable operators to define the states of the network in a manner consistent with a Markov process. This yields a Dirac-Von Neumann formulation consistent with quantum mechanics. Moreover, the formulation presented here has the advantage of having a computational efficiency devoid of the number of layers in the network. This, paired with the natural efficiency of quantum computing, can imply a significant improvement in efficiency, particularly for deep networks. Finally, but not least, the developments here are quite general in nature since the approach presented here can also be used for quantum-inspired neural networks implemented on conventional computers.
Energy-Based Reranking: Improving Neural Machine Translation Using Energy-Based Models
Authors: Sumanta Bhattacharyya, Amirmohammad Rooshenas, Subhajit Naskar, Simeng Sun, Mohit Iyyer, Andrew McCallum
Date: 2020-09-20 00:00:00
Proceeding: ACL 2021 5
Tasks: Computational Efficiency, Machine Translation, NMT, Reranking, Re-Ranking, Sentence, Translation

The discrepancy between maximum likelihood estimation (MLE) and task measures such as BLEU score has been studied before for autoregressive neural machine translation (NMT) and resulted in alternative training algorithms (Ranzato et al., 2016; Norouzi et al., 2016; Shen et al., 2016; Wu et al., 2018). However, MLE training remains the de facto approach for autoregressive NMT because of its computational efficiency and stability. Despite this mismatch between the training objective and task measure, we notice that the samples drawn from an MLE-based trained NMT support the desired distribution -- there are samples with much higher BLEU score comparing to the beam decoding output. To benefit from this observation, we train an energy-based model to mimic the behavior of the task measure (i.e., the energy-based model assigns lower energy to samples with higher BLEU score), which is resulted in a re-ranking algorithm based on the samples drawn from NMT: energy-based re-ranking (EBR). We use both marginal energy models (over target sentence) and joint energy models (over both source and target sentences). Our EBR with the joint energy model consistently improves the performance of the Transformer-based NMT: +4 BLEU points on IWSLT'14 German-English, +3.0 BELU points on Sinhala-English, +1.2 BLEU on WMT'16 English-German tasks.
Dissecting Lottery Ticket Transformers: Structural and Behavioral Study of Sparse Neural Machine Translation
Authors: Rajiv Movva, Jason Y. Zhao
Date: 2020-09-17 00:00:00
Proceeding: EMNLP (BlackboxNLP) 2020 11
Tasks: Machine Translation, NMT, Translation

Recent work on the lottery ticket hypothesis has produced highly sparse Transformers for NMT while maintaining BLEU. However, it is unclear how such pruning techniques affect a model's learned representations. By probing Transformers with more and more low-magnitude weights pruned away, we find that complex semantic information is first to be degraded. Analysis of internal activations reveals that higher layers diverge most over the course of pruning, gradually becoming less complex than their dense counterparts. Meanwhile, early layers of sparse models begin to perform more encoding. Attention mechanisms remain remarkably consistent as sparsity increases.
Augmented Natural Language for Generative Sequence Labeling
Authors: Ben Athiwaratkun, Cicero Nogueira dos santos, Jason Krone, Bing Xiang
Date: 2020-09-15 00:00:00
Proceeding: EMNLP 2020 11
Tasks: intent-classification, Intent Classification, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Sentence

We propose a generative framework for joint sequence labeling and sentence-level classification. Our model performs multiple sequence labeling tasks at once using a single, shared natural language output space. Unlike prior discriminative methods, our model naturally incorporates label semantics and shares knowledge across tasks. Our framework is general purpose, performing well on few-shot, low-resource, and high-resource tasks. We demonstrate these advantages on popular named entity recognition, slot labeling, and intent classification benchmarks. We set a new state-of-the-art for few-shot slot labeling, improving substantially upon the previous 5-shot ($75.0\% \rightarrow 90.9\%$) and 1-shot ($70.4\% \rightarrow 81.0\%$) state-of-the-art results. Furthermore, our model generates large improvements ($46.27\% \rightarrow 63.83\%$) in low-resource slot labeling over a BERT baseline by incorporating label semantics. We also maintain competitive results on high-resource tasks, performing within two points of the state-of-the-art on all tasks and setting a new state-of-the-art on the SNIPS dataset.
RecoBERT: A Catalog Language Model for Text-Based Recommendations
Authors: Itzik Malkiel, Oren Barkan, Avi Caciularu, Noam Razin, Ori Katz, Noam Koenigstein
Date: 2020-09-25 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Language Modeling, Language Modelling

Language models that utilize extensive self-supervised pre-training from unlabeled text, have recently shown to significantly advance the state-of-the-art performance in a variety of language understanding tasks. However, it is yet unclear if and how these recent models can be harnessed for conducting text-based recommendations. In this work, we introduce RecoBERT, a BERT-based approach for learning catalog-specialized language models for text-based item recommendations. We suggest novel training and inference procedures for scoring similarities between pairs of items, that don't require item similarity labels. Both the training and the inference techniques were designed to utilize the unlabeled structure of textual catalogs, and minimize the discrepancy between them. By incorporating four scores during inference, RecoBERT can infer text-based item-to-item similarities more accurately than other techniques. In addition, we introduce a new language understanding task for wine recommendations using similarities based on professional wine reviews. As an additional contribution, we publish annotated recommendations dataset crafted by human wine experts. Finally, we evaluate RecoBERT and compare it to various state-of-the-art NLP models on wine and fashion recommendations tasks.
Virtual Proximity Citation (VCP): A Supervised Deep Learning Method to Relate Uncited Papers On Grounds of Citation Proximity
Authors: Rohit Rawat
Date: 2020-09-25 00:00:00
Tasks: Articles

Citation based approaches have seen good progress for recommending research papers using citations in the paper. Citation proximity analysis which uses the in-text citation proximity to find relatedness between two research papers is better than co-citation analysis and bibliographic analysis. However, one common problem which exists in each approach is that paper should be well cited. If documents are not cited properly or not cited at all, then using these approaches will not be helpful. To overcome the problem, this paper discusses the approach Virtual Citation Proximity (VCP) which uses Siamese Neural Network along with the notion of citation proximity analysis and content-based filtering. To train this model, the actual distance between the two citations in a document is used as ground truth, this distance is the word count between the two citations. VCP is trained on Wikipedia articles for which the actual word count is available which is used to calculate the similarity between the documents. This can be used to calculate relatedness between two documents in a way they would have been cited in the proximity even if the documents are uncited. This approach has shown a great improvement in predicting proximity with basic neural networks over the approach which uses the Average Citation Proximity index value as the ground truth. This can be improved by using a complex neural network and proper hyper tuning of parameters.
A Diagnostic Study of Explainability Techniques for Text Classification
Authors: Pepa Atanasova, Jakob Grue Simonsen, Christina Lioma, Isabelle Augenstein
Date: 2020-09-25 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Classification, Diagnostic, General Classification, text-classification, Text Classification

Recent developments in machine learning have introduced models that approach human performance at the cost of increased architectural complexity. Efforts to make the rationales behind the models' predictions transparent have inspired an abundance of new explainability techniques. Provided with an already trained model, they compute saliency scores for the words of an input instance. However, there exists no definitive guide on (i) how to choose such a technique given a particular application task and model architecture, and (ii) the benefits and drawbacks of using each such technique. In this paper, we develop a comprehensive list of diagnostic properties for evaluating existing explainability techniques. We then employ the proposed list to compare a set of diverse explainability techniques on downstream text classification tasks and neural network architectures. We also compare the saliency scores assigned by the explainability techniques with human annotations of salient input regions to find relations between a model's performance and the agreement of its rationales with human ones. Overall, we find that the gradient-based explanations perform best across tasks and model architectures, and we present further insights into the properties of the reviewed explainability techniques.
Learning to Match Jobs with Resumes from Sparse Interaction Data using Multi-View Co-Teaching Network
Authors: Shuqing Bian, Xu Chen, Wayne Xin Zhao, Kun Zhou, Yupeng Hou, Yang song, Tao Zhang, Ji-Rong Wen
Date: 2020-09-25 00:00:00
Tasks: Text Matching

With the ever-increasing growth of online recruitment data, job-resume matching has become an important task to automatically match jobs with suitable resumes. This task is typically casted as a supervised text matching problem. Supervised learning is powerful when the labeled data is sufficient. However, on online recruitment platforms, job-resume interaction data is sparse and noisy, which affects the performance of job-resume match algorithms. To alleviate these problems, in this paper, we propose a novel multi-view co-teaching network from sparse interaction data for job-resume matching. Our network consists of two major components, namely text-based matching model and relation-based matching model. The two parts capture semantic compatibility in two different views, and complement each other. In order to address the challenges from sparse and noisy data, we design two specific strategies to combine the two components. First, two components share the learned parameters or representations, so that the original representations of each component can be enhanced. More importantly, we adopt a co-teaching mechanism to reduce the influence of noise in training data. The core idea is to let the two components help each other by selecting more reliable training instances. The two strategies focus on representation enhancement and data enhancement, respectively. Compared with pure text-based matching models, the proposed approach is able to learn better data representations from limited or even sparse interaction data, which is more resistible to noise in training data. Experiment results have demonstrated that our model is able to outperform state-of-the-art methods for job-resume matching.
Process mining classification with a weightless neural network
Authors: Rafael Garcia Barbastefano, Maria Clara Lippi, Diego Carvalho
Date: 2020-09-25 00:00:00
Tasks: Classification, General Classification

Using a weightless neural network architecture WiSARD we propose a straightforward graph to retina codification to represent business process graph flows avoiding kernels, and we present how WiSARD outperforms the classification performance with small training sets in the process mining context.
Online Learning of Non-Markovian Reward Models
Authors: Gavin Rens, Jean-François Raskin, Raphaël Reynouad, Giuseppe Marra
Date: 2020-09-26 00:00:00
Tasks: Active Learning

There are situations in which an agent should receive rewards only after having accomplished a series of previous tasks, that is, rewards are non-Markovian. One natural and quite general way to represent history-dependent rewards is via a Mealy machine, a finite state automaton that produces output sequences from input sequences. In our formal setting, we consider a Markov decision process (MDP) that models the dynamics of the environment in which the agent evolves and a Mealy machine synchronized with this MDP to formalize the non-Markovian reward function. While the MDP is known by the agent, the reward function is unknown to the agent and must be learned. Our approach to overcome this challenge is to use Angluin's $L^*$ active learning algorithm to learn a Mealy machine representing the underlying non-Markovian reward machine (MRM). Formal methods are used to determine the optimal strategy for answering so-called membership queries posed by $L^*$. Moreover, we prove that the expected reward achieved will eventually be at least as much as a given, reasonable value provided by a domain expert. We evaluate our framework on three problems. The results show that using $L^*$ to learn an MRM in a non-Markovian reward decision process is effective.
A Multi-Agent System for Solving the Dynamic Capacitated Vehicle Routing Problem with Stochastic Customers using Trajectory Data Mining
Authors: Juan Camilo Fonseca-Galindo, Gabriela de Castro Surita, José Maia Neto, Cristiano Leite de Castro, André Paim Lemos
Date: 2020-09-26 00:00:00

The worldwide growth of e-commerce has created new challenges for logistics companies, one of which is being able to deliver products quickly and at low cost, which reflects directly in the way of sorting packages, needing to eliminate steps such as storage and batch creation. Our work presents a multi-agent system that uses trajectory data mining techniques to extract territorial patterns and use them in the dynamic creation of last-mile routes. The problem can be modeled as a Dynamic Capacitated Vehicle Routing Problem (VRP) with Stochastic Customer, being therefore NP-HARD, what makes its implementation unfeasible for many packages. The work's main contribution is to solve this problem only depending on the Warehouse system configurations and not on the number of packages processed, which is appropriate for Big Data scenarios commonly present in the delivery of e-commerce products. Computational experiments were conducted for single and multi depot instances. Due to its probabilistic nature, the proposed approach presented slightly lower performances when compared to the static VRP algorithm. However, the operational gains that our solution provides making it very attractive for situations in which the routes must be set dynamically.
A Column Generation based Heuristic for the Tail Assignment Problem
Authors: Akash Sambrekar, El Mehdi Er Raqabi
Date: 2020-09-25 00:00:00

This article proposes an efficient heuristic in accelerating the column generation by parallel resolution of pricing problems for aircrafts in the tail assignment problem (TAP). The approach is able to achieve considerable improvement in resolution time for real life test instances from two major Indian air carriers. The different restrictions on individual aircraft for maintenance routing as per aviation regulatory bodies are considered in this paper. We also present a variable fixing heuristic to improve the integrality of the solution. The hybridization of constraint programming and column generation was substantial in accelerating the resolution process.
Towards a Natural Language Query Processing System
Authors: Chantal Montgomery, Haruna Isah, Farhana Zulkernine
Date: 2020-09-25 00:00:00
Tasks: Information Retrieval, Management, Retrieval

Tackling the information retrieval gap between non-technical database end-users and those with the knowledge of formal query languages has been an interesting area of data management and analytics research. The use of natural language interfaces to query information from databases offers the opportunity to bridge the communication challenges between end-users and systems that use formal query languages. Previous research efforts mainly focused on developing structured query interfaces to relational databases. However, the evolution of unstructured big data such as text, images, and video has exposed the limitations of traditional structured query interfaces. While the existing web search tools prove the popularity and usability of natural language query, they return complete documents and web pages instead of focused query responses and are not applicable to database systems. This paper reports our study on the design and development of a natural language query interface to a backend relational database. The novelty in the study lies in defining a graph database as a middle layer to store necessary metadata needed to transform a natural language query into structured query language that can be executed on backend databases. We implemented and evaluated our approach using a restaurant dataset. The translation results for some sample queries yielded a 90% accuracy rate.
Pareto-Optimal Bit Allocation for Collaborative Intelligence
Authors: Saeed Ranjbar Alvar, Ivan V. Bajić
Date: 2020-09-25 00:00:00

In recent studies, collaborative intelligence (CI) has emerged as a promising framework for deployment of Artificial Intelligence (AI)-based services on mobile/edge devices. In CI, the AI model (a deep neural network) is split between the edge and the cloud, and intermediate features are sent from the edge sub-model to the cloud sub-model. In this paper, we study bit allocation for feature coding in multi-stream CI systems. We model task distortion as a function of rate using convex surfaces similar to those found in distortion-rate theory. Using such models, we are able to provide closed-form bit allocation solutions for single-task systems and scalarized multi-task systems. Moreover, we provide analytical characterization of the full Pareto set for 2-stream k-task systems, and bounds on the Pareto set for 3-stream 2-task systems. Analytical results are examined on a variety of DNN models from the literature to demonstrate wide applicability of the results
Proceedings of the Sixteenth International Workshop on the ACL2 Theorem Prover and its Applications
Authors: Grant Passmore, Ruben Gamboa
Date: 2020-09-26 00:00:00

This volume contains a selection of papers presented at the 16th International Workshop on the ACL2 Theorem Prover and its Applications (ACL2-2020). The workshops are the premier technical forum for presenting research and experiences related to ACL2.
I Like to Move It: 6D Pose Estimation as an Action Decision Process
Authors: Benjamin Busam, Hyun Jun Jung, Nassir Navab
Date: 2020-09-26 00:00:00
Tasks: 6D Pose Estimation, Object, Pose Estimation, Pose Prediction, Pose Retrieval, Retrieval

Object pose estimation is an integral part of robot vision and AR. Previous 6D pose retrieval pipelines treat the problem either as a regression task or discretize the pose space to classify. We change this paradigm and reformulate the problem as an action decision process where an initial pose is updated in incremental discrete steps that sequentially move a virtual 3D rendering towards the correct solution. A neural network estimates likely moves from a single RGB image iteratively and determines so an acceptable final pose. In comparison to other approaches that train object-specific pose models, we learn a decision process. This allows for a lightweight architecture while it naturally generalizes to unseen objects. A coherent stop action for process termination enables dynamic reduction of the computation cost if there are insignificant changes in a video sequence. Instead of a static inference time, we thereby automatically increase the runtime depending on the object motion. Robustness and accuracy of our action decision network are evaluated on Laval and YCB video scenes where we significantly improve the state-of-the-art.
Measure Utility, Gain Trust: Practical Advice for XAI Researcher
Authors: Brittany Davis, Maria Glenski, William Sealy, Dustin Arendt
Date: 2020-09-27 00:00:00
Tasks: BIG-bench Machine Learning, Explainable Artificial Intelligence (XAI)

Research into the explanation of machine learning models, i.e., explainable AI (XAI), has seen a commensurate exponential growth alongside deep artificial neural networks throughout the past decade. For historical reasons, explanation and trust have been intertwined. However, the focus on trust is too narrow, and has led the research community astray from tried and true empirical methods that produced more defensible scientific knowledge about people and explanations. To address this, we contribute a practical path forward for researchers in the XAI field. We recommend researchers focus on the utility of machine learning explanations instead of trust. We outline five broad use cases where explanations are useful and, for each, we describe pseudo-experiments that rely on objective empirical measurements and falsifiable hypotheses. We believe that this experimental rigor is necessary to contribute to scientific knowledge in the field of XAI.
Zero-shot Multi-Domain Dialog State Tracking Using Descriptive Rules
Authors: Edgar Altszyler, Pablo Brusco, Nikoletta Basiou, John Byrnes, Dimitra Vergyri
Date: 2020-09-17 00:00:00
Tasks: Descriptive, dialog state tracking

In this work, we present a framework for incorporating descriptive logical rules in state-of-the-art neural networks, enabling them to learn how to handle unseen labels without the introduction of any new training data. The rules are integrated into existing networks without modifying their architecture, through an additional term in the network's loss function that penalizes states of the network that do not obey the designed rules. As a case of study, the framework is applied to an existing neural-based Dialog State Tracker. Our experiments demonstrate that the inclusion of logical rules allows the prediction of unseen labels, without deteriorating the predictive capacity of the original system.