pwc-archive/papers-with-abstracts huggingface dataset:
Finding teams that balance expert load and task coverage
Authors: Sofia Maria Nikolakaki, Mingxiang Cai, Evimaria Terzi
Date: 2020-11-03 00:00:00

The rise of online labor markets (e.g., Freelancer, Guru and Upwork) has ignited a lot of research on team formation, where experts acquiring different skills form teams to complete tasks. The core idea in this line of work has been the strict requirement that the team of experts assigned to complete a given task should contain a superset of the skills required by the task. However, in many applications the required skills are often a wishlist of the entity that posts the task and not all of the skills are absolutely necessary. Thus, in our setting we relax the complete coverage requirement and we allow for tasks to be partially covered by the formed teams, assuming that the quality of task completion is proportional to the fraction of covered skills per task. At the same time, we assume that when multiple tasks need to be performed, the less the load of an expert the better the performance. We combine these two high-level objectives into one and define the BalancedTA problem. We also consider a generalization of this problem where each task consists of required and optional skills. In this setting, our objective is the same under the constraint that all required skills should be covered. From the technical point of view, we show that the BalancedTA problem (and its variant) is NP-hard and design efficient heuristics for solving it in practice. Using real datasets from three online market places, Freelancer, Guru and Upwork we demonstrate the efficiency of our methods and the practical utility of our framework.
Geometry Perspective Of Estimating Learning Capability Of Neural Networks
Authors: Ankan Dutta, Arnab Rakshit
Date: 2020-11-03 00:00:00
Tasks: Learning Theory

The paper uses statistical and differential geometric motivation to acquire prior information about the learning capability of an artificial neural network on a given dataset. The paper considers a broad class of neural networks with generalized architecture performing simple least square regression with stochastic gradient descent (SGD). The system characteristics at two critical epochs in the learning trajectory are analyzed. During some epochs of the training phase, the system reaches equilibrium with the generalization capability attaining a maximum. The system can also be coherent with localized, non-equilibrium states, which is characterized by the stabilization of the Hessian matrix. The paper proves that neural networks with higher generalization capability will have a slower convergence rate. The relationship between the generalization capability with the stability of the neural network has also been discussed. By correlating the principles of high-energy physics with the learning theory of neural networks, the paper establishes a variant of the Complexity-Action conjecture from an artificial neural network perspective.
Bandits in Matching Markets: Ideas and Proposals for Peer Lending
Authors: Soumajyoti Sarkar
Date: 2020-10-30 00:00:00
Tasks: Decision Making, Fairness, Sequential Decision Making

Motivated by recent applications of sequential decision making in matching markets, in this paper we attempt at formulating and abstracting market designs for P2P lending. We describe a paradigm to set the stage for how peer to peer investments can be conceived from a matching market perspective, especially when both borrower and lender preferences are respected. We model these specialized markets as an optimization problem and consider different utilities for agents on both sides of the market while also understanding the impact of equitable allocations to borrowers. We devise a technique based on sequential decision making that allow the lenders to adjust their choices based on the dynamics of uncertainty from competition over time and that also impacts the rewards in return for their investments. Using simulated experiments we show the dynamics of the regret based on the optimal borrower-lender matching and find that the lender regret depends on the initial preferences set by the lenders which could affect their learning over decision making steps.
Hierarchical Multitask Learning Approach for BERT
Authors: Çağla Aksoy, Alper Ahmetoğlu, Tunga Güngör
Date: 2020-10-17 00:00:00
Tasks: Language Modeling, Language Modelling, Natural Language Inference, Question Answering, Sentence

Recent works show that learning contextualized embeddings for words is beneficial for downstream tasks. BERT is one successful example of this approach. It learns embeddings by solving two tasks, which are masked language model (masked LM) and the next sentence prediction (NSP). The pre-training of BERT can also be framed as a multitask learning problem. In this work, we adopt hierarchical multitask learning approaches for BERT pre-training. Pre-training tasks are solved at different layers instead of the last layer, and information from the NSP task is transferred to the masked LM task. Also, we propose a new pre-training task bigram shift to encode word order information. We choose two downstream tasks, one of which requires sentence-level embeddings (textual entailment), and the other requires contextualized embeddings of words (question answering). Due to computational restrictions, we use the downstream task data instead of a large dataset for the pre-training to see the performance of proposed models when given a restricted dataset. We test their performance on several probing tasks to analyze learned embeddings. Our results show that imposing a task hierarchy in pre-training improves the performance of embeddings.
Auxiliary Sequence Labeling Tasks for Disfluency Detection
Authors: Dongyub Lee, Byeongil Ko, Myeong Cheol Shin, Taesun Whang, Daniel Lee, Eun Hwa Kim, EungGyun Kim, Jaechoon Jo
Date: 2020-10-24 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, POS, speech-recognition, Speech Recognition

Detecting disfluencies in spontaneous speech is an important preprocessing step in natural language processing and speech recognition applications. Existing works for disfluency detection have focused on designing a single objective only for disfluency detection, while auxiliary objectives utilizing linguistic information of a word such as named entity or part-of-speech information can be effective. In this paper, we focus on detecting disfluencies on spoken transcripts and propose a method utilizing named entity recognition (NER) and part-of-speech (POS) as auxiliary sequence labeling (SL) tasks for disfluency detection. First, we investigate cases that utilizing linguistic information of a word can prevent mispredicting important words and can be helpful for the correct detection of disfluencies. Second, we show that training a disfluency detection model with auxiliary SL tasks can improve its F-score in disfluency detection. Then, we analyze which auxiliary SL tasks are influential depending on baseline models. Experimental results on the widely used English Switchboard dataset show that our method outperforms the previous state-of-the-art in disfluency detection.
A Framework for Sequential Planning in Multi-Agent Settings
Authors: P. Doshi, P. J. Gmytrasiewicz
Date: 2011-09-09 00:00:00
Tasks: Decision Making

This paper extends the framework of partially observable Markov decision processes (POMDPs) to multi-agent settings by incorporating the notion of agent models into the state space. Agents maintain beliefs over physical states of the environment and over models of other agents, and they use Bayesian updates to maintain their beliefs over time. The solutions map belief states to actions. Models of other agents may include their belief states and are related to agent types considered in games of incomplete information. We express the agents autonomy by postulating that their models are not directly manipulable or observable by other agents. We show that important properties of POMDPs, such as convergence of value iteration, the rate of convergence, and piece-wise linearity and convexity of the value functions carry over to our framework. Our approach complements a more traditional approach to interactive settings which uses Nash equilibria as a solution paradigm. We seek to avoid some of the drawbacks of equilibria which may be non-unique and do not capture off-equilibrium behaviors. We do so at the cost of having to represent, process and continuously revise models of other agents. Since the agents beliefs may be arbitrarily nested, the optimal solutions to decision making problems are only asymptotically computable. However, approximate belief updates and approximately optimal plans are computable. We illustrate our framework using a simple application domain, and we show examples of belief updates and value functions.
Finites elements and signal processing analysis of eccentricity and demagnetization faults in PMSM drivetrains: Approach for diagnosis
Authors: Naourez Ben Hadj, Manel Krichen, Rafik Neji
Date: 2020-11-09 00:00:00
Tasks: Fault Diagnosis

Today, interest in automotive applications notably Hybrid Electric Vehicles (HEV) has risen due to environmental concerns and the modern society's energetic dependence. Consequently, it is necessary to study and implement in these vehicle fault diagnosis systems which allow them to be more reliable and safe enhancing its sustainability. Several research works have been focused on Fault Diagnosis approaches for electric machines due to it's influence in the automotive system availability. Our work fits into this context, firstly with a review of faults and diagnosis methods of the permanent magnet synchronous motor (PMSM) and secondly by using analytical-numerical approach based on finite element method and signal processing (FEMM-MATLAB coupling) in order to detect and analysis of eccentricity and partial demagnetization faults for surface mounted PMSM. So, this work mainly includes fault performance, harmonic component and signal spectrum characteristics, timefrequency analysis technique. The outcomes obtained are further compared with the motor characteristics under healthy operation to have distinct detection to faults.
Balancing the Payment System
Authors: Tomaž Fleischman, Paolo Dini
Date: 2020-11-06 00:00:00
Tasks: Management

The increasingly complex economic and financial environment in which we live makes the management of liquidity in payment systems and the economy in general a persistent challenge. New technologies are making it possible to address this challenge through alternative solutions that complement and strengthen existing payment systems. For example, the interbank balancing method can also be applied to private payment systems, complementary currencies, and trade credit clearing systems to provide better liquidity and risk management. In this paper we introduce the concept of a balanced payment system and demonstrate the effects of balancing on a small example. We show how to construct a balanced payment subsystem that can be settled in full and, therefore, that can be removed from the payment system to achieve liquidity-saving and payments gridlock resolution. We also briefly introduce a generalization of a payment system and of the method to balance it in the form of a specific application (Tetris Core Technologies), whose wider adoption could contribute to the financial stability of and better management of liquidity and risk for the whole economy.
A Survey of Point-of-interest Recommendation in Location-based Social Networks
Authors: Shenglin Zhao, Irwin King, Michael R. Lyu
Date: 2016-07-03 00:00:00
Tasks: Movie Recommendation, Recommendation Systems

Point-of-interest (POI) recommendation that suggests new places for users to visit arises with the popularity of location-based social networks (LBSNs). Due to the importance of POI recommendation in LBSNs, it has attracted much academic and industrial interest. In this paper, we offer a systematic review of this field, summarizing the contributions of individual efforts and exploring their relations. We discuss the new properties and challenges in POI recommendation, compared with traditional recommendation problems, e.g., movie recommendation. Then, we present a comprehensive review in three aspects: influential factors for POI recommendation, methodologies employed for POI recommendation, and different tasks in POI recommendation. Specifically, we propose three taxonomies to classify POI recommendation systems. First, we categorize the systems by the influential factors check-in characteristics, including the geographical information, social relationship, temporal influence, and content indications. Second, we categorize the systems by the methodology, including systems modeled by fused methods and joint methods. Third, we categorize the systems as general POI recommendation and successive POI recommendation by subtle differences in the recommendation task whether to be bias to the recent check-in. For each category, we summarize the contributions and system features, and highlight the representative work. Moreover, we discuss the available data sets and the popular metrics. Finally, we point out the possible future directions in this area and conclude this survey.
State Dependence and Unobserved Heterogeneity in the Extensive Margin of Trade
Authors: Julian Hinz, Amrei Stammann, Joschka Wanner
Date: 2020-04-27 00:00:00

We study the role and drivers of persistence in the extensive margin of bilateral trade. Motivated by a stylized heterogeneous firms model of international trade with market entry costs, we consider dynamic three-way fixed effects binary choice models and study the corresponding incidental parameter problem. The standard maximum likelihood estimator is consistent under asymptotics where all panel dimensions grow at a constant rate, but it has an asymptotic bias in its limiting distribution, invalidating inference even in situations where the bias appears to be small. Thus, we propose two different bias-corrected estimators. Monte Carlo simulations confirm their desirable statistical properties. We apply these estimators in a reassessment of the most commonly studied determinants of the extensive margin of trade. Both true state dependence and unobserved heterogeneity contribute considerably to trade persistence and taking this persistence into account matters significantly in identifying the effects of trade policies on the extensive margin.
Improving Neural Network Training in Low Dimensional Random Bases
Authors: Frithjof Gressmann, Zach Eaton-Rosen, Carlo Luschi
Date: 2020-11-09 00:00:00
Proceeding: NeurIPS 2020 12

Stochastic Gradient Descent (SGD) has proven to be remarkably effective in optimizing deep neural networks that employ ever-larger numbers of parameters. Yet, improving the efficiency of large-scale optimization remains a vital and highly active area of research. Recent work has shown that deep neural networks can be optimized in randomly-projected subspaces of much smaller dimensionality than their native parameter space. While such training is promising for more efficient and scalable optimization schemes, its practical application is limited by inferior optimization performance. Here, we improve on recent random subspace approaches as follows: Firstly, we show that keeping the random projection fixed throughout training is detrimental to optimization. We propose re-drawing the random subspace at each step, which yields significantly better performance. We realize further improvements by applying independent projections to different parts of the network, making the approximation more efficient as network dimensionality grows. To implement these experiments, we leverage hardware-accelerated pseudo-random number generation to construct the random projections on-demand at every optimization step, allowing us to distribute the computation of independent random directions across multiple workers with shared random seeds. This yields significant reductions in memory and is up to 10 times faster for the workloads in question.
An Empirical Study on Text-Independent Speaker Verification based on the GE2E Method
Authors: Soroosh Tayebi Arasteh
Date: 2020-11-10 00:00:00
Tasks: Speaker Recognition, Speaker Verification, Text-Independent Speaker Verification

While many researchers in the speaker recognition area have started to replace the former classical state-of-the-art methods with deep learning techniques, some of the traditional i-vector-based methods are still state-of-the-art in the context of text-independent speaker verification. Google's Generalized End-to-End Loss for Speaker Verification (GE2E), a deep learning-based technique using long short-term memory units, has recently gained a lot of attention due to its speed in convergence and generalization. In this study, we aim at further studying the GE2E method and comparing different scenarios in order to investigate all of its aspects. Various experiments including the effects of a random sampling of test and enrollment utterances, test utterance duration, and the number of enrollment utterances are discussed in this article. Furthermore, we compare the GE2E method with the baseline state-of-the-art i-vector-based methods for text-independent speaker verification and show that it outperforms them by resulting in lower error rates while being end-to-end and requiring less training time for convergence.
Adaptive Learning of Rank-One Models for Efficient Pairwise Sequence Alignment
Authors: Govinda M. Kamath, Tavor Z. Baharav, Ilan Shomorony
Date: 2020-11-09 00:00:00
Proceeding: NeurIPS 2020 12

Pairwise alignment of DNA sequencing data is a ubiquitous task in bioinformatics and typically represents a heavy computational burden. State-of-the-art approaches to speed up this task use hashing to identify short segments (k-mers) that are shared by pairs of reads, which can then be used to estimate alignment scores. However, when the number of reads is large, accurately estimating alignment scores for all pairs is still very costly. Moreover, in practice, one is only interested in identifying pairs of reads with large alignment scores. In this work, we propose a new approach to pairwise alignment estimation based on two key new ingredients. The first ingredient is to cast the problem of pairwise alignment estimation under a general framework of rank-one crowdsourcing models, where the workers' responses correspond to k-mer hash collisions. These models can be accurately solved via a spectral decomposition of the response matrix. The second ingredient is to utilise a multi-armed bandit algorithm to adaptively refine this spectral estimator only for read pairs that are likely to have large alignments. The resulting algorithm iteratively performs a spectral decomposition of the response matrix for adaptively chosen subsets of the read pairs.
f-IRL: Inverse Reinforcement Learning via State Marginal Matching
Authors: Tianwei Ni, Harshit Sikchi, YuFei Wang, Tejus Gupta, Lisa Lee, Benjamin Eysenbach
Date: 2020-11-09 00:00:00
Tasks: Imitation Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Imitation learning is well-suited for robotic tasks where it is difficult to directly program the behavior or specify a cost for optimal control. In this work, we propose a method for learning the reward function (and the corresponding policy) to match the expert state density. Our main result is the analytic gradient of any f-divergence between the agent and expert state distribution w.r.t. reward parameters. Based on the derived gradient, we present an algorithm, f-IRL, that recovers a stationary reward function from the expert density by gradient descent. We show that f-IRL can learn behaviors from a hand-designed target state density or implicitly through expert observations. Our method outperforms adversarial imitation learning methods in terms of sample efficiency and the required number of expert trajectories on IRL benchmarks. Moreover, we show that the recovered reward function can be used to quickly solve downstream tasks, and empirically demonstrate its utility on hard-to-explore tasks and for behavior transfer across changes in dynamics.
ATCN: Resource-Efficient Processing of Time Series on Edge
Authors: Mohammadreza Baharani, Hamed Tabkhi
Date: 2020-11-10 00:00:00
Tasks: General Classification, Heartbeat Classification, Time Series, Time Series Analysis, Time Series Prediction

This paper presents a scalable deep learning model called Agile Temporal Convolutional Network (ATCN) for high-accurate fast classification and time series prediction in resource-constrained embedded systems. ATCN is a family of compact networks with formalized hyperparameters that enable application-specific adjustments to be made to the model architecture. It is primarily designed for embedded edge devices with very limited performance and memory, such as wearable biomedical devices and real-time reliability monitoring systems. ATCN makes fundamental improvements over the mainstream temporal convolutional neural networks, including residual connections to increase the network depth and accuracy, and the incorporation of separable depth-wise convolution to reduce the computational complexity of the model. As part of the present work, two ATCN families, namely T0, and T1 are also presented and evaluated on different ranges of embedded processors - Cortex-M7 and Cortex-A57 processor. An evaluation of the ATCN models against the best-in-class InceptionTime and MiniRocket shows that ATCN almost maintains accuracy while improving the execution time on a broad range of embedded and cyber-physical applications with demand for real-time processing on the embedded edge. At the same time, in contrast to existing solutions, ATCN is the first time-series classifier based on deep learning that can be run bare-metal on embedded microcontrollers (Cortex-M7) with limited computational performance and memory capacity while delivering state-of-the-art accuracy.
Personalized Query Rewriting in Conversational AI Agents
Authors: Alireza Roshan-Ghias, Clint Solomon Mathialagan, Pragaash Ponnusamy, Lambert Mathias, Chenlei Guo
Date: 2020-11-09 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Diversity, Form, Natural Language Understanding, Retrieval, speech-recognition, Speech Recognition, Spoken Language Understanding

Spoken language understanding (SLU) systems in conversational AI agents often experience errors in the form of misrecognitions by automatic speech recognition (ASR) or semantic gaps in natural language understanding (NLU). These errors easily translate to user frustrations, particularly so in recurrent events e.g. regularly toggling an appliance, calling a frequent contact, etc. In this work, we propose a query rewriting approach by leveraging users' historically successful interactions as a form of memory. We present a neural retrieval model and a pointer-generator network with hierarchical attention and show that they perform significantly better at the query rewriting task with the aforementioned user memories than without. We also highlight how our approach with the proposed models leverages the structural and semantic diversity in ASR's output towards recovering users' intents.