pwc-archive/papers-with-abstracts huggingface dataset:
Domain Adaptation for Semantic Parsing
Authors: Zechang Li, Yuxuan Lai, Yansong Feng, Dongyan Zhao
Date: 2020-06-23 00:00:00
Tasks: Domain Adaptation, Semantic Parsing

Recently, semantic parsing has attracted much attention in the community. Although many neural modeling efforts have greatly improved the performance, it still suffers from the data scarcity issue. In this paper, we propose a novel semantic parser for domain adaptation, where we have much fewer annotated data in the target domain compared to the source domain. Our semantic parser benefits from a two-stage coarse-to-fine framework, thus can provide different and accurate treatments for the two stages, i.e., focusing on domain invariant and domain specific information, respectively. In the coarse stage, our novel domain discrimination component and domain relevance attention encourage the model to learn transferable domain general structures. In the fine stage, the model is guided to concentrate on domain related details. Experiments on a benchmark dataset show that our method consistently outperforms several popular domain adaptation strategies. Additionally, we show that our model can well exploit limited target data to capture the difference between the source and target domain, even when the target domain has far fewer training instances.
Combining Neural Language Models for WordSense Induction
Authors: Nikolay Arefyev, Boris Sheludko, Tatiana Aleksashina
Date: 2020-06-23 00:00:00
Tasks: Word Sense Induction

Word sense induction (WSI) is the problem of grouping occurrences of an ambiguous word according to the expressed sense of this word. Recently a new approach to this task was proposed, which generates possible substitutes for the ambiguous word in a particular context using neural language models, and then clusters sparse bag-of-words vectors built from these substitutes. In this work, we apply this approach to the Russian language and improve it in two ways. First, we propose methods of combining left and right contexts, resulting in better substitutes generated. Second, instead of fixed number of clusters for all ambiguous words we propose a technique for selecting individual number of clusters for each word. Our approach established new state-of-the-art level, improving current best results of WSI for the Russian language on two RUSSE 2018 datasets by a large margin.
Primary quantization matrix estimation of double compressed JPEG images via CNN
Authors: Yakun Niu, Benedetta Tondi, Yao Zhao, Mauro Barni
Date: 2019-08-09 00:00:00
Tasks: Quantization

Available model-based techniques for the estimation of the primary quantization matrix in double-compressed JPEG images work only under specific conditions regarding the relationship between the first and second compression quality factors, and the alignment of the first and second JPEG compression grids. In this paper, we propose a single CNN-based estimation technique that can work under a very general range of settings. We do so, by adapting a dense CNN network to the problem at hand. Particular attention is paid to the choice of the loss function. Experimental results highlight several advantages of the new method, including: i) capability of working under very general conditions, ii) improved performance in terms of MSE and accuracy especially in the non-aligned case, iii) better spatial resolution due to the ability of providing good results also on small image patches.
A General Class of Transfer Learning Regression without Implementation Cost
Authors: Shunya Minami, Song Liu, Stephen Wu, Kenji Fukumizu, Ryo Yoshida
Date: 2020-06-23 00:00:00
Tasks: Density Ratio Estimation, regression, Transfer Learning

We propose a novel framework that unifies and extends existing methods of transfer learning (TL) for regression. To bridge a pretrained source model to the model on a target task, we introduce a density-ratio reweighting function, which is estimated through the Bayesian framework with a specific prior distribution. By changing two intrinsic hyperparameters and the choice of the density-ratio model, the proposed method can integrate three popular methods of TL: TL based on cross-domain similarity regularization, a probabilistic TL using the density-ratio estimation, and fine-tuning of pretrained neural networks. Moreover, the proposed method can benefit from its simple implementation without any additional cost; the regression model can be fully trained using off-the-shelf libraries for supervised learning in which the original output variable is simply transformed to a new output variable. We demonstrate its simplicity, generality, and applicability using various real data applications.
Non-Parametric Graph Learning for Bayesian Graph Neural Networks
Authors: Soumyasundar Pal, Saber Malekmohammadi, Florence Regol, Yingxue Zhang, Yishi Xu, Mark Coates
Date: 2020-06-23 00:00:00
Tasks: Graph Learning, Link Prediction, Node Classification

Graphs are ubiquitous in modelling relational structures. Recent endeavours in machine learning for graph-structured data have led to many architectures and learning algorithms. However, the graph used by these algorithms is often constructed based on inaccurate modelling assumptions and/or noisy data. As a result, it fails to represent the true relationships between nodes. A Bayesian framework which targets posterior inference of the graph by considering it as a random quantity can be beneficial. In this paper, we propose a novel non-parametric graph model for constructing the posterior distribution of graph adjacency matrices. The proposed model is flexible in the sense that it can effectively take into account the output of graph-based learning algorithms that target specific tasks. In addition, model inference scales well to large graphs. We demonstrate the advantages of this model in three different problem settings: node classification, link prediction and recommendation.
When Do Neural Networks Outperform Kernel Methods?
Authors: Behrooz Ghorbani, Song Mei, Theodor Misiakiewicz, Andrea Montanari
Date: 2020-06-24 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: image-classification, Image Classification

For a certain scaling of the initialization of stochastic gradient descent (SGD), wide neural networks (NN) have been shown to be well approximated by reproducing kernel Hilbert space (RKHS) methods. Recent empirical work showed that, for some classification tasks, RKHS methods can replace NNs without a large loss in performance. On the other hand, two-layers NNs are known to encode richer smoothness classes than RKHS and we know of special examples for which SGD-trained NN provably outperform RKHS. This is true even in the wide network limit, for a different scaling of the initialization. How can we reconcile the above claims? For which tasks do NNs outperform RKHS? If covariates are nearly isotropic, RKHS methods suffer from the curse of dimensionality, while NNs can overcome it by learning the best low-dimensional representation. Here we show that this curse of dimensionality becomes milder if the covariates display the same low-dimensional structure as the target function, and we precisely characterize this tradeoff. Building on these results, we present the spiked covariates model that can capture in a unified framework both behaviors observed in earlier work. We hypothesize that such a latent low-dimensional structure is present in image classification. We test numerically this hypothesis by showing that specific perturbations of the training distribution degrade the performances of RKHS methods much more significantly than NNs.
Design and Evaluation of Personalized Free Trials
Authors: Hema Yoganarasimhan, Ebrahim Barzegary, Abhishek Pani
Date: 2020-06-24 00:00:00

Free trial promotions, where users are given a limited time to try the product for free, are a commonly used customer acquisition strategy in the Software as a Service (SaaS) industry. We examine how trial length affect users' responsiveness, and seek to quantify the gains from personalizing the length of the free trial promotions. Our data come from a large-scale field experiment conducted by a leading SaaS firm, where new users were randomly assigned to 7, 14, or 30 days of free trial. First, we show that the 7-day trial to all consumers is the best uniform policy, with a 5.59% increase in subscriptions. Next, we develop a three-pronged framework for personalized policy design and evaluation. Using our framework, we develop seven personalized targeting policies based on linear regression, lasso, CART, random forest, XGBoost, causal tree, and causal forest, and evaluate their performances using the Inverse Propensity Score (IPS) estimator. We find that the personalized policy based on lasso performs the best, followed by the one based on XGBoost. In contrast, policies based on causal tree and causal forest perform poorly. We then link a method's effectiveness in designing policy with its ability to personalize the treatment sufficiently without over-fitting (i.e., capture spurious heterogeneity). Next, we segment consumers based on their optimal trial length and derive some substantive insights on the drivers of user behavior in this context. Finally, we show that policies designed to maximize short-run conversions also perform well on long-run outcomes such as consumer loyalty and profitability.
Simple and Scalable Parallelized Bayesian Optimization
Authors: Masahiro Nomura
Date: 2020-06-24 00:00:00
Tasks: Bayesian Optimization, BIG-bench Machine Learning, Hyperparameter Optimization

In recent years, leveraging parallel and distributed computational resources has become essential to solve problems of high computational cost. Bayesian optimization (BO) has shown attractive results in those expensive-to-evaluate problems such as hyperparameter optimization of machine learning algorithms. While many parallel BO methods have been developed to search efficiently utilizing these computational resources, these methods assumed synchronous settings or were not scalable. In this paper, we propose a simple and scalable BO method for asynchronous parallel settings. Experiments are carried out with a benchmark function and hyperparameter optimization of multi-layer perceptrons, which demonstrate the promising performance of the proposed method.
Distribution-Based Invariant Deep Networks for Learning Meta-Features
Authors: Gwendoline de Bie, Herilalaina Rakotoarison, Gabriel Peyré, Michèle Sebag
Date: 2020-06-24 00:00:00
Tasks: Benchmarking, General Classification, regression

Recent advances in deep learning from probability distributions successfully achieve classification or regression from distribution samples, thus invariant under permutation of the samples. The first contribution of the paper is to extend these neural architectures to achieve invariance under permutation of the features, too. The proposed architecture, called Dida, inherits the NN properties of universal approximation, and its robustness w.r.t. Lipschitz-bounded transformations of the input distribution is established. The second contribution is to empirically and comparatively demonstrate the merits of the approach on two tasks defined at the dataset level. On both tasks, Dida learns meta-features supporting the characterization of a (labelled) dataset. The first task consists of predicting whether two dataset patches are extracted from the same initial dataset. The second task consists of predicting whether the learning performance achieved by a hyper-parameter configuration under a fixed algorithm (ranging in k-NN, SVM, logistic regression and linear classifier with SGD) dominates that of another configuration, for a dataset extracted from the OpenML benchmarking suite. On both tasks, Dida outperforms the state of the art: DSS (Maron et al., 2020) and Dataset2Vec (Jomaa et al., 2019) architectures, as well as the models based on the hand-crafted meta-features of the literature.
Slice Sampling for General Completely Random Measures
Authors: Peiyuan Zhu, Alexandre Bouchard-Côté, Trevor Campbell
Date: 2020-06-24 00:00:00

Completely random measures provide a principled approach to creating flexible unsupervised models, where the number of latent features is infinite and the number of features that influence the data grows with the size of the data set. Due to the infinity the latent features, posterior inference requires either marginalization---resulting in dependence structures that prevent efficient computation via parallelization and conjugacy---or finite truncation, which arbitrarily limits the flexibility of the model. In this paper we present a novel Markov chain Monte Carlo algorithm for posterior inference that adaptively sets the truncation level using auxiliary slice variables, enabling efficient, parallelized computation without sacrificing flexibility. In contrast to past work that achieved this on a model-by-model basis, we provide a general recipe that is applicable to the broad class of completely random measure-based priors. The efficacy of the proposed algorithm is evaluated on several popular nonparametric models, demonstrating a higher effective sample size per second compared to algorithms using marginalization as well as a higher predictive performance compared to models employing fixed truncations.
Where does active travel fit within local community narratives of mobility space and place?
Authors: Alec Biehl, Ying Chen, Karla Sanabria-Veaz, David Uttal, Amanda Stathopoulos
Date: 2019-05-07 00:00:00
Tasks: Cultural Vocal Bursts Intensity Prediction, Sentiment Analysis

Encouraging sustainable mobility patterns is at the forefront of policymaking at all scales of governance as the collective consciousness surrounding climate change continues to expand. Not every community, however, possesses the necessary economic or socio-cultural capital to encourage modal shifts away from private motorized vehicles towards active modes. The current literature on `soft' policy emphasizes the importance of tailoring behavior change campaigns to individual or geographic context. Yet, there is a lack of insight and appropriate tools to promote active mobility and overcome transport disadvantage from the local community perspective. The current study investigates the promotion of walking and cycling adoption using a series of focus groups with local residents in two geographic communities, namely Chicago's (1) Humboldt Park neighborhood and (2) suburb of Evanston. The research approach combines traditional qualitative discourse analysis with quantitative text-mining tools, namely topic modeling and sentiment analysis. The analysis uncovers the local mobility culture, embedded norms and values associated with acceptance of active travel modes in different communities. We observe that underserved populations within diverse communities view active mobility simultaneously as a necessity and as a symbol of privilege that is sometimes at odds with the local culture. The mixed methods approach to analyzing community member discourses is translated into policy findings that are either tailored to local context or broadly applicable to curbing automobile dominance. Overall, residents of both Humboldt Park and Evanston envision a society in which multimodalism replaces car-centrism, but differences in the local physical and social environments would and should influence the manner in which overarching policy objectives are met.
Attentional Graph Convolutional Networks for Knowledge Concept Recommendation in MOOCs in a Heterogeneous View
Authors: Shen Wang, Jibing Gong, Jinlong Wang, Wenzheng Feng, Hao Peng, Jie Tang, Philip S. Yu
Date: 2020-06-23 00:00:00
Tasks: Graph Neural Network, Representation Learning

Massive open online courses are becoming a modish way for education, which provides a large-scale and open-access learning opportunity for students to grasp the knowledge. To attract students' interest, the recommendation system is applied by MOOCs providers to recommend courses to students. However, as a course usually consists of a number of video lectures, with each one covering some specific knowledge concepts, directly recommending courses overlook students'interest to some specific knowledge concepts. To fill this gap, in this paper, we study the problem of knowledge concept recommendation. We propose an end-to-end graph neural network-based approach calledAttentionalHeterogeneous Graph Convolutional Deep Knowledge Recommender(ACKRec) for knowledge concept recommendation in MOOCs. Like other recommendation problems, it suffers from sparsity issues. To address this issue, we leverage both content information and context information to learn the representation of entities via graph convolution network. In addition to students and knowledge concepts, we consider other types of entities (e.g., courses, videos, teachers) and construct a heterogeneous information network to capture the corresponding fruitful semantic relationships among different types of entities and incorporate them into the representation learning process. Specifically, we use meta-path on the HIN to guide the propagation of students' preferences. With the help of these meta-paths, the students' preference distribution with respect to a candidate knowledge concept can be captured. Furthermore, we propose an attention mechanism to adaptively fuse the context information from different meta-paths, in order to capture the different interests of different students. The promising experiment results show that the proposedACKRecis able to effectively recommend knowledge concepts to students pursuing online learning in MOOCs.
Adversarial Soft Advantage Fitting: Imitation Learning without Policy Optimization
Authors: Paul Barde, Julien Roy, Wonseok Jeon, Joelle Pineau, Christopher Pal, Derek Nowrouzezahrai
Date: 2020-06-23 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Imitation Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Adversarial Imitation Learning alternates between learning a discriminator -- which tells apart expert's demonstrations from generated ones -- and a generator's policy to produce trajectories that can fool this discriminator. This alternated optimization is known to be delicate in practice since it compounds unstable adversarial training with brittle and sample-inefficient reinforcement learning. We propose to remove the burden of the policy optimization steps by leveraging a novel discriminator formulation. Specifically, our discriminator is explicitly conditioned on two policies: the one from the previous generator's iteration and a learnable policy. When optimized, this discriminator directly learns the optimal generator's policy. Consequently, our discriminator's update solves the generator's optimization problem for free: learning a policy that imitates the expert does not require an additional optimization loop. This formulation effectively cuts by half the implementation and computational burden of Adversarial Imitation Learning algorithms by removing the Reinforcement Learning phase altogether. We show on a variety of tasks that our simpler approach is competitive to prevalent Imitation Learning methods.
Learning Potentials of Quantum Systems using Deep Neural Networks
Authors: Arijit Sehanobish, Hector H. Corzo, Onur Kara, David van Dijk
Date: 2020-06-23 00:00:00

Attempts to apply Neural Networks (NN) to a wide range of research problems have been ubiquitous and plentiful in recent literature. Particularly, the use of deep NNs for understanding complex physical and chemical phenomena has opened a new niche of science where the analysis tools from Machine Learning (ML) are combined with the computational concepts of the natural sciences. Reports from this unification of ML have presented evidence that NNs can learn classical Hamiltonian mechanics. This application of NNs to classical physics and its results motivate the following question: Can NNs be endowed with inductive biases through observation as means to provide insights into quantum phenomena? In this work, this question is addressed by investigating possible approximations for reconstructing the Hamiltonian of a quantum system in an unsupervised manner by using only limited information obtained from the system's probability distribution.
Non-Convex Structured Phase Retrieval
Authors: Namrata Vaswani
Date: 2020-06-23 00:00:00
Tasks: Astronomy, Retrieval, valid

Phase retrieval (PR), also sometimes referred to as quadratic sensing, is a problem that occurs in numerous signal and image acquisition domains ranging from optics, X-ray crystallography, Fourier ptychography, sub-diffraction imaging, and astronomy. In each of these domains, the physics of the acquisition system dictates that only the magnitude (intensity) of certain linear projections of the signal or image can be measured. Without any assumptions on the unknown signal, accurate recovery necessarily requires an over-complete set of measurements. The only way to reduce the measurements/sample complexity is to place extra assumptions on the unknown signal/image. A simple and practically valid set of assumptions is obtained by exploiting the structure inherently present in many natural signals or sequences of signals. Two commonly used structural assumptions are (i) sparsity of a given signal/image or (ii) a low rank model on the matrix formed by a set, e.g., a time sequence, of signals/images. Both have been explored for solving the PR problem in a sample-efficient fashion. This article describes this work, with a focus on non-convex approaches that come with sample complexity guarantees under simple assumptions. We also briefly describe other different types of structural assumptions that have been used in recent literature.
Connectivity-informed Drainage Network Generation using Deep Convolution Generative Adversarial Networks
Authors: Sung Eun Kim, Yongwon Seo, Junshik Hwang, Hongkyu Yoon, Jonghyun Lee
Date: 2020-06-16 00:00:00

Stochastic network modeling is often limited by high computational costs to generate a large number of networks enough for meaningful statistical evaluation. In this study, Deep Convolutional Generative Adversarial Networks (DCGANs) were applied to quickly reproduce drainage networks from the already generated network samples without repetitive long modeling of the stochastic network model, Gibb's model. In particular, we developed a novel connectivity-informed method that converts the drainage network images to the directional information of flow on each node of the drainage network, and then transform it into multiple binary layers where the connectivity constraints between nodes in the drainage network are stored. DCGANs trained with three different types of training samples were compared; 1) original drainage network images, 2) their corresponding directional information only, and 3) the connectivity-informed directional information. Comparison of generated images demonstrated that the novel connectivity-informed method outperformed the other two methods by training DCGANs more effectively and better reproducing accurate drainage networks due to its compact representation of the network complexity and connectivity. This work highlights that DCGANs can be applicable for high contrast images common in earth and material sciences where the network, fractures, and other high contrast features are important.