pwc-archive/papers-with-abstracts huggingface dataset:
nTreeClus: a Tree-based Sequence Encoder for Clustering Categorical Series
Authors: Hadi Jahanshahi, Mustafa Gokce Baydogan
Date: 2021-02-20 00:00:00
Tasks: Clustering, Time Series, Time Series Analysis

The overwhelming presence of categorical/sequential data in diverse domains emphasizes the importance of sequence mining. The challenging nature of sequences proves the need for continuing research to find a more accurate and faster approach providing a better understanding of their (dis)similarities. This paper proposes a new Model-based approach for clustering sequence data, namely nTreeClus. The proposed method deploys Tree-based Learners, k-mers, and autoregressive models for categorical time series, culminating with a novel numerical representation of the categorical sequences. Adopting this new representation, we cluster sequences, considering the inherent patterns in categorical time series. Accordingly, the model showed robustness to its parameter. Under different simulated scenarios, nTreeClus improved the baseline methods for various internal and external cluster validation metrics for up to 10.7% and 2.7%, respectively. The empirical evaluation using synthetic and real datasets, protein sequences, and categorical time series showed that nTreeClus is competitive or superior to most state-of-the-art algorithms.
Multilingual Answer Sentence Reranking via Automatically Translated Data
Authors: Thuy Vu, Alessandro Moschitti
Date: 2021-02-20 00:00:00
Tasks: Question Answering, Reranking, Sentence

We present a study on the design of multilingual Answer Sentence Selection (AS2) models, which are a core component of modern Question Answering (QA) systems. The main idea is to transfer data, created from one resource rich language, e.g., English, to other languages, less rich in terms of resources. The main findings of this paper are: (i) the training data for AS2 translated into a target language can be used to effectively fine-tune a Transformer-based model for that language; (ii) one multilingual Transformer model it is enough to rank answers in multiple languages; and (iii) mixed-language question/answer pairs can be used to fine-tune models to select answers from any language, where the input question is just in one language. This highly reduces the complexity and technical requirement of a multilingual QA system. Our experiments validate the findings above, showing a modest drop, at most 3%, with respect to the state-of-the-art English model.
Entity Structure Within and Throughout: Modeling Mention Dependencies for Document-Level Relation Extraction
Authors: Benfeng Xu, Quan Wang, Yajuan Lyu, Yong Zhu, Zhendong Mao
Date: 2021-02-20 00:00:00
Tasks: Document-level Relation Extraction, Relation, Relation Extraction

Entities, as the essential elements in relation extraction tasks, exhibit certain structure. In this work, we formulate such structure as distinctive dependencies between mention pairs. We then propose SSAN, which incorporates these structural dependencies within the standard self-attention mechanism and throughout the overall encoding stage. Specifically, we design two alternative transformation modules inside each self-attention building block to produce attentive biases so as to adaptively regularize its attention flow. Our experiments demonstrate the usefulness of the proposed entity structure and the effectiveness of SSAN. It significantly outperforms competitive baselines, achieving new state-of-the-art results on three popular document-level relation extraction datasets. We further provide ablation and visualization to show how the entity structure guides the model for better relation extraction. Our code is publicly available.
Game Mechanic Alignment Theory and Discovery
Authors: Michael Cerny Green, Ahmed Khalifa, Philip Bontrager, Rodrigo Canaan, Julian Togelius
Date: 2021-02-20 00:00:00

We present a new concept called Game Mechanic Alignment theory as a way to organize game mechanics through the lens of systemic rewards and agential motivations. By disentangling player and systemic influences, mechanics may be better identified for use in an automated tutorial generation system, which could tailor tutorials for a particular playstyle or player. Within, we apply this theory to several well-known games to demonstrate how designers can benefit from it, we describe a methodology for how to estimate "mechanic alignment", and we apply this methodology on multiple games in the GVGAI framework. We discuss how effectively this estimation captures agential motivations and systemic rewards and how our theory could be used as an alternative way to find mechanics for tutorial generation.
CDA: a Cost Efficient Content-based Multilingual Web Document Aligner
Authors: Thuy Vu, Alessandro Moschitti
Date: 2021-02-20 00:00:00
Proceeding: EACL 2021 2
Tasks: Machine Translation, Translation

We introduce a Content-based Document Alignment approach (CDA), an efficient method to align multilingual web documents based on content in creating parallel training data for machine translation (MT) systems operating at the industrial level. CDA works in two steps: (i) projecting documents of a web domain to a shared multilingual space; then (ii) aligning them based on the similarity of their representations in such space. We leverage lexical translation models to build vector representations using TF-IDF. CDA achieves performance comparable with state-of-the-art systems in the WMT-16 Bilingual Document Alignment Shared Task benchmark while operating in multilingual space. Besides, we created two web-scale datasets to examine the robustness of CDA in an industrial setting involving up to 28 languages and millions of documents. The experiments show that CDA is robust, cost-effective, and is significantly superior in (i) processing large and noisy web data and (ii) scaling to new and low-resourced languages.
GMLight: Lighting Estimation via Geometric Distribution Approximation
Authors: Fangneng Zhan, Yingchen Yu, Changgong Zhang, Rongliang Wu, WenBo Hu, Shijian Lu, Feiying Ma, Xuansong Xie, Ling Shao
Date: 2021-02-20 00:00:00
Tasks: Lighting Estimation, regression

Inferring the scene illumination from a single image is an essential yet challenging task in computer vision and computer graphics. Existing works estimate lighting by regressing representative illumination parameters or generating illumination maps directly. However, these methods often suffer from poor accuracy and generalization. This paper presents Geometric Mover's Light (GMLight), a lighting estimation framework that employs a regression network and a generative projector for effective illumination estimation. We parameterize illumination scenes in terms of the geometric light distribution, light intensity, ambient term, and auxiliary depth, which can be estimated by a regression network. Inspired by the earth mover's distance, we design a novel geometric mover's loss to guide the accurate regression of light distribution parameters. With the estimated light parameters, the generative projector synthesizes panoramic illumination maps with realistic appearance and high-frequency details. Extensive experiments show that GMLight achieves accurate illumination estimation and superior fidelity in relighting for 3D object insertion. The codes are available at \href{https://github.com/fnzhan/Illumination-Estimation}{https://github.com/fnzhan/Illumination-Estimation}.
Machine Translation Customization via Automatic Training Data Selection from the Web
Authors: Thuy Vu, Alessandro Moschitti
Date: 2021-02-20 00:00:00
Tasks: Machine Translation, Translation

Machine translation (MT) systems, especially when designed for an industrial setting, are trained with general parallel data derived from the Web. Thus, their style is typically driven by word/structure distribution coming from the average of many domains. In contrast, MT customers want translations to be specialized to their domain, for which they are typically able to provide text samples. We describe an approach for customizing MT systems on specific domains by selecting data similar to the target customer data to train neural translation models. We build document classifiers using monolingual target data, e.g., provided by the customers to select parallel training data from Web crawled data. Finally, we train MT models on our automatically selected data, obtaining a system specialized to the target domain. We tested our approach on the benchmark from WMT-18 Translation Task for News domains enabling comparisons with state-of-the-art MT systems. The results show that our models outperform the top systems while using less data and smaller models.
Towards Automatic Evaluation of Dialog Systems: A Model-Free Off-Policy Evaluation Approach
Authors: Haoming Jiang, Bo Dai, Mengjiao Yang, Tuo Zhao, Wei Wei
Date: 2021-02-20 00:00:00
Proceeding: EMNLP 2021 11
Tasks: Model-based Reinforcement Learning, Off-policy evaluation, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Text Generation

Reliable automatic evaluation of dialogue systems under an interactive environment has long been overdue. An ideal environment for evaluating dialog systems, also known as the Turing test, needs to involve human interaction, which is usually not affordable for large-scale experiments. Though researchers have attempted to use metrics (e.g., perplexity, BLEU) in language generation tasks or some model-based reinforcement learning methods (e.g., self-play evaluation) for automatic evaluation, these methods only show a very weak correlation with the actual human evaluation in practice. To bridge such a gap, we propose a new framework named ENIGMA for estimating human evaluation scores based on recent advances of off-policy evaluation in reinforcement learning. ENIGMA only requires a handful of pre-collected experience data, and therefore does not involve human interaction with the target policy during the evaluation, making automatic evaluations feasible. More importantly, ENIGMA is model-free and agnostic to the behavior policies for collecting the experience data (see details in Section 2), which significantly alleviates the technical difficulties of modeling complex dialogue environments and human behaviors. Our experiments show that ENIGMA significantly outperforms existing methods in terms of correlation with human evaluation scores.
Learning Neural Generative Dynamics for Molecular Conformation Generation
Authors: Minkai Xu, Shitong Luo, Yoshua Bengio, Jian Peng, Jian Tang
Date: 2021-02-20 00:00:00
Proceeding: ICLR 2021 1
Tasks: valid

We study how to generate molecule conformations (i.e., 3D structures) from a molecular graph. Traditional methods, such as molecular dynamics, sample conformations via computationally expensive simulations. Recently, machine learning methods have shown great potential by training on a large collection of conformation data. Challenges arise from the limited model capacity for capturing complex distributions of conformations and the difficulty in modeling long-range dependencies between atoms. Inspired by the recent progress in deep generative models, in this paper, we propose a novel probabilistic framework to generate valid and diverse conformations given a molecular graph. We propose a method combining the advantages of both flow-based and energy-based models, enjoying: (1) a high model capacity to estimate the multimodal conformation distribution; (2) explicitly capturing the complex long-range dependencies between atoms in the observation space. Extensive experiments demonstrate the superior performance of the proposed method on several benchmarks, including conformation generation and distance modeling tasks, with a significant improvement over existing generative models for molecular conformation sampling.
Generalization bounds for graph convolutional neural networks via Rademacher complexity
Authors: Shaogao Lv
Date: 2021-02-20 00:00:00
Tasks: Generalization Bounds

This paper aims at studying the sample complexity of graph convolutional networks (GCNs), by providing tight upper bounds of Rademacher complexity for GCN models with a single hidden layer. Under regularity conditions, theses derived complexity bounds explicitly depend on the largest eigenvalue of graph convolution filter and the degree distribution of the graph. Again, we provide a lower bound of Rademacher complexity for GCNs to show optimality of our derived upper bounds. Taking two commonly used examples as representatives, we discuss the implications of our results in designing graph convolution filters an graph distribution.
Elastic Similarity and Distance Measures for Multivariate Time Series
Authors: Ahmed Shifaz, Charlotte Pelletier, Francois Petitjean, Geoffrey I. Webb
Date: 2021-02-20 00:00:00
Tasks: Classification, Dynamic Time Warping, General Classification, Outlier Detection, Time Series, Time Series Analysis, Time Series Classification

This paper contributes multivariate versions of seven commonly used elastic similarity and distance measures for time series data analytics. Elastic similarity and distance measures are a class of similarity measures that can compensate for misalignments in the time axis of time series data. We adapt two existing strategies used in a multivariate version of the well-known Dynamic Time Warping (DTW), namely, Independent and Dependent DTW, to these seven measures. While these measures can be applied to various time series analysis tasks, we demonstrate their utility on multivariate time series classification using the nearest neighbor classifier. On 23 well-known datasets, we demonstrate that each of the measures but one achieves the highest accuracy relative to others on at least one dataset, supporting the value of developing a suite of multivariate similarity and distance measures. We also demonstrate that there are datasets for which either the dependent versions of all measures are more accurate than their independent counterparts or vice versa. In addition, we also construct a nearest neighbor-based ensemble of the measures and show that it is competitive to other state-of-the-art single-strategy multivariate time series classifiers.
On Single-User Interactive Beam Alignment in Next Generation Systems: A Deep Learning Viewpoint
Authors: Abbas Khalili, Sundeep Rangan, Elza Erkip
Date: 2021-02-20 00:00:00

Communication in high frequencies such as millimeter wave and terahertz suffer from high path-loss and intense shadowing which necessitates beamforming for reliable data transmission. On the other hand, at high frequencies the channels are sparse and consist of few spatial clusters. Therefore, beam alignment (BA) strategies are used to find the direction of these channel clusters and adjust the width of the beam used for data transmission. In this work, a single-user uplink scenario where the channel has one dominant cluster is considered. It is assumed that the user transmits a set of BA packets over a fixed duration. Meanwhile, the base-station (BS) uses different probing beams to scan different angular regions. Since the BS measurements are noisy, it is not possible to find a narrow beam that includes the angle of arrival (AoA) of the user with probability one. Therefore, the BS allocates a narrow beam to the user which includes the AoA of the user with a predetermined error probability while minimizing the expected beamwidth of the allocated beam. Due to intractability of this noisy BA problem, here this problem is posed as an end-to-end optimization of a deep neural network (DNN) and effects of different loss functions are discussed and investigated. It is observed that the proposed DNN based BA, at high SNRs, achieves a performance close to that of the optimal BA when there is no-noise and for all SNRs, outperforms state-of-the-art.
ALMA: Alternating Minimization Algorithm for Clustering Mixture Multilayer Network
Authors: Xing Fan, Marianna Pensky, Feng Yu, Teng Zhang
Date: 2021-02-20 00:00:00
Tasks: Clustering, Stochastic Block Model, Tensor Decomposition

The paper considers a Mixture Multilayer Stochastic Block Model (MMLSBM), where layers can be partitioned into groups of similar networks, and networks in each group are equipped with a distinct Stochastic Block Model. The goal is to partition the multilayer network into clusters of similar layers, and to identify communities in those layers. Jing et al. (2020) introduced the MMLSBM and developed a clustering methodology, TWIST, based on regularized tensor decomposition. The present paper proposes a different technique, an alternating minimization algorithm (ALMA), that aims at simultaneous recovery of the layer partition, together with estimation of the matrices of connection probabilities of the distinct layers. Compared to TWIST, ALMA achieves higher accuracy both theoretically and numerically.
Logarithmic Regret in Feature-based Dynamic Pricing
Authors: Jianyu Xu, Yu-Xiang Wang
Date: 2021-02-20 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Marketing

Feature-based dynamic pricing is an increasingly popular model of setting prices for highly differentiated products with applications in digital marketing, online sales, real estate and so on. The problem was formally studied as an online learning problem [Javanmard & Nazerzadeh, 2019] where a seller needs to propose prices on the fly for a sequence of $T$ products based on their features $x$ while having a small regret relative to the best -- "omniscient" -- pricing strategy she could have come up with in hindsight. We revisit this problem and provide two algorithms (EMLP and ONSP) for stochastic and adversarial feature settings, respectively, and prove the optimal $O(d\log{T})$ regret bounds for both. In comparison, the best existing results are $O\left(\min\left\{\frac{1}{\lambda_{\min}^2}\log{T}, \sqrt{T}\right\}\right)$ and $O(T^{2/3})$ respectively, with $\lambda_{\min}$ being the smallest eigenvalue of $\mathbb{E}[xx^T]$ that could be arbitrarily close to $0$. We also prove an $\Omega(\sqrt{T})$ information-theoretic lower bound for a slightly more general setting, which demonstrates that "knowing-the-demand-curve" leads to an exponential improvement in feature-based dynamic pricing.
Hard-Attention for Scalable Image Classification
Authors: Athanasios Papadopoulos, Paweł Korus, Nasir Memon
Date: 2021-02-20 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Classification, Deep Attention, General Classification, Hard Attention, image-classification, Image Classification

Can we leverage high-resolution information without the unsustainable quadratic complexity to input scale? We propose Traversal Network (TNet), a novel multi-scale hard-attention architecture, which traverses image scale-space in a top-down fashion, visiting only the most informative image regions along the way. TNet offers an adjustable trade-off between accuracy and complexity, by changing the number of attended image locations. We compare our model against hard-attention baselines on ImageNet, achieving higher accuracy with less resources (FLOPs, processing time and memory). We further test our model on fMoW dataset, where we process satellite images of size up to $896 \times 896$ px, getting up to $2.5$x faster processing compared to baselines operating on the same resolution, while achieving higher accuracy as well. TNet is modular, meaning that most classification models could be adopted as its backbone for feature extraction, making the reported performance gains orthogonal to benefits offered by existing optimized deep models. Finally, hard-attention guarantees a degree of interpretability to our model's predictions, without any extra cost beyond inference. Code is available at $\href{https://github.com/Tpap/TNet}{github.com/Tpap/TNet}$.
CKNet: A Convolutional Neural Network Based on Koopman Operator for Modeling Latent Dynamics from Pixels
Authors: Yongqian Xiao, Xin Xu, QianLi Lin
Date: 2021-02-19 00:00:00
Tasks: MuJoCo

With the development of end-to-end control based on deep learning, it is important to study new system modeling techniques to realize dynamics modeling with high-dimensional inputs. In this paper, a novel Koopman-based deep convolutional network, called CKNet, is proposed to identify latent dynamics from raw pixels. CKNet learns an encoder and decoder to play the role of the Koopman eigenfunctions and modes, respectively. The Koopman eigenvalues can be approximated by eigenvalues of the learned state transition matrix. The deterministic convolutional Koopman network (DCKNet) and the variational convolutional Koopman network (VCKNet) are proposed to span some subspace for approximating the Koopman operator respectively. Because CKNet is trained under the constraints of the Koopman theory, the identified latent dynamics is in a linear form and has good interpretability. Besides, the state transition and control matrices are trained as trainable tensors so that the identified dynamics is also time-invariant. We also design an auxiliary weight term for reducing multi-step linearity and prediction losses. Experiments were conducted on two offline trained and four online trained nonlinear forced dynamical systems with continuous action spaces in Gym and Mujoco environment respectively, and the results show that identified dynamics are adequate for approximating the latent dynamics and generating clear images. Especially for offline trained cases, this work confirms CKNet from a novel perspective that we visualize the evolutionary processes of the latent states and the Koopman eigenfunctions with DCKNet and VCKNet separately to each task based on the same episode and results demonstrate that different approaches learn similar features in shapes.
Linear Classifiers in Product Space Forms
Authors: Puoya Tabaghi, Chao Pan, Eli Chien, Jianhao Peng, Olgica Milenkovic
Date: 2021-02-19 00:00:00

Embedding methods for product spaces are powerful techniques for low-distortion and low-dimensional representation of complex data structures. Here, we address the new problem of linear classification in product space forms -- products of Euclidean, spherical, and hyperbolic spaces. First, we describe novel formulations for linear classifiers on a Riemannian manifold using geodesics and Riemannian metrics which generalize straight lines and inner products in vector spaces. Second, we prove that linear classifiers in $d$-dimensional space forms of any curvature have the same expressive power, i.e., they can shatter exactly $d+1$ points. Third, we formalize linear classifiers in product space forms, describe the first known perceptron and support vector machine classifiers for such spaces and establish rigorous convergence results for perceptrons. Moreover, we prove that the Vapnik-Chervonenkis dimension of linear classifiers in a product space form of dimension $d$ is \emph{at least} $d+1$. We support our theoretical findings with simulations on several datasets, including synthetic data, image data, and single-cell RNA sequencing (scRNA-seq) data. The results show that classification in low-dimensional product space forms for scRNA-seq data offers, on average, a performance improvement of $\sim15\%$ when compared to that in Euclidean spaces of the same dimension.