pwc-archive/papers-with-abstracts huggingface dataset:
UniTE -- The Best of Both Worlds: Unifying Function-Fitting and Aggregation-Based Approaches to Travel Time and Travel Speed Estimation
Authors: Tobias Skovgaard Jepsen, Christian S. Jensen, Thomas Dyhre Nielsen
Date: 2021-04-27 00:00:00
Tasks: Travel Time Estimation

Travel time or speed estimation are part of many intelligent transportation applications. Existing estimation approaches rely on either function fitting or aggregation and represent different trade-offs between generalizability and accuracy. Function-fitting approaches learn functions that map feature vectors of, e.g., routes, to travel time or speed estimates, which enables generalization to unseen routes. However, mapping functions are imperfect and offer poor accuracy in practice. Aggregation-based approaches instead form estimates by aggregating historical data, e.g., traversal data for routes. This enables very high accuracy given sufficient data. However, they rely on simplistic heuristics when insufficient data is available, yielding poor generalizability. We present a Unifying approach to Travel time and speed Estimation (UniTE) that combines function-fitting and aggregation-based approaches into a unified framework that aims to achieve the generalizability of function-fitting approaches and the accuracy of aggregation-based approaches. An empirical study finds that an instance of UniTE can improve the accuracies of travel speed distribution and travel time estimation by $40-64\%$ and $3-23\%$, respectively, compared to using function fitting or aggregation alone
Building-GAN: Graph-Conditioned Architectural Volumetric Design Generation
Authors: Kai-Hung Chang, Chin-Yi Cheng, Jieliang Luo, Shingo Murata, Mehdi Nourbakhsh, Yoshito Tsuji
Date: 2021-04-27 00:00:00
Proceeding: ICCV 2021 10
Tasks: 3D geometry, Diversity, Graph Neural Network, Layout Generation

Volumetric design is the first and critical step for professional building design, where architects not only depict the rough 3D geometry of the building but also specify the programs to form a 2D layout on each floor. Though 2D layout generation for a single story has been widely studied, there is no developed method for multi-story buildings. This paper focuses on volumetric design generation conditioned on an input program graph. Instead of outputting dense 3D voxels, we propose a new 3D representation named voxel graph that is both compact and expressive for building geometries. Our generator is a cross-modal graph neural network that uses a pointer mechanism to connect the input program graph and the output voxel graph, and the whole pipeline is trained using the adversarial framework. The generated designs are evaluated qualitatively by a user study and quantitatively using three metrics: quality, diversity, and connectivity accuracy. We show that our model generates realistic 3D volumetric designs and outperforms previous methods and baselines.
Inductive Program Synthesis over Noisy Datasets using Abstraction Refinement Based Optimization
Authors: Shivam Handa, Martin Rinard
Date: 2021-04-27 00:00:00
Tasks: Program Synthesis

We present a new synthesis algorithm to solve program synthesis over noisy datasets, i.e., data that may contain incorrect/corrupted input-output examples. Our algorithm uses an abstraction refinement based optimization process to synthesize programs which optimize the tradeoff between the loss over the noisy dataset and the complexity of the synthesized program. The algorithm uses abstractions to divide the search space of programs into subspaces by computing an abstract value that represents outputs for all programs in a subspace. The abstract value allows our algorithm to compute, for each subspace, a sound approximate lower bound of the loss over all programs in the subspace. It iteratively refines these abstractions to further subdivide the space into smaller subspaces, prune subspaces that do not contain an optimal program, and eventually synthesize an optimal program. We implemented this algorithm in a tool called Rose. We compare Rose to a current state-of-the-art noisy program synthesis system using the SyGuS 2018 benchmark suite. Our evaluation demonstrates that Rose significantly outperforms this previous system: on two noisy benchmark program synthesis problems sets drawn from the SyGus 2018 benchmark suite, Rose delivers speedups of up to 1587 and 81.7, with median speedups of 20.5 and 81.7. Rose also terminates on 20 (out of 54) and 4 (out of 11) more benchmark problems than the previous system. Both Rose and the previous system synthesize programs that are optimal over the provided noisy data sets. For the majority of the problems in the benchmark sets ($272$ out of $286$), the synthesized programs also produce correct outputs for all inputs in the original (unseen) noise-free data set. These results highlight the benefits that Rose can deliver for effective noisy program synthesis.
Multi-fairness under class-imbalance
Authors: Arjun Roy, Vasileios Iosifidis, Eirini Ntoutsi
Date: 2021-04-27 00:00:00
Tasks: Attribute, Decision Making, Fairness

Recent studies showed that datasets used in fairness-aware machine learning for multiple protected attributes (referred to as multi-discrimination hereafter) are often imbalanced. The class-imbalance problem is more severe for the often underrepresented protected group (e.g. female, non-white, etc.) in the critical minority class. Still, existing methods focus only on the overall error-discrimination trade-off, ignoring the imbalance problem, thus amplifying the prevalent bias in the minority classes. Therefore, solutions are needed to solve the combined problem of multi-discrimination and class-imbalance. To this end, we introduce a new fairness measure, Multi-Max Mistreatment (MMM), which considers both (multi-attribute) protected group and class membership of instances to measure discrimination. To solve the combined problem, we propose a boosting approach that incorporates MMM-costs in the distribution update and post-training selects the optimal trade-off among accurate, balanced, and fair solutions. The experimental results show the superiority of our approach against state-of-the-art methods in producing the best balanced performance across groups and classes and the best accuracy for the protected groups in the minority class.
A Dual Process Model for Optimizing Cross Entropy in Neural Networks
Authors: Stefan Jaeger
Date: 2021-04-27 00:00:00

Minimizing cross-entropy is a widely used method for training artificial neural networks. Many training procedures based on backpropagation use cross-entropy directly as their loss function. Instead, this theoretical essay investigates a dual process model with two processes, in which one process minimizes the Kullback-Leibler divergence while its dual counterpart minimizes the Shannon entropy. Postulating that learning consists of two dual processes complementing each other, the model defines an equilibrium state for both processes in which the loss function assumes its minimum. An advantage of the proposed model is that it allows deriving the optimal learning rate and momentum weight to update network weights for backpropagation. Furthermore, the model introduces the golden ratio and complex numbers as important new concepts in machine learning.
MULTIMODAL ANALYSIS: Informed content estimation and audio source separation
Authors: Gabriel Meseguer-Brocal
Date: 2021-04-27 00:00:00
Tasks: Audio Source Separation

This dissertation proposes the study of multimodal learning in the context of musical signals. Throughout, we focus on the interaction between audio signals and text information. Among the many text sources related to music that can be used (e.g. reviews, metadata, or social network feedback), we concentrate on lyrics. The singing voice directly connects the audio signal and the text information in a unique way, combining melody and lyrics where a linguistic dimension complements the abstraction of musical instruments. Our study focuses on the audio and lyrics interaction for targeting source separation and informed content estimation.
Early Classification of Time Series is Meaningful
Authors: Youssef Achenchabe, Alexis Bondu, Antoine Cornuéjols, Vincent Lemaire
Date: 2021-04-27 00:00:00
Tasks: Classification, Early  Classification, General Classification, Time Series, Time Series Analysis

Many approaches have been proposed for early classification of time series in light of its significance in a wide range of applications including healthcare, transportation and finance. However, recently a preprint saved on Arxiv claim that all research done for almost 20 years now on the Early Classification of Time Series is useless, or, at the very least, ill-oriented because severely lacking a strong ground. In this paper, we answer in detail the main issues and misunderstandings raised by the authors of the preprint, and propose directions to further expand the fields of application of early classification of time series.
Autotuning PolyBench Benchmarks with LLVM Clang/Polly Loop Optimization Pragmas Using Bayesian Optimization (extended version)
Authors: Xingfu Wu, Michael Kruse, Prasanna Balaprakash, Hal Finkel, Paul Hovland, Valerie Taylor, Mary Hall
Date: 2021-04-27 00:00:00
Tasks: Bayesian Optimization

In this paper, we develop a ytopt autotuning framework that leverages Bayesian optimization to explore the parameter space search and compare four different supervised learning methods within Bayesian optimization and evaluate their effectiveness. We select six of the most complex PolyBench benchmarks and apply the newly developed LLVM Clang/Polly loop optimization pragmas to the benchmarks to optimize them. We then use the autotuning framework to optimize the pragma parameters to improve their performance. The experimental results show that our autotuning approach outperforms the other compiling methods to provide the smallest execution time for the benchmarks syr2k, 3mm, heat-3d, lu, and covariance with two large datasets in 200 code evaluations for effectively searching the parameter spaces with up to 170,368 different configurations. We find that the Floyd-Warshall benchmark did not benefit from autotuning because Polly uses heuristics to optimize the benchmark to make it run much slower. To cope with this issue, we provide some compiler option solutions to improve the performance. Then we present loop autotuning without a user's knowledge using a simple mctree autotuning framework to further improve the performance of the Floyd-Warshall benchmark. We also extend the ytopt autotuning framework to tune a deep learning application.
Learning low bending and low distortion manifold embeddings
Authors: Juliane Braunsmann, Marko Rajković, Martin Rumpf, Benedikt Wirth
Date: 2021-04-27 00:00:00

Autoencoders are a widespread tool in machine learning to transform high-dimensional data into a lowerdimensional representation which still exhibits the essential characteristics of the input. The encoder provides an embedding from the input data manifold into a latent space which may then be used for further processing. For instance, learning interpolation on the manifold may be simplified via the new manifold representation in latent space. The efficiency of such further processing heavily depends on the regularity and structure of the embedding. In this article, the embedding into latent space is regularized via a loss function that promotes an as isometric and as flat embedding as possible. The required training data comprises pairs of nearby points on the input manifold together with their local distance and their local Frechet average. This regularity loss functional even allows to train the encoder on its own. The loss functional is computed via a Monte Carlo integration which is shown to be consistent with a geometric loss functional defined directly on the embedding map. Numerical tests are performed using image data that encodes different data manifolds. The results show that smooth manifold embeddings in latent space are obtained. These embeddings are regular enough such that interpolation between not too distant points on the manifold is well approximated by linear interpolation in latent space.
Efficient channel charting via phase-insensitive distance computation
Authors: Luc Le Magoarou
Date: 2021-04-27 00:00:00
Tasks: Dimensionality Reduction, Scheduling

Channel charting is an unsupervised learning task whose objective is to encode channels so that the obtained representation reflects the relative spatial locations of the corresponding users. It has many potential applications, ranging from user scheduling to proactive handover. In this paper, a channel charting method is proposed, based on a distance measure specifically designed to reduce the effect of small scale fading, which is an irrelevant phenomenon with respect to the channel charting task. A nonlinear dimensionality reduction technique aimed at preserving local distances (Isomap) is then applied to actually get the channel representation. The approach is empirically validated on realistic synthetic \new{multipath} MIMO channels, achieving better results than previously proposed approaches, at a lower cost.
Structured Sparse Non-negative Matrix Factorization with L20-Norm for scRNA-seq Data Analysis
Authors: Wenwen Min, Taosheng Xu, Xiang Wan, Tsung-Hui Chang
Date: 2021-04-27 00:00:00
Tasks: Clustering, Dimensionality Reduction, feature selection

Non-negative matrix factorization (NMF) is a powerful tool for dimensionality reduction and clustering. Unfortunately, the interpretation of the clustering results from NMF is difficult, especially for the high-dimensional biological data without effective feature selection. In this paper, we first introduce a row-sparse NMF with $\ell_{2,0}$-norm constraint (NMF_$\ell_{20}$), where the basis matrix $W$ is constrained by the $\ell_{2,0}$-norm, such that $W$ has a row-sparsity pattern with feature selection. It is a challenge to solve the model, because the $\ell_{2,0}$-norm is non-convex and non-smooth. Fortunately, we prove that the $\ell_{2,0}$-norm satisfies the Kurdyka-\L{ojasiewicz} property. Based on the finding, we present a proximal alternating linearized minimization algorithm and its monotone accelerated version to solve the NMF_$\ell_{20}$ model. In addition, we also present a orthogonal NMF with $\ell_{2,0}$-norm constraint (ONMF_$\ell_{20}$) to enhance the clustering performance by using a non-negative orthogonal constraint. We propose an efficient algorithm to solve ONMF_$\ell_{20}$ by transforming it into a series of constrained and penalized matrix factorization problems. The results on numerical and scRNA-seq datasets demonstrate the efficiency of our methods in comparison with existing methods.
A unified framework for Hamiltonian deep neural networks
Authors: Clara L. Galimberti, Liang Xu, Giancarlo Ferrari Trecate
Date: 2021-04-27 00:00:00

Training deep neural networks (DNNs) can be difficult due to the occurrence of vanishing/exploding gradients during weight optimization. To avoid this problem, we propose a class of DNNs stemming from the time discretization of Hamiltonian systems. The time-invariant version of the corresponding Hamiltonian models enjoys marginal stability, a property that, as shown in previous works and for specific DNNs architectures, can mitigate convergence to zero or divergence of gradients. In the present paper, we formally study this feature by deriving and analysing the backward gradient dynamics in continuous time. The proposed Hamiltonian framework, besides encompassing existing networks inspired by marginally stable ODEs, allows one to derive new and more expressive architectures. The good performance of the novel DNNs is demonstrated on benchmark classification problems, including digit recognition using the MNIST dataset.
One Backward from Ten Forward, Subsampling for Large-Scale Deep Learning
Authors: Chaosheng Dong, Xiaojie Jin, Weihao Gao, Yijia Wang, Hongyi Zhang, Xiang Wu, Jianchao Yang, Xiaobing Liu
Date: 2021-04-27 00:00:00

Deep learning models in large-scale machine learning systems are often continuously trained with enormous data from production environments. The sheer volume of streaming training data poses a significant challenge to real-time training subsystems and ad-hoc sampling is the standard practice. Our key insight is that these deployed ML systems continuously perform forward passes on data instances during inference, but ad-hoc sampling does not take advantage of this substantial computational effort. Therefore, we propose to record a constant amount of information per instance from these forward passes. The extra information measurably improves the selection of which data instances should participate in forward and backward passes. A novel optimization framework is proposed to analyze this problem and we provide an efficient approximation algorithm under the framework of Mini-batch gradient descent as a practical solution. We also demonstrate the effectiveness of our framework and algorithm on several large-scale classification and regression tasks, when compared with competitive baselines widely used in industry.
Detection of Fake Users in SMPs Using NLP and Graph Embeddings
Authors: Manojit Chakraborty, Shubham Das, Radhika Mamidi
Date: 2021-04-27 00:00:00
Tasks: Graph Representation Learning, Representation Learning

Social Media Platforms (SMPs) like Facebook, Twitter, Instagram etc. have large user base all around the world that generates huge amount of data every second. This includes a lot of posts by fake and spam users, typically used by many organisations around the globe to have competitive edge over others. In this work, we aim at detecting such user accounts in Twitter using a novel approach. We show how to distinguish between Genuine and Spam accounts in Twitter using a combination of Graph Representation Learning and Natural Language Processing techniques.
Property Inference Attacks on Convolutional Neural Networks: Influence and Implications of Target Model's Complexity
Authors: Mathias P. M. Parisot, Balazs Pejo, Dayana Spagnuelo
Date: 2021-04-27 00:00:00

Machine learning models' goal is to make correct predictions for specific tasks by learning important properties and patterns from data. By doing so, there is a chance that the model learns properties that are unrelated to its primary task. Property Inference Attacks exploit this and aim to infer from a given model (\ie the target model) properties about the training dataset seemingly unrelated to the model's primary goal. If the training data is sensitive, such an attack could lead to privacy leakage. This paper investigates the influence of the target model's complexity on the accuracy of this type of attack, focusing on convolutional neural network classifiers. We perform attacks on models that are trained on facial images to predict whether someone's mouth is open. Our attacks' goal is to infer whether the training dataset is balanced gender-wise. Our findings reveal that the risk of a privacy breach is present independently of the target model's complexity: for all studied architectures, the attack's accuracy is clearly over the baseline. We discuss the implication of the property inference on personal data in the light of Data Protection Regulations and Guidelines.
Generating Lead Sheets with Affect: A Novel Conditional seq2seq Framework
Authors: Dimos Makris, Kat R. Agres, Dorien Herremans
Date: 2021-04-27 00:00:00
Tasks: Machine Translation, Music Generation, NMT

The field of automatic music composition has seen great progress in the last few years, much of which can be attributed to advances in deep neural networks. There are numerous studies that present different strategies for generating sheet music from scratch. The inclusion of high-level musical characteristics (e.g., perceived emotional qualities), however, as conditions for controlling the generation output remains a challenge. In this paper, we present a novel approach for calculating the valence (the positivity or negativity of the perceived emotion) of a chord progression within a lead sheet, using pre-defined mood tags proposed by music experts. Based on this approach, we propose a novel strategy for conditional lead sheet generation that allows us to steer the music generation in terms of valence, phrasing, and time signature. Our approach is similar to a Neural Machine Translation (NMT) problem, as we include high-level conditions in the encoder part of the sequence-to-sequence architectures used (i.e., long-short term memory networks, and a Transformer network). We conducted experiments to thoroughly analyze these two architectures. The results show that the proposed strategy is able to generate lead sheets in a controllable manner, resulting in distributions of musical attributes similar to those of the training dataset. We also verified through a subjective listening test that our approach is effective in controlling the valence of a generated chord progression.
Confined Gradient Descent: Privacy-preserving Optimization for Federated Learning
Authors: Yanjun Zhang, Guangdong Bai, Xue Li, Surya Nepal, Ryan K L Ko
Date: 2021-04-27 00:00:00
Tasks: Federated Learning, Privacy Preserving

Federated learning enables multiple participants to collaboratively train a model without aggregating the training data. Although the training data are kept within each participant and the local gradients can be securely synthesized, recent studies have shown that such privacy protection is insufficient. The global model parameters that have to be shared for optimization are susceptible to leak information about training data. In this work, we propose Confined Gradient Descent (CGD) that enhances privacy of federated learning by eliminating the sharing of global model parameters. CGD exploits the fact that a gradient descent optimization can start with a set of discrete points and converges to another set at the neighborhood of the global minimum of the objective function. It lets the participants independently train on their local data, and securely share the sum of local gradients to benefit each other. We formally demonstrate CGD's privacy enhancement over traditional FL. We prove that less information is exposed in CGD compared to that of traditional FL. CGD also guarantees desired model accuracy. We theoretically establish a convergence rate for CGD. We prove that the loss of the proprietary models learned for each participant against a model learned by aggregated training data is bounded. Extensive experimental results on two real-world datasets demonstrate the performance of CGD is comparable with the centralized learning, with marginal differences on validation loss (mostly within 0.05) and accuracy (mostly within 1%).