pwc-archive/papers-with-abstracts huggingface dataset:
Oil and Gas Pipeline Monitoring during COVID-19 Pandemic via Unmanned Aerial Vehicle
Authors: Myssar Jabbar Hammood Al-Battbootti, Iuliana Marin, Nicolae Goga, Ramona Popa
Date: 2021-11-15 00:00:00

The vast network of oil and gas transmission pipelines requires periodic monitoring for maintenance and hazard inspection to avoid equipment failure and potential accidents. The severe COVID-19 pandemic situation forced the companies to shrink the size of their teams. One risk which is faced on-site is represented by the uncontrolled release of flammable oil and gas. Among many inspection methods, the unmanned aerial vehicle system contains flexibility and stability. Unmanned aerial vehicles can transfer data in real-time, while they are doing their monitoring tasks. The current article focuses on unmanned aerial vehicles equipped with optical sensing and artificial intelligence, especially image recognition with deep learning techniques for pipeline surveillance. Unmanned aerial vehicles can be used for regular patrolling duties to identify and capture images and videos of the area of interest. Places that are hard to reach will be accessed faster, cheaper and with less risk. The current paper is based on the idea of capturing video and images of drone-based inspections, which can discover several potential hazardous problems before they become dangerous. Damage can emerge as a weakening of the cladding on the external pipe insulation. There can also be the case when the thickness of piping through external corrosion can occur. The paper describes a survey completed by experts from the oil and gas industry done for finding the functional and non-functional requirements of the proposed system.
2nd Place Solution to Facebook AI Image Similarity Challenge Matching Track
Authors: SeungKee Jeon
Date: 2021-11-15 00:00:00
Tasks: Self-Supervised Learning

This paper presents the 2nd place solution to the Facebook AI Image Similarity Challenge : Matching Track on DrivenData. The solution is based on self-supervised learning, and Vision Transformer(ViT). The main breaktrough comes from concatenating query and reference image to form as one image and asking ViT to directly predict from the image if query image used reference image. The solution scored 0.8291 Micro-average Precision on the private leaderboard.
Quantum-Assisted Support Vector Regression
Authors: Archismita Dalal, Mohsen Bagherimehrab, Barry C. Sanders
Date: 2021-11-17 00:00:00
Tasks: Facial Landmark Detection, regression, Stock Market Prediction, Weather Forecasting

A popular machine-learning model for regression tasks, including stock-market prediction, weather forecasting and real-estate pricing, is the classical support vector regression (SVR). However, a practically realisable quantum SVR remains to be formulated. We devise annealing-based algorithms, namely simulated and quantum-classical hybrid, for training two SVR models and compare their empirical performances against the SVR implementation of Python's scikit-learn package for facial-landmark detection (FLD), a particular use case for SVR. Our method is to derive a quadratic-unconstrained-binary formulation for the optimisation problem used for training a SVR model and solve this problem using annealing. Using D-Wave's hybrid solver, we construct a quantum-assisted SVR model, thereby demonstrating a slight advantage over classical models regarding FLD accuracy. Furthermore, we observe that annealing-based SVR models predict landmarks with lower variances compared to the SVR models trained by gradient-based methods. Our work is a proof-of-concept example for applying quantum-assisted SVR to a supervised-learning task with a small training dataset.
Fast BATLLNN: Fast Box Analysis of Two-Level Lattice Neural Networks
Authors: James Ferlez, Haitham Khedr, Yasser Shoukry
Date: 2021-11-17 00:00:00
Tasks: Vocal Bursts Valence Prediction

In this paper, we present the tool Fast Box Analysis of Two-Level Lattice Neural Networks (Fast BATLLNN) as a fast verifier of box-like output constraints for Two-Level Lattice (TLL) Neural Networks (NNs). In particular, Fast BATLLNN can verify whether the output of a given TLL NN always lies within a specified hyper-rectangle whenever its input constrained to a specified convex polytope (not necessarily a hyper-rectangle). Fast BATLLNN uses the unique semantics of the TLL architecture and the decoupled nature of box-like output constraints to dramatically improve verification performance relative to known polynomial-time verification algorithms for TLLs with generic polytopic output constraints. In this paper, we evaluate the performance and scalability of Fast BATLLNN, both in its own right and compared to state-of-the-art NN verifiers applied to TLL NNs. Fast BATLLNN compares very favorably to even the fastest NN verifiers, completing our synthetic TLL test bench more than 400x faster than its nearest competitor.
Differentially Private Federated Learning on Heterogeneous Data
Authors: Maxence Noble, Aurélien Bellet, Aymeric Dieuleveut
Date: 2021-11-17 00:00:00
Tasks: Federated Learning

Federated Learning (FL) is a paradigm for large-scale distributed learning which faces two key challenges: (i) efficient training from highly heterogeneous user data, and (ii) protecting the privacy of participating users. In this work, we propose a novel FL approach (DP-SCAFFOLD) to tackle these two challenges together by incorporating Differential Privacy (DP) constraints into the popular SCAFFOLD algorithm. We focus on the challenging setting where users communicate with a "honest-but-curious" server without any trusted intermediary, which requires to ensure privacy not only towards a third-party with access to the final model but also towards the server who observes all user communications. Using advanced results from DP theory, we establish the convergence of our algorithm for convex and non-convex objectives. Our analysis clearly highlights the privacy-utility trade-off under data heterogeneity, and demonstrates the superiority of DP-SCAFFOLD over the state-of-the-art algorithm DP-FedAvg when the number of local updates and the level of heterogeneity grow. Our numerical results confirm our analysis and show that DP-SCAFFOLD provides significant gains in practice.
Universal Inference Meets Random Projections: A Scalable Test for Log-concavity
Authors: Robin Dunn, Aditya Gangrade, Larry Wasserman, Aaditya Ramdas
Date: 2021-11-17 00:00:00
Tasks: valid

Shape constraints yield flexible middle grounds between fully nonparametric and fully parametric approaches to modeling distributions of data. The specific assumption of log-concavity is motivated by applications across economics, survival modeling, and reliability theory. However, there do not currently exist valid tests for whether the underlying density of given data is log-concave. The recent universal inference methodology provides a valid test. The universal test relies on maximum likelihood estimation (MLE), and efficient methods already exist for finding the log-concave MLE. This yields the first test of log-concavity that is provably valid in finite samples in any dimension, for which we also establish asymptotic consistency results. Empirically, we find that a random projections approach that converts the d-dimensional testing problem into many one-dimensional problems can yield high power, leading to a simple procedure that is statistically and computationally efficient.
Preference Communication in Multi-Objective Normal-Form Games
Authors: Willem Röpke, Diederik M. Roijers, Ann Nowé, Roxana Rădulescu
Date: 2021-11-17 00:00:00
Tasks: Form

We consider preference communication in two-player multi-objective normal-form games. In such games, the payoffs resulting from joint actions are vector-valued. Taking a utility-based approach, we assume there exists a utility function for each player which maps vectors to scalar utilities and consider agents that aim to maximise the utility of expected payoff vectors. As agents typically do not know their opponent's utility function or strategy, they must learn policies to interact with each other. Inspired by Stackelberg games, we introduce four novel preference communication protocols to aid agents in arriving at adequate solutions. Each protocol describes a specific approach for one agent to communicate preferences over their actions and how another agent responds. Additionally, to study when communication emerges, we introduce a communication protocol where agents must learn when to communicate. These protocols are subsequently evaluated on a set of five benchmark games against baseline agents that do not communicate. We find that preference communication can alter the learning process and lead to the emergence of cyclic policies which had not been previously observed in this setting. We further observe that the resulting policies can heavily depend on the characteristics of the game that is played. Lastly, we find that communication naturally emerges in both cooperative and self-interested settings.
Rapping-Singing Voice Synthesis based on Phoneme-level Prosody Control
Authors: Konstantinos Markopoulos, Nikolaos Ellinas, Alexandra Vioni, Myrsini Christidou, Panos Kakoulidis, Georgios Vamvoukakis, Georgia Maniati, June Sig Sung, Hyoungmin Park, Pirros Tsiakoulis, Aimilios Chalamandaris
Date: 2021-11-17 00:00:00
Tasks: Singing Voice Synthesis, valid

In this paper, a text-to-rapping/singing system is introduced, which can be adapted to any speaker's voice. It utilizes a Tacotron-based multispeaker acoustic model trained on read-only speech data and which provides prosody control at the phoneme level. Dataset augmentation and additional prosody manipulation based on traditional DSP algorithms are also investigated. The neural TTS model is fine-tuned to an unseen speaker's limited recordings, allowing rapping/singing synthesis with the target's speaker voice. The detailed pipeline of the system is described, which includes the extraction of the target pitch and duration values from an a capella song and their conversion into target speaker's valid range of notes before synthesis. An additional stage of prosodic manipulation of the output via WSOLA is also investigated for better matching the target duration values. The synthesized utterances can be mixed with an instrumental accompaniment track to produce a complete song. The proposed system is evaluated via subjective listening tests as well as in comparison to an available alternate system which also aims to produce synthetic singing voice from read-only training data. Results show that the proposed approach can produce high quality rapping/singing voice with increased naturalness.
Interpretable Models via Pairwise permutations algorithm
Authors: Troy Maaslandand, João Pereira, Diogo Bastos, Marcus de Goffau, Max Nieuwdorp, Aeilko H. Zwinderman, Evgeni Levin
Date: 2021-11-17 00:00:00
Tasks: Feature Importance

One of the most common pitfalls often found in high dimensional biological data sets are correlations between the features. This may lead to statistical and machine learning methodologies overvaluing or undervaluing these correlated predictors, while the truly relevant ones are ignored. In this paper, we will define a new method called \textit{pairwise permutation algorithm} (PPA) with the aim of mitigating the correlation bias in feature importance values. Firstly, we provide a theoretical foundation, which builds upon previous work on permutation importance. PPA is then applied to a toy data set, where we demonstrate its ability to correct the correlation effect. We further test PPA on a microbiome shotgun dataset, to show that the PPA is already able to obtain biological relevant biomarkers.
Interpreting multi-variate models with setPCA
Authors: Nordine Aouni, Luc Linders, David Robinson, Len Vandelaer, Jessica Wiezorek, Geetesh Gupta, Rachel Cavill
Date: 2021-11-17 00:00:00

Principal Component Analysis (PCA) and other multi-variate models are often used in the analysis of "omics" data. These models contain much information which is currently neither easily accessible nor interpretable. Here we present an algorithmic method which has been developed to integrate this information with existing databases of background knowledge, stored in the form of known sets (for instance genesets or pathways). To make this accessible we have produced a Graphical User Interface (GUI) in Matlab which allows the overlay of known set information onto the loadings plot and thus improves the interpretability of the multi-variate model. For each known set the optimal convex hull, covering a subset of elements from the known set, is found through a search algorithm and displayed. In this paper we discuss two main topics; the details of the search algorithm for the optimal convex hull for this problem and the GUI interface which is freely available for download for academic use.
Smart Data Representations: Impact on the Accuracy of Deep Neural Networks
Authors: Oliver Neumann, Nicole Ludwig, Marian Turowski, Benedikt Heidrich, Veit Hagenmeyer, Ralf Mikut
Date: 2021-11-17 00:00:00
Tasks: Time Series, Time Series Analysis, Time Series Forecasting

Deep Neural Networks are able to solve many complex tasks with less engineering effort and better performance. However, these networks often use data for training and evaluation without investigating its representation, i.e.~the form of the used data. In the present paper, we analyze the impact of data representations on the performance of Deep Neural Networks using energy time series forecasting. Based on an overview of exemplary data representations, we select four exemplary data representations and evaluate them using two different Deep Neural Network architectures and three forecasting horizons on real-world energy time series. The results show that, depending on the forecast horizon, the same data representations can have a positive or negative impact on the accuracy of Deep Neural Networks.
Unsupervised Spectral Unmixing For Telluric Correction Using A Neural Network Autoencoder
Authors: Rune D. Kjærsgaard, Aaron Bello-Arufe, Alexander D. Rathcke, Lars A. Buchhave, Line K. H. Clemmensen
Date: 2021-11-17 00:00:00

The absorption of light by molecules in the atmosphere of Earth is a complication for ground-based observations of astrophysical objects. Comprehensive information on various molecular species is required to correct for this so called telluric absorption. We present a neural network autoencoder approach for extracting a telluric transmission spectrum from a large set of high-precision observed solar spectra from the HARPS-N radial velocity spectrograph. We accomplish this by reducing the data into a compressed representation, which allows us to unveil the underlying solar spectrum and simultaneously uncover the different modes of variation in the observed spectra relating to the absorption of $\mathrm{H_2O}$ and $\mathrm{O_2}$ in the atmosphere of Earth. We demonstrate how the extracted components can be used to remove $\mathrm{H_2O}$ and $\mathrm{O_2}$ tellurics in a validation observation with similar accuracy and at less computational expense than a synthetic approach with molecfit.
Cross-lingual Low Resource Speaker Adaptation Using Phonological Features
Authors: Georgia Maniati, Nikolaos Ellinas, Konstantinos Markopoulos, Georgios Vamvoukakis, June Sig Sung, Hyoungmin Park, Aimilios Chalamandaris, Pirros Tsiakoulis
Date: 2021-11-17 00:00:00
Tasks: Speech Synthesis

The idea of using phonological features instead of phonemes as input to sequence-to-sequence TTS has been recently proposed for zero-shot multilingual speech synthesis. This approach is useful for code-switching, as it facilitates the seamless uttering of foreign text embedded in a stream of native text. In our work, we train a language-agnostic multispeaker model conditioned on a set of phonologically derived features common across different languages, with the goal of achieving cross-lingual speaker adaptation. We first experiment with the effect of language phonological similarity on cross-lingual TTS of several source-target language combinations. Subsequently, we fine-tune the model with very limited data of a new speaker's voice in either a seen or an unseen language, and achieve synthetic speech of equal quality, while preserving the target speaker's identity. With as few as 32 and 8 utterances of target speaker data, we obtain high speaker similarity scores and naturalness comparable to the corresponding literature. In the extreme case of only 2 available adaptation utterances, we find that our model behaves as a few-shot learner, as the performance is similar in both the seen and unseen adaptation language scenarios.
Surrogate-Assisted Genetic Algorithm for Wrapper Feature Selection
Authors: Mohammed Ghaith Altarabichi, Sławomir Nowaczyk, Sepideh Pashami, Peyman Sheikholharam Mashhad
Date: 2021-11-17 00:00:00
Tasks: feature selection

Feature selection is an intractable problem, therefore practical algorithms often trade off the solution accuracy against the computation time. In this paper, we propose a novel multi-stage feature selection framework utilizing multiple levels of approximations, or surrogates. Such a framework allows for using wrapper approaches in a much more computationally efficient way, significantly increasing the quality of feature selection solutions achievable, especially on large datasets. We design and evaluate a Surrogate-Assisted Genetic Algorithm (SAGA) which utilizes this concept to guide the evolutionary search during the early phase of exploration. SAGA only switches to evaluating the original function at the final exploitation phase. We prove that the run-time upper bound of SAGA surrogate-assisted stage is at worse equal to the wrapper GA, and it scales better for induction algorithms of high order of complexity in number of instances. We demonstrate, using 14 datasets from the UCI ML repository, that in practice SAGA significantly reduces the computation time compared to a baseline wrapper Genetic Algorithm (GA), while converging to solutions of significantly higher accuracy. Our experiments show that SAGA can arrive at near-optimal solutions three times faster than a wrapper GA, on average. We also showcase the importance of evolution control approach designed to prevent surrogates from misleading the evolutionary search towards false optima.
Sampling To Improve Predictions For Underrepresented Observations In Imbalanced Data
Authors: Rune D. Kjærsgaard, Manja G. Grønberg, Line K. H. Clemmensen
Date: 2021-11-17 00:00:00

Data imbalance is common in production data, where controlled production settings require data to fall within a narrow range of variation and data are collected with quality assessment in mind, rather than data analytic insights. This imbalance negatively impacts the predictive performance of models on underrepresented observations. We propose sampling to adjust for this imbalance with the goal of improving the performance of models trained on historical production data. We investigate the use of three sampling approaches to adjust for imbalance. The goal is to downsample the covariates in the training data and subsequently fit a regression model. We investigate how the predictive power of the model changes when using either the sampled or the original data for training. We apply our methods on a large biopharmaceutical manufacturing data set from an advanced simulation of penicillin production and find that fitting a model using the sampled data gives a small reduction in the overall predictive performance, but yields a systematically better performance on underrepresented observations. In addition, the results emphasize the need for alternative, fair, and balanced model evaluations.
Exploring Unsupervised Learning Methods for Automated Protocol Analysis
Authors: Arijit Dasgupta, Yi-Xue Yan, Clarence Ong, Jenn-Yue Teo, Chia-Wei Lim
Date: 2021-11-17 00:00:00
Tasks: Clustering, Management

The ability to analyse and differentiate network protocol traffic is crucial for network resource management to provide differentiated services by Telcos. Automated Protocol Analysis (APA) is crucial to significantly improve efficiency and reduce reliance on human experts. There are numerous automated state-of-the-art unsupervised methods for clustering unknown protocols in APA. However, many such methods have not been sufficiently explored using diverse test datasets. Thus failing to demonstrate their robustness to generalise. This study proposed a comprehensive framework to evaluate various combinations of feature extraction and clustering methods in APA. It also proposed a novel approach to automate selection of dataset dependent model parameters for feature extraction, resulting in improved performance. Promising results of a novel field-based tokenisation approach also led to our proposal of a novel automated hybrid approach for feature extraction and clustering of unknown protocols in APA. Our proposed hybrid approach performed the best in 7 out of 9 of the diverse test datasets, thus displaying the robustness to generalise across diverse unknown protocols. It also outperformed the unsupervised clustering technique in state-of-the-art open-source APA tool, NETZOB in all test datasets.
High Quality Streaming Speech Synthesis with Low, Sentence-Length-Independent Latency
Authors: Nikolaos Ellinas, Georgios Vamvoukakis, Konstantinos Markopoulos, Aimilios Chalamandaris, Georgia Maniati, Panos Kakoulidis, Spyros Raptis, June Sig Sung, Hyoungmin Park, Pirros Tsiakoulis
Date: 2021-11-17 00:00:00
Tasks: CPU, Decoder, Sentence, Speech Synthesis, text-to-speech, Text to Speech, Vocal Bursts Intensity Prediction

This paper presents an end-to-end text-to-speech system with low latency on a CPU, suitable for real-time applications. The system is composed of an autoregressive attention-based sequence-to-sequence acoustic model and the LPCNet vocoder for waveform generation. An acoustic model architecture that adopts modules from both the Tacotron 1 and 2 models is proposed, while stability is ensured by using a recently proposed purely location-based attention mechanism, suitable for arbitrary sentence length generation. During inference, the decoder is unrolled and acoustic feature generation is performed in a streaming manner, allowing for a nearly constant latency which is independent from the sentence length. Experimental results show that the acoustic model can produce feature sequences with minimal latency about 31 times faster than real-time on a computer CPU and 6.5 times on a mobile CPU, enabling it to meet the conditions required for real-time applications on both devices. The full end-to-end system can generate almost natural quality speech, which is verified by listening tests.