pwc-archive/papers-with-abstracts huggingface dataset:
Massively Parallel Genetic Optimization through Asynchronous Propagation of Populations
Authors: Oskar Taubert, Marie Weiel, Daniel Coquelin, Anis Farshian, Charlotte Debus, Alexander Schug, Achim Streit, Markus Götz
Date: 2023-01-20 00:00:00
Tasks: global-optimization

We present Propulate, an evolutionary optimization algorithm and software package for global optimization and in particular hyperparameter search. For efficient use of HPC resources, Propulate omits the synchronization after each generation as done in conventional genetic algorithms. Instead, it steers the search with the complete population present at time of breeding new individuals. We provide an MPI-based implementation of our algorithm, which features variants of selection, mutation, crossover, and migration and is easy to extend with custom functionality. We compare Propulate to the established optimization tool Optuna. We find that Propulate is up to three orders of magnitude faster without sacrificing solution accuracy, demonstrating the efficiency and efficacy of our lazy synchronization approach. Code and documentation are available at https://github.com/Helmholtz-AI-Energy/propulate
MTGP: Combining Metamorphic Testing and Genetic Programming
Authors: Dominik Sobania, Martin Briesch, Philipp Röchner, Franz Rothlauf
Date: 2023-01-20 00:00:00
Tasks: Program Synthesis

Genetic programming is an evolutionary approach known for its performance in program synthesis. However, it is not yet mature enough for a practical use in real-world software development, since usually many training cases are required to generate programs that generalize to unseen test cases. As in practice, the training cases have to be expensively hand-labeled by the user, we need an approach to check the program behavior with a lower number of training cases. Metamorphic testing needs no labeled input/output examples. Instead, the program is executed multiple times, first on a given (randomly generated) input, followed by related inputs to check whether certain user-defined relations between the observed outputs hold. In this work, we suggest MTGP, which combines metamorphic testing and genetic programming and study its performance and the generalizability of the generated programs. Further, we analyze how the generalizability depends on the number of given labeled training cases. We find that using metamorphic testing combined with labeled training cases leads to a higher generalization rate than the use of labeled training cases alone in almost all studied configurations. Consequently, we recommend researchers to use metamorphic testing in their systems if the labeling of the training data is expensive.
ETLP: Event-based Three-factor Local Plasticity for online learning with neuromorphic hardware
Authors: Fernando M. Quintana, Fernando Perez-Peña, Pedro L. Galindo, Emre O. Neftci, Elisabetta Chicca, Lyes Khacef
Date: 2023-01-19 00:00:00

Neuromorphic perception with event-based sensors, asynchronous hardware and spiking neurons is showing promising results for real-time and energy-efficient inference in embedded systems. The next promise of brain-inspired computing is to enable adaptation to changes at the edge with online learning. However, the parallel and distributed architectures of neuromorphic hardware based on co-localized compute and memory imposes locality constraints to the on-chip learning rules. We propose in this work the Event-based Three-factor Local Plasticity (ETLP) rule that uses (1) the pre-synaptic spike trace, (2) the post-synaptic membrane voltage and (3) a third factor in the form of projected labels with no error calculation, that also serve as update triggers. We apply ETLP with feedforward and recurrent spiking neural networks on visual and auditory event-based pattern recognition, and compare it to Back-Propagation Through Time (BPTT) and eProp. We show a competitive performance in accuracy with a clear advantage in the computational complexity for ETLP. We also show that when using local plasticity, threshold adaptation in spiking neurons and a recurrent topology are necessary to learn spatio-temporal patterns with a rich temporal structure. Finally, we provide a proof of concept hardware implementation of ETLP on FPGA to highlight the simplicity of its computational primitives and how they can be mapped into neuromorphic hardware for online learning with low-energy consumption and real-time interaction.
Improving Reflexive Surfaces Efficiency with Genetic Algorithms
Authors: A. Steklain, M. Adames, F. Ganacim
Date: 2023-01-03 00:00:00
Tasks: Position

We propose using a Genetic Algorithm to improve the efficiency of reflexive surfaces in devices where the receiver's position is different from the classic parabolic antenna. With this technique, we show that we can improve the efficiency of the ARAPUCA photodetector.
Multi-Armed Bandits and Quantum Channel Oracles
Authors: Simon Buchholz, Jonas M. Kübler, Bernhard Schölkopf
Date: 2023-01-20 00:00:00
Tasks: Multi-Armed Bandits, reinforcement-learning, Reinforcement Learning (RL)

Multi-armed bandits are one of the theoretical pillars of reinforcement learning. Recently, the investigation of quantum algorithms for multi-armed bandit problems was started, and it was found that a quadratic speed-up (in query complexity) is possible when the arms and the randomness of the rewards of the arms can be queried in superposition. Here we introduce further bandit models where we only have limited access to the randomness of the rewards, but we can still query the arms in superposition. We show that then the query complexity is the same as for classical algorithms. This generalizes the prior result that no speed-up is possible for unstructured search when the oracle has positive failure probability.
Parametrization Cookbook: A set of Bijective Parametrizations for using Machine Learning methods in Statistical Inference
Authors: Jean-Benoist Leger
Date: 2023-01-19 00:00:00
Tasks: GPU

We present in this paper a way to transform a constrained statistical inference problem into an unconstrained one in order to be able to use modern computational methods, such as those based on automatic differentiation, GPU computing, stochastic gradients with mini-batch. Unlike the parametrizations classically used in Machine Learning, the parametrizations introduced here are all bijective and are even diffeomorphisms, thus allowing to keep the important properties from a statistical inference point of view, first of all identifiability. This cookbook presents a set of recipes to use to transform a constrained problem into a unconstrained one. For an easy use of parametrizations, this paper is at the same time a cookbook, and a Python package allowing the use of parametrizations with numpy, but also JAX and PyTorch, as well as a high level and expressive interface allowing to easily describe a parametrization to transform a difficult problem of statistical inference into an easier problem addressable with modern optimization tools.
The Evolution of Web Search User Interfaces -- An Archaeological Analysis of Google Search Engine Result Pages
Authors: B. Oliveira, C. T. Lopes
Date: 2023-01-20 00:00:00
Tasks: Layout Design

Web search engines have marked everyone's life by transforming how one searches and accesses information. Search engines give special attention to the user interface, especially search engine result pages (SERP). The well-known ''10 blue links'' list has evolved into richer interfaces, often personalized to the search query, the user, and other aspects. More than 20 years later, the literature has not adequately portrayed this development. We present a study on the evolution of SERP interfaces during the last two decades using Google Search as a case study. We used the most searched queries by year to extract a sample of SERP from the Internet Archive. Using this dataset, we analyzed how SERP evolved in content, layout, design (e.g., color scheme, text styling, graphics), navigation, and file size. We have also analyzed the user interface design patterns associated with SERP elements. We found that SERP are becoming more diverse in terms of elements, aggregating content from different verticals and including more features that provide direct answers. This systematic analysis portrays evolution trends in search engine user interfaces and, more generally, web design. We expect this work will trigger other, more specific studies that can take advantage of our dataset.
AdaEnsemble: Learning Adaptively Sparse Structured Ensemble Network for Click-Through Rate Prediction
Authors: YaChen Yan, Liubo Li
Date: 2023-01-06 00:00:00
Tasks: Click-Through Rate Prediction, Mixture-of-Experts, Recommendation Systems

Learning feature interactions is crucial to success for large-scale CTR prediction in recommender systems and Ads ranking. Researchers and practitioners extensively proposed various neural network architectures for searching and modeling feature interactions. However, we observe that different datasets favor different neural network architectures and feature interaction types, suggesting that different feature interaction learning methods may have their own unique advantages. Inspired by this observation, we propose AdaEnsemble: a Sparsely-Gated Mixture-of-Experts (SparseMoE) architecture that can leverage the strengths of heterogeneous feature interaction experts and adaptively learns the routing to a sparse combination of experts for each example, allowing us to build a dynamic hierarchy of the feature interactions of different types and orders. To further improve the prediction accuracy and inference efficiency, we incorporate the dynamic early exiting mechanism for feature interaction depth selection. The AdaEnsemble can adaptively choose the feature interaction depth and find the corresponding SparseMoE stacking layer to exit and compute prediction from. Therefore, our proposed architecture inherits the advantages of the exponential combinations of sparsely gated experts within SparseMoE layers and further dynamically selects the optimal feature interaction depth without executing deeper layers. We implement the proposed AdaEnsemble and evaluate its performance on real-world datasets. Extensive experiment results demonstrate the efficiency and effectiveness of AdaEnsemble over state-of-the-art models.
Class-Dependent Label-Noise Learning with Cycle-Consistency Regularization Feature Space
Authors: De Cheng, Yixiong Ning, Nannan Wang, Xinbo Gao, Heng Yang, Yuxuan Du, Bo Han, Tongliang Liu
Date: 2022-11-01 00:00:00
Proceeding: NIPS 2022 11

In label-noise learning, estimating the transition matrix plays an important role in building statistically consistent classifier. Current state-of-the-art consistent estimator for the transition matrix has been developed under the newly proposed sufficiently scattered assumption, through incorporating the minimum volume constraint of the transition matrix T into label-noise learning. To compute the volume of  T, it heavily relies on the estimated  noisy class posterior. However, the estimation error of the noisy class posterior could usually be large as deep learning methods tend to easily overfit the noisy labels. Then, directly minimizing the volume of such obtained T could lead the transition matrix to be poorly estimated.  Therefore, how to reduce the side-effects of the inaccurate noisy class posterior has become the bottleneck of such method. In this paper, we creatively propose to estimate the transition matrix under the forward-backward cycle-consistency regularization, of which we have greatly reduced the dependency of estimating the transition matrix T on the noisy class posterior. We show that the cycle-consistency regularization helps to minimize the volume of the transition matrix T indirectly without exploiting the estimated noisy class posterior, which could further encourage the estimated transition matrix T to converge to its optimal solution. Extensive experimental results consistently justify the effectiveness of the proposed method, on reducing the estimation error of the transition matrix and greatly boosting the classification performance.
A Semi-supervised Sensing Rate Learning based CMAB Scheme to Combat COVID-19 by Trustful Data Collection in the Crowd
Authors: Jianheng Tang, Kejia Fan, Wenxuan Xie, Luomin Zeng, Feijiang Han, Guosheng Huang, Tian Wang, Anfeng Liu, Shaobo Zhang
Date: 2023-01-17 00:00:00

The recruitment of trustworthy and high-quality workers is an important research issue for MCS. Previous studies either assume that the qualities of workers are known in advance, or assume that the platform knows the qualities of workers once it receives their collected data. In reality, to reduce costs and thus maximize revenue, many strategic workers do not perform their sensing tasks honestly and report fake data to the platform, which is called False data attacks. And it is very hard for the platform to evaluate the authenticity of the received data. In this paper, an incentive mechanism named Semi-supervision based Combinatorial Multi-Armed Bandit reverse Auction (SCMABA) is proposed to solve the recruitment problem of multiple unknown and strategic workers in MCS. First, we model the worker recruitment as a multi-armed bandit reverse auction problem and design an UCB-based algorithm to separate the exploration and exploitation, regarding the Sensing Rates (SRs) of recruited workers as the gain of the bandit. Next, a Semi-supervised Sensing Rate Learning (SSRL) approach is proposed to quickly and accurately obtain the workers' SRs, which consists of two phases, supervision and self-supervision. Last, SCMABA is designed organically combining the SRs acquisition mechanism with multi-armed bandit reverse auction, where supervised SR learning is used in the exploration, and the self-supervised one is used in the exploitation. We theoretically prove that our SCMABA achieves truthfulness and individual rationality and exhibits outstanding performances of the SCMABA mechanism through in-depth simulations of real-world data traces.
Enabling Astronaut Self-Scheduling using a Robust Advanced Modelling and Scheduling system: an assessment during a Mars analogue mission
Authors: Michael Saint-Guillain, Jean Vanderdonckt, Nicolas Burny, Vladimir Pletser, Tiago Vaquero, Steve Chien, Alexander Karl, Jessica Marquez, John Karasinski, Cyril Wain, Audrey Comein, Ignacio S. Casla, Jean Jacobs, Julien Meert, Cheyenne Chamart, Sirga Drouet, Julie Manon
Date: 2023-01-14 00:00:00
Tasks: Scheduling

Human long duration exploration missions (LDEMs) raise a number of technological challenges. This paper addresses the question of the crew autonomy: as the distances increase, the communication delays and constraints tend to prevent the astronauts from being monitored and supported by a real time ground control. Eventually, future planetary missions will necessarily require a form of astronaut self-scheduling. We study the usage of a computer decision-support tool by a crew of analog astronauts, during a Mars simulation mission conducted at the Mars Desert Research Station (MDRS, Mars Society) in Utah. The proposed tool, called Romie, belongs to the new category of Robust Advanced Modelling and Scheduling (RAMS) systems. It allows the crew members (i) to visually model their scientific objectives and constraints, (ii) to compute near-optimal operational schedules while taking uncertainty into account, (iii) to monitor the execution of past and current activities, and (iv) to modify scientific objectives/constraints w.r.t. unforeseen events and opportunistic science. In this study, we empirically measure how the astronauts, who are novice planners, perform at using such a tool when self-scheduling under the realistic assumptions of a simulated Martian planetary habitat.
Asymptotic Behavior of Zero-Forcing Precoding based on Imperfect Channel Knowledge for Massive MISO FDD Systems
Authors: Donia Ben Amor, Michael Joham, Wolfgang Utschick
Date: 2023-01-20 00:00:00

In this work, we study the asymptotic behavior of the zero-forcing precoder based on the least squares (LS) and the linear minimum mean-square error (LMMSE) channel estimates for the downlink (DL) of a frequency-division-duplex (FDD) massive multiple-input-single-output (MISO) system. We show analytically the rather surprising result that zero-forcing precoding based on the LS estimate leads asymptotically to an interference-free transmission, even if the number of pilots used for DL channel training is less than the number of antennas available at the base station (BS). Although the LMMSE channel estimate exhibits a better quality in terms of the MSE due to the exploitation of the channel statistics, we show that in the case of contaminated channel observations, zero-forcing based on the LMMSE is unable to eliminate the inter-user interference in the asymptotic limit of high DL transmit powers. In order for the results to hold, mild conditions on the channel probing phase are assumed. The validity of our analytical results is demonstrated through numerical simulations for different scenarios.
Latent Autoregressive Source Separation
Authors: Emilian Postolache, Giorgio Mariani, Michele Mancusi, Andrea Santilli, Luca Cosmo, Emanuele Rodolà
Date: 2023-01-09 00:00:00
Tasks: Dimensionality Reduction

Autoregressive models have achieved impressive results over a wide range of domains in terms of generation quality and downstream task performance. In the continuous domain, a key factor behind this success is the usage of quantized latent spaces (e.g., obtained via VQ-VAE autoencoders), which allow for dimensionality reduction and faster inference times. However, using existing pre-trained models to perform new non-trivial tasks is difficult since it requires additional fine-tuning or extensive training to elicit prompting. This paper introduces LASS as a way to perform vector-quantized Latent Autoregressive Source Separation (i.e., de-mixing an input signal into its constituent sources) without requiring additional gradient-based optimization or modifications of existing models. Our separation method relies on the Bayesian formulation in which the autoregressive models are the priors, and a discrete (non-parametric) likelihood function is constructed by performing frequency counts over latent sums of addend tokens. We test our method on images and audio with several sampling strategies (e.g., ancestral, beam search) showing competitive results with existing approaches in terms of separation quality while offering at the same time significant speedups in terms of inference time and scalability to higher dimensional data.
Minimum Time Control of a Gantry Crane System with Rate Constraints
Authors: Adrian Stein, Tarunraj Singh
Date: 2023-01-20 00:00:00

This paper focuses on the development of minimum time control profiles for point-to-point motion of a gantry crane system in the presence of uncertainties in modal parameters. Assuming that the velocity of the trolley of the crane can be commanded and is subject to limits, an optimal control problem is posed to determine the bang-off-bang control profile to transition the system from a point of rest to the terminal states with no residual vibrations. Both undamped and underdamped systems are considered and the variation of the structure of the optimal control profiles as a function of the final displacement is studied. As the magnitude of the rigid body displacement is increased, the collapse and birthing of switches in the optimal control profile are observed and explained. Robustness to uncertainties in modal parameters is accounted for by forcing the state sensitivities at the terminal time to zero. The observation that the time-optimal control profile merges with the robust time-optimal control is noted for specific terminal displacements and the migration of zeros of the time-delay filter parameterizing the optimal control profile are used to explain this counter intuitive result. A two degree of freedom gantry crane system is used to experimentally validate the observations of the numerical studies and the tradeoff of increase in maneuver time to the reduction of residual vibrations is experimentally illustrated.
Physics-guided neural networks for feedforward control with input-to-state stability guarantees
Authors: Max Bolderman, Hans Butler, Sjirk Koekebakker, Eelco van Horssen, Ramidin Kamidi, Theresa Spaan-Burke, Nard Strijbosch, Mircea Lazar
Date: 2023-01-20 00:00:00
Tasks: Friction

The increasing demand on precision and throughput within high-precision mechatronics industries requires a new generation of feedforward controllers with higher accuracy than existing, physics-based feedforward controllers. As neural networks are universal approximators, they can in principle yield feedforward controllers with a higher accuracy, but suffer from bad extrapolation outside the training data set, which makes them unsafe for implementation in industry. Motivated by this, we develop a novel physics-guided neural network (PGNN) architecture that structurally merges a physics-based layer and a black-box neural layer in a single model. The parameters of the two layers are simultaneously identified, while a novel regularization cost function is used to prevent competition among layers and to preserve consistency of the physics-based parameters. Moreover, in order to ensure stability of PGNN feedforward controllers, we develop sufficient conditions for analyzing or imposing (during training) input-to-state stability of PGNNs, based on novel, less conservative Lipschitz bounds for neural networks. The developed PGNN feedforward control framework is validated on a real-life, high-precision industrial linear motor used in lithography machines, where it reaches a factor 2 improvement with respect to physics-based mass-friction feedforward and it significantly outperforms alternative neural network based feedforward controllers.
Optimality-preserving Reduction of Chemical Reaction Networks
Authors: Kim G. Larsen, Daniele Toller, Mirco Tribastone, Max Tschaikowski, Andrea Vandin
Date: 2023-01-20 00:00:00
Tasks: Epidemiology

Across many disciplines, chemical reaction networks (CRNs) are an established population model defined as a system of coupled nonlinear ordinary differential equations. In many applications, for example, in systems biology and epidemiology, CRN parameters such as the kinetic reaction rates can be used as control inputs to steer the system toward a given target. Unfortunately, the resulting optimal control problem is nonlinear, therefore, computationally very challenging. We address this issue by introducing an optimality-preserving reduction algorithm for CRNs. The algorithm partitions the original state variables into a reduced set of macro-variables for which one can define a reduced optimal control problem from which one can exactly recover the solution of the original control problem. Notably, the reduction algorithm runs with polynomial time complexity in the size of the CRN. We use this result to reduce reachability and control problems of large-scale protein-interaction networks and vaccination models with hundreds of thousands of state variables.