pwc-archive/papers-with-abstracts huggingface dataset:
Language models of protein sequences at the scale of evolution enable accurate structure prediction
Authors: Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Sal Candido, Alexander Rives
Date: 2022-07-21 00:00:00
Proceeding: bioRxiv 2022 7
Tasks: Language Modeling, Language Modelling

Large language models have recently been shown to develop emergent capabilities with scale, going beyond simple pattern matching to perform higher level reasoning and generate lifelike images and text. While language models trained on protein sequences have been studied at a smaller scale, little is known about what they learn about biology as they are scaled up. In this work we train models up to 15 billion parameters, the largest language models of proteins to be evaluated to date. We find that as models are scaled they learn information enabling the prediction of the three-dimensional structure of a protein at the resolution of individual atoms. We present ESMFold for high accuracy end-to-end atomic level structure prediction directly from the individual sequence of a protein. ESMFold has similar accuracy to AlphaFold2 and RoseTTAFold for sequences with low perplexity that are well understood by the language model. ESMFold inference is an order of magnitude faster than AlphaFold2, enabling exploration of the structural space of metagenomic proteins in practical timescales.
The Poisson binomial mechanism for secure and private federated learning
Authors: Wei-Ning Chen, Ayfer Özgür, Peter Kairouz
Date: 2022-07-09 00:00:00
Tasks: Federated Learning

We introduce the Poisson Binomial mechanism (PBM), a discrete differential privacy mechanism for distributed mean estimation (DME) with applications to federated learning and analytics. We provide a tight analysis of its privacy guarantees, showing that it achieves the same privacy-accuracy trade-offs as the continuous Gaussian mechanism. Our analysis is based on a novel bound on the R\'enyi divergence of two Poisson binomial distributions that may be of independent interest. Unlike previous discrete DP schemes based on additive noise, our mechanism encodes local information into a parameter of the binomial distribution, and hence the output distribution is discrete with bounded support. Moreover, the support does not increase as the privacy budget $\varepsilon \rightarrow 0$ as in the case of additive schemes which require the addition of more noise to achieve higher privacy; on the contrary, the support becomes smaller as $\varepsilon \rightarrow 0$. The bounded support enables us to combine our mechanism with secure aggregation (SecAgg), a multi-party cryptographic protocol, without the need of performing modular clipping which results in an unbiased estimator of the sum of the local vectors. This in turn allows us to apply it in the private FL setting and provide an upper bound on the convergence rate of the SGD algorithm. Moreover, since the support of the output distribution becomes smaller as $\varepsilon \rightarrow 0$, the communication cost of our scheme decreases with the privacy constraint $\varepsilon$, outperforming all previous distributed DP schemes based on additive noise in the high privacy or low communication regimes.
Online Evasion Attacks on Recurrent Models:The Power of Hallucinating the Future
Authors: Byunggill Joe, Insik Shin, Jihun Hamm
Date: 2022-07-08 00:00:00
Tasks: Autonomous Driving

Recurrent models are frequently being used in online tasks such as autonomous driving, and a comprehensive study of their vulnerability is called for. Existing research is limited in generality only addressing application-specific vulnerability or making implausible assumptions such as the knowledge of future input. In this paper, we present a general attack framework for online tasks incorporating the unique constraints of the online setting different from offline tasks. Our framework is versatile in that it covers time-varying adversarial objectives and various optimization constraints, allowing for a comprehensive study of robustness. Using the framework, we also present a novel white-box attack called Predictive Attack that `hallucinates' the future. The attack achieves 98 percent of the performance of the ideal but infeasible clairvoyant attack on average. We validate the effectiveness of the proposed framework and attacks through various experiments.
Bayesian Hyperparameter Optimization for Deep Neural Network-Based Network Intrusion Detection
Authors: Mohammad Masum, Hossain Shahriar, Hisham Haddad, Md Jobair Hossain Faruk, Maria Valero, Md Abdullah Khan, Mohammad A. Rahman, Muhaiminul I. Adnan, Alfredo Cuzzocrea
Date: 2022-07-07 00:00:00
Tasks: Bayesian Optimization, Hyperparameter Optimization, Intrusion Detection, Network Intrusion Detection

Traditional network intrusion detection approaches encounter feasibility and sustainability issues to combat modern, sophisticated, and unpredictable security attacks. Deep neural networks (DNN) have been successfully applied for intrusion detection problems. The optimal use of DNN-based classifiers requires careful tuning of the hyper-parameters. Manually tuning the hyperparameters is tedious, time-consuming, and computationally expensive. Hence, there is a need for an automatic technique to find optimal hyperparameters for the best use of DNN in intrusion detection. This paper proposes a novel Bayesian optimization-based framework for the automatic optimization of hyperparameters, ensuring the best DNN architecture. We evaluated the performance of the proposed framework on NSL-KDD, a benchmark dataset for network intrusion detection. The experimental results show the framework's effectiveness as the resultant DNN architecture demonstrates significantly higher intrusion detection performance than the random search optimization-based approach in terms of accuracy, precision, recall, and f1-score.
Transfer Learning of wav2vec 2.0 for Automatic Lyric Transcription
Authors: Longshen Ou, Xiangming Gu, Ye Wang
Date: 2022-07-20 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Self-Supervised Learning, speech-recognition, Speech Recognition, Transfer Learning

Automatic speech recognition (ASR) has progressed significantly in recent years due to the emergence of large-scale datasets and the self-supervised learning (SSL) paradigm. However, as its counterpart problem in the singing domain, the development of automatic lyric transcription (ALT) suffers from limited data and degraded intelligibility of sung lyrics. To fill in the performance gap between ALT and ASR, we attempt to exploit the similarities between speech and singing. In this work, we propose a transfer-learning-based ALT solution that takes advantage of these similarities by adapting wav2vec 2.0, an SSL ASR model, to the singing domain. We maximize the effectiveness of transfer learning by exploring the influence of different transfer starting points. We further enhance the performance by extending the original CTC model to a hybrid CTC/attention model. Our method surpasses previous approaches by a large margin on various ALT benchmark datasets. Further experiments show that, with even a tiny proportion of training data, our method still achieves competitive performance.
Direct and Residual Subspace Decomposition of Spatial Room Impulse Responses
Authors: Thomas Deppisch, Sebastià V. Amengual Garí, Paul Calamia, Jens Ahrens
Date: 2022-07-20 00:00:00
Tasks: Direction of Arrival Estimation

Psychoacoustic experiments have shown that directional properties of the direct sound, salient reflections, and the late reverberation of an acoustic room response can have a distinct influence on the auditory perception of a given room. Spatial room impulse responses (SRIRs) capture those properties and thus are used for direction-dependent room acoustic analysis and virtual acoustic rendering. This work proposes a subspace method that decomposes SRIRs into a direct part, which comprises the direct sound and the salient reflections, and a residual, to facilitate enhanced analysis and rendering methods by providing individual access to these components. The proposed method is based on the generalized singular value decomposition and interprets the residual as noise that is to be separated from the other components of the reverberation. Large generalized singular values are attributed to the direct part, which is then obtained as a low-rank approximation of the SRIR. By advancing from the end of the SRIR toward the beginning while iteratively updating the residual estimate, the method adapts to spatio-temporal variations of the residual. The method is evaluated using a spatio-spectral error measure and simulated SRIRs of different rooms, microphone arrays, and ratios of direct sound to residual energy. The proposed method creates lower errors than existing approaches in all tested scenarios, including a scenario with two simultaneous reflections. A case study with measured SRIRs shows the applicability of the method under real-world acoustic conditions. A reference implementation is provided.
Phenomenon-Signal Model: Formalisation, Graph and Application
Authors: Hans Nikolaus Beck, Nayel Fabian Salem, Veronica Haber, Matthias Rauschenbach, Jan Reich
Date: 2022-07-20 00:00:00
Tasks: model, valid

Considering information as the basis of action, it may be of interest to examine the flow and acquisition of information between the actors in traffic. The central question is: Which signals does an automated driving system (which will be referred to as an automaton in the remainder of this paper) in traffic have to receive, decode or send in road traffic in order to act safely and in a manner that is compliant with valid standards. The phenomenon-signal model (PSM) is a method for structuring the problem area and for analysing and describing this very signal flow. The aim of this paper is to explain the basics, the structure and the application of this method.
Implementacion de un sistema IoT de bajo costo para el monitoreo de la calidad del aire en El Salvador
Authors: Omar Otoniel Flores-Cortez, Ronny Adalberto Cortez, Veronica Rosa
Date: 2022-07-20 00:00:00

Environmental pollution is a factor that represents a significant health risk. In El Salvador, The entity in charge of monitoring air quality is the ministry of Environment and Natural Resources, currently said ministry only has 3 stations for monitoring air quality throughout the territory of the country. The main objective of this work was the application of internet of things techniques in the design and implementation of a station of remote particulate pollution monitoring in the surrounding air. As well as setting up an Internet of Things (IoT) platform for the deployment of the data collected by the remote station. The development methodology of this research was based on the Model of Reference Architecture for IoT systems, which has as based on the development of system prototypes based on the correct choice of components available and suitable for the specific environment or application. For this case, they were used as inputs for the electronic hardware and Esp32 controller together with a Wemos-Lolin development board together with a sensor of particulate matter contamination PMS5003; by the side of software for the IoT platform was implemented a storage system, graphics and website based on low-cost tools The main result obtained in this work was an IoT prototype of an electronic station that allows monitoring the levels of contamination by the material of particles in the environment, whose data are accessible from any device with internet access through a site web, another result of this work is the configuring an IoT platform or cloud to connect wireless with the electronic station, the storage of the data produced by it and a web visualization stage.
${\mathcal K}$-monotonicity and feedback synthesis for incrementally stable networks
Authors: Yu Kawano, Fulvio Forni
Date: 2022-07-20 00:00:00

We discuss the role of monotonicity in enabling numerically tractable modular control design for networked nonlinear systems. We first show that the variational systems of monotone systems can be embedded into positive systems. Utilizing this embedding, we show how to solve a network stabilization problem by enforcing monotonicity and exponential dissipativity of the network sub-components. Such modular approach leads to a design algorithm based on a sequence of linear programming problems.
Anomaly Detection of Smart Metering System for Power Management with Battery Storage System/Electric Vehicle
Authors: Sangkeum Lee, Sarvar Hussain Nengroo, Hojun Jin, Yoonmee Doh, Chungho Lee, Taewook Heo, Dongsoo Har
Date: 2022-07-20 00:00:00
Tasks: Anomaly Detection, Management, Missing Values

A novel smart metering technique capable of anomaly detection was proposed for real-time home power management system. Smart meter data generated in real-time was obtained from 900 households of single apartments. To detect outliers and missing values in smart meter data, a deep learning model, the autoencoder, consisting of a graph convolutional network and bidirectional long short-term memory network, was applied to the smart metering technique. Power management based on the smart metering technique was performed by multi-objective optimization in the presence of a battery storage system and an electric vehicle. The results of the power management employing the proposed smart metering technique indicate a reduction in electricity cost and amount of power supplied by the grid compared to the results of power management without anomaly detection.
Benchmarking tools for a priori identifiability analysis
Authors: Xabier Rey Barreiro, Alejandro F. Villaverde
Date: 2022-07-20 00:00:00
Tasks: Benchmarking

The structural identifiability and the observability of a model determine the possibility of inferring its parameters and states by observing its outputs. These properties should be analysed before attempting to calibrate a model. Unfortunately, such \textit{a priori} analysis can be challenging, since it requires symbolic calculations that often have a high computational cost. In recent years a number of software tools have been developed for this task, mostly in the systems biology community but also in other disciplines. These tools have vastly different features and capabilities, and a critical assessment of their performance is still lacking. Here we present a comprehensive study of the computational resources available for analysing structural identifiability. We consider 12 software tools developed in 7 programming languages (Matlab, Maple, Mathematica, Julia, Python, Reduce, and Maxima), and evaluate their performance using a set of 25 case studies created from 21 models. Our results reveal their strengths and weaknesses, provide guidelines for choosing the most appropriate tool for a given problem, and highlight opportunities for future developments.
Optimal Trajectory Planning for Connected and Automated Vehicles in Lane-free Traffic with Vehicle Nudging
Authors: Venkata Karteek Yanumula, Panagiotis Typaldos, Dimitrios Troullinos, Milad Malekzadeh, Ioannis Papamichail, Markos Papageorgiou
Date: 2022-07-20 00:00:00
Tasks: Model Predictive Control, Trajectory Planning

The paper presents a movement strategy for Connected and Automated Vehicles (CAVs) in a lane-free traffic environment with vehicle nudging by use of an optimal control approach. State-dependent constraints on control inputs are considered to ensure that the vehicle moves within the road boundaries and to prevent collisions. An objective function, comprising various weighted sub-objectives, is designed, whose minimization leads to vehicle advancement at the desired speed, when possible, while avoiding obstacles. A nonlinear optimal control problem (OCP) is formulated for the minimization of the objective function subject to constraints for each vehicle. A computationally efficient Feasible Direction Algorithm (FDA) is called, on event-triggered basis, to compute in real-time the numerical solution for finite time-horizons within a Model Predictive Control (MPC) framework. The approach is applied to each vehicle on the road, while running simulations on a lane-free ring-road, for a wide range of vehicle densities and different types of vehicles. From the simulations, which create myriads of driving episodes for each involved vehicle, it is observed that the proposed approach is highly efficient in delivering safe, comfortable and efficient vehicle trajectories, as well as high traffic flow outcomes. The approach is under investigation for further use in various lane-free road infrastructures for CAV traffic.
Unified Grid-Forming Control of PMSG Wind Turbines for Fast Frequency Response and MPPT
Authors: Xue Lyu, Irina Subotić, Dominic Groß
Date: 2022-07-19 00:00:00
Tasks: Point Tracking

In this work we present a novel dual-port grid-forming control strategy, for permanent magnet synchronous generator wind turbines with back-to-back voltage source converters, that unifies the entire range of functions from maximum power point tracking (MPPT) to providing inertia and fast frequency response without explicit mode switching between grid-following and grid-forming control. The controls impose a well-defined AC voltage at the grid-side converter (GSC) and the machine-side converter (MSC) AC terminals and explicitly stabilize the DC-link capacitor voltage through both GSC and MSC. The wind turbine's kinetic energy storage and curtailment are adjusted through a combination of implicit rotor speed control and pitch angle control and directly determine the operating mode and level of grid support. Moreover, we provide analytical small-signal stability conditions for a simplified system and explicitly characterize the relationship between control gains, curtailment, and the wind turbines steady-state response. Finally, a detailed simulation study is used to validate the results and compared the proposed control with state-of-the-art MPPT control.
Chance-Constrained AC Optimal Power Flow for Unbalanced Distribution Grids
Authors: Kshitij Girigoudar, Ashley M. Hou, Line A. Roald
Date: 2022-07-19 00:00:00
Tasks: Load Forecasting

The growing penetration of distributed energy resources (DERs) is leading to continually changing operating conditions, which need to be managed efficiently by distribution grid operators. The intermittent nature of DERs such as solar photovoltaic (PV) systems as well as load forecasting errors not only increase uncertainty in the grid, but also pose significant power quality challenges such as voltage unbalance and voltage magnitude violations. This paper leverages a chance-constrained optimization approach to reduce the impact of uncertainty on distribution grid operation. We first present the chance-constrained optimal power flow (CC-OPF) problem for distribution grids and discuss a reformulation based on constraint tightening that does not require any approximations or relaxations of the three-phase AC power flow equations. We then propose two iterative solution algorithms capable of efficiently solving the reformulation. In the case studies, the performance of both algorithms is analyzed by running simulations on the IEEE 13-bus test feeder using real PV and load measurement data. The simulation results indicate that both methods are able to enforce the chance constraints in in- and out-of-sample evaluations.
Neuronal growth on high-aspect-ratio diamond nanopillar arrays for biosensing applications
Authors: Elena Losero, Somanath Jagannath, Maurizio Pezzoli, Valentin Goblot, Hossein Babashah, Hilal A. Lashuel, Christophe Galland, Niels Quack
Date: 2022-07-11 00:00:00

Monitoring neuronal activity with simultaneously high spatial and temporal resolution in living cell cultures is crucial to advance understanding of the development and functioning of our brain, and to gain further insights in the origin of brain disorders. While it has been demonstrated that the quantum sensing capabilities of nitrogen-vacancy (NV) centers in diamond allow real time detection of action potentials from large neurons in marine invertebrates, quantum monitoring of mammalian neurons (presenting much smaller dimensions and thus producing much lower signal and requiring higher spatial resolution) has hitherto remained elusive. In this context, diamond nanostructuring can offer the opportunity to boost the diamond platform sensitivity to the required level. However, a comprehensive analysis of the impact of a nanostructured diamond surface on the neuronal viability and growth was lacking. Here, we pattern a single crystal diamond surface with large-scale nanopillar arrays and we successfully demonstrate growth of a network of living and functional primary mouse hippocampal neurons on it. Our study on geometrical parameters reveals preferential growth along the nanopillar grid axes with excellent physical contact between cell membrane and nanopillar apex. Our results suggest that neuron growth can be tailored on diamond nanopillars to realize a nanophotonic quantum sensing platform for wide-field and label-free neuronal activity recording with sub-cellular resolution.
Spatio-temporal models of infectious disease with high rates of asymptomatic transmission
Authors: Aminur Rahman, Angela Peace, Ramesh Kesawan, Souparno Ghosh
Date: 2022-07-20 00:00:00
Tasks: parameter estimation

The surprisingly mercurial Covid-19 pandemic has highlighted the need to not only accelerate research on infectious disease, but to also study them using novel techniques and perspectives. A major contributor to the difficulty of containing the current pandemic is due to the highly asymptomatic nature of the disease. In this investigation, we develop a modeling framework to study the spatio-temporal evolution of diseases with high rates of asymptomatic transmission, and we apply this framework to a hypothetical country with mathematically tractable geography; namely, square counties uniformly organized into a rectangle. We first derive a model for the temporal dynamics of susceptible, infected, and recovered populations, which is applied at the county level. Next we use likelihood-based parameter estimation to derive temporally varying disease transmission parameters on the state-wide level. While these two methods give us some spatial structure and show the effects of behavioral and policy changes, they miss the evolution of hot zones that have caused significant difficulties in resource allocation during the current pandemic. It is evident that the distribution of cases will not be stagnantly based on the population density, as with many other diseases, but will continuously evolve. We model this as a diffusive process where the diffusivity is spatially varying based on the population distribution, and temporally varying based on the current number of simulated asymptomatic cases. With this final addition coupled to the SIR model with temporally varying transmission parameters, we capture the evolution of "hot zones" in our hypothetical setup.
Testing for a Threshold in Models with Endogenous Regressors
Authors: Mario P. Rothfelder, Otilia Boldea
Date: 2022-07-20 00:00:00

We show by simulation that the test for an unknown threshold in models with endogenous regressors - proposed in Caner and Hansen (2004) - can exhibit severe size distortions both in small and in moderately large samples, pertinent to empirical applications. We propose three new tests that rectify these size distortions. The first test is based on GMM estimators. The other two are based on unconventional 2SLS estimators, that use additional information about the linearity (or lack of linearity) of the first stage. Just like the test in Caner and Hansen (2004), our tests are non-pivotal, and we prove their bootstrap validity. The empirical application revisits the question in Ramey and Zubairy (2018) whether government spending multipliers are larger in recessions, but using tests for an unknown threshold. Consistent with Ramey and Zubairy (2018), we do not find strong evidence that these multipliers are larger in recessions.