pwc-archive/papers-with-abstracts huggingface dataset:
Communication-Efficient TeraByte-Scale Model Training Framework for Online Advertising
Authors: Weijie Zhao, Xuewu Jiao, Mingqing Hu, Xiaoyun Li, Xiangyu Zhang, Ping Li
Date: 2022-01-05 00:00:00
Tasks: Click-Through Rate Prediction, CPU, GPU

Click-Through Rate (CTR) prediction is a crucial component in the online advertising industry. In order to produce a personalized CTR prediction, an industry-level CTR prediction model commonly takes a high-dimensional (e.g., 100 or 1000 billions of features) sparse vector (that is encoded from query keywords, user portraits, etc.) as input. As a result, the model requires Terabyte scale parameters to embed the high-dimensional input. Hierarchical distributed GPU parameter server has been proposed to enable GPU with limited memory to train the massive network by leveraging CPU main memory and SSDs as secondary storage. We identify two major challenges in the existing GPU training framework for massive-scale ad models and propose a collection of optimizations to tackle these challenges: (a) the GPU, CPU, SSD rapidly communicate with each other during the training. The connections between GPUs and CPUs are non-uniform due to the hardware topology. The data communication route should be optimized according to the hardware topology; (b) GPUs in different computing nodes frequently communicates to synchronize parameters. We are required to optimize the communications so that the distributed system can become scalable. In this paper, we propose a hardware-aware training workflow that couples the hardware topology into the algorithm design. To reduce the extensive communication between computing nodes, we introduce a $k$-step model merging algorithm for the popular Adam optimizer and provide its convergence rate in non-convex optimization. To the best of our knowledge, this is the first application of $k$-step adaptive optimization method in industrial-level CTR model training. The numerical results on real-world data confirm that the optimized system design considerably reduces the training time of the massive model, with essentially no loss in accuracy.
Attention-Based Recommendation On Graphs
Authors: Taher Hekmatfar, Saman Haratizadeh, Parsa Razban, Sama Goliaei
Date: 2022-01-04 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems

Graph Neural Networks (GNN) have shown remarkable performance in different tasks. However, there are a few studies about GNN on recommender systems. GCN as a type of GNNs can extract high-quality embeddings for different entities in a graph. In a collaborative filtering task, the core problem is to find out how informative an entity would be for predicting the future behavior of a target user. Using an attention mechanism, we can enable GCNs to do such an analysis when the underlying data is modeled as a graph. In this study, we proposed GARec as a model-based recommender system that applies an attention mechanism along with a spatial GCN on a recommender graph to extract embeddings for users and items. The attention mechanism tells GCN how much a related user or item should affect the final representation of the target entity. We compared the performance of GARec against some baseline algorithms in terms of RMSE. The presented method outperforms existing model-based, non-graph neural networks and graph neural networks in different MovieLens datasets.
Concept drift-tolerant transfer learning in dynamic environments.
Authors: Cuie Yang, Yiu-ming Cheung, Jinliang Ding, Kay Chen Tan
Date: 2021-02-10 00:00:00
Proceeding: Journal - IEEE Transactions on Neural Networks and Learning Systems 2021 2
Tasks: Transfer Learning

Existing transfer learning methods that focus on
problems in stationary environments are not usually applicable
to dynamic environments, where concept drift may occur. To the
best of our knowledge, the concept drift-tolerant transfer learn-
ing (CDTL), whose major challenge is the need to adapt the
target model and knowledge of source domains to the changing
environments, has yet to be well explored in the literature.
This article, therefore, proposes a hybrid ensemble approach to
deal with the CDTL problem provided that data in the target
domain are generated in a streaming chunk-by-chunk manner
from nonstationary environments. At each time step, a class-
wise weighted ensemble is presented to adapt the model of
target domains to new environments. It assigns a weight vector
for each classifier generated from the previous data chunks to
allow each class of the current data leveraging historical knowl-
edge independently. Then, a domain-wise weighted ensemble is
introduced to combine the source and target models to select
useful knowledge of each domain. The source models are updated
with the source instances performed by the proposed adaptive
weighted CORrelation ALignment (AW-CORAL). AW-CORAL
iteratively minimizes domain discrepancy meanwhile decreases
the effect of unrelated source instances. In this way, positive
knowledge of source domains can be potentially promoted while
negative knowledge is reduced. Empirical studies on synthetic
and real benchmark data sets demonstrate the effectiveness of
the proposed algorithm.
A computational framework for modelling infectious disease policy based on age and household structure with applications to the COVID-19 pandemic
Authors: Joe Hilton, Heather Riley, Lorenzo Pellis, Rabia Aziza, Samuel P. C. Brand, Ivy K. Kombe, John Ojal, Andrea Parisi, Matt J. Keeling, D. James Nokes, Robert Manson-Sawko, Thomas House
Date: 2022-01-14 00:00:00

The widespread, and in many countries unprecedented, use of non-pharmaceutical interventions (NPIs) during the COVID-19 pandemic has highlighted the need for mathematical models which can estimate the impact of these measures while accounting for the highly heterogeneous risk profile of COVID-19. Models accounting either for age structure or the household structure necessary to explicitly model many NPIs are commonly used in infectious disease modelling, but models incorporating both levels of structure present substantial computational and mathematical challenges due to their high dimensionality. Here we present a modelling framework for the spread of an epidemic that includes explicit representation of age structure and household structure. Our model is formulated in terms of tractable systems of ordinary differential equations for which we provide an open-source Python implementation. Such tractability leads to significant benefits for model calibration, exhaustive evaluation of possible parameter values, and interpretability of results. We demonstrate the flexibility of our model through four policy case studies, where we quantify the likely benefits of the following measures which were either considered or implemented in the UK during the current COVID-19 pandemic: control of within- and between-household mixing through NPIs; formation of support bubbles during lockdown periods; out-of-household isolation (OOHI); and temporary relaxation of NPIs during holiday periods. Our ordinary differential equation formulation and associated analysis demonstrate that multiple dimensions of risk stratification and social structure can be incorporated into infectious disease models without sacrificing mathematical tractability. This model and its software implementation expand the range of tools available to infectious disease policy analysts.
Empirical Analysis of EIP-1559: Transaction Fees, Waiting Time, and Consensus Security
Authors: Yulin Liu, Yuxuan Lu, Kartik Nayak, Fan Zhang, Luyao Zhang, Yinhong Zhao
Date: 2022-01-14 00:00:00

A transaction fee mechanism (TFM) is an essential component of a blockchain protocol. However, a systematic evaluation of the real-world impact of TFMs is still absent. Using rich data from the Ethereum blockchain, the mempool, and exchanges, we study the effect of EIP-1559, one of the earliest-deployed TFMs that depart from the traditional first-price auction paradigm. We conduct a rigorous and comprehensive empirical study to examine its causal effect on blockchain transaction fee dynamics, transaction waiting times, and consensus security. Our results show that EIP-1559 improves the user experience by mitigating intrablock differences in the gas price paid and reducing users' waiting times. However, EIP-1559 has only a small effect on gas fee levels and consensus security. In addition, we find that when Ether's price is more volatile, the waiting time is significantly higher. We also verify that a larger block size increases the presence of siblings. These findings suggest new directions for improving TFMs.
Multiple Genome Analytics Framework: The Case of All SARS-CoV-2 Complete Variants
Authors: Konstantinos Xylogiannopoulos
Date: 2022-01-13 00:00:00
Tasks: All

Pattern detection and string matching are fundamental problems in computer science and the accelerated expansion of bioinformatics and computational biology have made them a core topic for both disciplines. The SARS-CoV-2 pandemic has made such problems more demanding with hundreds or thousands of new genome variants discovered every week, because of constant mutations, and there is a desperate need for fast and accurate analyses. The requirement for computational tools for genomic analyses, such as sequence alignment, is very important, although, in most cases the resources and computational power required are enormous. The presented Multiple Genome Analytics Framework combines data structures and algorithms, specifically built for text mining and pattern detection, that can help to efficiently address several computational biology and bioinformatics problems concurrently with minimal resources. A single execution of advanced algorithms, with space and time complexity O(nlogn), is enough to acquire knowledge on all repeated patterns that exist in multiple genome sequences and this information can be used from other meta-algorithms for further meta-analyses. The potential of the proposed framework is demonstrated with the analysis of more than 300,000 SARS-CoV-2 genome sequences and the detection of all repeated patterns with length up to 60 nucleotides in these sequences. These results have been used to provide answers to questions such as common patterns among all variants, sequence alignment, palindromes and tandem repeats detection, different organism genome comparisons, polymerase chain reaction primers detection, etc.
Molecular Noise In Synaptic Communication
Authors: Sebastian Lotter, Maximilian Schäfer, Robert Schober
Date: 2022-01-14 00:00:00

In synaptic molecular communication (MC), the activation of postsynaptic receptors by neurotransmitters (NTs) is governed by a stochastic reaction-diffusion process. This randomness of synaptic MC contributes to the randomness of the electrochemical downstream signal in the postsynaptic cell, called postsynaptic membrane potential (PSP). Since the randomness of the PSP is relevant for neural computation and learning, characterizing the statistics of the PSP is critical. However, the statistical characterization of the synaptic reaction-diffusion process is difficult because the reversible bi-molecular reaction of NTs with receptors renders the system nonlinear. Consequently, there is currently no model available which characterizes the impact of the statistics of postsynaptic receptor activation on the PSP. In this work, we propose a novel statistical model for the synaptic reaction-diffusion process in terms of the chemical master equation (CME). We further propose a novel numerical method which allows to compute the CME efficiently and we use this method to characterize the statistics of the PSP. Finally, we present results from stochastic particle-based computer simulations which validate the proposed models. We show that the biophysical parameters governing synaptic transmission shape the autocovariance of the receptor activation and, ultimately, the statistics of the PSP. Our results suggest that the processing of the synaptic signal by the postsynaptic cell effectively mitigates synaptic noise while the statistical characteristics of the synaptic signal are preserved. The results presented in this paper contribute to a better understanding of the impact of the randomness of synaptic signal transmission on neuronal information processing.
Trajectory and Transmit Power Optimization for IRS-Assisted UAV Communication under Malicious Jamming
Authors: Zhi Ji, Wendong Yang, Xinrong Guan, Xiao Zhao, Guoxin Li, Qingqing Wu
Date: 2022-01-14 00:00:00

In this letter, we investigate an unmanned aerial vehicle (UAV) communication system, where an intelligent reflecting surface (IRS) is deployed to assist in the transmission from a ground node (GN) to the UAV in the presence of a jammer. We aim to maximize the average rate of the UAV communication by jointly optimizing the GN's transmit power, the IRS's passive beamforming and the UAV's trajectory. However, the formulated problem is difficult to solve due to the non-convex objective function and the coupled optimization variables. Thus, to tackle it, we propose an alternating optimization (AO) based algorithm by exploiting the successive convex approximation (SCA) and semidefinite relaxation (SDR) techniques. Simulation results show that the proposed algorithm can significantly improve the average rate compared with the benchmark algorithms. Moreover, it also shows that when the jamming power is large and the number of IRS elements is relatively small, deploying the IRS near the jammer outperforms deploying it near the GN, and vice versa.
A 1D-0D-3D coupled model for simulating blood flow and transport processes in breast tissue
Authors: Marvin Fritz, Tobias Köppl, J. Tinsley Oden, Andreas Wagner, Barbara Wohlmuth, Chengyue Wu
Date: 2022-01-14 00:00:00

In this work, we present mixed dimensional models for simulating blood flow and transport processes in breast tissue and the vascular tree supplying it. These processes are considered, to start from the aortic inlet to the capillaries and tissue of the breast. Large variations in biophysical properties and flow conditions exist in this system necessitating the use of different flow models for different geometries and flow regimes. Large variations in biophysical properties and flow conditions exist in this system necessitating the use of different flow models for different geometries and flow regimes. In total, we consider four different model types. First, a system of 1D nonlinear hyperbolic PDEs is considered to simulate blood flow in larger arteries with highly elastic vessel walls. Second, we assign 1D linearized hyperbolic PDEs to model the smaller arteries with stiffer vessel walls. The third model type consists of ODE systems (0D models). It is used to model the arterioles and peripheral circulation. Finally, homogenized 3D porous media models are considered to simulate flow and transport in capillaries and tissue within the breast volume. Sink terms are used to account for the influence of the venous and lymphatic systems. Combining the four model types, we obtain two different 1D-0D-3D coupled models for simulating blood flow and transport processes: The first 1D-0D-3D model covers the whole path from the aorta to the breast, while the second model is a sub-model obtained by restriction to breast vasculature and tissue making possible a significant reduction in computational cost. Several numerical experiments are conducted that demonstrate realistic flow simulations compared to existing data on blood flow in human breast and vascular system.
Evaluating Muscle Synergies with EMG Data and Physics Simulation in the Neurorobotics Platform
Authors: Benedikt Feldotto, Cristian Soare, Alois Knoll, Piyanee Sriya, Sarah Astill, Marc de Kamps, Samit Chakrabarty
Date: 2022-01-14 00:00:00
Tasks: Electromyography (EMG)

Although we can measure muscle activity and analyze their activation patterns, we understand little about how individual muscles affect the joint torque generated. It is known that they are controlled by circuits in the spinal cord, a system much less well understood than the cortex. Knowing the contribution of the muscles towards a joint torque would improve our understanding of human limb control. We present a novel framework to examine the control of biomechanics using physics simulations informed by electromyography (EMG) data. These signals drive a virtual musculoskeletal model in the Neurorobotics Platform (NRP), which we then use to evaluate resulting joint torques. We use our framework to analyze raw EMG data collected during an isometric knee extension study to identify synergies that drive a musculoskeletal lower limb model. The resulting knee torques are used as a reference for genetic algorithms (GA) to generate new simulated activation patterns. On the platform the GA finds solutions that generate torques matching those observed. Possible solutions include synergies that are similar to those extracted from the human study. In addition, the GA finds activation patterns that are different from the the biological ones while still producing the same knee torque. The NRP forms a highly modular integrated simulation platform allowing these in silico experiments. We argue that our framework allows for research of the neurobiomechanical control of muscles during tasks, which would otherwise not be possible.
A Study of Transducer based End-to-End ASR with ESPnet: Architecture, Auxiliary Loss and Decoding Strategies
Authors: Florian Boyer, Yusuke Shinohara, Takaaki Ishii, Hirofumi Inaguma, Shinji Watanabe
Date: 2022-01-14 00:00:00
Tasks: Multi-Task Learning

In this study, we present recent developments of models trained with the RNN-T loss in ESPnet. It involves the use of various architectures such as recently proposed Conformer, multi-task learning with different auxiliary criteria and multiple decoding strategies, including our own proposition. Through experiments and benchmarks, we show that our proposed systems can be competitive against other state-of-art systems on well-known datasets such as LibriSpeech and AISHELL-1. Additionally, we demonstrate that these models are promising against other already implemented systems in ESPnet in regards to both performance and decoding speed, enabling the possibility to have powerful systems for a streaming task. With these additions, we hope to expand the usefulness of the ESPnet toolkit for the research community and also give tools for the ASR industry to deploy our systems in realistic and production environments.
Value of Fleet Vehicle Grid in Providing Transmission System Operator Services
Authors: Cormac O'Malley, Marko Aunedi, Fei Teng, Goran Strbac
Date: 2022-01-14 00:00:00
Tasks: Scheduling

In this paper a new aggregated model for electric vehicle (EV) fleets is presented that considers their daily and weekly usage patterns. A frequency-constrained stochastic unit commitment model is employed to optimally schedule EV charging and discharging as well as the provision of frequency response (FR) in an electricity system, while respecting the vehicles' energy requirements and driving schedules. Through case studies we demonstrate that an EV with vehicle to grid (V2G) capability can reduce system costs in a future GB electricity grid by up to {\pounds}12,000 per year, and reduce CO2 emissions by 60 tonnes per year, mainly due to reduced curtailment of wind power. The paper also quantifies the changes in the benefits of fleet V2G resulting from variations in FR delivery time, the penetration of wind or uptake of alternative flexibility providers. Finally , a battery degradation model dependent on an EV's state of charge is proposed and implemented in the stochastic scheduling problem. It enables significant degradation cost reductions of 16% with only a 0.4% reduction of an EV's system value.
An Almost Globally Stable Adaptive Phase-Locked Loop for Synchronization of a Grid-Connected Voltage Source Converter
Authors: Daniele Zonetti, Alexey Bobtsov, Romeo Ortega, Nikolay Nikolaev, Oriol Gomis-Bellmunt
Date: 2022-01-14 00:00:00

In this paper we are interested in the problem of adaptive synchronization of a voltage source converter with a possibly weak grid with unknown angle and frequency, but knowledge of its parameters. To guarantee a suitable synchronization with the angle of the three-phase grid voltage we design an adaptive observer for such a signal requiring measurements only at the point of common coupling. Then we propose two alternative certainty-equivalent, adaptive phase-locked loops that ensure the angle estimation error goes to zero for almost all initial conditions. Although well-known, for the sake of completeness, we also present a PI controller with feedforward action that ensures the converter currents converge to an arbitrary desired value. Relevance of the theoretical results and their robustness to variation of the grid parameters are thoroughly discussed and validated in the challenging scenario of a converter connected to a grid with low short-circuit-ratio.
Bi-level Volt/VAR Optimization in Distribution Networks with Smart PV Inverters
Authors: Yao Long, Daniel S. Kirschen
Date: 2022-01-14 00:00:00

Optimal Volt/VAR control (VVC) in distribution networks relies on an effective coordination between the conventional utility-owned mechanical devices and the smart residential photovoltaic (PV) inverters. Typically, a central controller carries out a periodic optimization and sends setpoints to the local controller of each device. However, instead of tracking centrally dispatched setpoints, smart PV inverters can cooperate on a much faster timescale to reach optimality within a PV inverter group. To accommodate such PV inverter groups in the VVC architecture, this paper proposes a bi-level optimization framework. The upper-level determines the setpoints of the mechanical devices to minimize the network active power losses, while the lower-level represents the coordinated actions that the inverters take for their own objectives. The interactions between these two levels are captured in the bi-level optimization, which is solved using the Karush-Kuhn-Tucker (KKT) conditions. This framework fully exploits the capabilities of the different types of voltage regulation devices and enables them to cooperatively optimize their goals. Case studies on typical distribution networks with field-recorded data demonstrate the effectiveness and advantages of the proposed approach.
Coherence resonance and stochastic synchronization in a small-world neural network: An interplay in the presence of spike-timing-dependent plasticity
Authors: Marius E. Yamakou, Estelle M. Inack
Date: 2022-01-14 00:00:00

Coherence resonance (CR), stochastic synchronization (SS), and spike-timing-dependent plasticity (STDP) are ubiquitous dynamical processes in biological neural networks. Whether there exists an optimal network and STDP configuration at which CR and SS are both pronounced is a fundamental question of interest that is still elusive. We expect such a configuration to enable the brain to make synergistic and optimal use of these phenomena to process information efficiently. This paper considers a small-world network of excitable Hodgkin-Huxley neurons driven by channel noise and STDP with an asymmetric Hebbian time window. Numerical results indicate specific network topology and STDP parameter intervals in which CR and SS can be simultaneously enhanced. Our results imply that an optimally tuned inherent background noise, STDP rule, and network topology can play a constructive role in enhancing both the time precision of firing and the synchronization in neural systems.