pwc-archive/papers-with-abstracts huggingface dataset:
A Simple yet Effective Framework for Active Learning to Rank
Authors: Qingzhong Wang, Haifang Li, Haoyi Xiong, Wen Wang, Jiang Bian, Yu Lu, Shuaiqiang Wang, Zhicong Cheng, Dejing Dou, Dawei Yin
Date: 2022-05-20 00:00:00
Tasks: Active Learning, Learning-To-Rank

While China has become the biggest online market in the world with around 1 billion internet users, Baidu runs the world largest Chinese search engine serving more than hundreds of millions of daily active users and responding billions queries per day. To handle the diverse query requests from users at web-scale, Baidu has done tremendous efforts in understanding users' queries, retrieve relevant contents from a pool of trillions of webpages, and rank the most relevant webpages on the top of results. Among these components used in Baidu search, learning to rank (LTR) plays a critical role and we need to timely label an extremely large number of queries together with relevant webpages to train and update the online LTR models. To reduce the costs and time consumption of queries/webpages labeling, we study the problem of Activ Learning to Rank (active LTR) that selects unlabeled queries for annotation and training in this work. Specifically, we first investigate the criterion -- Ranking Entropy (RE) characterizing the entropy of relevant webpages under a query produced by a sequence of online LTR models updated by different checkpoints, using a Query-By-Committee (QBC) method. Then, we explore a new criterion namely Prediction Variances (PV) that measures the variance of prediction results for all relevant webpages under a query. Our empirical studies find that RE may favor low-frequency queries from the pool for labeling while PV prioritizing high-frequency queries more. Finally, we combine these two complementary criteria as the sample selection strategies for active learning. Extensive experiments with comparisons to baseline algorithms show that the proposed approach could train LTR models achieving higher Discounted Cumulative Gain (i.e., the relative improvement {\Delta}DCG4=1.38%) with the same budgeted labeling efforts.
GDSRec: Graph-Based Decentralized Collaborative Filtering for Social Recommendation
Authors: Jiajia Chen, Xin Xin, Xianfeng Liang, Xiangnan He, Jun Liu
Date: 2022-05-20 00:00:00
Tasks: Collaborative Filtering

Generating recommendations based on user-item interactions and user-user social relations is a common use case in web-based systems. These connections can be naturally represented as graph-structured data and thus utilizing graph neural networks (GNNs) for social recommendation has become a promising research direction. However, existing graph-based methods fails to consider the bias offsets of users (items). For example, a low rating from a fastidious user may not imply a negative attitude toward this item because the user tends to assign low ratings in common cases. Such statistics should be considered into the graph modeling procedure. While some past work considers the biases, we argue that these proposed methods only treat them as scalars and can not capture the complete bias information hidden in data. Besides, social connections between users should also be differentiable so that users with similar item preference would have more influence on each other. To this end, we propose Graph-Based Decentralized Collaborative Filtering for Social Recommendation (GDSRec). GDSRec treats the biases as vectors and fuses them into the process of learning user and item representations. The statistical bias offsets are captured by decentralized neighborhood aggregation while the social connection strength is defined according to the preference similarity and then incorporated into the model design. We conduct extensive experiments on two benchmark datasets to verify the effectiveness of the proposed model. Experimental results show that the proposed GDSRec achieves superior performance compared with state-of-the-art related baselines. Our implementations are available in \url{https://github.com/MEICRS/GDSRec}.
Safe Exploration in Markov Decision Processes
Authors: Teodor Mihai Moldovan, Pieter Abbeel
Date: 2012-05-22 00:00:00
Tasks: Safe Exploration

In environments with uncertain dynamics exploration is necessary to learn how to perform well. Existing reinforcement learning algorithms provide strong exploration guarantees, but they tend to rely on an ergodicity assumption. The essence of ergodicity is that any state is eventually reachable from any other state by following a suitable policy. This assumption allows for exploration algorithms that operate by simply favoring states that have rarely been visited before. For most physical systems this assumption is impractical as the systems would break before any reasonable exploration has taken place, i.e., most physical systems don't satisfy the ergodicity assumption. In this paper we address the need for safe exploration methods in Markov decision processes. We first propose a general formulation of safety through ergodicity. We show that imposing safety by restricting attention to the resulting set of guaranteed safe policies is NP-hard. We then present an efficient algorithm for guaranteed safe, but potentially suboptimal, exploration. At the core is an optimization formulation in which the constraints restrict attention to a subset of the guaranteed safe policies and the objective favors exploration policies. Our framework is compatible with the majority of previously proposed exploration methods, which rely on an exploration bonus. Our experiments, which include a Martian terrain exploration problem, show that our method is able to explore better than classical exploration methods.
Policy Gradients with Variance Related Risk Criteria
Authors: Dotan Di Castro, Aviv Tamar, Shie Mannor
Date: 2012-06-27 00:00:00
Tasks: Reinforcement Learning (RL)

Managing risk in dynamic decision problems is of cardinal importance in many fields such as finance and process control. The most common approach to defining risk is through various variance related criteria such as the Sharpe Ratio or the standard deviation adjusted reward. It is known that optimizing many of the variance related risk criteria is NP-hard. In this paper we devise a framework for local policy gradient style algorithms for reinforcement learning for variance related criteria. Our starting point is a new formula for the variance of the cost-to-go in episodic tasks. Using this formula we develop policy gradient algorithms for criteria that involve both the expected cost and the variance of the cost. We prove the convergence of these algorithms to local minima and demonstrate their applicability in a portfolio planning problem.
Curiosity-Driven Energy-Efficient Worker Scheduling in Vehicular Crowdsourcing: A Deep Reinforcement Learning Approach
Authors: Chi Harold Liu, Yinuo Zhao, Zipeng Dai, Ye Yuan, Guoren Wang, Dapeng Wu, Kin K. Leung
Date: 2020-04-24 00:00:00
Proceeding: ICDE 2020 4
Tasks: Deep Reinforcement Learning, Fairness, Reinforcement Learning (RL), Scheduling

Spatial crowdsourcing (SC) utilizes the potential of a crowd to accomplish certain location based tasks. Although worker scheduling has been well studied recently, most existing works only focus on the static deployment of workers but ignore their temporal movement continuity. In this paper, we explicitly consider the use of unmanned vehicular workers, e.g., drones and driverless cars, which are more controllable and can be deployed in remote or dangerous areas to carry on long-term and hash tasks as a vehicular crowdsourcing (VC) campaign. We propose a novel deep reinforcement learning (DRL) approach for curiosity-driven energy-efficient worker scheduling, called "DRL-CEWS", to achieve an optimal trade-off between maximizing the collected amount of data and coverage fairness, and minimizing the overall energy consumption of workers. Specifically, we first utilize a chief-employee distributed computational architecture to stabilize and facilitate the training process. Then, we propose a spatial curiosity model with a sparse reward mechanism to help derive the optimal policy in large crowdsensing space with unevenly distributed data. Extensive simulation results show that DRL-CEWS outperforms the state-of-the-art methods and baselines, and we also visualize the benefits curiosity model brings and show the impact of two hyperparameters.
Visualizing and Explaining Language Models
Authors: Adrian M. P. Braşoveanu, Răzvan Andonie
Date: 2022-04-30 00:00:00
Tasks: Deep Learning, Language Modeling, Language Modelling

During the last decade, Natural Language Processing has become, after Computer Vision, the second field of Artificial Intelligence that was massively changed by the advent of Deep Learning. Regardless of the architecture, the language models of the day need to be able to process or generate text, as well as predict missing words, sentences or relations depending on the task. Due to their black-box nature, such models are difficult to interpret and explain to third parties. Visualization is often the bridge that language model designers use to explain their work, as the coloring of the salient words and phrases, clustering or neuron activations can be used to quickly understand the underlying models. This paper showcases the techniques used in some of the most popular Deep Learning for NLP visualizations, with a special focus on interpretability and explainability.
Low power communication signal enhancement method of Internet of things based on nonlocal mean denoising
Authors: Mingchuan Tian, Jizheng Liu
Date: 2022-05-14 00:00:00
Tasks: Denoising

In order to improve the transmission effect of low-power communication signal of Internet of things and compress the enhancement time of low-power communication signal, this paper designs a low-power communication signal enhancement method of Internet of things based on nonlocal mean denoising. Firstly, the residual of one-dimensional communication layer is pre processed by convolution core to obtain the residual of one-dimensional communication layer; Then, according to the two classification recognition method, the noise reduction signal feature recognition of the low-power communication signal of the Internet of things is realized, the non local mean noise reduction algorithm is used to remove the low-power communication signal of the Internet of things, and the weight value between similar blocks is calculated according to the European distance method. Finally, the low-power communication signal enhancement of the Internet of things is realized by the non local mean value denoising method. The experimental results show that the communication signal enhancement time overhead of this method is low, which is always less than 2.6s. The lowest bit error rate after signal enhancement is about 1%, and the signal-to-noise ratio is up to 18 dB, which shows that this method can achieve signal enhancement.
Audio Declipping with (Weighted) Analysis Social Sparsity
Authors: Pavel Záviška, Pavel Rajmic
Date: 2022-05-20 00:00:00
Tasks: Audio declipping

We develop the analysis (cosparse) variant of the popular audio declipping algorithm of Siedenburg et al. (2014). Furthermore, we extend both the old and the new variants by the possibility of weighting the time-frequency coefficients. We examine the audio reconstruction performance of several combinations of weights and shrinkage operators. The weights are shown to improve the reconstruction quality in some cases; however, the best scores achieved by the non-weighted methods are not surpassed with the help of weights. Yet, the analysis Empirical Wiener (EW) shrinkage was able to reach the quality of a computationally more expensive competitor, the Persistent Empirical Wiener (PEW). Moreover, the proposed analysis variant incorporating PEW slightly outperforms the synthesis counterpart in terms of an auditorily motivated metric.
Voice Activity Projection: Self-supervised Learning of Turn-taking Events
Authors: Erik Ekstedt, Gabriel Skantze
Date: 2022-05-19 00:00:00
Tasks: Self-Supervised Learning

The modeling of turn-taking in dialog can be viewed as the modeling of the dynamics of voice activity of the interlocutors. We extend prior work and define the predictive task of Voice Activity Projection, a general, self-supervised objective, as a way to train turn-taking models without the need of labeled data. We highlight a theoretical weakness with prior approaches, arguing for the need of modeling the dependency of voice activity events in the projection window. We propose four zero-shot tasks, related to the prediction of upcoming turn-shifts and backchannels, and show that the proposed model outperforms prior work.
End-to-End Zero-Shot Voice Conversion with Location-Variable Convolutions
Authors: Wonjune Kang, Mark Hasegawa-Johnson, Deb Roy
Date: 2022-05-19 00:00:00
Tasks: Speech Synthesis, Style Transfer, Voice Conversion

Zero-shot voice conversion is becoming an increasingly popular research topic, as it promises the ability to transform speech to sound like any speaker. However, relatively little work has been done on end-to-end methods for this task, which are appealing because they remove the need for a separate vocoder to generate audio from intermediate features. In this work, we propose LVC-VC, an end-to-end zero-shot voice conversion model that uses location-variable convolutions (LVCs) to jointly model the conversion and speech synthesis processes. LVC-VC utilizes carefully designed input features that have disentangled content and speaker information, and it uses a neural vocoder-like architecture that utilizes LVCs to efficiently combine them and perform voice conversion while directly synthesizing time domain audio. Experiments show that our model achieves especially well balanced performance between voice style transfer and speech intelligibility compared to several baselines.
Data-driven Distributed Control to Scale EV Integration into Power Grid
Authors: Emin Ucer, Mithat Kisacikoglu
Date: 2022-05-20 00:00:00
Tasks: Fairness

Electric vehicles (EVs) are finally making their way onto the roads, but the challenges concerning long charging times and impact on congestion of the power distribution grid are still not resolved. Proposed solutions depend on heavy communication and rigorous computation and mostly need real-time connectivity for optimal operation; thereby, they are not scalable. With the availability of historical measurement data, EV chargers can take better-informed actions while staying mostly off-line. This study develops a distributed and data-driven congestion detection methodology together with the Additive Increase Multiplicative Decrease (AIMD) algorithm to control mass EV charging in a distribution grid. The proposed distributed AIMD algorithm performs very closely to the ideal AIMD in terms of fairness and congestion handling, and its communication need is significantly low. The results can provide crucial insights on how data can be used to reveal the inner dynamics and structure of the power grid and help develop more advanced data-driven algorithms for grid integrated power electronics control.
Stochastic MPC with robustness to bounded parametric uncertainty
Authors: Elena Arcari, Andrea Iannelli, Andrea Carron, Melanie N. Zeilinger
Date: 2022-05-20 00:00:00
Tasks: Model Predictive Control

The performance of model-based control techniques strongly depends on the quality of the employed dynamics model. If strong guarantees are desired, it is therefore common to robustly treat all possible sources of uncertainty, such as model inaccuracies or external disturbances. This, however, can result in overly conservative control strategies. In this paper, we present a stochastic model predictive control approach for discrete-time LTI systems subject to bounded parametric uncertainty and potentially unbounded stochastic additive noise. The proposed scheme makes use of homothetic tubes along the prediction horizon for a robust treatment of parametric uncertainty. Stochastic noise is handled by non-conservatively tightening constraints using the concept of probabilistic reachable sets (PRS). In order to accommodate all possible parametric uncertainties, we provide a strategy for generating "robustified" PRS based only on first and second moments of the noise sequence. In the case of quadratic cost functions, and under a further i.i.d. assumption on the noise distribution, we also provide an average asymptotic performance bound for the l2-norm of the closed-loop state. Finally, we demonstrate our scheme on both an illustrative example, and in a building temperature control problem.
Near-field focusing using phased arrays with dynamic polarization control
Authors: Nitin Jonathan Myers, Yanki Aslan, Geethu Joseph
Date: 2022-05-20 00:00:00

Phased arrays in near-field communication allow the transmitter to focus wireless signals in a small region around the receiver. Proper focusing is achieved by carefully tuning the phase shifts and the polarization of the signals transmitted from the phased array. In this paper, we study the impact of polarization on near-field focusing and investigate the use of dynamic polarization control (DPC) phased arrays in this context. Our studies indicate that the optimal polarization configuration for near-field focusing varies spatially across the antenna array. Such a spatial variation motivates the need for DPC phased arrays which allow independent polarization control across different antennas. We show using simulations that DPC phased arrays in the near-field achieve a higher received signal-to-noise ratio than conventional switched- or dual-polarization phased arrays.
Approximate Dynamic Programming for Constrained Linear Systems: A Piecewise Quadratic Approximation Approach
Authors: Kanghui He, Shengling Shi, Ton van den Boom, Bart De Schutter
Date: 2022-05-20 00:00:00
Tasks: Model Predictive Control

Approximate dynamic programming (ADP) faces challenges in dealing with constraints in control problems. Model predictive control (MPC) is, in comparison, well-known for its accommodation of constraints and stability guarantees, although its computation is sometimes prohibitive. This paper introduces an approach combining the two methodologies to overcome their individual limitations. The predictive control law for constrained linear quadratic regulation (CLQR) problems has been proven to be piecewise affine (PWA) while the value function is piecewise quadratic. We exploit these formal results from MPC to design an ADP method for CLQR problems. A novel convex and piecewise quadratic neural network with a local-global architecture is proposed to provide an accurate approximation of the value function, which is used as the cost-to-go function in the online dynamic programming problem. An efficient decomposition algorithm is developed to speed up the online computation. Rigorous stability analysis of the closed-loop system is conducted for the proposed control scheme under the condition that a good approximation of the value function is achieved. Comparative simulations are carried out to demonstrate the potential of the proposed method in terms of online computation and optimality.
Distributed Optimization in Distribution Systems with Grid-Forming and Grid-Supporting Inverters
Authors: Rabayet Sadnan, Anamika Dubey
Date: 2022-05-20 00:00:00
Tasks: Decision Making, Distributed Computing, Distributed Optimization, Problem Decomposition

With massive penetrations of active grid-edge technologies, distributed computing and optimization paradigm has gained significant attention to solve distribution-level optimal power flow (OPF) problems. However, the application of generic distributed optimization techniques to OPF problems leads to a very large number of macro-iterations or communication rounds among the distributed computing agents delaying the decision-making process or resulting in suboptimal solutions. Moreover, the existing distribution-level OPF problems typically model inverter-interfaced distributed energy resources (DERs) as grid-following inverters; grid-supporting and grid-forming functionalities have not been explicitly considered. The added complexities introduced by different inverter models require further attention to developing an appropriate model for new types of inverter-based DERs and computationally-tractable OPF algorithms. In this paper, we expand the distribution-level OPF model to include a combination of the grid-forming, grid-supporting, grid-following inverter-based DERs and also present the application of a domain-specific problem decomposition and distributed algorithm for the topologically radial power distribution systems to efficiently solve distribution-level OPF problem.
A Subspace Method for Time Series Anomaly Detection in Cyber-Physical Systems
Authors: Fredy Vides, Esteban Segura, Carlos Vargas-Agüero
Date: 2022-05-20 00:00:00
Tasks: Anomaly Detection, Time Series, Time Series Analysis, Time Series Anomaly Detection

Time series anomaly detection is an important process for system monitoring and model switching, among other applications in cyber-physical systems. In this document, we present a fast subspace method for time series anomaly detection, with a relatively low computational cost, that has been designed for anomaly detection in real sensor signals corresponding to dynamical systems. We also present some general results corresponding to the theoretical foundations of our method, together with a prototypical algorithm to for time series anomaly detection. Some numerical examples corresponding to applications of the prototypical algorithm are presented, and some computational tools based on the theory and algorithms presented in this paper, are provided.
Long Run Incremental Cost (LRIC) Distribution Network Pricing in UK, advising China's Distribution Network
Authors: Asad Mujeeb, Wang Peng
Date: 2022-05-20 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning (RL)

Electricity distribution network system is considered one of the key component of the modern electrical power system. Due to increase in the energy demand, penetration of renewable energy resources into the power system has been extensively increasing in recent years. More and more distributed generations (DGs) are joining the distribution network to create balance in the power system and meet the supply and demand of consumers. Today, large amount of DGs inclusion in the distribution network system has completely modernized power system resulting in a decentralize electricity market. Hence, Government of UK is pressurizing 14 distribution network operators (DNOs) to include more DGs into their distribution network system. DGs inclusion in the network system might be helpful due to many factors, but it creates many challenges for distribution network system in the long term. The network security is realized to be one of the challenge that impact the efficiency of accurate calculation and distribution of network pricing among consumers. To address the aforementioned issue, this research analysed the network security on the basis of Long run incremental cost (LRIC) pricing to balance and reduce the network pricing for the DNOs in UK. However, this study presented an approach of Deep reinforcement learning (DRL) also called deep reinforcement learning algorithm (DQN) to optimize the reactive power values in the network to balance and reduce the network pricing while keeping the network security. The method considers IEEE14 bus as its mathematical model and practically simulates the method in MATLAB using DQN algorithm pseudo codes. The network security has been analysed with and without security factor before and after the nodal injection into the network.