pwc-archive/papers-with-abstracts huggingface dataset:
Don't Pay Attention to the Noise: Learning Self-supervised Representations of Light Curves with a Denoising Time Series Transformer
Authors: Mario Morvan, Nikolaos Nikolaou, Kai Hou Yip, Ingo Waldmann
Date: 2022-07-06 00:00:00
Tasks: Denoising, Time Series, Time Series Analysis

Astrophysical light curves are particularly challenging data objects due to the intensity and variety of noise contaminating them. Yet, despite the astronomical volumes of light curves available, the majority of algorithms used to process them are still operating on a per-sample basis. To remedy this, we propose a simple Transformer model -- called Denoising Time Series Transformer (DTST) -- and show that it excels at removing the noise and outliers in datasets of time series when trained with a masked objective, even when no clean targets are available. Moreover, the use of self-attention enables rich and illustrative queries into the learned representations. We present experiments on real stellar light curves from the Transiting Exoplanet Space Satellite (TESS), showing advantages of our approach compared to traditional denoising techniques.
Adaptive deep learning for nonlinear time series models
Authors: Daisuke Kurisu, Riku Fukami, Yuta Koike
Date: 2022-07-06 00:00:00
Tasks: Deep Learning, regression, Time Series, Time Series Analysis, Time Series Regression

In this paper, we develop a general theory for adaptive nonparametric estimation of the mean function of a non-stationary and nonlinear time series model using deep neural networks (DNNs). We first consider two types of DNN estimators, non-penalized and sparse-penalized DNN estimators, and establish their generalization error bounds for general non-stationary time series. We then derive minimax lower bounds for estimating mean functions belonging to a wide class of nonlinear autoregressive (AR) models that include nonlinear generalized additive AR, single index, and threshold AR models. Building upon the results, we show that the sparse-penalized DNN estimator is adaptive and attains the minimax optimal rates up to a poly-logarithmic factor for many nonlinear AR models. Through numerical simulations, we demonstrate the usefulness of the DNN methods for estimating nonlinear AR models with intrinsic low-dimensional structures and discontinuous or rough mean functions, which is consistent with our theory.
Conditional Distribution Function Estimation Using Neural Networks for Censored and Uncensored Data
Authors: Bingqing Hu, Bin Nan
Date: 2022-07-06 00:00:00
Proceeding: NeurIPS 2023 11

Most work in neural networks focuses on estimating the conditional mean of a continuous response variable given a set of covariates.In this article, we consider estimating the conditional distribution function using neural networks for both censored and uncensored data. The algorithm is built upon the data structure particularly constructed for the Cox regression with time-dependent covariates. Without imposing any model assumption, we consider a loss function that is based on the full likelihood where the conditional hazard function is the only unknown nonparametric parameter, for which unconstraint optimization methods can be applied. Through simulation studies, we show the proposed method possesses desirable performance, whereas the partial likelihood method and the traditional neural networks with $L_2$ loss yield biased estimates when model assumptions are violated. We further illustrate the proposed method with several real-world data sets. The implementation of the proposed methods is made available at https://github.com/bingqing0729/NNCDE.
SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrieval
Authors: Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, Furu Wei
Date: 2022-07-06 00:00:00
Tasks: Language Modeling, Language Modelling, Passage Retrieval, Retrieval

In this paper, we propose SimLM (Similarity matching with Language Model pre-training), a simple yet effective pre-training method for dense passage retrieval. It employs a simple bottleneck architecture that learns to compress the passage information into a dense vector through self-supervised pre-training. We use a replaced language modeling objective, which is inspired by ELECTRA, to improve the sample efficiency and reduce the mismatch of the input distribution between pre-training and fine-tuning. SimLM only requires access to unlabeled corpus, and is more broadly applicable when there are no labeled data or queries. We conduct experiments on several large-scale passage retrieval datasets, and show substantial improvements over strong baselines under various settings. Remarkably, SimLM even outperforms multi-vector approaches such as ColBERTv2 which incurs significantly more storage cost. Our code and model check points are available at https://github.com/microsoft/unilm/tree/master/simlm .
Ask Me What You Need: Product Retrieval using Knowledge from GPT-3
Authors: Su Young Kim, Hyeonjin Park, Kyuyong Shin, Kyung-Min Kim
Date: 2022-07-06 00:00:00
Tasks: Common Sense Reasoning, Question Answering, Retrieval

As online merchandise become more common, many studies focus on embedding-based methods where queries and products are represented in the semantic space. These methods alleviate the problem of vocab mismatch between the language of queries and products. However, past studies usually dealt with queries that precisely describe the product, and there still exists the need to answer imprecise queries that may require common sense knowledge, i.e., 'what should I get my mom for Mother's Day.' In this paper, we propose a GPT-3 based product retrieval system that leverages the knowledge-base (KB) of GPT-3 for question answering; users do not need to know the specific illustrative keywords for a product when querying. Our method tunes prompt tokens of GPT-3 to prompt knowledge and render answers that are mapped directly to products without further processing. Our method shows consistent performance improvement on two real-world and one public dataset, compared to the baseline methods. We provide an in-depth discussion on leveraging GPT-3 knowledge into a question answering based retrieval system.
Re-weighting Negative Samples for Model-Agnostic Matching
Authors: Jiazhen Lou, Hong Wen, Fuyu Lv, Jing Zhang, Tengfei Yuan, Zhao Li
Date: 2022-07-06 00:00:00
Tasks: model, Multi-Task Learning, Recommendation Systems

Recommender Systems (RS), as an efficient tool to discover users' interested items from a very large corpus, has attracted more and more attention from academia and industry. As the initial stage of RS, large-scale matching is fundamental yet challenging. A typical recipe is to learn user and item representations with a two-tower architecture and then calculate the similarity score between both representation vectors, which however still struggles in how to properly deal with negative samples. In this paper, we find that the common practice that randomly sampling negative samples from the entire space and treating them equally is not an optimal choice, since the negative samples from different sub-spaces at different stages have different importance to a matching model. To address this issue, we propose a novel method named Unbiased Model-Agnostic Matching Approach (UMA$^2$). It consists of two basic modules including 1) General Matching Model (GMM), which is model-agnostic and can be implemented as any embedding-based two-tower models; and 2) Negative Samples Debias Network (NSDN), which discriminates negative samples by borrowing the idea of Inverse Propensity Weighting (IPW) and re-weighs the loss in GMM. UMA$^2$ seamlessly integrates these two modules in an end-to-end multi-task learning framework. Extensive experiments on both real-world offline dataset and online A/B test demonstrate its superiority over state-of-the-art methods.
Combining Topic Modeling with Grounded Theory: Case Studies of Project Collaboration
Authors: Eyyub Can Odacioglu, Lihong Zhang, Richard Allmendinger
Date: 2022-06-28 00:00:00
Tasks: Management

This paper proposes an Artificial Intelligence (AI) Grounded Theory for management studies. We argue that this novel and rigorous approach that embeds topic modelling will lead to the latent knowledge to be found. We illustrate this abductive method using 51 case studies of collaborative innovation published by Project Management Institute (PMI). Initial results are presented and discussed that include 40 topics, 6 categories, 4 of which are core categories, and two new theories of project collaboration.
A Dataset on Malicious Paper Bidding in Peer Review
Authors: Steven Jecmen, Minji Yoon, Vincent Conitzer, Nihar B. Shah, Fei Fang
Date: 2022-06-24 00:00:00
Tasks: Descriptive

In conference peer review, reviewers are often asked to provide "bids" on each submitted paper that express their interest in reviewing that paper. A paper assignment algorithm then uses these bids (along with other data) to compute a high-quality assignment of reviewers to papers. However, this process has been exploited by malicious reviewers who strategically bid in order to unethically manipulate the paper assignment, crucially undermining the peer review process. For example, these reviewers may aim to get assigned to a friend's paper as part of a quid-pro-quo deal. A critical impediment towards creating and evaluating methods to mitigate this issue is the lack of any publicly-available data on malicious paper bidding. In this work, we collect and publicly release a novel dataset to fill this gap, collected from a mock conference activity where participants were instructed to bid either honestly or maliciously. We further provide a descriptive analysis of the bidding behavior, including our categorization of different strategies employed by participants. Finally, we evaluate the ability of each strategy to manipulate the assignment, and also evaluate the performance of some simple algorithms meant to detect malicious bidding. The performance of these detection algorithms can be taken as a baseline for future research on detecting malicious bidding.
Learning Classifier Systems for Self-Explaining Socio-Technical-Systems
Authors: Michael Heider, Helena Stegherr, Richard Nordsieck, Jörg Hähner
Date: 2022-07-01 00:00:00
Tasks: Decision Making

In socio-technical settings, operators are increasingly assisted by decision support systems. By employing these, important properties of socio-technical systems such as self-adaptation and self-optimization are expected to improve further. To be accepted by and engage efficiently with operators, decision support systems need to be able to provide explanations regarding the reasoning behind specific decisions. In this paper, we propose the usage of Learning Classifier Systems, a family of rule-based machine learning methods, to facilitate transparent decision making and highlight some techniques to improve that. We then present a template of seven questions to assess application-specific explainability needs and demonstrate their usage in an interview-based case study for a manufacturing scenario. We find that the answers received did yield useful insights for a well-designed LCS model and requirements to have stakeholders actively engage with an intelligent agent.
Improving Streaming End-to-End ASR on Transformer-based Causal Models with Encoder States Revision Strategies
Authors: Zehan Li, Haoran Miao, Keqi Deng, Gaofeng Cheng, Sanli Tian, Ta Li, Yonghong Yan
Date: 2022-07-06 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Knowledge Distillation, speech-recognition, Speech Recognition

There is often a trade-off between performance and latency in streaming automatic speech recognition (ASR). Traditional methods such as look-ahead and chunk-based methods, usually require information from future frames to advance recognition accuracy, which incurs inevitable latency even if the computation is fast enough. A causal model that computes without any future frames can avoid this latency, but its performance is significantly worse than traditional methods. In this paper, we propose corresponding revision strategies to improve the causal model. Firstly, we introduce a real-time encoder states revision strategy to modify previous states. Encoder forward computation starts once the data is received and revises the previous encoder states after several frames, which is no need to wait for any right context. Furthermore, a CTC spike position alignment decoding algorithm is designed to reduce time costs brought by the revision strategy. Experiments are all conducted on Librispeech datasets. Fine-tuning on the CTC-based wav2vec2.0 model, our best method can achieve 3.7/9.2 WERs on test-clean/other sets, which is also competitive with the chunk-based methods and the knowledge distillation methods.
mu-synthesis-based Generalized Robust Framework for Grid-following and Grid-forming Inverters
Authors: Soham Chakraborty, Sourav Patel, Murti V Salapaka
Date: 2022-07-06 00:00:00

Grid-following and grid-forming inverters are integral components of microgrids and for integration of renewable energy sources with the grid. For grid following inverters, which need to emulate controllable current sources, a significant challenge is to address the large uncertainty of the grid impedance. For grid forming inverters, which need to emulate a controllable voltage source, large uncertainty due to varying loads has to be addressed. In this article, a mu-synthesis-based robust control design methodology, where performance under quantified uncertainty is guaranteed, is developed under a unified approach for both grid-following and grid-forming inverters. The control objectives, while designing the proposed optimal controllers, are: i) reference tracking, disturbance rejection, harmonic compensation capability with sufficient LCL resonance damping under large variations of grid impedance uncertainty for grid-following inverters; ii) reference tracking, disturbance rejection, harmonic compensation capability with enhanced dynamic response under large variations of equivalent loading uncertainty for grid-forming inverters. A combined system-in-the-loop (SIL), controller hardware-in-the-loop (CHIL) and power hardware-in-the-loop (PHIL) based experimental validation on 10 kVA microgrid system with two physical inverter systems is conducted in order to evaluate the efficacy and viability of the proposed controllers.
Modeling and Analysis of Utilizing Cryptocurrency Mining for Demand Flexibility in Electric Energy Systems: A Synthetic Texas Grid Case Study
Authors: Ali Menati, Kiyeob Lee, Le Xie
Date: 2022-07-06 00:00:00

The electricity sector is facing the dual challenge of supporting increasing level of demand electrification while substantially reducing its carbon footprint. Among electricity demands, the energy consumption of cryptocurrency mining data centers has witnessed significant growth worldwide. If well-coordinated, these data centers could be tailor-designed to aggressively absorb the increasing uncertainties of energy supply and, in turn, provide valuable grid-level services in the electricity market. In this paper, we study the impact of integrating new cryptocurrency mining loads into Texas power grid and the potential profit of utilizing demand flexibility from cryptocurrency mining facilities in the electricity market. We investigate different demand response programs available for data centers and quantify the annual profit of cryptocurrency mining units participating in these programs. We perform our simulations using a synthetic 2000 bus ERCOT grid model, along with added cryptocurrency mining loads on top of the real-world demand profiles in Texas. Our preliminary results show that depending on the size and location of these new loads, we observe different impacts on the ERCOT electricity market, where they could increase the electricity prices and incur more fluctuations in a highly non-uniform manner.
Adaptive Identification with Guaranteed Performance Under Saturated-Observation and Non-Persistent Excitation
Authors: Lantian Zhang, Lei Guo
Date: 2022-07-06 00:00:00

This paper investigates the adaptive identification and prediction problems for stochastic dynamical systems with saturated observations, which arise from various fields in engineering and social systems, but up to now still lack comprehensive theoretical studies including performance guarantees needed in practical applications. With this impetus, the paper has made the following main contributions: (i) To introduce a two-step Quasi-Newton (TSQN) algorithm to improve the performance of the identification, which is applicable to a typical class of nonlinear stochastic systems with outputs observed under possibly varying saturation. (ii) To establish the global convergence of both the parameter estimators and adaptive predictors and to prove the asymptotic normality, under the weakest possible non-persistent excitation (PE) condition, which can be applied to stochastic feedback systems with general non-stationary and correlated system signals or data. (iii) To establish useful probabilistic estimation error bounds for any given finite length of data, using either martingale inequalities or Monte Carlo experiments. A numerical example is also provided to illustrate the performance of the proposed identification algorithm.
A Review of High-Performance Computing and Parallel Techniques Applied to Power Systems Optimization
Authors: Ahmed Al-Shafei, Hamidreza Zareipour, Yankai Cao
Date: 2022-07-06 00:00:00
Tasks: Cloud Computing, CPU, GPU, State Estimation

The accelerating technological landscape and drive towards net-zero emission made the power system grow in scale and complexity. Serial computational approaches for grid planning and operation struggle to execute necessary calculations within reasonable times. Resorting to high-performance and parallel computing approaches has become paramount. Moreover, the ambitious plans for the future grid and IoT integration make a shift towards utilizing Cloud computing inevitable. This article recounts the dawn of parallel computation and its appearance in power system studies, reviewing the most recent literature and research on exploiting the available computational resources and technologies today. The article starts with a brief introduction to the field. The relevant hardware and paradigms are then explained thoroughly in this article providing a base for the reader to understand the literature. Later, parallel power system studies are reviewed, reciting the study development from older papers up to the 21st century, emphasizing the most impactful work of the last decade. The studies included system stability studies, state estimation and power system operation, and market optimization. The reviews are split into \ac{CPU} based,\ac{GPU} based, and Cloud-based studies. Finally, the state-of-the-art is discussed, highlighting the issue of standardization and the future of computation in power system studies.
Interference Aware Path Planning for Mobile Robots in mmWave Multi Cell Networks
Authors: Yijing Ren, Vasilis Friderikos
Date: 2022-07-05 00:00:00

The emerging beyond 5G and envisioned 6G wireless networks are considered as key enablers in supporting a diversified set of applications for industrial mobile robots (MRs). The scenario under investigation in this paper relates to mobile robots that autonomously roam in an industrial floor and perform a variety of tasks at different locations whilst utilizing high directivity beamformers in mmWave small cells. In such scenarios, the potential close proximity of mobile robots connected to different base stations, may cause excessive levels of interference having as a net result a decrease in the overall achievable data rate in the network. To resolve this issue, a novel mixed integer linear programming formulation is proposed where the trajectory of the mobile robots is considered jointly with the interference level at different beam sectors. Therefore, creating a low interference path for each mobile robot in the industrial floor. A wide set of numerical investigations reveal that the proposed path planning optimization approach for the mmWave connected mobile robots can improve the overall achievable throughput by up to 31% compared to an interference oblivious scheme, without penalizing the overall travelling time.
STOCHOS: Stochastic Opportunistic Maintenance Scheduling For Offshore Wind Farms
Authors: Petros Papadopoulos, David W. Coit, Ahmed Aziz Ezzat
Date: 2022-07-05 00:00:00
Tasks: Scheduling

Despite the promising outlook, the numerous economic and environmental benefits of offshore wind energy are still compromised by its high operations and maintenance (O&M) expenditures. On one hand, offshore-specific challenges such as site remoteness, harsh weather, transportation requirements, and production losses, significantly inflate the O&M costs relative to land-based wind farms. On the other hand, the uncertainties in weather conditions, asset degradation, and electricity prices largely constrain the farm operator's ability to identify the time windows at which maintenance is possible, let alone optimal. In response, we propose STOCHOS, short for the stochastic holistic opportunistic scheduler--a maintenance scheduling approach tailored to address the unique challenges and uncertainties in offshore wind farms. Given probabilistic forecasts of key environmental and operational parameters, STOCHOS optimally schedules the offshore maintenance tasks by harnessing the opportunities that arise due to favorable weather conditions, on-site maintenance resources, and maximal operating revenues. STOCHOS is formulated as a two-stage stochastic mixed integer linear program, which we solve using a scenario-based rolling horizon algorithm that aligns with the industrial practice. Tested on real-world data from the U.S. North Atlantic where several offshore wind farms are in-development, STOCHOS demonstrates considerable improvements relative to prevalent maintenance benchmarks, across various O&M metrics, including total cost, downtime, resource utilization, and maintenance interruptions.
Numerical Approximation Methods for Antenna Radiation Patterns for Motus Wildlife Tracking Systems
Authors: Erik Carlson, Douglas Gobeille, Robert Deluca, Pam Loring
Date: 2022-06-28 00:00:00

As plans for offshore wind energy development increase in the US, the developing methods to monitor migratory birds and bats offshore is an important area of research. To contribute to this research, current guidance recommends the deployment of Motus Wildlife Tracking stations by each developer for pre- and post- construction monitoring. To understand the characteristics of each of the stations, calibration techniques are recommended for each deployment. Despite this, previous attempts to calibrate these stations has failed to provide sufficient detail to allow for high-resolution tracking techniques. In this paper, we introduce an affordable and robust methodology to calibrate stations in the off-shore environment and develop processes to turn this raw calibration data into a numerical approximation of the antennas radiation pattern.