pwc-archive/papers-with-abstracts huggingface dataset:
Efficient and Generalizable Tuning Strategies for Stochastic Gradient MCMC
Authors: Jeremie Coullon, Leah South, Christopher Nemeth
Date: 2021-05-27 00:00:00
Tasks: Bayesian Inference

Stochastic gradient Markov chain Monte Carlo (SGMCMC) is a popular class of algorithms for scalable Bayesian inference. However, these algorithms include hyperparameters such as step size or batch size that influence the accuracy of estimators based on the obtained posterior samples. As a result, these hyperparameters must be tuned by the practitioner and currently no principled and automated way to tune them exists. Standard MCMC tuning methods based on acceptance rates cannot be used for SGMCMC, thus requiring alternative tools and diagnostics. We propose a novel bandit-based algorithm that tunes the SGMCMC hyperparameters by minimizing the Stein discrepancy between the true posterior and its Monte Carlo approximation. We provide theoretical results supporting this approach and assess various Stein-based discrepancies. We support our results with experiments on both simulated and real datasets, and find that this method is practical for a wide range of applications.
A Non-asymptotic Approach to Best-Arm Identification for Gaussian Bandits
Authors: Antoine Barrier, Aurélien Garivier, Tomáš Kocák
Date: 2021-05-27 00:00:00

We propose a new strategy for best-arm identification with fixed confidence of Gaussian variables with bounded means and unit variance. This strategy, called Exploration-Biased Sampling, is not only asymptotically optimal: it is to the best of our knowledge the first strategy with non-asymptotic bounds that asymptotically matches the sample complexity.But the main advantage over other algorithms like Track-and-Stop is an improved behavior regarding exploration: Exploration-Biased Sampling is biased towards exploration in a subtle but natural way that makes it more stable and interpretable. These improvements are allowed by a new analysis of the sample complexity optimization problem, which yields a faster numerical resolution scheme and several quantitative regularity results that we believe of high independent interest.
Statistical Depth Meets Machine Learning: Kernel Mean Embeddings and Depth in Functional Data Analysis
Authors: George Wynne, Stanislav Nagy
Date: 2021-05-26 00:00:00
Tasks: BIG-bench Machine Learning

Statistical depth is the act of gauging how representative a point is compared to a reference probability measure. The depth allows introducing rankings and orderings to data living in multivariate, or function spaces. Though widely applied and with much experimental success, little theoretical progress has been made in analysing functional depths. This article highlights how the common $h$-depth and related statistical depths for functional data can be viewed as a kernel mean embedding, a technique used widely in statistical machine learning. This connection facilitates answers to open questions regarding statistical properties of functional depths, as well as it provides a link between the depth and empirical characteristic function based procedures for functional data.
A functionality taxonomy for document search engines
Authors: Rik D. T. Janssen, Henderik A. Proper
Date: 2021-05-27 00:00:00

In this paper a functionality taxonomy for document search engines is proposed. It can be used to assess the features of a search engine, to position search engines relative to each other, or to select which search engine 'fits' a certain situation. One is able to identify areas for improvement. During development, we were guided by the viewpoint of the user. We use the word `search engine' in the broadest sense possible, including library and web based (meta) search engines. The taxonomy distinguishes seven functionality areas: an indexing service, user profiling, query composition, query execution, result presentation, result refinement, and history keeping. Each of these relates and provides services to other functionality areas. It can be extended whenever necessary. To illustrate the validity of our taxonomy, it has been used for comparing various document search engines existing today (ACM Digital Library, PiCarta, Copernic, AltaVista, Google, and GuideBeam). It appears that the functionality aspects covered by our taxonomy can be used for describing these search engines.
A data-driven strategy to combine word embeddings in information retrieval
Authors: Alfredo Silva, Marcelo Mendoza
Date: 2021-05-26 00:00:00
Tasks: Ad-Hoc Information Retrieval, Descriptive, Information Retrieval, Retrieval, Word Embeddings

Word embeddings are vital descriptors of words in unigram representations of documents for many tasks in natural language processing and information retrieval. The representation of queries has been one of the most critical challenges in this area because it consists of a few terms and has little descriptive capacity. Strategies such as average word embeddings can enrich the queries' descriptive capacity since they favor the identification of related terms from the continuous vector representations that characterize these approaches. We propose a data-driven strategy to combine word embeddings. We use Idf combinations of embeddings to represent queries, showing that these representations outperform the average word embeddings recently proposed in the literature. Experimental results on benchmark data show that our proposal performs well, suggesting that data-driven combinations of word embeddings are a promising line of research in ad-hoc information retrieval.
Learning to Quantize Deep Neural Networks: A Competitive-Collaborative Approach
Authors: Md Fahim Faysal Khan
Date: 2020-07-01 00:00:00
Tasks: Quantization

Reducing the model size and computation costs for dedicated
AI accelerator designs, neural network quantization methods have at-
tracted momentous attention recently. Unfortunately, merely minimizing
quantization loss using constant discretization causes accuracy deterio-
ration. In this paper, we propose an iterative accuracy-driven learning
framework of competitive-collaborative quantization (CCQ) to gradually
adapt the bit-precision of each individual layer. Orthogonal to prior
quantization policies working with full precision for the first and last
layers of the network, CCQ offers layer-wise competition for any target
quantization policy with holistic layer fine-tuning to recover accuracy,
where the state-of-the-art networks can be entirely quantized without
any significant accuracy degradation.
RL-GRIT: Reinforcement Learning for Grammar Inference
Authors: Walt Woods
Date: 2021-05-17 00:00:00
Tasks: Constituency Parsing, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

When working to understand usage of a data format, examples of the data format are often more representative than the format's specification. For example, two different applications might use very different JSON representations, or two PDF-writing applications might make use of very different areas of the PDF specification to realize the same rendered content. The complexity arising from these distinct origins can lead to large, difficult-to-understand attack surfaces, presenting a security concern when considering both exfiltration and data schizophrenia. Grammar inference can aid in describing the practical language generator behind examples of a data format. However, most grammar inference research focuses on natural language, not data formats, and fails to support crucial features such as type recursion. We propose a novel set of mechanisms for grammar inference, RL-GRIT, and apply them to understanding de facto data formats. After reviewing existing grammar inference solutions, it was determined that a new, more flexible scaffold could be found in Reinforcement Learning (RL). Within this work, we lay out the many algorithmic changes required to adapt RL from its traditional, sequential-time environment to the highly interdependent environment of parsing. The result is an algorithm which can demonstrably learn recursive control structures in simple data formats, and can extract meaningful structure from fragments of the PDF format. Whereas prior work in grammar inference focused on either regular languages or constituency parsing, we show that RL can be used to surpass the expressiveness of both classes, and offers a clear path to learning context-sensitive languages. The proposed algorithm can serve as a building block for understanding the ecosystems of de facto data formats.
Enhance Multimodal Model Performance with Data Augmentation: Facebook Hateful Meme Challenge Solution
Authors: Yang Li, Zinc Zhang, Hutchin Huang
Date: 2021-05-25 00:00:00
Tasks: Data Augmentation, Deep Learning

Hateful content detection is one of the areas where deep learning can and should make a significant difference. The Hateful Memes Challenge from Facebook helps fulfill such potential by challenging the contestants to detect hateful speech in multi-modal memes using deep learning algorithms. In this paper, we utilize multi-modal, pre-trained models VilBERT and Visual BERT. We improved models' performance by adding training datasets generated from data augmentation. Enlarging the training data set helped us get a more than 2% boost in terms of AUROC with the Visual BERT model. Our approach achieved 0.7439 AUROC along with an accuracy of 0.7037 on the challenge's test set, which revealed remarkable progress.
Intelligent Reflecting Surface-assisted Free-space Optical Communications
Authors: Vahid Jamali, Hedieh Ajam, Marzieh Najafi, Bernhard Schmauss, Robert Schober, H. Vincent Poor
Date: 2021-05-13 00:00:00

Free-space optical (FSO) systems are able to offer the high data-rate, secure, and cost-efficient communication links required for applications such as wireless front- and backhauling for 5G and 6G communication networks. Despite the substantial advancement of FSO systems over the past decades, the requirement of a line-of-sight connection between transmitter and receiver remains a key limiting factor for their deployment. In this paper, we discuss the potential role of intelligent reflecting surfaces (IRSs) as a solution to relax this requirement. We present an overview of existing optical IRS technologies; compare optical IRSs with radio-frequency IRSs and optical relays; and identify various open problems for future research on IRS-assisted FSO communications.
Compositional Thinking in Cyberphysical Systems Theory
Authors: Georgios Bakirtzis, Eswaran Subrahmanian, Cody H. Fleming
Date: 2021-05-27 00:00:00

Engineering safe and secure cyber-physical systems requires system engineers to develop and maintain a number of model views, both dynamic and static, which can be seen as algebras. We posit that verifying the composition of requirement, behavioral, and architectural models using category theory gives rise to a strictly compositional interpretation of cyber-physical systems theory, which can assist in the modeling and analysis of safety-critical cyber-physical systems.
An Improved Measure of Musical Noise Based on Spectral Kurtosis
Authors: Matteo Torcoli
Date: 2021-05-27 00:00:00
Tasks: Speech Enhancement

Audio processing methods operating on a time-frequency representation of the signal can introduce unpleasant sounding artifacts known as musical noise. These artifacts are observed in the context of audio coding, speech enhancement, and source separation. The change in kurtosis of the power spectrum introduced during the processing was shown to correlate with the human perception of musical noise in the context of speech enhancement, leading to the proposal of measures based on it. These baseline measures are here shown to correlate with human perception only in a limited manner. As ground truth for the human perception, the results from two listening tests are considered: one involving audio coding and one involving source separation. Simple but effective perceptually motivated improvements are proposed and the resulting new measure is shown to clearly outperform the baselines in terms of correlation with the results of both listening tests. Moreover, with respect to the listening test on musical noise in audio coding, the exhibited correlation is nearly as good as the one exhibited by the Artifact-related Perceptual Score (APS), which was found to be the best objective measure for this task. The APS is however computationally very expensive. The proposed measure is easily computed, requiring only a fraction of the computational cost of the APS.
Improving RNN-T ASR Accuracy Using Context Audio
Authors: Andreas Schwarz, Ilya Sklyar, Simon Wiesler
Date: 2020-11-20 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

We present a training scheme for streaming automatic speech recognition (ASR) based on recurrent neural network transducers (RNN-T) which allows the encoder network to learn to exploit context audio from a stream, using segmented or partially labeled sequences of the stream during training. We show that the use of context audio during training and inference can lead to word error rate reductions of more than 6% in a realistic production setting for a voice assistant ASR system. We investigate the effect of the proposed training approach on acoustically challenging data containing background speech and present data points which indicate that this approach helps the network learn both speaker and environment adaptation. To gain further insight into the ability of a long short-term memory (LSTM) based ASR encoder to exploit long-term context, we also visualize RNN-T loss gradients with respect to the input.
OpenSerDes: An Open Source Process-Portable All-Digital Serial Link
Authors: Gaurav Kumar K, Baibhab Chatterjee, Shreyas Sen
Date: 2021-05-27 00:00:00
Tasks: All

In the last decade, the growing influence of open source software has necessitated the need to reduce the abstraction levels in hardware design. Open source hardware significantly reduces the development time, increasing the probability of first-pass success and enable developers to optimize software solutions based on hardware features, thereby reducing the design costs. The recent introduction of open source Process Development Kit (OpenPDK) by Skywater technologies in June 2020 has eliminated the barriers to Application-Specific Integrated Circuit (ASIC) design, which is otherwise considered expensive and not easily accessible. The OpenPDK is the first concrete step towards achieving the goal of open source circuit blocks that can be imported to reuse and modify in ASIC design. With process technologies scaling down for better performance, the need for entirely digital designs, which can be synthesized in any standard Automatic Place-and-Route (APR) tool, has increased considerably, for mapping physical design to the new process technology. This work presents the first open source all-digital Serializer/Deserializer (SerDes) for multi-GHz serial links designed using Skywater OpenPDK 130nm process node. To ensure that the design is fully synthesizable, the SerDes uses CMOS inverter-based drivers at the Tx, while the Rx front end comprises a resistive feedback inverter as a sensing element, followed by sampling elements. A fully digital oversampling CDR at the Rx recovers the Tx clock for proper decoding of data bits. The physical design flow utilizes OpenLANE, which is an end-to-end tool for generating GDS from RTL. Virtuoso has been used for extracting parasitics for post-layout simulations, which exhibit the SerDes functionality at 2 Gbps for 34 dB channel loss while consuming 438 mW power. The GDS and netlist files of the SerDes are uploaded in a GitHub repository for public access.
High Tension Lines: Predicting robustness of high-voltage power-grids to cascading failure using network embedding
Authors: Jonathan Bourne
Date: 2021-05-27 00:00:00
Tasks: Graph Embedding, Network Embedding, Time Series Analysis, Vocal Bursts Intensity Prediction

This paper explores whether graph embedding methods can be used as a tool for analysing the robustness of power-grids within the framework of network science. The paper focuses on the strain elevation tension spring embedding (SETSe) algorithm and compares it to node2vec and Deep Graph Infomax, and the measures mean edge capacity and line load. These five methods are tested on how well they can predict the collapse point of the giant component of a network under random attack. The analysis uses seven power-grid networks, ranging from 14 to 2000 nodes. In total, 3456 load profiles are created for each network by loading the edges of the network to have a range of tolerances and concentrating network capacity into fewer edges. One hundred random attack sequences are generated for each load profile, and the mean number of attacks required for the giant component to collapse for each profile is recorded. The relationship between the embedding values for each load profile and the mean collapse point is then compared across all five methods. It is found that only SETSe and line load perform well as proxies for robustness with $R^2 = 0.89$ for both measures. When tested on a time series normal operating conditions line load performs exceptionally well ($R=0.99$). However, the SETSe algorithm provides valuable qualitative insight into the state of the power-grid by leveraging its method local smoothing and global weighting of node features to provide an interpretable geographical embedding. This paper shows that graph representation algorithms can be used to analyse network properties such as robustness to cascading failure attacks, even when the network is embedded at node level.
Optimal control of centrifugal spreader
Authors: Franz Rußwurm, Pavel Osinenko, Stefan Streif
Date: 2021-05-27 00:00:00
Tasks: Model Predictive Control

Achieving an evenly distributed fertilization spread pattern is a complex technical task. A corresponding control algorithm must account for the tractor movement, the settings of the spreader, the prescribed dosage as well as machine constraints. It dictates, in particular, the fertilization process needs be estimated ahead to achieve an optimized spread pattern. The presented work is concerned with the development of a predictive control scheme for optimized fertilizer application using modeling of the tractor moving on the field and the spread pattern in form of a crescent behind the tractor. In particular, the form of the spread pattern is modeled via four normal distributions, two for each side of the pattern. The control goal is to achieve a desired fertilizer distribution on the field. The study presents three algorithms for comparison: a one-step optimization and two approaches using model-predictive control, one with a simplified model of the spread pattern in the prediction horizon, and one with a comprehensive one model, respectively. The best results are obtained with model-predictive control using the comprehensive model.
Revisiting Grid-Forming and Grid-Following Inverters: A Duality Theory
Authors: Yitong Li, Yunjie Gu, Timothy C. Green
Date: 2021-05-27 00:00:00

Power electronic converters for integrating renewable energy resources into power systems can be divided into grid-forming and grid-following inverters. They possess certain similarities, but several important differences, which means that the relationship between them is quite subtle and sometimes obscure. In this article, a new perspective based on duality is proposed to create new insights. It successfully unifies the grid interfacing and synchronization characteristics of the two inverter types in a symmetric, elegant, and technology-neutral form. Analysis shows that the grid-forming and grid-following inverters are duals of each other in several ways including a) synchronization controllers: frequency droop control and phase-locked loop (PLL); b) grid-interfacing characteristics: current-following voltage-forming and voltage-following current-forming; c) swing characteristics: current-angle swing and voltage-angle swing; d) inner-loop controllers: output impedance shaping and output admittance shaping; and e) grid strength compatibility: strong-grid instability and weak-grid instability. The swing equations are also derived in dual form, which reveal the dynamic interaction between the grid strength, the synchronization controllers, and the inner-loop controllers. Insights are generated into cases of poor stability in both small-signal and transient/large-signal. The theoretical analysis and simulation results are used to illustrate cases for simple single-inverter-infinite-bus systems and a multi-inverter power network.
Target-Dependent Chemical Species Tomography with Hybrid Meshing of Sensing Regions
Authors: Rui Zhang, Jingjing Si, Godwin Enemali, Yong Bao, Chang Liu
Date: 2021-02-10 00:00:00

This paper develops a hybrid-size meshing scheme for target-dependent imaging in Chemical Species Tomography (CST). The traditional implementation of CST generally places the target field in the central region of laser sensing, the so-called Region of Interest (RoI), with uniform-size meshes. The centre of the RoI locates at the midpoint between the laser emitters and receivers, while the size of the RoI is empirically determined by the optical layout. A too small RoI cannot make the most use of laser beams, while a too large one leads to much severer rank deficiency in CST. To solve the above-mentioned issues, we introduce hybrid-size meshing, for the first time, by reforming the density of the pixels in the entire sensing region of CST. This development alleviates the ill-posedness of the CST inverse problem by detailing the target flow field with dense pixels in the RoI and fully considering the complete physical absorption model with sparse pixels out of the RoI. The proposed scheme was both numerically and experimentally validated using a CST sensor with 32 laser beams using a variety of computational tomographic algorithms. The images reconstructed using the hybrid-size meshing scheme show better accuracy and finer profile of the target flow, compared with those reconstructed using the traditionally uniform-size meshing. The proposed hybrid-size meshing scheme significantly facilitates the industrial application of CST towards practical combustors, in which the combustion zone is bypassed by cooling air. In these scenarios, the proposed scheme can better characterise the combustion zone with dense meshes, while maintaining the integrity of the physical model by considering the absorption in the bypass air with sparse meshes.
A High-Dynamic-Range Digital RF-Over-Fiber Link for MRI Receive Coils Using Delta-Sigma Modulation
Authors: Mingdong Fan, Robert W. Brown, Xi Gao, Soumyajit Mandal, Labros Petropoulos, Xiaoyu Yang, Shinya Handa, Hiroyuki Fujita
Date: 2021-05-27 00:00:00

The coaxial cables commonly used to connect RF coil arrays with the control console of an MRI scanner are susceptible to electromagnetic coupling. As the number of RF channel increases, such coupling could result in severe heating and pose a safety concern. Non-conductive transmission solutions based on fiber-optic cables are considered to be one of the alternatives, but are limited by the high dynamic range ($>80$~dB) of typical MRI signals. A new digital fiber-optic transmission system based on delta-sigma modulation (DSM) is developed to address this problem. A DSM-based optical link is prototyped using off-the-shelf components and bench-tested at different signal oversampling rates (OSR). An end-to-end dynamic range (DR) of 81~dB, which is sufficient for typical MRI signals, is obtained over a bandwidth of 200~kHz, which corresponds to $OSR=50$. A fully-integrated custom fourth-order continuous-time DSM (CT-DSM) is designed in 180~nm CMOS technology to enable transmission of full-bandwidth MRI signals (up to 1~MHz) with adequate DR. Initial electrical test results from this custom chip are also presented.