pwc-archive/papers-with-abstracts huggingface dataset:
The TAP free energy for high-dimensional linear regression
Authors: Jiaze Qiu, Subhabrata Sen
Date: 2022-03-14 00:00:00
Tasks: regression, Vocal Bursts Intensity Prediction

We derive a variational representation for the log-normalizing constant of the posterior distribution in Bayesian linear regression with a uniform spherical prior and an i.i.d. Gaussian design. We work under the "proportional" asymptotic regime, where the number of observations and the number of features grow at a proportional rate. This rigorously establishes the Thouless-Anderson-Palmer (TAP) approximation arising from spin glass theory, and proves a conjecture of Krzakala et. al. (2014) in the special case of the spherical prior.
Accelerated Bayesian SED Modeling using Amortized Neural Posterior Estimation
Authors: ChangHoon Hahn, Peter Melchior
Date: 2022-03-14 00:00:00
Tasks: Bayesian Inference, CPU

State-of-the-art spectral energy distribution (SED) analyses use a Bayesian framework to infer the physical properties of galaxies from observed photometry or spectra. They require sampling from a high-dimensional space of SED model parameters and take $>10-100$ CPU hours per galaxy, which renders them practically infeasible for analyzing the $billions$ of galaxies that will be observed by upcoming galaxy surveys ($e.g.$ DESI, PFS, Rubin, Webb, and Roman). In this work, we present an alternative scalable approach to rigorous Bayesian inference using Amortized Neural Posterior Estimation (ANPE). ANPE is a simulation-based inference method that employs neural networks to estimate the posterior probability distribution over the full range of observations. Once trained, it requires no additional model evaluations to estimate the posterior. We present, and publicly release, ${\rm SED}{flow}$, an ANPE method to produce posteriors of the recent Hahn et al. (2022) SED model from optical photometry. ${\rm SED}{flow}$ takes ${\sim}1$ $second~per~galaxy$ to obtain the posterior distributions of 12 model parameters, all of which are in excellent agreement with traditional Markov Chain Monte Carlo sampling results. We also apply ${\rm SED}{flow}$ to 33,884 galaxies in the NASA-Sloan Atlas and publicly release their posteriors: see https://changhoonhahn.github.io/SEDflow.
Strategic Thinking for Sustainable Practice-Centered Computing
Authors: Myriam Lewkowicz
Date: 2022-03-15 00:00:00

This book was a trigger for me to reflect on all the projects we have conducted at the local, regional, national, and European levels around healthcare, addressing the social isolation of elderlies, the social support among caregivers, and the coordination issues of professionals willing to take care for patients at their home. Since Ina Wagner interviewed me in Paris, the situation evolved in France as a new healthcare law was voted that emphasizes the importance of cooperation among healthcare practitioners, the need for IT support, and the key role of local territories (which is not obvious in France, that is a very centralized country). The reflection that started with this book definitely helped me to make decisions in this evolving context, and I am grateful to Claudia M{\"u}ller who, through an interview, facilitated me to realize what are, from my experience, the main issues that have to be dealt with when aiming at ensuring a sustainable practice-based computing approach. In this epilogue, I briefly list and illustrate these main issues, hoping they could support other sustainable experiences.
On the Advances and Challenges of Adaptive Online Testing
Authors: Da Xu, Bo Yang
Date: 2022-03-15 00:00:00

In recent years, the interest in developing adaptive solutions for online testing has grown significantly in the industry. While the advances related to this relative new technology have been developed in multiple domains, it lacks in the literature a systematic and complete treatment of the procedure that involves exploration, inference, and analysis. This short paper aims to develop a comprehensive understanding of adaptive online testing, including various building blocks and analytical results. We also address the latest developments, research directions, and challenges that have been less mentioned in the literature.
Distributed-Memory Sparse Kernels for Machine Learning
Authors: Vivek Bharadwaj, Aydın Buluç, James Demmel
Date: 2022-03-15 00:00:00
Tasks: BIG-bench Machine Learning, Collaborative Filtering, Graph Embedding

Sampled Dense Times Dense Matrix Multiplication (SDDMM) and Sparse Times Dense Matrix Multiplication (SpMM) appear in diverse settings, such as collaborative filtering, document clustering, and graph embedding. Frequently, the SDDMM output becomes the input sparse matrix for a subsequent SpMM operation. Existing work has focused on shared memory parallelization of these primitives. While there has been extensive analysis of communication-minimizing distributed 1.5D algorithms for SpMM, no such analysis exists for SDDMM or the back-to-back sequence of SDDMM and SpMM, termed FusedMM. We show that distributed memory 1.5D and 2.5D algorithms for SpMM can be converted to algorithms for SDDMM with identical communication costs and input / output data layouts. Further, we give two communication-eliding strategies to reduce costs further for FusedMM kernels: either reusing the replication of an input dense matrix for the SDDMM and SpMM in sequence, or fusing the local SDDMM and SpMM kernels. We benchmark FusedMM algorithms on Cori, a Cray XC40 at LBNL, using Erdos-Renyi random matrices and large real-world sparse matrices. On 256 nodes with 68 cores each, 1.5D FusedMM algorithms using either communication eliding approach can save at least 30% of time spent exclusively in communication compared to executing a distributed-memory SpMM and SDDMM kernel in sequence. On real-world matrices with hundreds of millions of edges, all of our algorithms exhibit at least a 10x speedup over the SpMM algorithm in PETSc. On these matrices, our communication-eliding techniques exhibit runtimes up to 1.6 times faster than an unoptimized sequence of SDDMM and SpMM. We embed and test the scaling of our algorithms in real-world applications, including collaborative filtering via alternating-least-squares and inference for attention-based graph neural networks.
UofA-Truth at Factify 2022 : Transformer And Transfer Learning Based Multi-Modal Fact-Checking
Authors: Abhishek Dhankar, Osmar R. Zaïane, Francois Bolduc
Date: 2022-01-28 00:00:00
Tasks: Fact Checking, Misinformation, Transfer Learning

Identifying fake news is a very difficult task, especially when considering the multiple modes of conveying information through text, image, video and/or audio. We attempted to tackle the problem of automated misinformation/disinformation detection in multi-modal news sources (including text and images) through our simple, yet effective, approach in the FACTIFY shared task at De-Factify@AAAI2022. Our model produced an F1-weighted score of 74.807%, which was the fourth best out of all the submissions. In this paper we will explain our approach to undertake the shared task.
Balanced Truncation Model Reduction with A Priori Error Bounds for LTI Systems with Nonzero Initial Value
Authors: Christian Schröder, Matthias Voigt
Date: 2020-06-03 00:00:00

In standard balanced truncation model order reduction, the initial condition is typically ignored in the reduction procedure and is assumed to be zero instead. However, such a reduced-order model may be a bad approximation to the full-order system, if the initial condition is not zero. In the literature there are several attempts for modified reduction methods at the price of having no error bound or only a posteriori error bounds which are often too expensive to evaluate. In this work we propose a new balancing procedure that is based on a shift transformation on the state. We first derive a joint projection reduced-order model in which the part of the system depending only on the input and the one depending only on the initial value are reduced at once and we prove an a priori error bound. With this result at hand, we derive a separate projection procedure in which the two parts are reduced separately. This gives the freedom to choose different reduction orders for the different subsystems. Moreover, we discuss how the reduced-order models can be constructed in practice. Since the error bounds are parameter-dependent we show how they can be optimized efficiently. We conclude this paper by comparing our results with the ones from the literature by a series of numerical experiments.
Fast and Accurate Linear Fitting for Incompletely Sampled Gaussian Function With a Long Tail
Authors: Kai Wu, J. Andrew Zhang, Y. Jay Guo
Date: 2022-03-15 00:00:00

Fitting experiment data onto a curve is a common signal processing technique to extract data features and establish the relationship between variables. Often, we expect the curve to comply with some analytical function and then turn data fitting into estimating the unknown parameters of a function. Among analytical functions for data fitting, Gaussian function is the most widely used one due to its extensive applications in numerous science and engineering fields. To name just a few, Gaussian function is highly popular in statistical signal processing and analysis, thanks to the central limit theorem [1]; Gaussian function frequently appears in the quantum harmonic oscillator, quantum field theory, optics, lasers, and many other theories and models in Physics [2]; moreover, Gaussian function is widely applied in chemistry for depicting molecular orbitals, in computer science for imaging processing and in artificial intelligence for defining neural networks.
MSCET: A Multi-Scenario Offloading Schedule for Biomedical Data Processing and Analysis in Cloud-Edge-Terminal Collaborative Vehicular Networks
Authors: Zhichen Ni, Honglong Chen, Zhe Li, Xiaomeng Wang, Na Yan, Weifeng Liu, Feng Xia
Date: 2022-02-16 00:00:00
Tasks: Edge-computing

With the rapid development of Artificial Intelligence (AI) and Internet of Things (IoTs), an increasing number of computation intensive or delay sensitive biomedical data processing and analysis tasks are produced in vehicles, bringing more and more challenges to the biometric monitoring of drivers. Edge computing is a new paradigm to solve these challenges by offloading tasks from the resource-limited vehicles to Edge Servers (ESs) in Road Side Units (RSUs). However, most of the traditional offloading schedules for vehicular networks concentrate on the edge, while some tasks may be too complex for ESs to process. To this end, we consider a collaborative vehicular network in which the cloud, edge and terminal can cooperate with each other to accomplish the tasks. The vehicles can offload the computation intensive tasks to the cloud to save the resource of edge. We further construct the virtual resource pool which can integrate the resource of multiple ESs since some regions may be covered by multiple RSUs. In this paper, we propose a Multi-Scenario offloading schedule for biomedical data processing and analysis in Cloud-Edge-Terminal collaborative vehicular networks called MSCET. The parameters of the proposed MSCET are optimized to maximize the system utility. We also conduct extensive simulations to evaluate the proposed MSCET and the results illustrate that MSCET outperforms other existing schedules.
Competition-Level Code Generation with AlphaCode
Authors: Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d'Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James Molloy, Daniel J. Mankowitz, Esme Sutherland Robson, Pushmeet Kohli, Nando de Freitas, Koray Kavukcuoglu, Oriol Vinyals
Date: 2022-02-08 00:00:00
Proceeding: DeepMind 2022 2
Tasks: Code Generation

Programming is a powerful and ubiquitous problem-solving tool. Developing systems that can assist programmers or even generate programs independently could make programming more productive and accessible, yet so far incorporating innovations in AI has proven challenging. Recent large-scale language models have demonstrated an impressive ability to generate code, and are now able to complete simple programming tasks. However, these models still perform poorly when evaluated on more complex, unseen problems that require problem-solving skills beyond simply translating instructions into code. For example, competitive programming problems which require an understanding of algorithms and complex natural language remain extremely challenging. To address this gap, we introduce AlphaCode, a system for code generation that can create novel solutions to these problems that require deeper reasoning. In simulated evaluations on recent programming competitions on the Codeforces platform, AlphaCode achieved on average a ranking of top 54.3% in competitions with more than 5,000 participants. We found that three key components were critical to achieve good and reliable performance: (1) an extensive and clean competitive programming dataset for training and evaluation, (2) large and efficient-to-sample transformer-based architectures, and (3) large-scale model sampling to explore the search space, followed by filtering based on program behavior to a small set of submissions.
Text-free non-parallel many-to-many voice conversion using normalising flows
Authors: Thomas Merritt, Abdelhamid Ezzerg, Piotr Biliński, Magdalena Proszewska, Kamil Pokora, Roberto Barra-Chicote, Daniel Korzekwa
Date: 2022-03-15 00:00:00
Tasks: Normalising Flows, Speech Synthesis, text-to-speech, Text to Speech, Text-To-Speech Synthesis, Voice Conversion

Non-parallel voice conversion (VC) is typically achieved using lossy representations of the source speech. However, ensuring only speaker identity information is dropped whilst all other information from the source speech is retained is a large challenge. This is particularly challenging in the scenario where at inference-time we have no knowledge of the text being read, i.e., text-free VC. To mitigate this, we investigate information-preserving VC approaches. Normalising flows have gained attention for text-to-speech synthesis, however have been under-explored for VC. Flows utilize invertible functions to learn the likelihood of the data, thus provide a lossless encoding of speech. We investigate normalising flows for VC in both text-conditioned and text-free scenarios. Furthermore, for text-free VC we compare pre-trained and jointly-learnt priors. Flow-based VC evaluations show no degradation between text-free and text-conditioned VC, resulting in improvements over the state-of-the-art. Also, joint-training of the prior is found to negatively impact text-free VC quality.
Leveraging Unimodal Self-Supervised Learning for Multimodal Audio-Visual Speech Recognition
Authors: Xichen Pan, Peiyu Chen, Yichen Gong, Helong Zhou, Xinbing Wang, Zhouhan Lin
Date: 2022-02-24 00:00:00
Proceeding: ACL 2022 5
Tasks: Audio-Visual Speech Recognition, Automatic Speech Recognition (ASR), Language Modelling, Lipreading, Lip Reading, Self-Supervised Learning, Sentence, speech-recognition, Speech Recognition, Visual Speech Recognition

Training Transformer-based models demands a large amount of data, while obtaining aligned and labelled data in multimodality is rather cost-demanding, especially for audio-visual speech recognition (AVSR). Thus it makes a lot of sense to make use of unlabelled unimodal data. On the other side, although the effectiveness of large-scale self-supervised learning is well established in both audio and visual modalities, how to integrate those pre-trained models into a multimodal scenario remains underexplored. In this work, we successfully leverage unimodal self-supervised learning to promote the multimodal AVSR. In particular, audio and visual front-ends are trained on large-scale unimodal datasets, then we integrate components of both front-ends into a larger multimodal framework which learns to recognize parallel audio-visual data into characters through a combination of CTC and seq2seq decoding. We show that both components inherited from unimodal self-supervised learning cooperate well, resulting in that the multimodal framework yields competitive results through fine-tuning. Our model is experimentally validated on both word-level and sentence-level tasks. Especially, even without an external language model, our proposed model raises the state-of-the-art performances on the widely accepted Lip Reading Sentences 2 (LRS2) dataset by a large margin, with a relative improvement of 30%.
Joint Time-Vertex Fractional Fourier Transform
Authors: Tuna Alikaşifoğlu, Bünyamin Kartal, Eray Özgünay, Aykut Koç
Date: 2022-03-15 00:00:00
Tasks: Denoising, Time Series, Time Series Analysis

Graph signal processing (GSP) facilitates the analysis of high-dimensional data on non-Euclidean domains by utilizing graph signals defined on graph vertices. In addition to static data, each vertex can provide continuous time-series signals, transforming graph signals into time-series signals on each vertex. The joint time-vertex Fourier transform (JFT) framework offers spectral analysis capabilities to analyze these joint time-vertex signals. Analogous to the fractional Fourier transform (FRT) extending the ordinary Fourier transform (FT), we introduce the joint time-vertex fractional Fourier transform (JFRT) as a generalization of JFT. The JFRT enables fractional analysis for joint time-vertex processing by extending Fourier analysis to fractional orders in both temporal and vertex domains. We theoretically demonstrate that JFRT generalizes JFT and maintains properties such as index additivity, reversibility, reduction to identity, and unitarity for specific graph topologies. Additionally, we derive Tikhonov regularization-based denoising in the JFRT domain, ensuring robust and well-behaved solutions. Comprehensive numerical experiments on synthetic and real-world datasets highlight the effectiveness of JFRT in denoising and clustering tasks that outperform state-of-the-art approaches.
Distributed Pinning Set Stabilization of Large-Scale Boolean Networks
Authors: Shiyong Zhu, Jianquan Lu, Liangjie Sun, Jinde Cao
Date: 2022-03-15 00:00:00

In this article, we design the distributed pinning controllers to globally stabilize a Boolean network (BN), specially a sparsely connected large-scale one, towards a preassigned subset of state space through the node-to-node message exchange. Given an appointed state set, system nodes are partitioned into two disjoint parts, which respectively gather the nodes whose states are fixed or arbitrary with respect to the given state set. With such node division, three parts of pinned nodes are selected and the state feedback controllers are accordingly designed such that the resulting BN satisfies three conditions: the states of the other nodes cannot affect the nodal dynamics of fixed-state nodes, the subgraph of network structure induced by the fixed-state nodes is acyclic, and the steady state of the subnetwork induced by the fixed-state nodes lies in the state set given beforehand. If the BN after control is acyclic, the stabilizing time is revealed to be no more than the length of the longest path in the current network structure plus one. This enables us to further design the pinning controllers with the constraint of stabilizing time. Noting that the overall procedure runs in an exponentially increasing time with respect to the largest number of functional variables in the dynamics of pinned nodes, the sparsely-connected large-scale BNs can be well addressed in a reasonable amount of time. Finally, we demonstrate the applications of our theoretical results in a T-LGL survival signal network with $29$ nodes and T-cell receptor signaling network with $90$ nodes.
Optimal Utilization Strategy of the LiFePO$_4$ Battery Storage
Authors: T. Sayfutdinov, P. Vorobev
Date: 2021-01-27 00:00:00
Tasks: Scheduling

The paper provides a comprehensive battery storage modelling approach, which accounts for operation- and degradation-aware characteristics, i.e., variable efficiency, internal resistance growth, and capacity fade. Based on the available experimental data from the literature, we build mixed-integer linear programming compatible lithium iron phosphate (LiFePO$_4$) battery model that can be used in problems related to various applications, i.e., power system, smart grid, and vehicular applications. Such formulation allows finding the globally optimal solution using off-the-shelf academic and commercial solvers. In the numerical study, the proposed modelling approach has been applied to realistic scenarios of peak-shaving, where the importance of considering the developed models is explicitly demonstrated. For instance, a time-varying operation strategy is required to obtain the optimal utilization of the LiFePO$_4$ battery storage. Particularly, during the battery operational lifetime its optimal average SoC may change by up to $20\%$, while the duration of charging process may increase by $75\%$. Finally, using the same LiFePO$_4$ benchmark model from the literature, we compare the results of using the proposed approach to the state-of-the-art in the optimal sizing and scheduling problems. The proposed approach led to a $12.1\%$ reduction of battery investment and operating costs compared to the state-of-the-art method.
Efficient Training of the Memristive Deep Belief Net Immune to Non-Idealities of the Synaptic Devices
Authors: Wei Wang, Barak Hoffer, Tzofnat Greenberg-Toledo, Yang Li, Minhui Zou, Eric Herbelin, Ronny Ronen, Xiaoxin Xu, Yulin Zhao, Jianguo Yang, Shahar Kvatinsky
Date: 2022-03-15 00:00:00

The tunability of conductance states of various emerging non-volatile memristive devices emulates the plasticity of biological synapses, making it promising in the hardware realization of large-scale neuromorphic systems. The inference of the neural network can be greatly accelerated by the vector-matrix multiplication (VMM) performed within a crossbar array of memristive devices in one step. Nevertheless, the implementation of the VMM needs complex peripheral circuits and the complexity further increases since non-idealities of memristive devices prevent precise conductance tuning (especially for the online training) and largely degrade the performance of the deep neural networks (DNNs). Here, we present an efficient online training method of the memristive deep belief net (DBN). The proposed memristive DBN uses stochastically binarized activations, reducing the complexity of peripheral circuits, and uses the contrastive divergence (CD) based gradient descent learning algorithm. The analog VMM and digital CD are performed separately in a mixed-signal hardware arrangement, making the memristive DBN high immune to non-idealities of synaptic devices. The number of write operations on memristive devices is reduced by two orders of magnitude. The recognition accuracy of 95%~97% can be achieved for the MNIST dataset using pulsed synaptic behaviors of various memristive synaptic devices.
Optimal selection and tracking of generalized Nash equilibria in monotone games
Authors: Emilio Benenati, Wicak Ananduta, Sergio Grammatico
Date: 2022-03-15 00:00:00

A fundamental open problem in monotone game theory is the computation of a specific generalized Nash equilibrium (GNE) among all the available ones, e.g. the optimal equilibrium with respect to a system-level objective. The existing GNE seeking algorithms have in fact convergence guarantees toward an arbitrary, possibly inefficient, equilibrium. In this paper, we solve this open problem by leveraging results from fixed-point selection theory and in turn derive distributed algorithms for the computation of an optimal GNE in monotone games. We then extend the technical results to the time-varying setting and propose an algorithm that tracks the sequence of optimal equilibria up to an asymptotic error, whose bound depends on the local computational capabilities of the agents.