pwc-archive/papers-with-abstracts huggingface dataset:
Over-the-Air Federated Learning Over MIMO Channels: A Sparse-Coded Multiplexing Approach
Authors: Chenxi Zhong, Xiaojun Yuan
Date: 2023-04-10 00:00:00
Tasks: Federated Learning

The communication bottleneck of over-the-air federated learning (OA-FL) lies in uploading the gradients of local learning models. In this paper, we study the reduction of the communication overhead in the gradients uploading by using the multiple-input multiple-output (MIMO) technique. We propose a novel sparse-coded multiplexing (SCoM) approach that employs sparse-coding compression and MIMO multiplexing to balance the communication overhead and the learning performance of the FL model. We derive an upper bound on the learning performance loss of the SCoM-based MIMO OA-FL scheme by quantitatively characterizing the gradient aggregation error. Based on the analysis results, we show that the optimal number of multiplexed data streams to minimize the upper bound on the FL learning performance loss is given by the minimum of the numbers of transmit and receive antennas. We then formulate an optimization problem for the design of precoding and post-processing matrices to minimize the gradient aggregation error. To solve this problem, we develop a low-complexity algorithm based on alternating optimization (AO) and alternating direction method of multipliers (ADMM), which effectively mitigates the impact of the gradient aggregation error. Numerical results demonstrate the superb performance of the proposed SCoM approach.
Signal Variation Metrics and Graph Fourier Transforms for Directed Graphs
Authors: Laura Shimabukuro, Antonio Ortega
Date: 2023-04-10 00:00:00

In this paper we consider the problem of constructing graph Fourier transforms (GFTs) for directed graphs (digraphs), with a focus on developing multiple GFT designs that can capture different types of variation over the digraph node-domain. Specifically, for any given digraph we propose three GFT designs based on the polar decomposition. Our method is closely related to existing polar decomposition based GFT designs, but with added interpretability in the digraph node-domain. Each of our proposed digraph GFTs has a clear node domain variation interpretation, so that one or more of the GFTs can be used to extract different insights from available graph signals. We demonstrate the benefits of our approach experimentally using M-block cyclic graphs, showing that the diffusion of signals on the graph leads to changes in the spectrum obtained from our proposed GFTs, but cannot be observed with the conventional GFT definition.
From Data-driven Learning to Physics-inspired Inferring: A Novel Mobile MIMO Channel Prediction Scheme Based on Neural ODE
Authors: Zhuoran Xiao, Zhaoyang Zhang, Zirui Chen, Zhaohui Yang, Chongwen Huang, Xiaoming Chen
Date: 2023-04-09 00:00:00
Tasks: Prediction

In this paper, we propose an innovative learning-based channel prediction scheme so as to achieve higher prediction accuracy and reduce the requirements of huge amounts and strict sequential format of channel data. Inspired by the idea of the neural ordinary differential equation (Neural ODE), we first prove that the channel prediction problem can be modeled as an ODE problem with a known initial value by analyzing the physical process of electromagnetic wave propagation within a mobile environment. Then, we design a novel physics-inspired spatial channel gradient network (SCGnet), which represents the derivative process of channel varying as a special neural network and can obtain the gradients at any relative displacement needed for the ODE solving. With the SCGnet, the static channel at any location served by the base station is accurately inferred through consecutive propagation and integration. Finally, we design an efficient recurrent positioning algorithm based on some prior knowledge of user mobility to obtain the velocity vector and propose an approximate Doppler compensation method to make up the instantaneous angular-delay domain channel. Only discrete historical channel data is needed for the training, whereas only a few fresh channel measurements are needed for the prediction, which ensures the scheme's practicability. Comprehensive evaluations show that the proposed scheme is most efficient in representing, learning, and predicting mobile wireless channels.
Overview of processing techniques for surface electromyography signals
Authors: Alejandra Manjarres-Triana, Juan Acevedo-Serna, Andrés A. Ramírez-Duque, Mario F. Jiménez, Edith Pulido-Herrera, John J. Villarejo Mayor
Date: 2023-04-08 00:00:00

Surface electromyography (sEMG) is a technology to assess muscle activation, which is an important component in applications related to diagnosis, treatment, progression assessment, and rehabilitation of specific individuals' conditions. Recently, sEMG potential has been shown, since it can be used in a non-invasive manner; nevertheless, it requires careful signal analysis to support health professionals reliably. This paper briefly described the basic concepts involved in the sEMG, such as the physiology of the muscles, the data acquisition, the signal processing techniques, and classification methods that may be used to identify disorders or signs of abnormalities according to muscular patterns. Specifically, classification methods encompass digital signal processing techniques and machine learning with high potential in the field. We hope that this work serves as an introduction to researchers interested in this field.
Estimation of Ground NO2 Measurements from Sentinel-5P Tropospheric Data through Categorical Boosting
Authors: Francesco Mauro, Luigi Russo, Fjoralba Janku, Alessandro Sebastianelli, Silvia Liberata Ullo
Date: 2023-04-08 00:00:00

This study aims to analyse the Nitrogen Dioxide (NO2) pollution in the Emilia Romagna Region (Northern Italy) during 2019, with the help of satellite retrievals from the Sentinel-5P mission of the European Copernicus Programme and ground-based measurements, obtained from the ARPA site (Regional Agency for the Protection of the Environment). The final goal is the estimation of ground NO2 measurements when only satellite data are available. For this task, we used a Machine Learning (ML) model, Categorical Boosting, which was demonstrated to work quite well and allowed us to achieve a Root-Mean-Square Error (RM SE) of 0.0242 over the 43 stations utilised to get the Ground Truth values. This procedure represents the starting point to understand which other actions must be taken to improve the final performance of the model and extend its validity.
AI-Driven Resource Allocation in Optical Wireless Communication Systems
Authors: Abdelrahman S. Elgamal, Osama Z. Aletri, Barzan A. Yosuf, Ahmad Adnan Qidan, Taisir El-Gorashi, Jaafar M. H. Elmirghani
Date: 2023-04-08 00:00:00
Tasks: Management, Reinforcement Learning (RL)

Visible light communication (VLC) is a promising solution to satisfy the extreme demands of emerging applications. VLC offers bandwidth that is orders of magnitude higher than what is offered by the radio spectrum, hence making best use of the resources is not a trivial matter. There is a growing interest to make next generation communication networks intelligent using AI based tools to automate the resource management and adapt to variations in the network automatically as opposed to conventional handcrafted schemes based on mathematical models assuming prior knowledge of the network. In this article, a reinforcement learning (RL) scheme is developed to intelligently allocate resources of an optical wireless communication (OWC) system in a HetNet environment. The main goal is to maximise the total reward of the system which is the sum rate of all users. The results of the RL scheme are compared with that of an optimization scheme that is based on Mixed Integer Linear Programming (MILP) model.
Energy Efficient Laser-Based Optical Wireless Communication Networks
Authors: Walter Zibusiso Ncube, Ahmad Adnan Qidan, Taisir El-Gorashi, Jaafar M. H. Elmirghani
Date: 2023-04-07 00:00:00

Rising data demands are a growing concern globally. The task at hand is to evolve current communication networks to support enhanced data rates while maintaining low latency, energy consumption and costs. To meet the above challenge, Optical Wireless Communication (OWC) technology is proposed as a solution to complement traditional Radio Frequency (RF) based communication systems. Recently, Vertical Cavity Surface Emitting Lasers (VCSELS) have been considered for data transmission in OWC indoor environments due to their ability to transmit power through narrow, near-circular beams to receivers. In this paper, we study the energy efficiency of a VCSEL-based OWC system in an indoor environment and compare it to that of a Light Emitting Diode (LED) based system. The main findings show that the VCSEL system performs better and has higher energy efficiency.
A stochastic control perspective on term structure models with roll-over risk
Authors: Claudio Fontana, Simone Pavarana, Wolfgang J. Runggaldier
Date: 2023-04-10 00:00:00

In this paper, we consider a generic interest rate market in the presence of roll-over risk, which generates spreads in spot/forward term rates. We do not require classical absence of arbitrage and rely instead on a minimal market viability assumption, which enables us to work in the context of the benchmark approach. In a Markovian setting, we extend the control theoretic approach of Gombani & Runggaldier (2013) and derive representations of spot/forward spreads as value functions of suitable stochastic optimal control problems, formulated under the real-world probability and with power-type objective functionals. We determine endogenously the funding-liquidity spread by relating it to the risk-sensitive optimization problem of a representative investor.
TemPL: A Novel Deep Learning Model for Zero-Shot Prediction of Protein Stability and Activity Based on Temperature-Guided Language Modeling
Authors: Pan Tan, Mingchen Li, Liang Zhang, Zhiqiang Hu, Liang Hong
Date: 2023-04-07 00:00:00
Tasks: Language Modeling, Language Modelling

We introduce TemPL, a novel deep learning approach for zero-shot prediction of protein stability and activity, harnessing temperature-guided language modeling. By assembling an extensive dataset of 96 million sequence-host bacterial strain optimal growth temperatures (OGTs) and {\Delta}Tm data for point mutations under consistent experimental conditions, we effectively compared TemPL with state-of-the-art models. Notably, TemPL demonstrated superior performance in predicting protein stability. An ablation study was conducted to elucidate the influence of OGT prediction and language modeling modules on TemPL's performance, revealing the importance of integrating both components. Consequently, TemPL offers considerable promise for protein engineering applications, facilitating the design of mutation sequences with enhanced stability and activity.
Robust optimized certainty equivalents and quantiles for loss positions with distribution uncertainty
Authors: Weiwei Li, Dejian Tian
Date: 2023-04-10 00:00:00

The paper investigates the robust optimized certainty equivalents and analyzes the relevant properties of them as risk measures for loss positions with distribution uncertainty. On this basis, the robust generalized quantiles are proposed and discussed. The robust expectiles with two specific penalization functions $\varphi_{1}$ and $\varphi_{2}$ are further considered respectively. The robust expectiles with $\varphi_{1}$ are proved to be coherent risk measures, and the dual representation theorems are established. In addition, the effect of penalization functions on the robust expectiles and its comparison with expectiles are examined and simulated numerically.
Controllable Motion Synthesis and Reconstruction with Autoregressive Diffusion Models
Authors: Wenjie Yin, Ruibo Tu, Hang Yin, Danica Kragic, Hedvig Kjellström, Mårten Björkman
Date: 2023-04-03 00:00:00
Tasks: Decoder, Motion Synthesis

Data-driven and controllable human motion synthesis and prediction are active research areas with various applications in interactive media and social robotics. Challenges remain in these fields for generating diverse motions given past observations and dealing with imperfect poses. This paper introduces MoDiff, an autoregressive probabilistic diffusion model over motion sequences conditioned on control contexts of other modalities. Our model integrates a cross-modal Transformer encoder and a Transformer-based decoder, which are found effective in capturing temporal correlations in motion and control modalities. We also introduce a new data dropout method based on the diffusion forward process to provide richer data representations and robust generation. We demonstrate the superior performance of MoDiff in controllable motion synthesis for locomotion with respect to two baselines and show the benefits of diffusion data dropout for robust synthesis and reconstruction of high-fidelity motion close to recorded data.
Training Recurrent Neural Networks via Forward Propagation Through Time
Authors: Anil Kag, Venkatesh Saligrama
Date: 2021-07-18 00:00:00
Proceeding: International Conference on Machine Learning 2021 7

Back-propagation through time (BPTT) has been widely used for training Recurrent Neural Networks (RNNs). BPTT updates RNN parameters on an instance by back-propagating the error in time over the entire sequence length, and as a result, leads to poor trainability due to the well-known gradient explosion/decay phenomena. While a number of prior works have proposed to mitigate vanishing/explosion effect through careful RNN architecture design, these RNN variants still train with BPTT. We propose a novel forward-propagation algorithm, FPTT, where at each time, for an instance, we update RNN parameters by optimizing an instantaneous risk function. Our proposed risk is a regularization penalty at time t  that evolves dynamically based on previously observed losses, and allows for RNN parameter updates to converge to a stationary solution of the empirical RNN objective. We consider both sequence-to-sequence as well as terminal loss problems. Empirically FPTT outperforms BPTT on a number of well-known benchmark tasks, thus enabling architectures like LSTMs to solve long range dependencies problems.
Scaffolding a Student to Instill Knowledge
Authors: Anil Kag, Durmus Alp Emre Acar, Aditya Gangrade, Venkatesh Saligrama
Date: 2023-05-01 00:00:00
Proceeding: International Conference on Learning Representations 2023 5
Tasks: Knowledge Distillation

We propose a novel knowledge distillation (KD) method to selectively instill teacher knowledge into a student model motivated by situations where the student's capacity is significantly smaller than that of the teachers. In vanilla KD, the teacher primarily sets a predictive target for the student to follow, and we posit that this target is overly optimistic due to the student's lack of capacity. We develop a novel scaffolding scheme where the teacher, in addition to setting a predictive target, also scaffolds the student's prediction by censoring hard-to-learn examples. Scaffolding utilizes the same information as the teacher's soft-max predictions as inputs, and in this sense, our proposal can be viewed as a natural variant of vanilla KD. We show on synthetic examples that censoring hard-examples leads to smoothening the student's loss landscape so that the student encounters fewer local minima. As a result, it has good generalization properties. Against vanilla KD, we achieve improved performance and are comparable to more intrusive techniques that leverage feature matching on benchmark datasets.
Efficient Edge Inference by Selective Query
Authors: Anil Kag, Igor Fedorov, Aditya Gangrade, Paul Whatmough, Venkatesh Saligrama
Date: 2023-05-01 00:00:00
Proceeding: International Conference on Learning Representations 2023 5

Edge devices provide inference on predictive tasks to many end-users. However, deploying deep neural networks that achieve state-of-the-art accuracy on these devices is infeasible due to edge resource constraints. Nevertheless, cloud-only processing, the de-facto standard, is also problematic, since uploading large amounts of data imposes severe communication bottlenecks. We propose a novel end-to-end hybrid learning framework that allows the edge to selectively query only those hard examples that the cloud can classify correctly. Our framework optimizes over neural architectures and trains edge predictors and routing models so that the overall accuracy remains high while minimizing the overall latency. Training a hybrid learner is difficult since we lack annotations of hard edge-examples. We introduce a novel proxy supervision in this context and show that our method adapts seamlessly and near optimally across different latency regimes. On the ImageNet dataset, our proposed method deployed on a micro-controller unit exhibits 25%  reduction in latency compared to cloud-only processing while suffering no excess loss.
SRSD: Rethinking Datasets of Symbolic Regression for Scientific Discovery
Authors: Yoshitomo Matsubara, Naoya Chiba, Ryo Igarashi, Yoshitaka Ushiku
Date: 2022-12-02 00:00:00
Proceeding: NeurIPS 2022 AI for Science: Progress and Promises 2022 12
Tasks: regression, scientific discovery, Symbolic Regression, tabular-regression

Symbolic Regression (SR) is a task of recovering mathematical expressions from given data and has been attracting attention from the research community to discuss its potential for scientific discovery. However, the community lacks datasets of symbolic regression for scientific discovery (SRSD) to discuss the potential of SR. To address the critical issue, we revisit datasets of SRSD to discuss the potential of symbolic regression for scientific discovery. Focused on a set of formulas used in the existing datasets based on Feynman Lectures on Physics, we recreate 120 datasets to discuss the performance of SRSD. For each of the 120 SRSD datasets, we carefully review the properties of the formula and its variables to design reasonably realistic sampling ranges of values so that our new SRSD datasets can be used for evaluating the potential of SRSD such as whether or not an SR method can (re)discover physical laws from such datasets. We conduct experiments on our new SRSD datasets using five state-of-the-art SR methods in SRBench, and the results show that the new SRSD datasets are more challenging than the original ones. Our datasets and code repository are publicly available.
Equivariant Graph Neural Networks for Charged Particle Tracking
Authors: Daniel Murnane, Savannah Thais, Ameya Thete
Date: 2023-04-11 00:00:00

Graph neural networks (GNNs) have gained traction in high-energy physics (HEP) for their potential to improve accuracy and scalability. However, their resource-intensive nature and complex operations have motivated the development of symmetry-equivariant architectures. In this work, we introduce EuclidNet, a novel symmetry-equivariant GNN for charged particle tracking. EuclidNet leverages the graph representation of collision events and enforces rotational symmetry with respect to the detector's beamline axis, leading to a more efficient model. We benchmark EuclidNet against the state-of-the-art Interaction Network on the TrackML dataset, which simulates high-pileup conditions expected at the High-Luminosity Large Hadron Collider (HL-LHC). Our results show that EuclidNet achieves near-state-of-the-art performance at small model scales (<1000 parameters), outperforming the non-equivariant benchmarks. This study paves the way for future investigations into more resource-efficient GNN models for particle tracking in HEP experiments.
Astroformer: More Data Might not be all you need for Classification
Authors: Rishit Dagli
Date: 2023-04-03 00:00:00
Tasks: All, Astronomy, Data Augmentation, Image Classification, Medical Image Classification

Recent advancements in areas such as natural language processing and computer vision rely on intricate and massive models that have been trained using vast amounts of unlabelled or partly labeled data and training or deploying these state-of-the-art methods to resource constraint environments has been a challenge. Galaxy morphologies are crucial to understanding the processes by which galaxies form and evolve. Efficient methods to classify galaxy morphologies are required to extract physical information from modern-day astronomy surveys. In this paper, we introduce Astroformer, a method to learn from less amount of data. We propose using a hybrid transformer-convolutional architecture drawing much inspiration from the success of CoAtNet and MaxViT. Concretely, we use the transformer-convolutional hybrid with a new stack design for the network, a different way of creating a relative self-attention layer, and pair it with a careful selection of data augmentation and regularization techniques. Our approach sets a new state-of-the-art on predicting galaxy morphologies from images on the Galaxy10 DECals dataset, a science objective, which consists of 17736 labeled images achieving 94.86% top-$1$ accuracy, beating the current state-of-the-art for this task by 4.62%. Furthermore, this approach also sets a new state-of-the-art on CIFAR-100 and Tiny ImageNet. We also find that models and training methods used for larger datasets would often not work very well in the low-data regime.
A Comprehensive Study on Object Detection Techniques in Unconstrained Environments
Authors: Hrishitva Patel
Date: 2023-04-11 00:00:00
Tasks: Object, object-detection, Object Detection

Object detection is a crucial task in computer vision that aims to identify and localize objects in images or videos. The recent advancements in deep learning and Convolutional Neural Networks (CNNs) have significantly improved the performance of object detection techniques. This paper presents a comprehensive study of object detection techniques in unconstrained environments, including various challenges, datasets, and state-of-the-art approaches. Additionally, we present a comparative analysis of the methods and highlight their strengths and weaknesses. Finally, we provide some future research directions to further improve object detection in unconstrained environments.
Those Aren't Your Memories, They're Somebody Else's: Seeding Misinformation in Chat Bot Memories
Authors: Conor Atkins, Benjamin Zi Hao Zhao, Hassan Jameel Asghar, Ian Wood, Mohamed Ali Kaafar
Date: 2023-04-06 00:00:00
Tasks: Misinformation

One of the new developments in chit-chat bots is a long-term memory mechanism that remembers information from past conversations for increasing engagement and consistency of responses. The bot is designed to extract knowledge of personal nature from their conversation partner, e.g., stating preference for a particular color. In this paper, we show that this memory mechanism can result in unintended behavior. In particular, we found that one can combine a personal statement with an informative statement that would lead the bot to remember the informative statement alongside personal knowledge in its long term memory. This means that the bot can be tricked into remembering misinformation which it would regurgitate as statements of fact when recalling information relevant to the topic of conversation. We demonstrate this vulnerability on the BlenderBot 2 framework implemented on the ParlAI platform and provide examples on the more recent and significantly larger BlenderBot 3 model. We generate 150 examples of misinformation, of which 114 (76%) were remembered by BlenderBot 2 when combined with a personal statement. We further assessed the risk of this misinformation being recalled after intervening innocuous conversation and in response to multiple questions relevant to the injected memory. Our evaluation was performed on both the memory-only and the combination of memory and internet search modes of BlenderBot 2. From the combinations of these variables, we generated 12,890 conversations and analyzed recalled misinformation in the responses. We found that when the chat bot is questioned on the misinformation topic, it was 328% more likely to respond with the misinformation as fact when the misinformation was in the long-term memory.