pwc-archive/papers-with-abstracts huggingface dataset:
Using Shallow Neural Networks with Functional Connectivity from EEG signals for Early Diagnosis of Alzheimer's and Frontotemporal Dementia
Authors: Zaineb Ajra, Binbin Xu, Gérard Dray, Jacky Montmain, Stéphane Perrey
Date: 2023-11-06 00:00:00
Tasks: EEG, Functional Connectivity

{Introduction: } Dementia is a neurological disorder associated with aging that can cause a loss of cognitive functions, impacting daily life. Alzheimer's disease (AD) is the most common cause of dementia, accounting for 50--70\% of cases, while frontotemporal dementia (FTD) affects social skills and personality. Electroencephalography (EEG) provides an effective tool to study the effects of AD on the brain. {Methods: } In this study, we propose to use shallow neural networks applied to two sets of features: spectral-temporal and functional connectivity using four methods. We compare three supervised machine learning techniques to the CNN models to classify EEG signals of AD / FTD and control cases. We also evaluate different measures of functional connectivity from common EEG frequency bands considering multiple thresholds. {Results and Discussion: } Results showed that the shallow CNN-based models achieved the highest accuracy of 94.54\% with AEC in test dataset when considering all connections, outperforming conventional methods and providing potentially an additional early dementia diagnosis tool. \url{https://doi.org/10.3389%2Ffneur.2023.1270405}
Energy Harvesting Maximization for Reconfigurable Intelligent Surfaces Using Amplitude Measurements
Authors: Morteza Tavana, Meysam Masoudi, Emil Björnson
Date: 2023-11-06 00:00:00

Energy harvesting can enable a reconfigurable intelligent surface (RIS) to self-sustain its operations without relying on external power sources. In this paper, we consider the problem of energy harvesting for RISs in the absence of coordination with the ambient RF source. We propose a series of sequential phase-alignment algorithms that maximize the received power based on only power measurements. We prove the convergence of the proposed algorithm to the optimal value for the noiseless scenario. However, for the noisy scenario, we propose a linear least squares estimator. We prove that within the class of linear estimators, the optimal set of measurement phases are equally-spaced phases. To evaluate the performance of the proposed method, we introduce a random phase update algorithm as a benchmark. Our simulation results show that the proposed algorithms outperform the random phase update method in terms of achieved power after convergence while requiring fewer measurements per phase update. Using simulations, we show that in a noiseless scenario with a discrete set of possible phase shifts for the RIS elements, the proposed method is sub-optimal, achieving a higher value than the random algorithm but not exactly the maximum feasible value that we obtained by exhaustive search.
Optimization of RIS Placement for Satellite-to-Ground Coverage Enhancement
Authors: Xingchen Liu, Liuxun Xue, Shu Sun, Meixia Tao
Date: 2023-11-06 00:00:00

In satellite-to-ground communication, ensuring reliable and efficient connectivity poses significant challenges. The reconfigurable intelligent surface (RIS) offers a promising solution due to its ability to manipulate wireless propagation environments and thus enhance communication performance. In this paper, we propose a method for optimizing the placement of RISs on building facets to improve satellite-to-ground communication coverage. We model satellite-to-ground communication with RIS assistance, considering the actual positions of buildings and ground users. The theoretical lower bound on the coverage enhancement in satellite-to-ground communication through large-scale RIS deployment is derived. Then a novel optimization framework for RIS placement is formulated, and a parallel genetic algorithm is employed to solve the problem. Simulation results demonstrate the superior performance of the proposed RIS deployment strategy in enhancing satellite communication coverage probability for non-line-of-sight users. The proposed framework can be applied to various architectural distributions, such as rural areas, towns, and cities, by adjusting parameter settings.
Perturbed examples reveal invariances shared by language models
Authors: Ruchit Rawal, Mariya Toneva
Date: 2023-11-07 00:00:00

The rapid growth in natural language processing (NLP) research has led to numerous new models, outpacing our understanding of how they compare to established ones. One major reason for this difficulty is saturating benchmarks, which may not well reflect differences in model performance in the wild. In this work, we introduce a novel framework to compare two NLP models by revealing their shared invariance to interpretable input perturbations targeting a specific linguistic capability. Via experiments on models from the same and different architecture families, this framework offers insights about how changes in models (e.g., distillation, size increase) affect linguistic capabilities. Furthermore, our framework enables evaluation of invariances between commercial black-box models (e.g., InstructGPT family) and models that are better understood (e.g., GPT-2). Across experiments, we observe that large language models share many invariances encoded by models of various sizes, whereas the invariances by large models are only shared by other large models. Possessing a wide variety of invariances may be key to the recent successes of large language models, and our framework can shed light on the types of invariances retained or emerging in new models. We make the code publicly available.
Channel Estimation and Training Design for Active RIS Aided Wireless Communications
Authors: Hao Chen, Nanxi Li, Ruizhe Long, Ying-Chang Liang
Date: 2023-11-06 00:00:00

Active reconfigurable intelligent surface (ARIS) is a newly emerging RIS technique that leverages radio frequency (RF) reflection amplifiers to empower phase-configurable reflection elements (REs) in amplifying the incident signal. Thereby, ARIS can enhance wireless communications with the strengthened ARIS-aided links. In this letter, we propose exploiting the signal amplification capability of ARIS for channel estimation, aiming to improve the estimation precision. Nevertheless, the signal amplification inevitably introduces the thermal noise at the ARIS, which can hinder the acquisition of accurate channel state information (CSI) with conventional channel estimation methods based on passive RIS (PRIS). To address this issue, we further investigate this ARIS-specific channel estimation problem and propose a least-square (LS) based channel estimator, whose performance can be further improved with the design on ARIS reflection patterns at the channel training phase. Based on the proposed LS channel estimator, we optimize the training reflection patterns to minimize the channel estimation error variance. Extensive simulation results show that our proposed design can achieve accurate channel estimation in the presence of the ARIS noises.
Breaking the Heavy-Tailed Noise Barrier in Stochastic Optimization Problems
Authors: Nikita Puchkin, Eduard Gorbunov, Nikolay Kutuzov, Alexander Gasnikov
Date: 2023-11-07 00:00:00
Tasks: Stochastic Optimization

We consider stochastic optimization problems with heavy-tailed noise with structured density. For such problems, we show that it is possible to get faster rates of convergence than $\mathcal{O}(K^{-2(\alpha - 1)/\alpha})$, when the stochastic gradients have finite moments of order $\alpha \in (1, 2]$. In particular, our analysis allows the noise norm to have an unbounded expectation. To achieve these results, we stabilize stochastic gradients, using smoothed medians of means. We prove that the resulting estimates have negligible bias and controllable variance. This allows us to carefully incorporate them into clipped-SGD and clipped-SSTM and derive new high-probability complexity bounds in the considered setup.
Computing Approximate $\ell_p$ Sensitivities
Date: 2023-11-07 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Dimensionality Reduction, regression, Sensitivity

Recent works in dimensionality reduction for regression tasks have introduced the notion of sensitivity, an estimate of the importance of a specific datapoint in a dataset, offering provable guarantees on the quality of the approximation after removing low-sensitivity datapoints via subsampling. However, fast algorithms for approximating $\ell_p$ sensitivities, which we show is equivalent to approximate $\ell_p$ regression, are known for only the $\ell_2$ setting, in which they are termed leverage scores. In this work, we provide efficient algorithms for approximating $\ell_p$ sensitivities and related summary statistics of a given matrix. In particular, for a given $n \times d$ matrix, we compute $\alpha$-approximation to its $\ell_1$ sensitivities at the cost of $O(n/\alpha)$ sensitivity computations. For estimating the total $\ell_p$ sensitivity (i.e. the sum of $\ell_p$ sensitivities), we provide an algorithm based on importance sampling of $\ell_p$ Lewis weights, which computes a constant factor approximation to the total sensitivity at the cost of roughly $O(\sqrt{d})$ sensitivity computations. Furthermore, we estimate the maximum $\ell_1$ sensitivity, up to a $\sqrt{d}$ factor, using $O(d)$ sensitivity computations. We generalize all these results to $\ell_p$ norms for $p > 1$. Lastly, we experimentally show that for a wide class of matrices in real-world datasets, the total sensitivity can be quickly approximated and is significantly smaller than the theoretical prediction, demonstrating that real-world datasets have low intrinsic effective dimensionality.
An Open Dataset Storage Standard for 6G Testbeds
Authors: Gilles Callebaut, Michiel Sandra, Christian Nelson, Thomas Wilding, Daan Delabie, Benjamin J. B. Deutschmann, William Tärneberg, Emma Fitzgerald, Anders J. Johansson, Liesbet Van der Perre
Date: 2023-11-05 00:00:00

The emergence of sixth-generation (6G) networks has spurred the development of novel testbeds, including sub-THz networks, cell-free systems, and 6G simulators. To maximize the benefits of these systems, it is crucial to make the generated data publicly available and easily reusable by others. Although data sharing has become a common practice, a lack of standardization hinders data accessibility and interoperability. In this study, we propose the Dataset Storage Standard (DSS) to address these challenges by facilitating data exchange and enabling convenient processing script creation in a testbed-agnostic manner. DSS supports both experimental and simulated data, allowing researchers to employ the same processing scripts and tools across different datasets. Unlike existing standardization efforts such as SigMF and NI RF Data Recording API, DSS provides a broader scope by accommodating a common definition file for testbeds and is not limited to RF data storage. The dataset format utilizes a hierarchical structure, with a tensor representation for specific experiment scenarios. In summary, DSS offers a comprehensive and flexible framework for enhancing the FAIR principles (Findability, Accessibility, Interoperability, and Reusability) in 6G testbeds, promoting open and efficient data sharing in the research community.
Kernel-, mean- and noise-marginalised Gaussian processes for exoplanet transits and $H_0$ inference
Authors: Namu Kroupa, David Yallup, Will Handley, Michael Hobson
Date: 2023-11-07 00:00:00
Tasks: Gaussian Processes

Using a fully Bayesian approach, Gaussian Process regression is extended to include marginalisation over the kernel choice and kernel hyperparameters. In addition, Bayesian model comparison via the evidence enables direct kernel comparison. The calculation of the joint posterior was implemented with a transdimensional sampler which simultaneously samples over the discrete kernel choice and their hyperparameters by embedding these in a higher-dimensional space, from which samples are taken using nested sampling. Kernel recovery and mean function inference were explored on synthetic data from exoplanet transit light curve simulations. Subsequently, the method was extended to marginalisation over mean functions and noise models and applied to the inference of the present-day Hubble parameter, $H_0$, from real measurements of the Hubble parameter as a function of redshift, derived from the cosmologically model-independent cosmic chronometer and $\Lambda$CDM-dependent baryon acoustic oscillation observations. The inferred $H_0$ values from the cosmic chronometers, baryon acoustic oscillations and combined datasets are $H_0= 66 \pm 6\, \mathrm{km}\,\mathrm{s}^{-1}\,\mathrm{Mpc}^{-1}$, $H_0= 67 \pm 10\, \mathrm{km}\,\mathrm{s}^{-1}\,\mathrm{Mpc}^{-1}$ and $H_0= 69 \pm 6\, \mathrm{km}\,\mathrm{s}^{-1}\,\mathrm{Mpc}^{-1}$, respectively. The kernel posterior of the cosmic chronometers dataset prefers a non-stationary linear kernel. Finally, the datasets are shown to be not in tension with $\ln R=12.17\pm 0.02$.
HyperS2V: A Framework for Structural Representation of Nodes in Hyper Networks
Authors: Shu Liu, Cameron Lai, Fujio Toriumi
Date: 2023-11-07 00:00:00

In contrast to regular (simple) networks, hyper networks possess the ability to depict more complex relationships among nodes and store extensive information. Such networks are commonly found in real-world applications, such as in social interactions. Learning embedded representations for nodes involves a process that translates network structures into more simplified spaces, thereby enabling the application of machine learning approaches designed for vector data to be extended to network data. Nevertheless, there remains a need to delve into methods for learning embedded representations that prioritize structural aspects. This research introduces HyperS2V, a node embedding approach that centers on the structural similarity within hyper networks. Initially, we establish the concept of hyper-degrees to capture the structural properties of nodes within hyper networks. Subsequently, a novel function is formulated to measure the structural similarity between different hyper-degree values. Lastly, we generate structural embeddings utilizing a multi-scale random walk framework. Moreover, a series of experiments, both intrinsic and extrinsic, are performed on both toy and real networks. The results underscore the superior performance of HyperS2V in terms of both interpretability and applicability to downstream tasks.
Multi-resolution Time-Series Transformer for Long-term Forecasting
Authors: Yitian Zhang, Liheng Ma, Soumyasundar Pal, Yingxue Zhang, Mark Coates
Date: 2023-11-07 00:00:00
Tasks: Time Series, Time Series Forecasting

The performance of transformers for time-series forecasting has improved significantly. Recent architectures learn complex temporal patterns by segmenting a time-series into patches and using the patches as tokens. The patch size controls the ability of transformers to learn the temporal patterns at different frequencies: shorter patches are effective for learning localized, high-frequency patterns, whereas mining long-term seasonalities and trends requires longer patches. Inspired by this observation, we propose a novel framework, Multi-resolution Time-Series Transformer (MTST), which consists of a multi-branch architecture for simultaneous modeling of diverse temporal patterns at different resolutions. In contrast to many existing time-series transformers, we employ relative positional encoding, which is better suited for extracting periodic components at different scales. Extensive experiments on several real-world datasets demonstrate the effectiveness of MTST in comparison to state-of-the-art forecasting techniques.
Exploiting Hybrid Terrestrial/LEO Satellite Systems for Rural Connectivity
Authors: Houcem Ben Salem, Nour Kouzayha, Ammar El Falou, Mohamed-Slim Alouini, Tareq Y. Al-Naffouri
Date: 2023-11-05 00:00:00

Satellite networks are playing an important role in realizing global seamless connectivity in beyond 5G and 6G wireless networks. In this paper, we develop a comprehensive analytical framework to assess the performance of hybrid terrestrial/satellite networks in providing rural connectivity. We assume that the terrestrial base stations are equipped with multiple-input-multiple-output (MIMO) technologies and that the user has the option to associate with a base station or a satellite to be served. Using tools from stochastic geometry, we derive tractable expressions for the coverage probability and average data rate and prove the accuracy of the derived expressions through Monte Carlo simulations. The obtained results capture the impact of the satellite constellation size, the terrestrial base station density, and the MIMO configuration parameters.
Do Language Models Learn Semantics of Code? A Case Study in Vulnerability Detection
Authors: Benjamin Steenhoek, Md Mahbubur Rahman, Shaila Sharmin, Wei Le
Date: 2023-11-07 00:00:00
Tasks: Vulnerability Detection

Recently, pretrained language models have shown state-of-the-art performance on the vulnerability detection task. These models are pretrained on a large corpus of source code, then fine-tuned on a smaller supervised vulnerability dataset. Due to the different training objectives and the performance of the models, it is interesting to consider whether the models have learned the semantics of code relevant to vulnerability detection, namely bug semantics, and if so, how the alignment to bug semantics relates to model performance. In this paper, we analyze the models using three distinct methods: interpretability tools, attention analysis, and interaction matrix analysis. We compare the models' influential feature sets with the bug semantic features which define the causes of bugs, including buggy paths and Potentially Vulnerable Statements (PVS). We find that (1) better-performing models also aligned better with PVS, (2) the models failed to align strongly to PVS, and (3) the models failed to align at all to buggy paths. Based on our analysis, we developed two annotation methods which highlight the bug semantics inside the model's inputs. We evaluated our approach on four distinct transformer models and four vulnerability datasets and found that our annotations improved the models' performance in the majority of settings - 11 out of 16, with up to 9.57 points improvement in F1 score compared to conventional fine-tuning. We further found that with our annotations, the models aligned up to 232% better to potentially vulnerable statements. Our findings indicate that it is helpful to provide the model with information of the bug semantics, that the model can attend to it, and motivate future work in learning more complex path-based bug semantics. Our code and data are available at https://figshare.com/s/4a16a528d6874aad51a0.
PIPO-Net: A Penalty-based Independent Parameters Optimization Deep Unfolding Network
Authors: Xiumei Li, Zhijie Zhang, Huang Bai, Ljubiša Stanković, Junpeng Hao, Junmei Sun
Date: 2023-11-04 00:00:00
Tasks: Compressive Sensing

Compressive sensing (CS) has been widely applied in signal and image processing fields. Traditional CS reconstruction algorithms have a complete theoretical foundation but suffer from the high computational complexity, while fashionable deep network-based methods can achieve high-accuracy reconstruction of CS but are short of interpretability. These facts motivate us to develop a deep unfolding network named the penalty-based independent parameters optimization network (PIPO-Net) to combine the merits of the above mentioned two kinds of CS methods. Each module of PIPO-Net can be viewed separately as an optimization problem with respective penalty function. The main characteristic of PIPO-Net is that, in each round of training, the learnable parameters in one module are updated independently from those of other modules. This makes the network more flexible to find the optimal solutions of the corresponding problems. Moreover, the mean-subtraction sampling and the high-frequency complementary blocks are developed to improve the performance of PIPO-Net. Experiments on reconstructing CS images demonstrate the effectiveness of the proposed PIPO-Net.
Time-Efficient Reinforcement Learning with Stochastic Stateful Policies
Authors: Firas Al-Hafez, Guoping Zhao, Jan Peters, Davide Tateo
Date: 2023-11-07 00:00:00
Tasks: continuous-control, Continuous Control, Imitation Learning, Inductive Bias, reinforcement-learning, Reinforcement Learning

Stateful policies play an important role in reinforcement learning, such as handling partially observable environments, enhancing robustness, or imposing an inductive bias directly into the policy structure. The conventional method for training stateful policies is Backpropagation Through Time (BPTT), which comes with significant drawbacks, such as slow training due to sequential gradient propagation and the occurrence of vanishing or exploding gradients. The gradient is often truncated to address these issues, resulting in a biased policy update. We present a novel approach for training stateful policies by decomposing the latter into a stochastic internal state kernel and a stateless policy, jointly optimized by following the stateful policy gradient. We introduce different versions of the stateful policy gradient theorem, enabling us to easily instantiate stateful variants of popular reinforcement learning and imitation learning algorithms. Furthermore, we provide a theoretical analysis of our new gradient estimator and compare it with BPTT. We evaluate our approach on complex continuous control tasks, e.g., humanoid locomotion, and demonstrate that our gradient estimator scales effectively with task complexity while offering a faster and simpler alternative to BPTT.
KPI Extraction from Maintenance Work Orders -- A Comparison of Expert Labeling, Text Classification and AI-Assisted Tagging for Computing Failure Rates of Wind Turbines
Authors: Marc-Alexander Lutz, Bastian Schäfermeier, Rachael Sexton, Michael Sharp, Alden Dima, Stefan Faulstich, Jagan Mohini Aluri
Date: 2023-11-07 00:00:00
Tasks: TAG, text-classification, Text Classification

Maintenance work orders are commonly used to document information about wind turbine operation and maintenance. This includes details about proactive and reactive wind turbine downtimes, such as preventative and corrective maintenance. However, the information contained in maintenance work orders is often unstructured and difficult to analyze, presenting challenges for decision-makers wishing to use it for optimizing operation and maintenance. To address this issue, this work compares three different approaches to calculate reliability by performance indicators from maintenance work orders. The first approach involves manual labeling of the maintenance work orders by domain experts, using the schema defined in an industrial guideline to assign the label accordingly. The second approach involves the development of a model that automatically labels the maintenance work orders using text classification methods. Through this method, we are able to achieve macro average and weighted average F1-Scores of 0.75 and 0.85 respectively. The third technique uses an AI-assisted tagging tool to tag and structure the raw maintenance information, together with a novel rule-based approach for extracting relevant maintenance work orders for failure rate calculation. In our experiments the AI-assisted tool leads to a 88% drop in tagging time in comparison to the other two approaches, while expert labeling and text classification are more accurate in KPI extraction. Overall, our findings make extracting maintenance information from maintenance work orders more efficient, enable the assessment of reliability key performance indicators and therefore support the optimization of wind turbine operation and maintenance.