pwc-archive/papers-with-abstracts huggingface dataset:
Enhanced graph-learning schemes driven by similar distributions of motifs
Authors: Samuel Rey, T. Mitchell Roddenberry, Santiago Segarra, Antonio G. Marques
Date: 2022-07-11 00:00:00
Tasks: Graph Learning, Inference Optimization

This paper looks at the task of network topology inference, where the goal is to learn an unknown graph from nodal observations. One of the novelties of the approach put forth is the consideration of prior information about the density of motifs of the unknown graph to enhance the inference of classical Gaussian graphical models. Dealing with the density of motifs directly constitutes a challenging combinatorial task. However, we note that if two graphs have similar motif densities, one can show that the expected value of a polynomial applied to their empirical spectral distributions will be similar. Guided by this, we first assume that we have a reference graph that is related to the sought graph (in the sense of having similar motif densities) and then, we exploit this relation by incorporating a similarity constraint and a regularization term in the network topology inference optimization problem. The (non-)convexity of the optimization problem is discussed and a computational efficient alternating majorization-minimization algorithm is designed. We assess the performance of the proposed method through exhaustive numerical experiments where different constraints are considered and compared against popular baselines algorithms on both synthetic and real-world datasets.
Novel Transceiver Design in Wideband Massive MIMO for Beam Squint Minimization
Authors: Liza Afeef, Abuu B. Kihero, Hüseyin Arslan
Date: 2022-07-11 00:00:00

When using ultra-wideband (UWB) signaling on massive multiple-input multiple-output (mMIMO) systems, the electromagnetic wave at each array element incurs an extra propagation delay comparable to (or larger than) the symbol duration, producing a shift in beam direction known as beam squint. The beam squinting problem degrades the array gain and reduces the system capacity. This letter proposes a novel transceiver design based on lens antenna subarray (LAS) and analog subband filters to compensate for the beam squinting effect. In particular, the proposed design aims to divide the UWB signal into narrowband beams and control them with a simplified exhaustive search-based precoding that is proposed to align the beam angle to the target direction. The design is analyzed in terms of beam gain, complexity, power consumption, and capacity, demonstrating significant performance enhancement with respect to the conventional system with uncompensated beam squinting problem.
Frequency-Angle Two-Dimensional Reflection Coefficient Modeling Based on Terahertz Channel Measurement
Authors: Zhaowei Chang, Jianhua Zhang, Pan Tang, Lei Tian, Li Yu, Guangyi Liu, Liang Xia
Date: 2022-07-11 00:00:00

Terahertz (THz) channel propagation characteristics are vital for the design, evaluation, and optimization for THz communication systems. Moreover, reflection plays a significant role in channel propagation. In this letter, the reflection coefficient of the THz channel is researched based on extensive measurement campaigns. Firstly, we set up the THz channel sounder from 220 to 320 GHz with the incident angle ranging from 10{\deg} to 80{\deg}. Based on the measured propagation loss, the reflection coefficients of five building materials, i.e., glass, tile, aluminium alloy, board, and plasterboard, are calculated separately for frequencies and incident angles. It is found that the lack of THz relative parameters leads to the Fresnel model of non-metallic materials can not fit the measured data well. Thus, we propose a frequency-angle two-dimensional reflection coefficient model by modifying the Fresnel model with the Lorenz and Drude model. The proposed model characterizes the frequency and incident angle for reflection coefficients and shows low root-mean-square error with the measured data. Generally, these results are useful for modeling THz channels.
A New Weighted Time Window-based Method to Detect B-point in Impedance Cardiogram
Authors: Nadica Miljković, Tomislav B. Šekara
Date: 2022-07-10 00:00:00

Background: Impedance Cardiogram (ICG) has impressive number of applications in healthcare. However, its wider adoption is excessively limited due to the well recognized challenges in ICG delineation. We present a simple, adaptive, and efficient method for the most demanding ICG delineation task - detection of less distinct B-point that marks the onset of the left ventricular ejection. Method: The core of the new method is transformation of ICG time series by weighted time window of an ICG segment preceding the maximal ICG peak (the C-point) aiming at the B-point enhancement. The resulting Modified B-point (MB-point) is then easily delineated. To evaluate the proposed workflow, the dataset comprising 20 healthy participants and 21065 B-points are manually annotated and openly shared with the software code. To the best of our knowledge, our ICG dataset has the largest number of annotated B-points. Detector performance is evaluated on two recorded segments with distinct noises, as well as on an open dataset comprising ICG recorded in another set of 24 healthy subjects. Results: The results showed that the method was superior to the competing methods when the tolerance for B-point detection was set to +/-150 ms in all cases and for both datasets (>99.4%). Conclusions: Proposed approach based on the weighted time windows presents a promising technique for reliable ICG delineation that is instrumental for appropriate feature extraction and ICG characterization. Further customization could incorporate labeling of other pathological ICG segments, other biomedical signals, as well as additional enhancements for improving robustness to noise.
Model-Driven Based Deep Unfolding Equalizer for Underwater Acoustic OFDM Communications
Authors: Hao Zhao, Cui Yang, Yalu Xu, Fei Ji, Miaowen Wen, Yankun Chen
Date: 2022-07-10 00:00:00

It is challenging to design an equalizer for the complex time-frequency doubly-selective channel. In this paper, we employ the deep unfolding approach to establish an equalizer for the underwater acoustic (UWA) orthogonal frequency division multiplexing (OFDM) system, namely UDNet. Each layer of UDNet is designed according to the classical minimum mean square error (MMSE) equalizer. Moreover, we consider the QPSK equalization as a four-classification task and adopt minimum Kullback-Leibler (KL) to achieve a smaller symbol error rate (SER) with the one-hot coding instead of the MMSE criterion. In addition, we introduce a sliding structure based on the banded approximation of the channel matrix to reduce the network size and aid UDNet to perform well for different-length signals without changing the network structure. Furthermore, we apply the measured at-sea doubly-selective UWA channel and offshore background noise to evaluate the proposed equalizer. Experimental results show that the proposed UDNet performs better with low computational complexity. Concretely, the SER of UDNet is nearly an order of magnitude lower than that of MMSE.
Matrix differentiation with diagrammatic notation
Authors: Kenji Nakahira
Date: 2022-07-10 00:00:00

We propose a diagrammatic notation for matrix differentiation. Our new notation enables us to derive formulas for matrix differentiation more easily than the usual matrix (or index) notation. We demonstrate the effectiveness of our notation through several examples.
Microwave Chirality Imaging for the Early Diagnosis of Neurological Degenerative Diseases
Authors: Wending Mai, Yifan Chen
Date: 2022-07-09 00:00:00

We propose a system to visualize the chirality of the protein in brains, which would be helpful to diagnose early neurological degenerative diseases in vivo. These neurological degenerative diseases often occur along with some mark proteins. By nanoparticle instilling and metamaterial technique, the chiral effect of the mark proteins is assumed to be manifest in microwave regime. Therefore, by detecting the transmission of cross-polarization, we could detect the chirality that rotates the microwave polarization angle. We developed a numerical method to simulate the electromagnetic response upon chiral (bi-isotropic) material. Then a numerical experiment was conduct with a numerical head phantom. A map of cross-polarized transmission magnitude can be reached by sweeping the antenna pair. The imaging results matches well with the distribution of chiral materials. It suggests that the proposed method would be capable of in vivo imaging of neurological degenerative disease using microwaves.
Schizophrenia detection based on EEG using Recurrent Auto-Encoder framework
Authors: Yihan Wu, Min Xia, Xiuzhu Wang, Yangsong Zhang
Date: 2022-07-09 00:00:00
Tasks: EEG, Electroencephalogram (EEG)

Schizophrenia (SZ) is a serious mental disorder that could seriously affect the patient's quality of life. In recent years, detection of SZ based on deep learning (DL) using electroencephalogram (EEG) has received increasing attention. In this paper, we proposed an end-to-end recurrent auto-encoder (RAE) model to detect SZ. In the RAE model, the raw data was input into one auto-encoder block, and the reconstructed data were recurrently input into the same block. The extracted code by auto-encoder block was simultaneously served as an input of a classifier block to discriminate SZ patients from healthy controls (HC). Evaluated on the dataset containing 14 SZ patients and 14 HC subjects, and the proposed method achieved an average classification accuracy of 81.81% in subject-independent experiment scenario. This study demonstrated that the structure of RAE is able to capture the differential features between SZ patients and HC subjects.
Temporal Properties of Enumerative Shaping: Autocorrelation and Energy Dispersion Index
Authors: Yunus Can Gültekin, Kaiquan Wu, Alex Alvarado
Date: 2022-07-08 00:00:00

We study the effective SNR behavior of various enumerative amplitude shaping algorithms. We show that their relative behavior can be explained via the temporal autocorrelation function or via the energy dispersion index.
Multilevel Geometric Optimization for Regularised Constrained Linear Inverse Problems
Authors: Sebastian Müller, Stefania Petra, Matthias Zisler
Date: 2022-07-11 00:00:00

We present a geometric multilevel optimization approach that smoothly incorporates box constraints. Given a box constrained optimization problem, we consider a hierarchy of models with varying discretization levels. Finer models are accurate but expensive to compute, while coarser models are less accurate but cheaper to compute. When working at the fine level, multilevel optimisation computes the search direction based on a coarser model which speeds up updates at the fine level. Moreover, exploiting geometry induced by the hierarchy the feasibility of the updates is preserved. In particular, our approach extends classical components of multigrid methods like restriction and prolongation to the Riemannian structure of our constraints.
Structural Inference of Networked Dynamical Systems with Universal Differential Equations
Authors: James Koch, Zhao Chen, Aaron Tuor, Jan Drgona, Draguna Vrabie
Date: 2022-07-11 00:00:00

Networked dynamical systems are common throughout science in engineering; e.g., biological networks, reaction networks, power systems, and the like. For many such systems, nonlinearity drives populations of identical (or near-identical) units to exhibit a wide range of nontrivial behaviors, such as the emergence of coherent structures (e.g., waves and patterns) or otherwise notable dynamics (e.g., synchrony and chaos). In this work, we seek to infer (i) the intrinsic physics of a base unit of a population, (ii) the underlying graphical structure shared between units, and (iii) the coupling physics of a given networked dynamical system given observations of nodal states. These tasks are formulated around the notion of the Universal Differential Equation, whereby unknown dynamical systems can be approximated with neural networks, mathematical terms known a priori (albeit with unknown parameterizations), or combinations of the two. We demonstrate the value of these inference tasks by investigating not only future state predictions but also the inference of system behavior on varied network topologies. The effectiveness and utility of these methods is shown with their application to canonical networked nonlinear coupled oscillators.
Consistency is the key to further mitigating catastrophic forgetting in continual learning
Authors: Prashant Bhat, Bahram Zonooz, Elahe Arani
Date: 2022-07-11 00:00:00
Tasks: Continual Learning, Self-Supervised Learning

Deep neural networks struggle to continually learn multiple sequential tasks due to catastrophic forgetting of previously learned tasks. Rehearsal-based methods which explicitly store previous task samples in the buffer and interleave them with the current task samples have proven to be the most effective in mitigating forgetting. However, Experience Replay (ER) does not perform well under low-buffer regimes and longer task sequences as its performance is commensurate with the buffer size. Consistency in predictions of soft-targets can assist ER in preserving information pertaining to previous tasks better as soft-targets capture the rich similarity structure of the data. Therefore, we examine the role of consistency regularization in ER framework under various continual learning scenarios. We also propose to cast consistency regularization as a self-supervised pretext task thereby enabling the use of a wide variety of self-supervised learning methods as regularizers. While simultaneously enhancing model calibration and robustness to natural corruptions, regularizing consistency in predictions results in lesser forgetting across all continual learning scenarios. Among the different families of regularizers, we find that stricter consistency constraints preserve previous task information in ER better.
Start Small, Think Big: On Hyperparameter Optimization for Large-Scale Knowledge Graph Embeddings
Authors: Adrian Kochsiek, Fritz Niesel, Rainer Gemulla
Date: 2022-07-11 00:00:00
Tasks: Graph Embedding, Hyperparameter Optimization, Knowledge Graph Embedding, Knowledge Graph Embeddings, Knowledge Graphs, Link Prediction

Knowledge graph embedding (KGE) models are an effective and popular approach to represent and reason with multi-relational data. Prior studies have shown that KGE models are sensitive to hyperparameter settings, however, and that suitable choices are dataset-dependent. In this paper, we explore hyperparameter optimization (HPO) for very large knowledge graphs, where the cost of evaluating individual hyperparameter configurations is excessive. Prior studies often avoided this cost by using various heuristics; e.g., by training on a subgraph or by using fewer epochs. We systematically discuss and evaluate the quality and cost savings of such heuristics and other low-cost approximation techniques. Based on our findings, we introduce GraSH, an efficient multi-fidelity HPO algorithm for large-scale KGEs that combines both graph and epoch reduction techniques and runs in multiple rounds of increasing fidelities. We conducted an experimental study and found that GraSH obtains state-of-the-art results on large graphs at a low cost (three complete training runs in total).
Dual Vision Transformer
Authors: Ting Yao, Yehao Li, Yingwei Pan, Yu Wang, Xiao-Ping Zhang, Tao Mei
Date: 2022-07-11 00:00:00

Prior works have proposed several strategies to reduce the computational cost of self-attention mechanism. Many of these works consider decomposing the self-attention procedure into regional and local feature extraction procedures that each incurs a much smaller computational complexity. However, regional information is typically only achieved at the expense of undesirable information lost owing to down-sampling. In this paper, we propose a novel Transformer architecture that aims to mitigate the cost issue, named Dual Vision Transformer (Dual-ViT). The new architecture incorporates a critical semantic pathway that can more efficiently compress token vectors into global semantics with reduced order of complexity. Such compressed global semantics then serve as useful prior information in learning finer pixel level details, through another constructed pixel pathway. The semantic pathway and pixel pathway are then integrated together and are jointly trained, spreading the enhanced self-attention information in parallel through both of the pathways. Dual-ViT is henceforth able to reduce the computational complexity without compromising much accuracy. We empirically demonstrate that Dual-ViT provides superior accuracy than SOTA Transformer architectures with reduced training complexity. Source code is available at \url{https://github.com/YehLi/ImageNetModel}.
From Correlation to Causation: Formalizing Interpretable Machine Learning as a Statistical Process
Authors: Lukas Klein, Mennatallah El-Assady, Paul F. Jäger
Date: 2022-07-11 00:00:00
Tasks: BIG-bench Machine Learning, Explainable Artificial Intelligence (XAI), Interpretable Machine Learning

Explainable AI (XAI) is a necessity in safety-critical systems such as in clinical diagnostics due to a high risk for fatal decisions. Currently, however, XAI resembles a loose collection of methods rather than a well-defined process. In this work, we elaborate on conceptual similarities between the largest subgroup of XAI, interpretable machine learning (IML), and classical statistics. Based on these similarities, we present a formalization of IML along the lines of a statistical process. Adopting this statistical view allows us to interpret machine learning models and IML methods as sophisticated statistical tools. Based on this interpretation, we infer three key questions, which we identify as crucial for the success and adoption of IML in safety-critical settings. By formulating these questions, we further aim to spark a discussion about what distinguishes IML from classical statistics and what our perspective implies for the future of the field.
HLT-MT: High-resource Language-specific Training for Multilingual Neural Machine Translation
Authors: Jian Yang, Yuwei Yin, Shuming Ma, Dongdong Zhang, Zhoujun Li, Furu Wei
Date: 2022-07-11 00:00:00
Tasks: Decoder, Machine Translation, Translation

Multilingual neural machine translation (MNMT) trained in multiple language pairs has attracted considerable attention due to fewer model parameters and lower training costs by sharing knowledge among multiple languages. Nonetheless, multilingual training is plagued by language interference degeneration in shared parameters because of the negative interference among different translation directions, especially on high-resource languages. In this paper, we propose the multilingual translation model with the high-resource language-specific training (HLT-MT) to alleviate the negative interference, which adopts the two-stage training with the language-specific selection mechanism. Specifically, we first train the multilingual model only with the high-resource pairs and select the language-specific modules at the top of the decoder to enhance the translation quality of high-resource directions. Next, the model is further trained on all available corpora to transfer knowledge from high-resource languages (HRLs) to low-resource languages (LRLs). Experimental results show that HLT-MT outperforms various strong baselines on WMT-10 and OPUS-100 benchmarks. Furthermore, the analytic experiments validate the effectiveness of our method in mitigating the negative interference in multilingual training.
UM4: Unified Multilingual Multiple Teacher-Student Model for Zero-Resource Neural Machine Translation
Authors: Jian Yang, Yuwei Yin, Shuming Ma, Dongdong Zhang, Shuangzhi Wu, Hongcheng Guo, Zhoujun Li, Furu Wei
Date: 2022-07-11 00:00:00
Tasks: Machine Translation, NMT, Translation

Most translation tasks among languages belong to the zero-resource translation problem where parallel corpora are unavailable. Multilingual neural machine translation (MNMT) enables one-pass translation using shared semantic space for all languages compared to the two-pass pivot translation but often underperforms the pivot-based method. In this paper, we propose a novel method, named as Unified Multilingual Multiple teacher-student Model for NMT (UM4). Our method unifies source-teacher, target-teacher, and pivot-teacher models to guide the student model for the zero-resource translation. The source teacher and target teacher force the student to learn the direct source to target translation by the distilled knowledge on both source and target sides. The monolingual corpus is further leveraged by the pivot-teacher model to enhance the student model. Experimental results demonstrate that our model of 72 directions significantly outperforms previous methods on the WMT benchmark.
Classification and Generation of real-world data with an Associative Memory Model
Authors: Rodrigo Simas, Luis Sa-Couto, Andreas Wichert
Date: 2022-07-11 00:00:00
Tasks: Decoder, Retrieval

Drawing from memory the face of a friend you have not seen in years is a difficult task. However, if you happen to cross paths, you would easily recognize each other. The biological memory is equipped with an impressive compression algorithm that can store the essential, and then infer the details to match perception. The Willshaw Memory is a simple abstract model for cortical computations which implements mechanisms of biological memories. Using our recently proposed sparse coding prescription for visual patterns, this model can store and retrieve an impressive amount of real-world data in a fault-tolerant manner. In this paper, we extend the capabilities of the basic Associative Memory Model by using a Multiple-Modality framework. In this setting, the memory stores several modalities (e.g., visual, or textual) of each pattern simultaneously. After training, the memory can be used to infer missing modalities when just a subset is perceived. Using a simple encoder-memory-decoder architecture, and a newly proposed iterative retrieval algorithm for the Willshaw Model, we perform experiments on the MNIST dataset. By storing both the images and labels as modalities, a single Memory can be used not only to retrieve and complete patterns but also to classify and generate new ones. We further discuss how this model could be used for other learning tasks, thus serving as a biologically-inspired framework for learning.
Assessing Ranking and Effectiveness of Evolutionary Algorithm Hyperparameters Using Global Sensitivity Analysis Methodologies
Authors: Varun Ojha, Jon Timmis, Giuseppe Nicosia
Date: 2022-07-11 00:00:00
Tasks: Evolutionary Algorithms, global-optimization, Sensitivity

We present a comprehensive global sensitivity analysis of two single-objective and two multi-objective state-of-the-art global optimization evolutionary algorithms as an algorithm configuration problem. That is, we investigate the quality of influence hyperparameters have on the performance of algorithms in terms of their direct effect and interaction effect with other hyperparameters. Using three sensitivity analysis methods, Morris LHS, Morris, and Sobol, to systematically analyze tunable hyperparameters of covariance matrix adaptation evolutionary strategy, differential evolution, non-dominated sorting genetic algorithm III, and multi-objective evolutionary algorithm based on decomposition, the framework reveals the behaviors of hyperparameters to sampling methods and performance metrics. That is, it answers questions like what hyperparameters influence patterns, how they interact, how much they interact, and how much their direct influence is. Consequently, the ranking of hyperparameters suggests their order of tuning, and the pattern of influence reveals the stability of the algorithms.