pwc-archive/papers-with-abstracts huggingface dataset:
Self-Validated Physics-Embedding Network: A General Framework for Inverse Modelling
Authors: Ruiyuan Kang, Dimitrios C. Kyritsis, Panos Liatsis
Date: 2022-10-12 00:00:00
Tasks: State Estimation

Physics-based inverse modeling techniques are typically restricted to particular research fields, whereas popular machine-learning-based ones are too data-dependent to guarantee the physical compatibility of the solution. In this paper, Self-Validated Physics-Embedding Network (SVPEN), a general neural network framework for inverse modeling is proposed. As its name suggests, the embedded physical forward model ensures that any solution that successfully passes its validation is physically reasonable. SVPEN operates in two modes: (a) the inverse function mode offers rapid state estimation as conventional supervised learning, and (b) the optimization mode offers a way to iteratively correct estimations that fail the validation process. Furthermore, the optimization mode provides SVPEN with reconfigurability i.e., replacing components like neural networks, physical models, and error calculations at will to solve a series of distinct inverse problems without pretraining. More than ten case studies in two highly nonlinear and entirely distinct applications: molecular absorption spectroscopy and Turbofan cycle analysis, demonstrate the generality, physical reliability, and reconfigurability of SVPEN. More importantly, SVPEN offers a solid foundation to use existing physical models within the context of AI, so as to striking a balance between data-driven and physics-driven models.
System theoretic approach of information processing in nested cellular automata
Authors: Jerzy Szynka
Date: 2022-10-12 00:00:00

The subject of this paper is the evolution of the concept of information processing in regular structures based on multi-level processing in nested cellular automata. The essence of the proposed model is a discrete space-time containing nested orthogonal space-times at its points. The factorization of the function describing the global behavior of a system is the key element of the mathematical description. Factorization describes the relations of physical connections, signal propagation times and signal processing to global behavior. In the model appear expressions similar to expressions used in the Special Relativity Theory.
A Novel Multi-Objective Velocity-Free Boolean Particle Swarm Optimization
Authors: Wei Quan, Denise Gorse
Date: 2022-10-12 00:00:00
Tasks: Diversity, Position

This paper extends boolean particle swarm optimization to a multi-objective setting, to our knowledge for the first time in the literature. Our proposed new boolean algorithm, MBOnvPSO, is notably simplified by the omission of a velocity update rule and has enhanced exploration ability due to the inclusion of a 'noise' term in the position update rule that prevents particles being trapped in local optima. Our algorithm additionally makes use of an external archive to store non-dominated solutions and implements crowding distance to encourage solution diversity. In benchmark tests, MBOnvPSO produced high quality Pareto fronts, when compared to benchmarked alternatives, for all of the multi-objective test functions considered, with competitive performance in search spaces with up to 600 discrete dimensions.
Explainable fMRI-based Brain Decoding via Spatial Temporal-pyramid Graph Convolutional Network
Authors: Ziyuan Ye, Youzhi Qu, Zhichao Liang, Mo Wang, Quanying Liu
Date: 2022-10-08 00:00:00
Tasks: Brain Decoding

Brain decoding, aiming to identify the brain states using neural activity, is important for cognitive neuroscience and neural engineering. However, existing machine learning methods for fMRI-based brain decoding either suffer from low classification performance or poor explainability. Here, we address this issue by proposing a biologically inspired architecture, Spatial Temporal-pyramid Graph Convolutional Network (STpGCN), to capture the spatial-temporal graph representation of functional brain activities. By designing multi-scale spatial-temporal pathways and bottom-up pathways that mimic the information process and temporal integration in the brain, STpGCN is capable of explicitly utilizing the multi-scale temporal dependency of brain activities via graph, thereby achieving high brain decoding performance. Additionally, we propose a sensitivity analysis method called BrainNetX to better explain the decoding results by automatically annotating task-related brain regions from the brain-network standpoint. We conduct extensive experiments on fMRI data under 23 cognitive tasks from Human Connectome Project (HCP) S1200. The results show that STpGCN significantly improves brain decoding performance compared to competing baseline models; BrainNetX successfully annotates task-relevant brain regions. Post hoc analysis based on these regions further validates that the hierarchical structure in STpGCN significantly contributes to the explainability, robustness and generalization of the model. Our methods not only provide insights into information representation in the brain under multiple cognitive tasks but also indicate a bright future for fMRI-based brain decoding.
iMedBot: A Web-based Intelligent Agent for Healthcare Related Prediction and Deep Learning
Authors: Chuhan Xu, Xia Jiang
Date: 2022-10-07 00:00:00

Background: Breast cancer is a multifactorial disease, genetic and environmental factors will affect its incidence probability. Breast cancer metastasis is one of the main cause of breast cancer related deaths reported by the American Cancer Society (ACS). Method: the iMedBot is a web application that we developed using the python Flask web framework and deployed on Amazon Web Services. It contains a frontend and a backend. The backend is supported by a python program we developed using the python Keras and scikit-learn packages, which can be used to learn deep feedforward neural network (DFNN) models. Result: the iMedBot can provide two main services: 1. it can predict 5-, 10-, or 15-year breast cancer metastasis based on a set of clinical information provided by a user. The prediction is done by using a set of DFNN models that were pretrained, and 2. It can train DFNN models for a user using user-provided dataset. The model trained will be evaluated using AUC and both the AUC value and the AUC ROC curve will be provided. Conclusion: The iMedBot web application provides a user-friendly interface for user-agent interaction in conducting personalized prediction and model training. It is an initial attempt to convert results of deep learning research into an online tool that may stir further research interests in this direction. Keywords: Deep learning, Breast Cancer, Web application, Model training.
Automatic Discovery of Composite SPMD Partitioning Strategies in PartIR
Authors: Sami Alabed, Dominik Grewe, Juliana Franco, Bart Chrzaszcz, Tom Natan, Tamara Norman, Norman A. Rink, Dimitrios Vytiniotis, Michael Schaarschmidt
Date: 2022-10-07 00:00:00

Large neural network models are commonly trained through a combination of advanced parallelism strategies in a single program, multiple data (SPMD) paradigm. For example, training large transformer models requires combining data, model, and pipeline partitioning; and optimizer sharding techniques. However, identifying efficient combinations for many model architectures and accelerator systems requires significant manual analysis. In this work, we present an automatic partitioner that identifies these combinations through a goal-oriented search. Our key findings are that a Monte Carlo Tree Search-based partitioner leveraging partition-specific compiler analysis directly into the search and guided goals matches expert-level strategies for various models.
Images as Weight Matrices: Sequential Image Generation Through Synaptic Learning Rules
Authors: Kazuki Irie, Jürgen Schmidhuber
Date: 2022-10-07 00:00:00
Tasks: Denoising, Image Generation, Inductive Bias

Work on fast weight programmers has demonstrated the effectiveness of key/value outer product-based learning rules for sequentially generating a weight matrix (WM) of a neural net (NN) by another NN or itself. However, the weight generation steps are typically not visually interpretable by humans, because the contents stored in the WM of an NN are not. Here we apply the same principle to generate natural images. The resulting fast weight painters (FPAs) learn to execute sequences of delta learning rules to sequentially generate images as sums of outer products of self-invented keys and values, one rank at a time, as if each image was a WM of an NN. We train our FPAs in the generative adversarial networks framework, and evaluate on various image datasets. We show how these generic learning rules can generate images with respectable visual quality without any explicit inductive bias for images. While the performance largely lags behind the one of specialised state-of-the-art image generators, our approach allows for visualising how synaptic learning rules iteratively produce complex connection patterns, yielding human-interpretable meaningful images. Finally, we also show that an additional convolutional U-Net (now popular in diffusion models) at the output of an FPA can learn one-step "denoising" of FPA-generated images to enhance their quality. Our code is public.
SGDA with shuffling: faster convergence for nonconvex-PŁ minimax optimization
Authors: Hanseul Cho, Chulhee Yun
Date: 2022-10-12 00:00:00

Stochastic gradient descent-ascent (SGDA) is one of the main workhorses for solving finite-sum minimax optimization problems. Most practical implementations of SGDA randomly reshuffle components and sequentially use them (i.e., without-replacement sampling); however, there are few theoretical results on this approach for minimax algorithms, especially outside the easier-to-analyze (strongly-)monotone setups. To narrow this gap, we study the convergence bounds of SGDA with random reshuffling (SGDA-RR) for smooth nonconvex-nonconcave objectives with Polyak-{\L}ojasiewicz (P{\L}) geometry. We analyze both simultaneous and alternating SGDA-RR for nonconvex-P{\L} and primal-P{\L}-P{\L} objectives, and obtain convergence rates faster than with-replacement SGDA. Our rates extend to mini-batch SGDA-RR, recovering known rates for full-batch gradient descent-ascent (GDA). Lastly, we present a comprehensive lower bound for GDA with an arbitrary step-size ratio, which matches the full-batch upper bound for the primal-P{\L}-P{\L} case.
Russian Web Tables: A Public Corpus of Web Tables for Russian Language Based on Wikipedia
Authors: Platon Fedorov, Alexey Mironov, George Chernishev
Date: 2022-10-03 00:00:00
Tasks: Knowledge Base Construction, Management, Question Answering

Corpora that contain tabular data such as WebTables are a vital resource for the academic community. Essentially, they are the backbone of any modern research in information management. They are used for various tasks of data extraction, knowledge base construction, question answering, column semantic type detection and many other. Such corpora are useful not only as a source of data, but also as a base for building test datasets. So far, there were no such corpora for the Russian language and this seriously hindered research in the aforementioned areas. In this paper, we present the first corpus of Web tables created specifically out of Russian language material. It was built via a special toolkit we have developed to crawl the Russian Wikipedia. Both the corpus and the toolkit are open-source and publicly available. Finally, we present a short study that describes Russian Wikipedia tables and their statistics.
Variational Open-Domain Question Answering
Authors: Valentin Liévin, Andreas Geert Motzfeldt, Ida Riis Jensen, Ole Winther
Date: 2022-09-23 00:00:00
Tasks: Language Modelling, MedQA, Multiple-choice, Multiple Choice Question Answering (MCQA), Open-Domain Question Answering, Question Answering, Retrieval, Variational Inference

Retrieval-augmented models have proven to be effective in natural language processing tasks, yet there remains a lack of research on their optimization using variational inference. We introduce the Variational Open-Domain (VOD) framework for end-to-end training and evaluation of retrieval-augmented models, focusing on open-domain question answering and language modelling. The VOD objective, a self-normalized estimate of the R\'enyi variational bound, approximates the task marginal likelihood and is evaluated under samples drawn from an auxiliary sampling distribution (cached retriever and/or approximate posterior). It remains tractable, even for retriever distributions defined on large corpora. We demonstrate VOD's versatility by training reader-retriever BERT-sized models on multiple-choice medical exam questions. On the MedMCQA dataset, we outperform the domain-tuned Med-PaLM by +5.3% despite using 2.500$\times$ fewer parameters. Our retrieval-augmented BioLinkBERT model scored 62.9% on the MedMCQA and 55.0% on the MedQA-USMLE. Last, we show the effectiveness of our learned retriever component in the context of medical semantic search.
Self-supervised Learning for Label-Efficient Sleep Stage Classification: A Comprehensive Evaluation
Authors: Emadeldeen Eldele, Mohamed Ragab, Zhenghua Chen, Min Wu, Chee-Keong Kwoh, XiaoLi Li
Date: 2022-10-10 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Self-Supervised Learning

The past few years have witnessed a remarkable advance in deep learning for EEG-based sleep stage classification (SSC). However, the success of these models is attributed to possessing a massive amount of labeled data for training, limiting their applicability in real-world scenarios. In such scenarios, sleep labs can generate a massive amount of data, but labeling these data can be expensive and time-consuming. Recently, the self-supervised learning (SSL) paradigm has shined as one of the most successful techniques to overcome the scarcity of labeled data. In this paper, we evaluate the efficacy of SSL to boost the performance of existing SSC models in the few-labels regime. We conduct a thorough study on three SSC datasets, and we find that fine-tuning the pretrained SSC models with only 5% of labeled data can achieve competitive performance to the supervised training with full labels. Moreover, self-supervised pretraining helps SSC models to be more robust to data imbalance and domain shift problems. The code is publicly available at \url{https://github.com/emadeldeen24/eval_ssl_ssc}.
Stacked Ensemble Machine Learning for Range-Separation Parameters
Authors: Cheng-Wei Ju, Ethan J. French, Nadav Geva, Alexander W. Kohn, and Zhou Lin
Date: 2021-09-24 00:00:00
Proceeding: The Journal of Physical Chemistry Letters 2021 9
Tasks: Diversity, Drug Discovery

Density functional theory-based high-throughput materials and drug discovery has achieved tremendous success in recent decades, but its power on organic semiconducting molecules suffered catastrophically from the self-interaction error until the nonempirical but expensive optimally tuned range-separated hybrid (OT-RSH) functionals were developed. An OT-RSH transitions from a short-range (semi)local functional to a long-range Hartree−Fock exchange at a distance characterized by a molecule-specific range-separation parameter (ω). Herein, we propose a stacked ensemble machine learning model that provides an accelerated alternative of OT-RSH based on system-dependent structural and electronic configurations. We trained ML-ωPBE, the first functional in our series, using a database of 1970 molecules with sufficient structural and functional diversity, and assessed its accuracy and efficiency using another 1956 molecules. Compared with nonempirical OT-ωPBE, ML-ωPBE reaches a mean absolute error of 0.00504a0−1 for optimal ω’s, reduces the computational cost by 2.66 orders of magnitude, and achieves comparable predictive power in optical properties.
An Energy-Efficient Spiking Neural Network for Finger Velocity Decoding for Implantable Brain-Machine Interface
Authors: Jiawei Liao, Lars Widmer, Xiaying Wang, Alfio Di Mauro, Samuel R. Nason-Tomaszewski, Cynthia A. Chestek, Luca Benini, Taekwang Jang
Date: 2022-10-07 00:00:00
Tasks: Decoder, regression

Brain-machine interfaces (BMIs) are promising for motor rehabilitation and mobility augmentation. High-accuracy and low-power algorithms are required to achieve implantable BMI systems. In this paper, we propose a novel spiking neural network (SNN) decoder for implantable BMI regression tasks. The SNN is trained with enhanced spatio-temporal backpropagation to fully leverage its ability in handling temporal problems. The proposed SNN decoder achieves the same level of correlation coefficient as the state-of-the-art ANN decoder in offline finger velocity decoding tasks, while it requires only 6.8% of the computation operations and 9.4% of the memory access.
Evaluation of Delay Uncertainty in PCB Interconnects Due to Fiber Weave Effect
Authors: Alex Manukovsky, Yuriy Shlepnev, Shimon Mordooch
Date: 2022-10-11 00:00:00

Delay Deviation Exceedance (DDE) and Differential Skew Exceedance (DSE) measures are proposed to quantify delay uncertainty in single-ended and differential PCB interconnects arising from Fiber Weave Effect (FWE). DDE and DSE are constructed with 3D EM analysis of traces over inhomogeneous dielectric with glass fiber bundles in resin. Measurements or 3D EM models for FWE are usually used for observations of delay or impedance dependency on position of traces over fiber bundles with the purpose to find the worst case scenario. This paper turns the results of 3D EM analysis into probabilistic measures of possible delay or skew uncertainty - probability to have delay deviation in single-ended interconnects over allowed limit (DDE measure) or to have skew over allowed limit in differential interconnects (DSE measure). The introduced measures allow formalizing the laminate selection process for parallel as well as for serial PCB interconnects.
Dynamic neuronal networks efficiently achieve classification in robotic interactions with real-world objects
Authors: Pakorn Uttayopas, Xiaoxiao Cheng, Udaya Bhaskar Rongala, Henrik Jörntell, Etienne Burdet
Date: 2022-10-12 00:00:00

Biological cortical networks are potentially fully recurrent networks without any distinct output layer, where recognition may instead rely on the distribution of activity across its neurons. Because such biological networks can have rich dynamics, they are well-designed to cope with dynamical interactions of the types that occur in nature, while traditional machine learning networks may struggle to make sense of such data. Here we connected a simple model neuronal network (based on the 'linear summation neuron model' featuring biologically realistic dynamics (LSM), consisting of 10 of excitatory and 10 inhibitory neurons, randomly connected) to a robot finger with multiple types of force sensors when interacting with materials of different levels of compliance. Scope: to explore the performance of the network on classification accuracy. Therefore, we compared the performance of the network output with principal component analysis of statistical features of the sensory data as well as its mechanical properties. Remarkably, even though the LSM was a very small and untrained network, and merely designed to provide rich internal network dynamics while the neuron model itself was highly simplified, we found that the LSM outperformed these other statistical approaches in terms of accuracy.
Can we use Common Voice to train a Multi-Speaker TTS system?
Authors: Sewade Ogun, Vincent Colotte, Emmanuel Vincent
Date: 2022-10-12 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Diversity, speech-recognition, Speech Recognition, text-to-speech, Text to Speech

Training of multi-speaker text-to-speech (TTS) systems relies on curated datasets based on high-quality recordings or audiobooks. Such datasets often lack speaker diversity and are expensive to collect. As an alternative, recent studies have leveraged the availability of large, crowdsourced automatic speech recognition (ASR) datasets. A major problem with such datasets is the presence of noisy and/or distorted samples, which degrade TTS quality. In this paper, we propose to automatically select high-quality training samples using a non-intrusive mean opinion score (MOS) estimator, WV-MOS. We show the viability of this approach for training a multi-speaker GlowTTS model on the Common Voice English dataset. Our approach improves the overall quality of generated utterances by 1.26 MOS point with respect to training on all the samples and by 0.35 MOS point with respect to training on the LibriTTS dataset. This opens the door to automatic TTS dataset curation for a wider range of languages.
Adversarial Speaker-Consistency Learning Using Untranscribed Speech Data for Zero-Shot Multi-Speaker Text-to-Speech
Authors: Byoung Jin Choi, Myeonghun Jeong, Minchan Kim, Sung Hwan Mun, Nam Soo Kim
Date: 2022-10-12 00:00:00
Tasks: text-to-speech, Text to Speech

Several recently proposed text-to-speech (TTS) models achieved to generate the speech samples with the human-level quality in the single-speaker and multi-speaker TTS scenarios with a set of pre-defined speakers. However, synthesizing a new speaker's voice with a single reference audio, commonly known as zero-shot multi-speaker text-to-speech (ZSM-TTS), is still a very challenging task. The main challenge of ZSM-TTS is the speaker domain shift problem upon the speech generation of a new speaker. To mitigate this problem, we propose adversarial speaker-consistency learning (ASCL). The proposed method first generates an additional speech of a query speaker using the external untranscribed datasets at each training iteration. Then, the model learns to consistently generate the speech sample of the same speaker as the corresponding speaker embedding vector by employing an adversarial learning scheme. The experimental results show that the proposed method is effective compared to the baseline in terms of the quality and speaker similarity in ZSM-TTS.
Cross-dataset COVID-19 Transfer Learning with Cough Detection, Cough Segmentation, and Data Augmentation
Authors: Bagus Tris Atmaja, Zanjabila, Suyanto, Akira Sasou
Date: 2022-10-12 00:00:00
Tasks: Data Augmentation, Transfer Learning

This paper addresses issues on cough-based COVID-19 detection. We propose a cross-dataset transfer learning approach to improve the performance of COVID-19 detection by incorporating cough detection, cough segmentation, and data augmentation. The first aimed at removing non-cough signals and cough signals with low probability. The second aimed at segregating several coughs in a waveform into individual coughs. The third aimed at increasing the number of samples for the deep learning model. These three processing blocks are important as our finding revealed a large margin of improvement relative to the baseline methods without these blocks. An ablation study is conducted to optimize hyperparameters and it was found that alpha mixup is an important factor among others in improving the model performance via this augmentation method. A summary of this study with previous studies on the same evaluation set was given to gain insights into different methods of cough-based COVID-19 detection.