pwc-archive/papers-with-abstracts huggingface dataset:
Parkinson’s Disease EMG Signal Prediction Using Neural Networks
Authors: Rafael Anicet Zanini, Esther Luna Colombini, Maria Claudia Ferrari de Castro
Date: 2019-10-06 00:00:00
Tasks: Electromyography (EMG), EMG Signal Prediction, Prediction

This paper proposes a comparison between different neural network models, using multilayer perceptron (MLPs) and recurrent neural network (RNN) models, for predicting Parkinson's disease electromyography (EMG) signals, to anticipate resulting resting tremor patterns. The experimental results indicate that the proposed models can adapt to different frequencies and amplitudes of tremor, and provide reasonable predictions for both EMG envelopes and EMG raw signals. Therefore, one could use these models as input for a control strategy for functional electrical stimulation (FES) devices used on tremor suppression, by dynamically predicting and improving FES control parameters based on tremor forecast.
A Mask-based Model for Mandarin Chinese Polyphone Disambiguation
Authors: Haiteng Zhang
Date: 2020-10-21 00:00:00
Tasks: Polyphone disambiguation, text-to-speech, Text to Speech

Polyphone disambiguation serves as an essential part of Mandarin text-to-speech (TTS) system. However, conventional system modeling the entire Pinyin set causes the case that prediction belongs to the unrelated polyphonic character instead of the current input one, which has negative impacts on TTS performance. To address this issue, we introduce a mask-based model for polyphone disambiguation. The model takes a mask vector extracted from the context as an extra input. In our model, the mask vector not only acts as a weighting factor in Weightedsoftmax to prevent the case of mis-prediction but also eliminates the contribution of non-candidate set to the overall loss. Moreover, to mitigate the uneven distribution of pronunciation, we introduce a new loss called Modified Focal Loss. The experimental result shows the effectiveness of the proposed mask based model. We also empirically studied the impact of Weighted-softmax and Modified Focal Loss. It was found that Weighted-softmax can effectively prevent the model from predicting outside the candidate set. Besides, Modified Focal Loss can reduce the adverse impacts of the uneven distribution of
pronunciation.
Estimation of Soil Moisture Applying Modified Dubois Model to Sentinel-1; A Regional Study from Central India
Authors: Abhilash Singh Kumar, Kumar Gaurav, Ganesh Gaurav  Kumar, Ganesh Kumar Meena, Shashi Kumar
Date: 2020-07-15 00:00:00
Tasks: Management

Surface soil moisture has a wide application in climate change, agronomy, water resources, and in many other domain of science and engineering. Measurement of soil moisture at high spatial and temporal resolution at regional and global scale is needed for the prediction of flood, drought, planning and management of agricultural productivity to ensure food security. Recent advancement in microwave remote sensing, especially after the launch of Sentinel operational satellites has enabled the scientific community to estimate soil moisture at higher spatial and temporal resolution with greater accuracy. This study evaluates the potential of Sentinel-1A satellite images to estimate soil moisture in a semi-arid region. Exactly at the time when satellite passes over the study area, we have collected soil samples at 37 different locations and measured the soil moisture from 5 cm below the ground surface using ML3 theta probe. We processed the soil samples in laboratory to obtain volumetric soil moisture using the oven dry method. We found soil moisture measured from calibrated theta probe and oven dry method are in good agreement with Root Mean Square Error (RMSE) 0.025 m 3 /m 3 and coefficient of determination (R 2 ) 0.85. We then processed Sentinel-1A images and applied modified Dubois model to calculate relative permittivity of the soil from the backscatter values ( σ∘ ). The volumetric soil moisture at each pixel is then calculated by applying the universal Topp’s model. Finally, we masked the pixels whose Normalised Difference Vegetation Index (NDVI) value is greater than 0.4 to generate soil moisture map as per the Dubois NDVI criterion. Our modelled soil moisture accord with the measured values with RMSE = 0.035 and R 2 = 0.75. We found a small bias in the modelled soil moisture ( 0.02m3/m3 ). However, this has reduced significantly ( 0.001m3/m3 ) after applying a bias correction based on Cumulative Distribution Function (CDF) matching. Our approach provides a first-order estimate of soil moisture from Sentinel-1A images in sparsely vegetated agricultural land.
A deep learning framework for Text-independent Writer Identification
Authors: Malihe Javidi, Mahdi Jampour
Date: 2020-10-01 00:00:00
Tasks: Deep Learning, Feature Engineering

Handwriting Writer Identification (HWI) refers to the process of handwriting text image analysis to identify the authorship of the documents. It has yielded promising results in various applications, including digital forensics, criminal purposes, exploring the writer of historical documents, etc. The complexity of the text image, especially in images with various handwriting makes the writer identification difficult. In this work, we propose an end-to-end system that relies on a straightforward yet well-designed deep network and very efficient feature extraction, emphasizing feature engineering. Our system is an extended version of ResNet by conjugating deep residual networks and a new traditional yet high-quality handwriting descriptor towards handwriting analysis. Our descriptor analyzes the handwriting thickness as a preliminary and essential feature for human handwriting characteristics. Our approach can also provide text-independent writer identification that we do not need to have the same handwriting content for learning our model. The proposed approach is evaluated and achieved consistent results on four public and well-known datasets of IAM, Firemaker, CVL, and CERUG-EN. We empirically demonstrate that our conjugated network outperforms the original ResNet, and it can work well for real-world applications in which patches with few letters exist.
CapsNet Regularization and its Conjugation with ResNet for Signature Identification
Authors: Mahdi Jampour, Saeid Abbaasi, Malihe Javidi
Date: 2021-01-29 00:00:00

We propose a new regularization term for CapsNet that significantly improves the generalization power of the original method from small training data while requiring much fewer parameters, making it suitable for large input images. We also propose a very efficient DNN architecture that integrates CapsNet with ResNet to obtain the advantages of the two architectures. CapsNet allows a powerful understanding of the objects' components and their positions, while ResNet provides efficient feature extraction and description. Our approach is general, and we demonstrate it on the problem of signature identification from images. To show our approach superiority, we provide several evaluations with different protocols. We also show that our approach outperforms the state-of-the-art on this problem  with thorough experiments on three publicly available datasets CEDAR, MCYT, and UTSig.
Downlink and Uplink Intelligent Reflecting Surface Aided Networks: NOMA and OMA
Authors: Yanyu Cheng, Kwok Hung Li, Yuanwei Liu, Kah Chan Teh, H. Vincent Poor
Date: 2020-05-03 00:00:00
Tasks: Diversity

Intelligent reflecting surfaces (IRSs) are envisioned to provide reconfigurable wireless environments for future communication networks. In this paper, both downlink and uplink IRS-aided non-orthogonal multiple access (NOMA) and orthogonal multiple access (OMA) networks are studied, in which an IRS is deployed to enhance the coverage by assisting a cell-edge user device (UD) to communicate with the base station (BS). To characterize system performance, new channel statistics of the BS-IRS-UD link with Nakagami-$m$ fading are investigated. For each scenario, the closed-form expressions for the outage probability and ergodic rate are derived. To gain further insight, the diversity order and high signal-to-noise ratio (SNR) slope for each scenario are obtained according to asymptotic approximations in the high-SNR regime. It is demonstrated that the diversity order is affected by the number of IRS reflecting elements and Nakagami fading parameters, but the high-SNR slope is not related to these parameters. Simulation results validate our analysis and reveal the superiority of the IRS over the full-duplex decode-and-forward relay.
Describing, modelling and forecasting the spatial and temporal spread of COVID-19 -- A short review
Authors: Julien Arino
Date: 2021-02-04 00:00:00

SARS-CoV-2 started propagating worldwide in January 2020 and has now reached virtually all communities on the planet. This short review provides evidence of this spread and documents modelling efforts undertaken to understand and forecast it, including a short section about the new variants that emerged in late 2020.
Safety during Transient Response in Direct Current Microgrids using Control Barrier Functions
Authors: K. C. Kosaraju, S. Sivaranjani, V. Gupta
Date: 2021-02-04 00:00:00

We consider the problem of guaranteeing that the transient voltages and currents stay within prescribed bounds in Direct Current (DC) microgrids, when the controller does not have access to accurate system dynamics due to the load being unknown and/or time-varying. To achieve this, we propose an optimization based controller design using control barrier functions. We show that the proposed controller has a decentralized structure and is robust with respect to the uncertainty in the precise values of the system parameters, such as the load.
A New Deep Neural Architecture Search Pipeline for Face Recognition
Authors: Ning Zhu, Zekuan Yu, Caixia Kou
Date: 2020-05-12 00:00:00
Tasks: AutoML, Face Recognition, image-classification, Image Classification, Neural Architecture Search, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

With the widespread popularity of electronic devices, the emergence of biometric technology has brought significant convenience to user authentication compared with the traditional password and mode unlocking. Among many biological characteristics, the face is a universal and irreplaceable feature with simple detection methods and good recognition accuracy. Face recognition is one of the main functions of electronic equipment propaganda. The previous work in this field mainly focused on converting loss function in traditional deep convolution neural networks without changing the network structure. With the development of AutoML, neural architecture search (NAS) has shown remarkable performance in image classification tasks. In this paper, we first propose a new deep neural architecture search pipeline combined with NAS technology and reinforcement learning strategy into face recognition. We quote the framework of NAS, which trains the child and controller networks alternately. At the same time, we optimize NAS by incorporating evaluation latency into rewards of reinforcement learning and utilize the policy gradient algorithm to search the architecture automatically with the cross-entropy loss. The network architectures we searched out have achieved state-of-the-art accuracy in the large-scale face dataset, which achieved 98.77% top-1 in the MS-Celeb-1M dataset and 99.89% in LFW dataset with relatively small network size.
Tracking fast and slow changes in synaptic weights from simultaneously observed pre- and postsynaptic spiking
Authors: Ganchao Wei, Ian H. Stevenson
Date: 2021-02-02 00:00:00

Synapses change on multiple timescales, ranging from milliseconds to minutes, due to a combination of both short- and long-term plasticity. Here we develop an extension of the common Generalized Linear Model to infer both short- and long-term changes in the coupling between a pre- and post-synaptic neuron based on observed spiking activity. We model short-term synaptic plasticity using additive effects that depend on the presynaptic spike timing, and we model long-term changes in both synaptic weight and baseline firing rate using point process adaptive smoothing. Using simulations, we first show that this model can accurately recover time-varying synaptic weights 1) for both depressing and facilitating synapses, 2) with a variety of long-term changes (including realistic changes, such as due to STDP), 3) with a range of pre- and post-synaptic firing rates, and 4) for both excitatory and inhibitory synapses. We then apply our model to two experimentally recorded putative synaptic connections. We find that simultaneously tracking fast changes in synaptic weights, slow changes in synaptic weights, and unexplained variations in baseline firing is essential. Omitting any one of these factors can lead to spurious inferences for the others. Altogether, this model provides a flexible framework for tracking short- and long-term variation in spike transmission.
Optimizing connection weights in neural networks using the whale optimization algorithm
Authors: Ibrahim Aljarah1, Hossam Faris1, Seyedali Mirjalili2
Date: 2016-11-21 00:00:00
Tasks: Stochastic Optimization

The learning process of artificial neural networks is considered as one of the most difficult challenges
in machine learning and has attracted many researchers
recently. The main difficulty of training a neural network
is the nonlinear nature and the unknown best set of main
controlling parameters (weights and biases). The main disadvantages of the conventional training algorithms are local
optima stagnation and slow convergence speed. This makes
stochastic optimization algorithm reliable alternative to alleviate these drawbacks. This work proposes a new training
algorithm based on the recently proposed whale optimization
algorithm (WOA). It has been proved that this algorithm is
able to solve a wide range of optimization problems and outperform the current algorithms. This motivated our attempts
to benchmark its performance in training feedforward neural
networks. For the first time in the literature, a set of 20
datasets with different levels of difficulty are chosen to test
the proposed WOA-based trainer. The results are verified by
comparisons with back-propagation algorithm and six evolutionary techniques. The qualitative and quantitative results
prove that the proposed trainer is able to outperform the current algorithms on the majority of datasets in terms of both
local optima avoidance and convergence speed.
Using Long Short-Term Memory (LSTM) and Internet of Things (IoT) for localized surface temperature forecasting in an urban environment
Authors: Manzhu Yu, Fangcao Xu, Weiming Hu, Jian Sun, Guido Cervone
Date: 2021-02-04 00:00:00

The rising temperature is one of the key indicators of a warming climate, and it can cause extensive stress to biological systems as well as built structures. Due to the heat island effect, it is most severe in urban environments compared to other landscapes due to the decrease in vegetation associated with a dense human-built environment. It is essential to adequately monitor the local temperature dynamics to mitigate risks associated with increasing temperatures, which can include short term strategy to protect people and animals, to long term strategy to how to build a new structure and cope with extreme events. Observed temperature is also a very important input for atmospheric models, and accurate data can lead to better future forecasts. Ambient temperature collected at ground level can have a higher variability when compared to regional weather forecasts, which fail to capture the local dynamics. There remains a clear need for an accurate air temperature prediction at the sub-urban scale at high temporal and spatial resolution. This research proposed a framework based on Long Short-Term Memory (LSTM) deep learning network to generate day-ahead hourly temperature forecast with high spatial resolution. A case study is shown which uses historical in-situ observations and Internet of Things (IoT) observations for New York City, USA. By leveraging the historical air temperature data from in-situ observations, the LSTM model can be exposed to more historical patterns that might not be present in the IoT observations. Meanwhile, by using IoT observations, the spatial resolution of air temperature predictions is significantly improved.
baller2vec: A Multi-Entity Transformer For Multi-Agent Spatiotemporal Modeling
Authors: Michael A. Alcorn, Anh Nguyen
Date: 2021-02-05 00:00:00
Proceeding: NeurIPS 2021 12

Multi-agent spatiotemporal modeling is a challenging task from both an algorithmic design and computational complexity perspective. Recent work has explored the efficacy of traditional deep sequential models in this domain, but these architectures are slow and cumbersome to train, particularly as model size increases. Further, prior attempts to model interactions between agents across time have limitations, such as imposing an order on the agents, or making assumptions about their relationships. In this paper, we introduce baller2vec, a multi-entity generalization of the standard Transformer that can, with minimal assumptions, simultaneously and efficiently integrate information across entities and time. We test the effectiveness of baller2vec for multi-agent spatiotemporal modeling by training it to perform two different basketball-related tasks: (1) simultaneously modeling the trajectories of all players on the court and (2) modeling the trajectory of the ball. Not only does baller2vec learn to perform these tasks well (outperforming a graph recurrent neural network with a similar number of parameters by a wide margin), it also appears to "understand" the game of basketball, encoding idiosyncratic qualities of players in its embeddings, and performing basketball-relevant functions with its attention heads.
Adversarial Robustness Study of Convolutional Neural Network for Lumbar Disk Shape Reconstruction from MR images
Authors: Jiasong Chen, Linchen Qian, Timur Urakov, Weiyong Gu, Liang Liang
Date: 2021-02-04 00:00:00
Tasks: Adversarial Robustness, Data Augmentation, Medical Image Analysis

Machine learning technologies using deep neural networks (DNNs), especially convolutional neural networks (CNNs), have made automated, accurate, and fast medical image analysis a reality for many applications, and some DNN-based medical image analysis systems have even been FDA-cleared. Despite the progress, challenges remain to build DNNs as reliable as human expert doctors. It is known that DNN classifiers may not be robust to noises: by adding a small amount of noise to an input image, a DNN classifier may make a wrong classification of the noisy image (i.e., in-distribution adversarial sample), whereas it makes the right classification of the clean image. Another issue is caused by out-of-distribution samples that are not similar to any sample in the training set. Given such a sample as input, the output of a DNN will become meaningless. In this study, we investigated the in-distribution (IND) and out-of-distribution (OOD) adversarial robustness of a representative CNN for lumbar disk shape reconstruction from spine MR images. To study the relationship between dataset size and robustness to IND adversarial attacks, we used a data augmentation method to create training sets with different levels of shape variations. We utilized the PGD-based algorithm for IND adversarial attacks and extended it for OOD adversarial attacks to generate OOD adversarial samples for model testing. The results show that IND adversarial training can improve the CNN robustness to IND adversarial attacks, and larger training datasets may lead to higher IND robustness. However, it is still a challenge to defend against OOD adversarial attacks.
Self-Supervised Deep Graph Embedding with High-Order Information Fusion for Community Discovery
Authors: Shuliang Xu, Shenglan Liu, Lin Feng
Date: 2021-02-05 00:00:00
Tasks: Attribute, Data Augmentation, Graph Embedding, Graph Neural Network, Representation Learning

Deep graph embedding is an important approach for community discovery. Deep graph neural network with self-supervised mechanism can obtain the low-dimensional embedding vectors of nodes from unlabeled and unstructured graph data. The high-order information of graph can provide more abundant structure information for the representation learning of nodes. However, most self-supervised graph neural networks only use adjacency matrix as the input topology information of graph and cannot obtain too high-order information since the number of layers of graph neural network is fairly limited. If there are too many layers, the phenomenon of over smoothing will appear. Therefore how to obtain and fuse high-order information of graph by a shallow graph neural network is an important problem. In this paper, a deep graph embedding algorithm with self-supervised mechanism for community discovery is proposed. The proposed algorithm uses self-supervised mechanism and different high-order information of graph to train multiple deep graph convolution neural networks. The outputs of multiple graph convolution neural networks are fused to extract the representations of nodes which include the attribute and structure information of a graph. In addition, data augmentation and negative sampling are introduced into the training process to facilitate the improvement of embedding result. The proposed algorithm and the comparison algorithms are conducted on the five experimental data sets. The experimental results show that the proposed algorithm outperforms the comparison algorithms on the most experimental data sets. The experimental results demonstrate that the proposed algorithm is an effective algorithm for community discovery.
NRTSI: Non-Recurrent Time Series Imputation
Authors: Siyuan Shan, Yang Li, Junier B. Oliva
Date: 2021-02-05 00:00:00
Tasks: Imputation, Time Series, Time Series Analysis

Time series imputation is a fundamental task for understanding time series with missing data. Existing methods either do not directly handle irregularly-sampled data or degrade severely with sparsely observed data. In this work, we reformulate time series as permutation-equivariant sets and propose a novel imputation model NRTSI that does not impose any recurrent structures. Taking advantage of the permutation equivariant formulation, we design a principled and efficient hierarchical imputation procedure. In addition, NRTSI can directly handle irregularly-sampled time series, perform multiple-mode stochastic imputation, and handle data with partially observed dimensions. Empirically, we show that NRTSI achieves state-of-the-art performance across a wide range of time series imputation benchmarks.