pwc-archive/papers-with-abstracts huggingface dataset:
TreeCaps: Tree-Based Capsule Networks for Source Code Processing
Authors: Nghi D. Q. Bui, Yijun Yu, Lingxiao Jiang
Date: 2020-09-05 00:00:00

Recently program learning techniques have been proposed to process source code based on syntactical structures (e.g., Abstract Syntax Trees) and/or semantic information (e.g., Dependency Graphs). Although graphs may be better at capturing various viewpoints of code semantics than trees, constructing graph inputs from code needs static code semantic analysis that may not be accurate and introduces noise during learning. Although syntax trees are precisely defined according to the language grammar and easier to construct and process than graphs, previous tree-based learning techniques have not been able to learn semantic information from trees to achieve better accuracy than graph-based techniques. We propose a new learning technique, named TreeCaps, by fusing together capsule networks with tree-based convolutional neural networks, to achieve learning accuracy higher than existing graph-based techniques while it is based only on trees. TreeCaps introduces novel variable-to-static routing algorithms into the capsule networks to compensate for the loss of previous routing algorithms. Aside from accuracy, we also find that TreeCaps is the most robust to withstand those semantic-preserving program transformations that change code syntax without modifying the semantics. Evaluated on a large number of Java and C/C++ programs, TreeCaps models outperform prior deep learning models of program source code, in terms of both accuracy and robustness for program comprehension tasks such as code functionality classification and function name prediction
Beyond Social Media Analytics: Understanding Human Behaviour and Deep Emotion using Self Structuring Incremental Machine Learning
Authors: Tharindu Bandaragoda
Date: 2020-09-05 00:00:00
Tasks: BIG-bench Machine Learning, Event Detection, Incremental Learning

This thesis develops a conceptual framework considering social data as representing the surface layer of a hierarchy of human social behaviours, needs and cognition which is employed to transform social data into representations that preserve social behaviours and their causalities. Based on this framework two platforms were built to capture insights from fast-paced and slow-paced social data. For fast-paced, a self-structuring and incremental learning technique was developed to automatically capture salient topics and corresponding dynamics over time. An event detection technique was developed to automatically monitor those identified topic pathways for significant fluctuations in social behaviours using multiple indicators such as volume and sentiment. This platform is demonstrated using two large datasets with over 1 million tweets. The separated topic pathways were representative of the key topics of each entity and coherent against topic coherence measures. Identified events were validated against contemporary events reported in news. Secondly for the slow-paced social data, a suite of new machine learning and natural language processing techniques were developed to automatically capture self-disclosed information of the individuals such as demographics, emotions and timeline of personal events. This platform was trialled on a large text corpus of over 4 million posts collected from online support groups. This was further extended to transform prostate cancer related online support group discussions into a multidimensional representation and investigated the self-disclosed quality of life of patients (and partners) against time, demographics and clinical factors. The capabilities of this extended platform have been demonstrated using a text corpus collected from 10 prostate cancer online support groups comprising of 609,960 prostate cancer discussions and 22,233 patients.
The Relationship between Deteriorating Mental Health Conditions and Longitudinal Behavioral Changes in Google and YouTube Usages among College Students in the United States during COVID-19: Observational Study
Authors: Anis Zaman, Boyu Zhang, Ehsan Hoque, Vincent Silenzio, Henry Kautz
Date: 2020-09-05 00:00:00

Mental health problems among the global population are worsened during the coronavirus disease (COVID-19). How individuals engage with online platforms such as Google Search and YouTube undergoes drastic shifts due to pandemic and subsequent lockdowns. Such ubiquitous daily behaviors on online platforms have the potential to capture and correlate with clinically alarming deteriorations in mental health profiles in a non-invasive manner. The goal of this study is to examine, among college students, the relationship between deteriorating mental health conditions and changes in user behaviors when engaging with Google Search and YouTube during COVID-19. This study recruited a cohort of 49 students from a U.S. college campus during January 2020 (prior to the pandemic) and measured the anxiety and depression levels of each participant. This study followed up with the same cohort during May 2020 (during the pandemic), and the anxiety and depression levels were assessed again. The longitudinal Google Search and YouTube history data were anonymized and collected. From individual-level Google Search and YouTube histories, we developed 5 signals that can quantify shifts in online behaviors during the pandemic. We then assessed the differences between groups with and without deteriorating mental health profiles in terms of these features. Significant features included late-night online activities, continuous usages, and time away from the internet, porn consumptions, and keywords associated with negative emotions, social activities, and personal affairs. Though further studies are required, our results demonstrated the feasibility of utilizing pervasive online data to establish non-invasive surveillance systems for mental health conditions that bypasses many disadvantages of existing screening methods.
Clustering COVID-19 Lung Scans
Authors: Jacob Householder, Andrew Householder, John Paul Gomez-Reed, Fredrick Park, Shuai Zhang
Date: 2020-09-05 00:00:00
Tasks: Clustering

With the ongoing COVID-19 pandemic, understanding the characteristics of the virus has become an important and challenging task in the scientific community. While tests do exist for COVID-19, the goal of our research is to explore other methods of identifying infected individuals. Our group applied unsupervised clustering techniques to explore a dataset of lungscans of COVID-19 infected, Viral Pneumonia infected, and healthy individuals. This is an important area to explore as COVID-19 is a novel disease that is currently being studied in detail. Our methodology explores the potential that unsupervised clustering algorithms have to reveal important hidden differences between COVID-19 and other respiratory illnesses. Our experiments use: Principal Component Analysis (PCA), K-Means++ (KM++) and the recently developed Robust Continuous Clustering algorithm (RCC). We evaluate the performance of KM++ and RCC in clustering COVID-19 lung scans using the Adjusted Mutual Information (AMI) score.
Efficient Computation of Higher Order 2D Image Moments using the Discrete Radon Transform
Authors: William Diggin, Michael Diggin
Date: 2020-09-04 00:00:00
Tasks: Position

Geometric moments and moment invariants of image artifacts have many uses in computer vision applications, e.g. shape classification or object position and orientation. Higher order moments are of interest to provide additional feature descriptors, to measure kurtosis or to resolve n-fold symmetry. This paper provides the method and practical application to extend an efficient algorithm, based on the Discrete Radon Transform, to generate moments greater than the 3rd order. The mathematical fundamentals are presented, followed by relevant implementation details. Results of scaling the algorithm based on image area and its computational comparison with a standard method demonstrate the efficacy of the approach.
Multiscale Context-Aware Ensemble Deep KELM for Efficient Hyperspectral Image Classification
Authors: Bobo Xi, Jiaojiao Li, Yunsong Li, Rui Song, Weiwei Sun, Qian Du.
Date: 2020-09-22 00:00:00
Tasks: Computational Efficiency, General Classification, Hyperspectral Image Classification, image-classification, Image Classification

Recently, multiscale spatial features have been widely utilized to improve the hyperspectral image (HSI) classification performance. However, fixed-size neighborhood involving the contextual information probably leads to misclassifications, especially for the boundary pixels. Additionally, it has been demonstrated that deep neural network (DNN) is practical to extract representative features for the classification tasks. Nevertheless, under the condition of high dimensionality versus small sample sizes, DNN tends to be over-fitting and it is generally time-consuming due to the deep-level feature learning process. To alleviate the aforementioned issues, we propose a multiscale context-aware ensemble deep kernel extreme learning machine (MSC-EDKELM) for efficient HSI classification. First, the scene of the HSI data set is over-segmented in multiscale via using the adaptive superpixel segmentation technique. Second, superpixel pattern (SP) and attentional neighboring superpixel pattern (ANSP) are generated by leveraging the superpixel maps, which can automatically comprise local and global contextual information, respectively. Afterward, an ensemble deep kernel extreme learning machine (EDKELM) is presented to investigate the deep-level characteristics in the SP and ANSP. Finally, the category of each pixel is accurately determined by the decision fusion and weighted output layer fusion strategy. Experimental results on four real-world HSI data sets demonstrate that the proposed frameworks outperform some classic and state-of-theart methods with high computational efficiency, which can be employed to serve real-time applications.
Deep Prototypical Networks with Hybrid Residual Attention for Hyperspectral Image Classification
Authors: Bobo Xi, Jiaojiao Li, Yunsong Li, Rui Song, Yanzi Shi, Songlin Liu, Qian Du
Date: 2020-06-25 00:00:00
Tasks: Classification, General Classification, Hyperspectral Image Classification, image-classification, Image Classification

Recently, convolutional neural networks (CNNs) have attracted enormous attention in pattern recognition and demonstrated excellent performance in hyperspectral image (HSI) classification. However, high-dimensional HSI dataset versus limited training samples is easy to cause the overfitting phenomenon in deep neural networks. Additionally, the intraclass distance of the embedding features extracted through the softmax-based CNNs may be greater than that of the interclass, which makes it difficult to further improve the classification accuracy. To address these issues, this article proposes a deep prototypical network with hybrid residual attention, which can effectively investigate the spectral-spatial information in the HSI. Specifically, in order to improve the generalization capability of the model, feature extraction with a hybrid residual attention module is presented to enhance the critical spectral-spatial features and suppress the useless ones in the classification task. Furthermore, a novel discriminant distance-based cross-entropy loss is proposed to increase the intraclass compactness, to obtain more superior results. Extensive experiments on three benchmark datasets are carried out to convincingly evaluate the proposed framework. With the generation of optimal prototypes representing each class and more discriminative embedding features, encouraging classification results are achieved compared with state-of-the-art methods.
Multimodal Emotion Recognition with Transformer-Based Self Supervised Feature Fusion
Authors: Shamane Siriwardhana ; Tharindu Kaluarachchi ; Mark Billinghurst ; Suranga Nanayakkara
Date: 2020-10-27 00:00:00
Tasks: Emotion Recognition, Multimodal Deep Learning, Multimodal Emotion Recognition, Multimodal Sentiment Analysis, Representation Learning, Self-Supervised Learning

Emotion Recognition is a challenging research area given its complex nature, and humans express emotional cues across various modalities such as language, facial expressions, and speech. Representation and fusion of features are the most crucial tasks in multimodal emotion recognition research. Self Supervised Learning (SSL) has become a prominent and influential research direction in representation learning, where researchers have access to pre-trained SSL models that represent different data modalities. For the first time in the literature, we represent three input modalities of text, audio (speech), and vision with features extracted from independently pre-trained SSL models in this paper. Given the high dimensional nature of SSL features, we introduce a novel Transformers and Attention-based fusion mechanism that can combine multimodal SSL features and achieve state-of-the-art results for the task of multimodal emotion recognition. We benchmark and evaluate our work to show that our model is robust and outperforms the state-of-the-art models on four datasets.
PowerModelsRestoration.jl: An Open-Source Framework for Exploring Power Network Restoration Algorithms
Authors: Noah Rhodes, David Fobes, Carleton Coffrin, Line Roald
Date: 2020-04-27 00:00:00

With the escalating frequency of extreme grid disturbances, such as natural disasters, comes an increasing need for efficient recovery plans. Algorithms for optimal power restoration play an important role in developing such plans, but also give rise to challenging mixed-integer nonlinear optimization problems, where tractable solution methods are not yet available. To assist in research on such solution methods, this work proposes PowerModelsRestoration, a flexible, open-source software framework for rapidly designing and testing power restoration algorithms. PowerModelsRestoration constructs a mathematical modeling layer for formalizing core restoration tasks that can be combined to develop complex workflows and high performance heuristics. The efficacy of the proposed framework is demonstrated by proof-of-concept studies on three established cases from the literature, focusing on single-phase positive sequence network models. The results demonstrate that PowerModelsRestoration reproduces the established literature, and for the first time provide an analysis of restoration with nonlinear power flow models, which have not been previously considered.
3-D Short-Range Imaging With Irregular {MIMO} Arrays Using {NUFFT-B}ased Range Migration Algorithm
Authors: J. Wang, P. Aubry and A. Yarovoy
Date: 2020-01-29 00:00:00
Tasks: Computational Efficiency

3-D imaging with irregular planar multiple-input-multiple-output (MIMO) arrays is discussed. Due to signal acquisition on irregular spatial sampling grids by using these antenna arrays, the fast Fourier transform (FFT)-based imaging algorithms cannot readily be used for image formation. To avoid the application of computationally intensive coherent summation algorithms such as filtered backprojection or Kirchhoff migration, we propose a nonuniform FFT (NUFFT)-based MIMO Range Migration Algorithm (i.e., NUFFT-based MIMO-RMA) for efficient microwave imaging. The algorithm exploits NUFFT to reconstruct the wavenumber-domain spectra related to each Fourier frequency. It is generic and applicable to 3-D imaging with irregular planar MIMO arrays. The effects of irregular spatial sampling and signal bandwidth on the imaging performance and computational efficiency of the proposed algorithm are analyzed. Finally, some numerical simulations and experimental results are presented to demonstrate the performance of the proposed imaging algorithm.
G-SimCLR: Self-Supervised Contrastive Learning with Guided Projection via Pseudo Labelling
Authors: Souradip Chakraborty, Aritra Roy Gosthipaty, Sayak Paul
Date: 2020-09-28 00:00:00
Tasks: Contrastive Learning, Denoising, Representation Learning, Self-Supervised Learning

In the realms of computer vision, it is evident that deep neural networks perform better in a supervised setting with a large amount of labeled data. The representations learned with supervision are not only of high quality but also helps the model in enhancing its accuracy. However, the collection and annotation of a large dataset are costly and time-consuming. To avoid the same, there has been a lot of research going on in the field of unsupervised visual representation learning especially in a self-supervised setting. Amongst the recent advancements in self-supervised methods for visual recognition, in SimCLR Chen et al. shows that good quality representations can indeed be learned without explicit supervision. In SimCLR, the authors maximize the similarity of augmentations of the same image and minimize the similarity of augmentations of different images. A linear classifier trained with the representations learned using this approach yields 76.5% top-1 accuracy on the ImageNet ILSVRC-2012 dataset. In this work, we propose that, with the normalized temperature-scaled cross-entropy (NT-Xent) loss function (as used in SimCLR), it is beneficial to not have images of the same category in the same batch. In an unsupervised setting, the information of images pertaining to the same category is missing. We use the latent space representation of a denoising autoencoder trained on the unlabeled dataset and cluster them with k-means to obtain pseudo labels. With this apriori information we batch images, where no two images from the same category are to be found. We report comparable performance enhancements on the CIFAR10 dataset and a subset of the ImageNet dataset. We refer to our method as G-SimCLR.
Deep Learning based Covert Attack Identification for Industrial Control Systems
Authors: Dan Li, Paritosh Ramanan, Nagi Gebraeel, Kamran Paynabar
Date: 2020-09-25 00:00:00
Tasks: Deep Learning, Time Series, Time Series Analysis

Cybersecurity of Industrial Control Systems (ICS) is drawing significant concerns as data communication increasingly leverages wireless networks. A lot of data-driven methods were developed for detecting cyberattacks, but few are focused on distinguishing them from equipment faults. In this paper, we develop a data-driven framework that can be used to detect, diagnose, and localize a type of cyberattack called covert attacks on smart grids. The framework has a hybrid design that combines an autoencoder, a recurrent neural network (RNN) with a Long-Short-Term-Memory (LSTM) layer, and a Deep Neural Network (DNN). This data-driven framework considers the temporal behavior of a generic physical system that extracts features from the time series of the sensor measurements that can be used for detecting covert attacks, distinguishing them from equipment faults, as well as localize the attack/fault. We evaluate the performance of the proposed method through a realistic simulation study on the IEEE 14-bus model as a typical example of ICS. We compare the performance of the proposed method with the traditional model-based method to show its applicability and efficacy.
Sequence-to-Sequence Load Disaggregation Using Multi-Scale Residual Neural Network
Authors: Gan Zhou, Zhi Li, Meng Fu, Yanjun Feng, Xingyao Wang, Chengwei Huang
Date: 2020-09-25 00:00:00
Tasks: Non-Intrusive Load Monitoring

With the increased demand on economy and efficiency of measurement technology, Non-Intrusive Load Monitoring (NILM) has received more and more attention as a cost-effective way to monitor electricity and provide feedback to users. Deep neural networks has been shown a great potential in the field of load disaggregation. In this paper, firstly, a new convolutional model based on residual blocks is proposed to avoid the degradation problem which traditional networks more or less suffer from when network layers are increased in order to learn more complex features. Secondly, we propose dilated convolution to curtail the excessive quantity of model parameters and obtain bigger receptive field, and multi-scale structure to learn mixed data features in a more targeted way. Thirdly, we give details about generating training and test set under certain rules. Finally, the algorithm is tested on real-house public dataset, UK-DALE, with three existing neural networks. The results are compared and analysed, the proposed model shows improvements on F1 score, MAE as well as model complexity across different appliances.
A little goes a long way: Improving toxic language classification despite data scarcity
Authors: Mika Juuti, Tommi Gröndahl, Adrian Flanagan, N. Asokan
Date: 2020-09-25 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Data Augmentation, General Classification

Detection of some types of toxic language is hampered by extreme scarcity of labeled training data. Data augmentation - generating new synthetic data from a labeled seed dataset - can help. The efficacy of data augmentation on toxic language classification has not been fully explored. We present the first systematic study on how data augmentation techniques impact performance across toxic language classifiers, ranging from shallow logistic regression architectures to BERT - a state-of-the-art pre-trained Transformer network. We compare the performance of eight techniques on very scarce seed datasets. We show that while BERT performed the best, shallow classifiers performed comparably when trained on data augmented with a combination of three techniques, including GPT-2-generated sentences. We discuss the interplay of performance and computational overhead, which can inform the choice of techniques under different constraints.
Developing FB Chatbot Based on Deep Learning Using RASA Framework for University Enquiries
Authors: Yurio Windiatmoko, Ahmad Fathan Hidayatullah, Ridho Rahmadi
Date: 2020-09-25 00:00:00
Tasks: Chatbot, Deep Learning

Smart systems for Universities powered by Artificial Intelligence have been massively developed to help humans in various tasks. The chatbot concept is not something new in today society which is developing with recent technology. College students or candidates of college students often need actual information like asking for something to customer service, especially during this pandemic, when it is difficult to have an immediate face-to-face meeting. Chatbots are functionally helping in several things such as curriculum information, admission for new students, schedule info for any lecture courses, students grade information, and some adding features for Muslim worships schedule, also weather forecast information. This Chatbot is developed by Deep Learning models, which was adopted by an artificial intelligence model that replicates human intelligence with some specific training schemes. This kind of Deep Learning is based on RNN which has some specific memory savings scheme for the Deep Learning Model, specifically this chatbot using LSTM which already integrates by RASA framework. LSTM is also known as Long Short Term Memory which efficiently saves some required memory but will remove some memory that is not needed. This Chatbot uses the FB platform because of the FB users have already reached up to 60.8% of its entire population in Indonesia. Here's the chatbot only focuses on case studies at campus of the Magister Informatics FTI University of Islamic Indonesia. This research is a first stage development within fairly sufficient simulate data.