pwc-archive/papers-with-abstracts huggingface dataset:
Wind Speed Prediction and Visualization Using Long Short-Term Memory Networks (LSTM)
Authors: Md Amimul Ehsan, Amir Shahirinia, Nian Zhang, Timothy Oladunni
Date: 2020-05-22 00:00:00
Tasks: Prediction, speed predict

Climate change is one of the most concerning issues of this century. Emission from electric power generation is a crucial factor that drives the concern to the next level. Renewable energy sources are widespread and available globally, however, one of the major challenges is to understand their characteristics in a more informative way. This paper proposes the prediction of wind speed that simplifies wind farm planning and feasibility study. Twelve artificial intelligence algorithms were used for wind speed prediction from collected meteorological parameters. The model performances were compared to determine the wind speed prediction accuracy. The results show a deep learning approach, long short-term memory (LSTM) outperforms other models with the highest accuracy of 97.8%.
Data-driven Efficient Solvers for Langevin Dynamics on Manifold in High Dimensions
Authors: Yuan Gao, Jian-Guo Liu, Nan Wu
Date: 2020-05-22 00:00:00

We study the Langevin dynamics of a physical system with manifold structure $\mathcal{M}\subset\mathbb{R}^p$ based on collected sample points $\{\mathsf{x}_i\}_{i=1}^n \subset \mathcal{M}$ that probe the unknown manifold $\mathcal{M}$. Through the diffusion map, we first learn the reaction coordinates $\{\mathsf{y}_i\}_{i=1}^n\subset \mathcal{N}$ corresponding to $\{\mathsf{x}_i\}_{i=1}^n$, where $\mathcal{N}$ is a manifold diffeomorphic to $\mathcal{M}$ and isometrically embedded in $\mathbb{R}^\ell$ with $\ell \ll p$. The induced Langevin dynamics on $\mathcal{N}$ in terms of the reaction coordinates captures the slow time scale dynamics such as conformational changes in biochemical reactions. To construct an efficient and stable approximation for the Langevin dynamics on $\mathcal{N}$, we leverage the corresponding Fokker-Planck equation on the manifold $\mathcal{N}$ in terms of the reaction coordinates $\mathsf{y}$. We propose an implementable, unconditionally stable, data-driven finite volume scheme for this Fokker-Planck equation, which automatically incorporates the manifold structure of $\mathcal{N}$. Furthermore, we provide a weighted $L^2$ convergence analysis of the finite volume scheme to the Fokker-Planck equation on $\mathcal{N}$. The proposed finite volume scheme leads to a Markov chain on $\{\mathsf{y}_i\}_{i=1}^n$ with an approximated transition probability and jump rate between the nearest neighbor points. After an unconditionally stable explicit time discretization, the data-driven finite volume scheme gives an approximated Markov process for the Langevin dynamics on $\mathcal{N}$ and the approximated Markov process enjoys detailed balance, ergodicity, and other good properties.
Customized Graph Neural Networks
Authors: Yiqi Wang, Yao Ma, Wei Jin, Chaozhuo Li, Charu Aggarwal, Jiliang Tang
Date: 2020-05-22 00:00:00
Tasks: Classification, General Classification, Graph Classification, Graph Neural Network

Recently, Graph Neural Networks (GNNs) have greatly advanced the task of graph classification. Typically, we first build a unified GNN model with graphs in a given training set and then use this unified model to predict labels of all the unseen graphs in the test set. However, graphs in the same dataset often have dramatically distinct structures, which indicates that a unified model may be sub-optimal given an individual graph. Therefore, in this paper, we aim to develop customized graph neural networks for graph classification. Specifically, we propose a novel customized graph neural network framework, i.e., Customized-GNN. Given a graph sample, Customized-GNN can generate a sample-specific model for this graph based on its structure. Meanwhile, the proposed framework is very general that can be applied to numerous existing graph neural network models. Comprehensive experiments on various graph classification benchmarks demonstrate the effectiveness of the proposed framework.
Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction
Authors: Laila Rasmy, Yang Xiang, Ziqian Xie, Cui Tao, Degui Zhi
Date: 2020-05-22 00:00:00
Tasks: Disease Prediction, Prediction

Deep learning (DL) based predictive models from electronic health records (EHR) deliver impressive performance in many clinical tasks. Large training cohorts, however, are often required to achieve high accuracy, hindering the adoption of DL-based models in scenarios with limited training data size. Recently, bidirectional encoder representations from transformers (BERT) and related models have achieved tremendous successes in the natural language processing domain. The pre-training of BERT on a very large training corpus generates contextualized embeddings that can boost the performance of models trained on smaller datasets. We propose Med-BERT, which adapts the BERT framework for pre-training contextualized embedding models on structured diagnosis data from 28,490,650 patients EHR dataset. Fine-tuning experiments are conducted on two disease-prediction tasks: (1) prediction of heart failure in patients with diabetes and (2) prediction of pancreatic cancer from two clinical databases. Med-BERT substantially improves prediction accuracy, boosting the area under receiver operating characteristics curve (AUC) by 2.02-7.12%. In particular, pre-trained Med-BERT substantially improves the performance of tasks with very small fine-tuning training sets (300-500 samples) boosting the AUC by more than 20% or equivalent to the AUC of 10 times larger training set. We believe that Med-BERT will benefit disease-prediction studies with small local training datasets, reduce data collection expenses, and accelerate the pace of artificial intelligence aided healthcare.
Do the Machine Learning Models on a Crowd Sourced Platform Exhibit Bias? An Empirical Study on Model Fairness
Authors: Sumon Biswas, Hridesh Rajan
Date: 2020-05-21 00:00:00
Tasks: Attribute, BIG-bench Machine Learning, Decision Making, Fairness, Model Optimization

Machine learning models are increasingly being used in important decision-making software such as approving bank loans, recommending criminal sentencing, hiring employees, and so on. It is important to ensure the fairness of these models so that no discrimination is made based on protected attribute (e.g., race, sex, age) while decision making. Algorithms have been developed to measure unfairness and mitigate them to a certain extent. In this paper, we have focused on the empirical evaluation of fairness and mitigations on real-world machine learning models. We have created a benchmark of 40 top-rated models from Kaggle used for 5 different tasks, and then using a comprehensive set of fairness metrics, evaluated their fairness. Then, we have applied 7 mitigation techniques on these models and analyzed the fairness, mitigation results, and impacts on performance. We have found that some model optimization techniques result in inducing unfairness in the models. On the other hand, although there are some fairness control mechanisms in machine learning libraries, they are not documented. The mitigation algorithm also exhibit common patterns such as mitigation in the post-processing is often costly (in terms of performance) and mitigation in the pre-processing stage is preferred in most cases. We have also presented different trade-off choices of fairness mitigation decisions. Our study suggests future research directions to reduce the gap between theoretical fairness aware algorithms and the software engineering methods to leverage them in practice.
Distributed Resource Scheduling for Large-Scale MEC Systems: A Multi-Agent Ensemble Deep Reinforcement Learning with Imitation Acceleration
Authors: Feibo Jiang, Li Dong, Kezhi Wang, Kun Yang, Cunhua Pan
Date: 2020-05-21 00:00:00
Tasks: Decision Making, Deep Reinforcement Learning, Edge-computing, Scheduling

We consider the optimization of distributed resource scheduling to minimize the sum of task latency and energy consumption for all the Internet of things devices (IoTDs) in a large-scale mobile edge computing (MEC) system. To address this problem, we propose a distributed intelligent resource scheduling (DIRS) framework, which includes centralized training relying on the global information and distributed decision making by each agent deployed in each MEC server. More specifically, we first introduce a novel multi-agent ensemble-assisted distributed deep reinforcement learning (DRL) architecture, which can simplify the overall neural network structure of each agent by partitioning the state space and also improve the performance of a single agent by combining decisions of all the agents. Secondly, we apply action refinement to enhance the exploration ability of the proposed DIRS framework, where the near-optimal state-action pairs are obtained by a novel L\'evy flight search. Finally, an imitation acceleration scheme is presented to pre-train all the agents, which can significantly accelerate the learning process of the proposed framework through learning the professional experience from a small amount of demonstration data. Extensive simulations are conducted to demonstrate that the proposed DIRS framework is efficient and outperforms the existing benchmark schemes.
SciSight: Combining faceted navigation and research group detection for COVID-19 exploratory scientific search
Authors: Tom Hope, Jason Portenoy, Kishore Vasan, Jonathan Borchardt, Eric Horvitz, Daniel S. Weld, Marti A. Hearst, Jevin West
Date: 2020-05-20 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Language Modelling

The COVID-19 pandemic has sparked unprecedented mobilization of scientists, generating a deluge of papers that makes it hard for researchers to keep track and explore new directions. Search engines are designed for targeted queries, not for discovery of connections across a corpus. In this paper, we present SciSight, a system for exploratory search of COVID-19 research integrating two key capabilities: first, exploring associations between biomedical facets automatically extracted from papers (e.g., genes, drugs, diseases, patient outcomes); second, combining textual and network information to search and visualize groups of researchers and their ties. SciSight has so far served over $15K$ users with over $42K$ page views and $13\%$ returns.
Estimating Differential Latent Variable Graphical Models with Applications to Brain Connectivity
Authors: Sen Na, Mladen Kolar, Oluwasanmi Koyejo
Date: 2019-09-12 00:00:00

Differential graphical models are designed to represent the difference between the conditional dependence structures of two groups, thus are of particular interest for scientific investigation. Motivated by modern applications, this manuscript considers an extended setting where each group is generated by a latent variable Gaussian graphical model. Due to the existence of latent factors, the differential network is decomposed into sparse and low-rank components, both of which are symmetric indefinite matrices. We estimate these two components simultaneously using a two-stage procedure: (i) an initialization stage, which computes a simple, consistent estimator, and (ii) a convergence stage, implemented using a projected alternating gradient descent algorithm applied to a nonconvex objective, initialized using the output of the first stage. We prove that given the initialization, the estimator converges linearly with a nontrivial, minimax optimal statistical error. Experiments on synthetic and real data illustrate that the proposed nonconvex procedure outperforms existing methods.
CERT: Contrastive Self-supervised Learning for Language Understanding
Authors: Hongchao Fang, Sicheng Wang, Meng Zhou, Jiayuan Ding, Pengtao Xie
Date: 2020-05-16 00:00:00
Tasks: Natural Language Understanding, Self-Supervised Learning, Sentence, Translation

Pretrained language models such as BERT, GPT have shown great effectiveness in language understanding. The auxiliary predictive tasks in existing pretraining approaches are mostly defined on tokens, thus may not be able to capture sentence-level semantics very well. To address this issue, we propose CERT: Contrastive self-supervised Encoder Representations from Transformers, which pretrains language representation models using contrastive self-supervised learning at the sentence level. CERT creates augmentations of original sentences using back-translation. Then it finetunes a pretrained language encoder (e.g., BERT) by predicting whether two augmented sentences originate from the same sentence. CERT is simple to use and can be flexibly plugged into any pretraining-finetuning NLP pipeline. We evaluate CERT on 11 natural language understanding tasks in the GLUE benchmark where CERT outperforms BERT on 7 tasks, achieves the same performance as BERT on 2 tasks, and performs worse than BERT on 2 tasks. On the averaged score of the 11 tasks, CERT outperforms BERT. The data and code are available at https://github.com/UCSD-AI4H/CERT
Sentiment Analysis for Education with R: packages, methods and practical applications
Authors: Michelangelo Misuraca, Alessia Forciniti, Germana Scepi, Maria Spano
Date: 2020-05-08 00:00:00
Tasks: Sentiment Analysis

Sentiment Analysis (SA) refers to a family of techniques at the crossroads of statistics, natural language processing, and computational linguistics. The primary goal is to detect the semantic orientation of individual opinions and comments expressed in written texts. There are several practical applications of SA in several domains. In an educational context, the use of this approach allows processing students' feedback, aiming at monitoring the teaching effectiveness of instructors and enhancing the learning experience. This paper wants to review the different R packages that can be used to carry on SA, comparing the implemented methods, discussing their characteristics, and showing how they perform by considering a simple example.
Personalized Early Stage Alzheimer's Disease Detection: A Case Study of President Reagan's Speeches
Authors: Ning Wang, Fan Luo, Vishal Peddagangireddy, K. P. Subbalakshmi, R. Chandramouli
Date: 2020-05-08 00:00:00
Proceeding: WS 2020 7
Tasks: Alzheimer's Disease Detection, Anomaly Detection, Clustering

Alzheimer`s disease (AD)-related global healthcare cost is estimated to be $1 trillion by 2050. Currently, there is no cure for this disease; however, clinical studies show that early diagnosis and intervention helps to extend the quality of life and inform technologies for personalized mental healthcare. Clinical research indicates that the onset and progression of Alzheimer`s disease lead to dementia and other mental health issues. As a result, the language capabilities of patient start to decline. In this paper, we show that machine learning-based unsupervised clustering of and anomaly detection with linguistic biomarkers are promising approaches for intuitive visualization and personalized early stage detection of Alzheimer`s disease. We demonstrate this approach on 10 year`s (1980 to 1989) of President Ronald Reagan`s speech data set. Key linguistic biomarkers that indicate early-stage AD are identified. Experimental results show that Reagan had early onset of Alzheimer`s sometime between 1983 and 1987. This finding is corroborated by prior work that analyzed his interviews using a statistical technique. The proposed technique also identifies the exact speeches that reflect linguistic biomarkers for early stage AD.
Inferring the source of official texts: can SVM beat ULMFiT?
Authors: Pedro Henrique Luz de Araujo, Teófilo Emidio de Campos, Marcelo Magalhães Silva de Sousa
Date: 2020-03-02 00:00:00
Proceeding: International Conference on Computational Processing of the Portuguese Language 2020 3
Tasks: Text Classification, Transfer Learning

Official Gazettes are a rich source of relevant information to the public. Their careful examination may lead to the detection of frauds and irregularities that may prevent mismanagement of public funds. This paper presents a dataset composed of documents from the Official Gazette of the Federal District, containing both samples with document source annotation and unlabeled ones. We train, evaluate and compare a transfer learning based model that uses ULMFiT with traditional bag-of-words models that use SVM and Naive Bayes as classifiers. We find the SVM to be competitive, its performance being marginally worse than the ULMFiT while having much faster train and inference time and being less computationally expensive. Finally, we conduct ablation analysis to assess the performance impact of the ULMFiT parts.
Density
Authors: Faisal Zaman, Ya Ping Wong, Boon Yian Ng
Date: 2017-02-01 00:00:00
Proceeding: 9th International Conference on Robotic, Vision, Signal Processing and Power Applications, Singapore 2017 2
Tasks: Clustering, Denoising, Density Estimation, Surface Reconstruction

Point cloud source data for surface reconstruction is usually contaminated with noise and outliers. To overcome this deﬁciency, a density-based point cloud denoising method is presented to remove outliers and noisy points. First, particle-swarm optimization technique is employed for automatically approximating optimal bandwidth of multivariate kernel density estimation to ensure the robust performance of density estimation. Then, mean-shift based clustering technique is used to remove outliers through a thresholding scheme. After removing outliers from the point cloud, bilateral mesh ﬁltering is applied to smooth the remaining points. The experimental results show that this approach, comparably, is robust and eﬃcient.
4D Visualization of Dynamic Events from Unconstrained Multi-View Videos
Authors: Aayush Bansal, Minh Vo, Yaser Sheikh, Deva Ramanan, Srinivasa Narasimhan
Date: 2020-05-27 00:00:00
Proceeding: CVPR 2020 6

We present a data-driven approach for 4D space-time visualization of dynamic events from videos captured by hand-held multiple cameras. Key to our approach is the use of self-supervised neural networks specific to the scene to compose static and dynamic aspects of an event. Though captured from discrete viewpoints, this model enables us to move around the space-time of the event continuously. This model allows us to create virtual cameras that facilitate: (1) freezing the time and exploring views; (2) freezing a view and moving through time; and (3) simultaneously changing both time and view. We can also edit the videos and reveal occluded objects for a given view if it is visible in any of the other views. We validate our approach on challenging in-the-wild events captured using up to 15 mobile cameras.
How to do Physics-based Learning
Authors: Michael Kellman, Michael Lustig, Laura Waller
Date: 2020-05-27 00:00:00

The goal of this tutorial is to explain step-by-step how to implement physics-based learning for the rapid prototyping of a computational imaging system. We provide a basic overview of physics-based learning, the construction of a physics-based network, and its reduction to practice. Specifically, we advocate exploiting the auto-differentiation functionality twice, once to build a physics-based network and again to perform physics-based learning. Thus, the user need only implement the forward model process for their system, speeding up prototyping time. We provide an open-source Pytorch implementation of a physics-based network and training procedure for a generic sparse recovery problem
On the Convergence of Gradient Descent Training for Two-layer ReLU-networks in the Mean Field Regime
Authors: Stephan Wojtowytsch
Date: 2020-05-27 00:00:00

We describe a necessary and sufficient condition for the convergence to minimum Bayes risk when training two-layer ReLU-networks by gradient descent in the mean field regime with omni-directional initial parameter distribution. This article extends recent results of Chizat and Bach to ReLU-activated networks and to the situation in which there are no parameters which exactly achieve MBR. The condition does not depend on the initalization of parameters and concerns only the weak convergence of the realization of the neural network, not its parameter distribution.
Stochastic Security: Adversarial Defense Using Long-Run Dynamics of Energy-Based Models
Authors: Mitch Hill, Jonathan Mitchell, Song-Chun Zhu
Date: 2020-05-27 00:00:00
Proceeding: ICLR 2021 1
Tasks: Adversarial Defense, Adversarial Purification, Robust classification

The vulnerability of deep networks to adversarial attacks is a central problem for deep learning from the perspective of both cognition and security. The current most successful defense method is to train a classifier using adversarial images created during learning. Another defense approach involves transformation or purification of the original input to remove adversarial signals before the image is classified. We focus on defending naturally-trained classifiers using Markov Chain Monte Carlo (MCMC) sampling with an Energy-Based Model (EBM) for adversarial purification. In contrast to adversarial training, our approach is intended to secure pre-existing and highly vulnerable classifiers. The memoryless behavior of long-run MCMC sampling will eventually remove adversarial signals, while metastable behavior preserves consistent appearance of MCMC samples after many steps to allow accurate long-run prediction. Balancing these factors can lead to effective purification and robust classification. We evaluate adversarial defense with an EBM using the strongest known attacks against purification. Our contributions are 1) an improved method for training EBM's with realistic long-run MCMC samples, 2) an Expectation-Over-Transformation (EOT) defense that resolves theoretical ambiguities for stochastic defenses and from which the EOT attack naturally follows, and 3) state-of-the-art adversarial defense for naturally-trained classifiers and competitive defense compared to adversarially-trained classifiers on Cifar-10, SVHN, and Cifar-100. Code and pre-trained models are available at https://github.com/point0bar1/ebm-defense.
NARMADA: Need and Available Resource Managing Assistant for Disasters and Adversities
Authors: Kaustubh Hiware, Ritam Dutt, Sayan Sinha, Sohan Patro, Kripabandhu Ghosh, Saptarshi Ghosh
Date: 2020-05-27 00:00:00
Proceeding: WS 2020 7
Tasks: Information Retrieval, Management, Retrieval

Although a lot of research has been done on utilising Online Social Media during disasters, there exists no system for a specific task that is critical in a post-disaster scenario -- identifying resource-needs and resource-availabilities in the disaster-affected region, coupled with their subsequent matching. To this end, we present NARMADA, a semi-automated platform which leverages the crowd-sourced information from social media posts for assisting post-disaster relief coordination efforts. The system employs Natural Language Processing and Information Retrieval techniques for identifying resource-needs and resource-availabilities from microblogs, extracting resources from the posts, and also matching the needs to suitable availabilities. The system is thus capable of facilitating the judicious management of resources during post-disaster relief operations.