pwc-archive/papers-with-abstracts huggingface dataset:
Automatic Opioid User Detection From Twitter: Transductive Ensemble Built On Different Meta-graph Based Similarities Over Heterogeneous Information Network
Authors: Yujie Fan, Yiming Zhang, Yanfang Y e∗, Xin Li
Date: 2018-07-01 00:00:00

Opioid (e.g., heroin and morphine) addiction has
become one of the largest and deadliest epidemics
in the United States. To combat such deadly epi-
demic, in this paper, we propose a novel frame-
work named HinOPU to automatically detect opi-
oid users from Twitter, which will assist in sharp-
ening our understanding toward the behavioral pro-
cess of opioid addiction and treatment. In HinOPU,
to model the users and the posted tweets as well
as their rich relationships, we introduce structured
heterogeneous information network (HIN) for rep-
resentation. Afterwards, we use meta-graph based
approach to characterize the semantic relatedness
over users; we then formulate different similari-
ties over users based on different meta-graphs on
HIN. To reduce the cost of acquiring labeled sam-
ples for supervised learning, we propose a trans-
ductive classification method to build the base clas-
sifiers based on different similarities formulated by
different meta-graphs. Then, to further improve the
detection accuracy, we construct an ensemble to
combine different predictions from different base
classifiers for opioid user detection. Comprehen-
sive experiments on real sample collections from
Twitter are conducted to validate the effectiveness
of HinOPU in opioid user detection by compar-
isons with other alternate methods.
New Approaches to Robust Inference on Market (Non-)Efficiency, Volatility Clustering and Nonlinear Dependence
Authors: Rustam Ibragimov, Rasmus Pedersen, Anton Skrobotov
Date: 2020-06-01 00:00:00
Tasks: Clustering, Time Series, Time Series Analysis, valid

Many financial and economic variables, including financial returns, exhibit nonlinear dependence, heterogeneity and heavy-tailedness. These properties may make problematic the analysis of (non-)efficiency and volatility clustering in economic and financial markets using traditional approaches that appeal to asymptotic normality of sample autocorrelation functions of returns and their squares. This paper presents new approaches to deal with the above problems. We provide the results that motivate the use of measures of market (non-)efficiency and volatility clustering based on (small) powers of absolute returns and their signed versions. We further provide new approaches to robust inference on the measures in the case of general time series, including GARCH-type processes. The approaches are based on robust $t-$statistics tests and new results on their applicability are presented. In the approaches, parameter estimates (e.g., estimates of measures of nonlinear dependence) are computed for groups of data, and the inference is based on $t-$statistics in the resulting group estimates. This results in valid robust inference under heterogeneity and dependence assumptions satisfied in real-world financial markets. Numerical results and empirical applications confirm the advantages and wide applicability of the proposed approaches.
A Plug-and-Play Priors Framework for Hyperspectral Unmixing
Authors: Min Zhao, Xiuheng Wang, Jie Chen, Wei Chen
Date: 2020-12-24 00:00:00
Tasks: Denoising, Hyperspectral Unmixing, Image Denoising

Spectral unmixing is a widely used technique in hyperspectral image processing and analysis. It aims to separate mixed pixels into the component materials and their corresponding abundances. Early solutions to spectral unmixing are performed independently on each pixel. Nowadays, investigating proper priors into the unmixing problem has been popular as it can significantly enhance the unmixing performance. However, it is non-trivial to handcraft a powerful regularizer, and complex regularizers may introduce extra difficulties in solving optimization problems in which they are involved. To address this issue, we present a plug-and-play (PnP) priors framework for hyperspectral unmixing. More specifically, we use the alternating direction method of multipliers (ADMM) to decompose the optimization problem into two iterative subproblems. One is a regular optimization problem depending on the forward model, and the other is a proximity operator related to the prior model and can be regarded as an image denoising problem. Our framework is flexible and extendable which allows a wide range of denoisers to replace prior models and avoids handcrafting regularizers. Experiments conducted on both synthetic data and real airborne data illustrate the superiority of the proposed strategy compared with other state-of-the-art hyperspectral unmixing methods.
The NRGTEN Python package: an extensible toolkit for coarse-grained normal mode analysis of proteins, nucleic acids, small molecules and their complexes
Authors: Olivier Mailhot, Rafael Najmanovich
Date: 2020-12-24 00:00:00

Summary: Coarse-grained normal mode analysis (NMA) is a fast computational technique to study the dynamics of biomolecules. Here we present the Najmanovich Research Group Toolkit for Elastic Networks (NRGTEN). NRGTEN is a Python toolkit that implements four different NMA models in addition to popular and novel metrics to benchmark and measure properties from these models. Furthermore, the toolkit is available as a public Python package and is easily extensible for the development or implementation of additional NMA models. The inclusion of the ENCoM model (Elastic Network Contact Model) developed in our group within NRGTEN is noteworthy, owing to its account for the specific chemical nature of atomic interactions. This makes possible some unique predictions of the effect of mutations, such as on stability (via changes in vibrational entropy differences), on the transition probability between different conformational states or on the flexibility profile of the whole macromolecule/complex (to study allostery and signalling). In addition, all NMA models can be used to generate conformational ensembles from a starting structure to aid in protein-protein, protein-ligand or other docking studies among applications. NRGTEN is freely available via a public Python package which can be easily installed on any modern machine and includes a detailed user guide hosted online. Availability and implementation: https://github.com/gregorpatof/nrgten_package/ Contact: rafael.najmanovich@umontreal.ca
Stochastic Pre-Event Preparation for Enhancing Resilience of Distribution Systems with High DER Penetration
Authors: Qianzhi Zhang, Zhaoyu Wang, Shanshan Ma, Anmar Arif
Date: 2020-12-24 00:00:00
Tasks: Scheduling

This paper proposes a stochastic optimal preparation and resource allocation method for upcoming extreme weather events in distribution systems, which can assist utilities to achieve faster and more efficient post-event restoration. With the objective of maximizing served load and minimizing operation cost, this paper develops a two-stage stochastic mixed-integer linear programming (SMILP) model. The first-stage determines the optimal positions and numbers of mobile resources, fuel resources, and labor resources. The second-stage considers network operational constraints and repair crew scheduling constraints. The proposed stochastic pre-event preparation model is solved by a scenario decomposition method, Progressive Hedging (PH), to ease the computational complexity introduced by a large number of scenarios. Furthermore, to show the impact of solar photovoltaic (PV) generation on system resilience, we consider three types of PV systems during power outage and compare the resilience improvements with different PV penetration levels. Numerical results from simulations on a large-scale (more than 10,000 nodes) distribution feeder have been used to validate the scalability and effectiveness of the proposed method.
Quickest Detection over Sensor Networks with Unknown Post-Change Distribution
Authors: Deniz Sargun, C. Emre Koksal
Date: 2020-12-23 00:00:00
Tasks: Change Detection

We propose a quickest change detection problem over sensor networks where both the subset of sensors undergoing a change and the local post-change distributions are unknown. Each sensor in the network observes a local discrete time random process over a finite alphabet. Initially, the observations are independent and identically distributed (i.i.d.) with known pre-change distributions independent from other sensors. At a fixed but unknown change point, a fixed but unknown subset of the sensors undergo a change and start observing samples from an unknown distribution. We assume the change can be quantified using concave (or convex) local statistics over the space of distributions. We propose an asymptotically optimal and computationally tractable stopping time for Lorden's criterion. Under this scenario, our proposed method uses a concave global cumulative sum (CUSUM) statistic at the fusion center and suppresses the most likely false alarms using information projection. Finally, we show some numerical results of the simulation of our algorithm for the problem described.
Federated Unlearning
Authors: Gaoyang Liu, Xiaoqiang Ma, Yang Yang, Chen Wang, Jiangchuan Liu
Date: 2020-12-27 00:00:00
Tasks: Data Poisoning, Federated Learning

Federated learning (FL) has recently emerged as a promising distributed machine learning (ML) paradigm. Practical needs of the "right to be forgotten" and countering data poisoning attacks call for efficient techniques that can remove, or unlearn, specific training data from the trained FL model. Existing unlearning techniques in the context of ML, however, are no longer in effect for FL, mainly due to the inherent distinction in the way how FL and ML learn from data. Therefore, how to enable efficient data removal from FL models remains largely under-explored. In this paper, we take the first step to fill this gap by presenting FedEraser, the first federated unlearning methodology that can eliminate the influence of a federated client's data on the global FL model while significantly reducing the time used for constructing the unlearned FL model.The basic idea of FedEraser is to trade the central server's storage for unlearned model's construction time, where FedEraser reconstructs the unlearned model by leveraging the historical parameter updates of federated clients that have been retained at the central server during the training process of FL. A novel calibration method is further developed to calibrate the retained updates, which are further used to promptly construct the unlearned model, yielding a significant speed-up to the reconstruction of the unlearned model while maintaining the model efficacy. Experiments on four realistic datasets demonstrate the effectiveness of FedEraser, with an expected speed-up of $4\times$ compared with retraining from the scratch. We envision our work as an early step in FL towards compliance with legal and ethical criteria in a fair and transparent manner.
Learning to predict synchronization of coupled oscillators on randomly generated graphs
Authors: Hardeep Bassi, Richard Yim, Rohith Kodukula, Joshua Vendrow, Cherlin Zhu, Hanbaek Lyu
Date: 2020-12-28 00:00:00

Suppose we are given a system of coupled oscillators on an unknown graph along with the trajectory of the system during some period. Can we predict whether the system will eventually synchronize? Even with a known underlying graph structure, this is an important yet analytically intractable question in general. In this work, we take an alternative approach to the synchronization prediction problem by viewing it as a classification problem based on the fact that any given system will eventually synchronize or converge to a non-synchronizing limit cycle. By only using some basic statistics of the underlying graphs such as edge density and diameter, our method can achieve perfect accuracy when there is a significant difference in the topology of the underlying graphs between the synchronizing and the non-synchronizing examples. However, in the problem setting where these graph statistics cannot distinguish the two classes very well (e.g., when the graphs are generated from the same random graph model), we find that pairing a few iterations of the initial dynamics along with the graph statistics as the input to our classification algorithms can lead to significant improvement in accuracy; far exceeding what is known by the classical oscillator theory. More surprisingly, we find that in almost all such settings, dropping out the basic graph statistics and training our algorithms with only initial dynamics achieves nearly the same accuracy. We demonstrate our method on three models of continuous and discrete coupled oscillators -- the Kuramoto model, Firefly Cellular Automata, and Greenberg-Hastings model. Finally, we also propose an "ensemble prediction" algorithm that successfully scales our method to large graphs by training on dynamics observed from multiple random subgraphs.
Inserting Information Bottlenecks for Attribution in Transformers
Authors: Zhiying Jiang, Raphael Tang, Ji Xin, Jimmy Lin
Date: 2020-12-27 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020

Pretrained transformers achieve the state of the art across tasks in natural language processing, motivating researchers to investigate their inner mechanisms. One common direction is to understand what features are important for prediction. In this paper, we apply information bottlenecks to analyze the attribution of each feature for prediction on a black-box model. We use BERT as the example and evaluate our approach both quantitatively and qualitatively. We show the effectiveness of our method in terms of attribution and the ability to provide insight into how information flows through layers. We demonstrate that our technique outperforms two competitive methods in degradation tests on four datasets. Code is available at https://github.com/bazingagin/IBA.
MeDAL: Medical Abbreviation Disambiguation Dataset for Natural Language Understanding Pretraining
Authors: Zhi Wen, Xing Han Lu, Siva Reddy
Date: 2020-12-27 00:00:00
Proceeding: EMNLP (ClinicalNLP) 2020 11
Tasks: Mortality Prediction, Natural Language Understanding

One of the biggest challenges that prohibit the use of many current NLP methods in clinical settings is the availability of public datasets. In this work, we present MeDAL, a large medical text dataset curated for abbreviation disambiguation, designed for natural language understanding pre-training in the medical domain. We pre-trained several models of common architectures on this dataset and empirically showed that such pre-training leads to improved performance and convergence speed when fine-tuning on downstream medical tasks.
A method to integrate and classify normal distributions
Authors: Abhranil Das, Wilson S Geisler
Date: 2020-12-23 00:00:00
Tasks: Dimensionality Reduction

Univariate and multivariate normal probability distributions are widely used when modeling decisions under uncertainty. Computing the performance of such models requires integrating these distributions over specific domains, which can vary widely across models. Besides some special cases, there exist no general analytical expressions, standard numerical methods or software for these integrals. Here we present mathematical results and open-source software that provide (i) the probability in any domain of a normal in any dimensions with any parameters, (ii) the probability density, cumulative distribution, and inverse cumulative distribution of any function of a normal vector, (iii) the classification errors among any number of normal distributions, the Bayes-optimal discriminability index and relation to the operating characteristic, (iv) dimension reduction and visualizations for such problems, and (v) tests for how reliably these methods may be used on given data. We demonstrate these tools with vision research applications of detecting occluding objects in natural scenes, and detecting camouflage.
Instance Segmentation of Industrial Point Cloud Data
Authors: Eva Agapaki, Ioannis Brilakis
Date: 2020-12-24 00:00:00
Tasks: Instance Segmentation, Segmentation, Semantic Segmentation

The challenge that this paper addresses is how to efficiently minimize the cost and manual labour for automatically generating object oriented geometric Digital Twins (gDTs) of industrial facilities, so that the benefits provide even more value compared to the initial investment to generate these models. Our previous work achieved the current state-of-the-art class segmentation performance (75% average accuracy per point and average AUC 90% in the CLOI dataset classes) as presented in (Agapaki and Brilakis 2020) and directly produces labelled point clusters of the most important to model objects (CLOI classes) from laser scanned industrial data. CLOI stands for C-shapes, L-shapes, O-shapes, I-shapes and their combinations. However, the problem of automated segmentation of individual instances that can then be used to fit geometric shapes remains unsolved. We argue that the use of instance segmentation algorithms has the theoretical potential to provide the output needed for the generation of gDTs. We solve instance segmentation in this paper through (a) using a CLOI-Instance graph connectivity algorithm that segments the point clusters of an object class into instances and (b) boundary segmentation of points that improves step (a). Our method was tested on the CLOI benchmark dataset (Agapaki et al. 2019) and segmented instances with 76.25% average precision and 70% average recall per point among all classes. This proved that it is the first to automatically segment industrial point cloud shapes with no prior knowledge other than the class point label and is the bedrock for efficient gDT generation in cluttered industrial point clouds.
Towards Real-World BCI: CCSPNet, A Compact Subject-Independent Motor Imagery Framework
Authors: Mahbod Nouri, Faraz Moradi, Hafez Ghaemi, Ali Motie Nasrabadi
Date: 2020-12-25 00:00:00
Tasks: Brain Computer Interface, Edge-computing, EEG, Electroencephalogram (EEG), Motor Imagery

A conventional brain-computer interface (BCI) requires a complete data gathering, training, and calibration phase for each user before it can be used. In recent years, a number of subject-independent (SI) BCIs have been developed. Many of these methods yield a weaker performance compared to the subject-dependent (SD) approach, and some are computationally expensive. A potential real-world application would greatly benefit from a more accurate, compact, and computationally efficient subject-independent BCI. In this work, we propose a novel subject-independent BCI framework, named CCSPNet (Convolutional Common Spatial Pattern Network) that is trained on the motor imagery (MI) paradigm of a large-scale electroencephalography (EEG) signals database consisting of 400 trials for every 54 subjects who perform two-class hand-movement MI tasks. The proposed framework applies a wavelet kernel convolutional neural network (WKCNN) and a temporal convolutional neural network (TCNN) in order to represent and extract the spectral features of EEG signals. A common spatial pattern (CSP) algorithm is implemented for spatial feature extraction, and the number of CSP features is reduced by a dense neural network. Finally, the class label is determined by a linear discriminant analysis (LDA) classifier. The CCSPNet evaluation results show that it is possible to have a compact BCI that achieves both SD and SI state-of-the-art performance comparable to complex and computationally expensive models.
ALP-KD: Attention-Based Layer Projection for Knowledge Distillation
Authors: Peyman Passban, Yimeng Wu, Mehdi Rezagholizadeh, Qun Liu
Date: 2020-12-27 00:00:00
Tasks: Knowledge Distillation

Knowledge distillation is considered as a training and compression strategy in which two neural networks, namely a teacher and a student, are coupled together during training. The teacher network is supposed to be a trustworthy predictor and the student tries to mimic its predictions. Usually, a student with a lighter architecture is selected so we can achieve compression and yet deliver high-quality results. In such a setting, distillation only happens for final predictions whereas the student could also benefit from teacher's supervision for internal components. Motivated by this, we studied the problem of distillation for intermediate layers. Since there might not be a one-to-one alignment between student and teacher layers, existing techniques skip some teacher layers and only distill from a subset of them. This shortcoming directly impacts quality, so we instead propose a combinatorial technique which relies on attention. Our model fuses teacher-side information and takes each layer's significance into consideration, then performs distillation between combined teacher layers and those of the student. Using our technique, we distilled a 12-layer BERT (Devlin et al. 2019) into 6-, 4-, and 2-layer counterparts and evaluated them on GLUE tasks (Wang et al. 2018). Experimental results show that our combinatorial approach is able to outperform other existing techniques.
Stochastic Approximation for Online Tensorial Independent Component Analysis
Authors: Chris Junchi Li, Michael I. Jordan
Date: 2020-12-28 00:00:00
Tasks: Dimensionality Reduction

Independent component analysis (ICA) has been a popular dimension reduction tool in statistical machine learning and signal processing. In this paper, we present a convergence analysis for an online tensorial ICA algorithm, by viewing the problem as a nonconvex stochastic approximation problem. For estimating one component, we provide a dynamics-based analysis to prove that our online tensorial ICA algorithm with a specific choice of stepsize achieves a sharp finite-sample error bound. In particular, under a mild assumption on the data-generating distribution and a scaling condition such that $d^4/T$ is sufficiently small up to a polylogarithmic factor of data dimension $d$ and sample size $T$, a sharp finite-sample error bound of $\tilde{O}(\sqrt{d/T})$ can be obtained.
Latent space models for multiplex networks with shared structure
Authors: Peter W. MacDonald, Elizaveta Levina, Ji Zhu
Date: 2020-12-28 00:00:00
Tasks: Stochastic Block Model

Latent space models are frequently used for modeling single-layer networks and include many popular special cases, such as the stochastic block model and the random dot product graph. However, they are not well-developed for more complex network structures, which are becoming increasingly common in practice. Here we propose a new latent space model for multiplex networks: multiple, heterogeneous networks observed on a shared node set. Multiplex networks can represent a network sample with shared node labels, a network evolving over time, or a network with multiple types of edges. The key feature of our model is that it learns from data how much of the network structure is shared between layers and pools information across layers as appropriate. We establish identifiability, develop a fitting procedure using convex optimization in combination with a nuclear norm penalty, and prove a guarantee of recovery for the latent positions as long as there is sufficient separation between the shared and the individual latent subspaces. We compare the model to competing methods in the literature on simulated networks and on a multiplex network describing the worldwide trade of agricultural products.