pwc-archive/papers-with-abstracts huggingface dataset:
Binary Black-box Evasion Attacks Against Deep Learning-based Static Malware Detectors with Adversarial Byte-Level Language Model
Authors: MohammadReza Ebrahimi, Ning Zhang, James Hu, Muhammad Taqi Raza, Hsinchun Chen
Date: 2020-12-14 00:00:00
Tasks: Deep Learning, Feature Engineering, Language Modeling, Language Modelling, Malware Analysis

Anti-malware engines are the first line of defense against malicious software. While widely used, feature engineering-based anti-malware engines are vulnerable to unseen (zero-day) attacks. Recently, deep learning-based static anti-malware detectors have achieved success in identifying unseen attacks without requiring feature engineering and dynamic analysis. However, these detectors are susceptible to malware variants with slight perturbations, known as adversarial examples. Generating effective adversarial examples is useful to reveal the vulnerabilities of such systems. Current methods for launching such attacks require accessing either the specifications of the targeted anti-malware model, the confidence score of the anti-malware response, or dynamic malware analysis, which are either unrealistic or expensive. We propose MalRNN, a novel deep learning-based approach to automatically generate evasive malware variants without any of these restrictions. Our approach features an adversarial example generation process, which learns a language model via a generative sequence-to-sequence recurrent neural network to augment malware binaries. MalRNN effectively evades three recent deep learning-based malware detectors and outperforms current benchmark methods. Findings from applying our MalRNN on a real dataset with eight malware categories are discussed.
Primer AI's Systems for Acronym Identification and Disambiguation
Authors: Nicholas Egan, John Bohannon
Date: 2020-12-14 00:00:00
Tasks: document understanding, Sentence, Sentence Embeddings, TAG

The prevalence of ambiguous acronyms make scientific documents harder to understand for humans and machines alike, presenting a need for models that can automatically identify acronyms in text and disambiguate their meaning. We introduce new methods for acronym identification and disambiguation: our acronym identification model projects learned token embeddings onto tag predictions, and our acronym disambiguation model finds training examples with similar sentence embeddings as test examples. Both of our systems achieve significant performance gains over previously suggested methods, and perform competitively on the SDU@AAAI-21 shared task leaderboard. Our models were trained in part on new distantly-supervised datasets for these tasks which we call AuxAI and AuxAD. We also identified a duplication conflict issue in the SciAD dataset, and formed a deduplicated version of SciAD that we call SciAD-dedupe. We publicly released all three of these datasets, and hope that they help the community make further strides in scientific document understanding.
Friedrichs Learning: Weak Solutions of Partial Differential Equations via Deep Learning
Authors: Fan Chen, Jianguo Huang, Chunmei Wang, Haizhao Yang
Date: 2020-12-15 00:00:00

This paper proposes Friedrichs learning as a novel deep learning methodology that can learn the weak solutions of PDEs via a minmax formulation, which transforms the PDE problem into a minimax optimization problem to identify weak solutions. The name "Friedrichs learning" is for highlighting the close relationship between our learning strategy and Friedrichs theory on symmetric systems of PDEs. The weak solution and the test function in the weak formulation are parameterized as deep neural networks in a mesh-free manner, which are alternately updated to approach the optimal solution networks approximating the weak solution and the optimal test function, respectively. Extensive numerical results indicate that our mesh-free method can provide reasonably good solutions to a wide range of PDEs defined on regular and irregular domains in various dimensions, where classical numerical methods such as finite difference methods and finite element methods may be tedious or difficult to be applied.
Applications of multivariate quasi-random sampling with neural networks
Authors: Marius Hofert, Avinash Prasad, Mu Zhu
Date: 2020-12-15 00:00:00

Generative moment matching networks (GMMNs) are suggested for modeling the cross-sectional dependence between stochastic processes. The stochastic processes considered are geometric Brownian motions and ARMA-GARCH models. Geometric Brownian motions lead to an application of pricing American basket call options under dependence and ARMA-GARCH models lead to an application of simulating predictive distributions. In both types of applications the benefit of using GMMNs in comparison to parametric dependence models is highlighted and the fact that GMMNs can produce dependent quasi-random samples with no additional effort is exploited to obtain variance reduction.
Chernoff Sampling for Active Testing and Extension to Active Regression
Authors: Subhojyoti Mukherjee, Ardhendu Tripathy, Robert Nowak
Date: 2020-12-15 00:00:00
Tasks: Active Learning, Experimental Design, regression

Active learning can reduce the number of samples needed to perform a hypothesis test and to estimate the parameters of a model. In this paper, we revisit the work of Chernoff that described an asymptotically optimal algorithm for performing a hypothesis test. We obtain a novel sample complexity bound for Chernoff's algorithm, with a non-asymptotic term that characterizes its performance at a fixed confidence level. We also develop an extension of Chernoff sampling that can be used to estimate the parameters of a wide variety of models and we obtain a non-asymptotic bound on the estimation error. We apply our extension of Chernoff sampling to actively learn neural network models and to estimate parameters in real-data linear and non-linear regression problems, where our approach performs favorably to state-of-the-art methods.
Detecting and Adapting to Irregular Distribution Shifts in Bayesian Online Learning
Authors: Aodong Li, Alex Boyd, Padhraic Smyth, Stephan Mandt
Date: 2020-12-15 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Autonomous Navigation, Change Point Detection

We consider the problem of online learning in the presence of distribution shifts that occur at an unknown rate and of unknown intensity. We derive a new Bayesian online inference approach to simultaneously infer these distribution shifts and adapt the model to the detected changes by integrating ideas from change point detection, switching dynamical systems, and Bayesian online learning. Using a binary 'change variable,' we construct an informative prior such that--if a change is detected--the model partially erases the information of past model updates by tempering to facilitate adaptation to the new data distribution. Furthermore, the approach uses beam search to track multiple change-point hypotheses and selects the most probable one in hindsight. Our proposed method is model-agnostic, applicable in both supervised and unsupervised learning settings, suitable for an environment of concept drifts or covariate drifts, and yields improvements over state-of-the-art Bayesian online learning approaches.
Enriched Annotations for Tumor Attribute Classification from Pathology Reports with Limited Labeled Data
Authors: Nick Altieri, Briton Park, Mara Olson, John DeNero, Anobel Odisho, Bin Yu
Date: 2020-12-15 00:00:00
Tasks: Attribute, General Classification, Interpretable Machine Learning

Precision medicine has the potential to revolutionize healthcare, but much of the data for patients is locked away in unstructured free-text, limiting research and delivery of effective personalized treatments. Generating large annotated datasets for information extraction from clinical notes is often challenging and expensive due to the high level of expertise needed for high quality annotations. To enable natural language processing for small dataset sizes, we develop a novel enriched hierarchical annotation scheme and algorithm, Supervised Line Attention (SLA), and apply this algorithm to predicting categorical tumor attributes from kidney and colon cancer pathology reports from the University of California San Francisco (UCSF). Whereas previous work only annotated document level labels, we in addition ask the annotators to enrich the traditional label by asking them to also highlight the relevant line or potentially lines for the final label, which leads to a 20% increase of annotation time required per document. With the enriched annotations, we develop a simple and interpretable machine learning algorithm that first predicts the relevant lines in the document and then predicts the tumor attribute. Our results show across the small dataset sizes of 32, 64, 128, and 186 labeled documents per cancer, SLA only requires half the number of labeled documents as state-of-the-art methods to achieve similar or better micro-f1 and macro-f1 scores for the vast majority of comparisons that we made. Accounting for the increased annotation time, this leads to a 40% reduction in total annotation time over the state of the art.
LSTM-based Space Occupancy Prediction towards Efficient Building Energy Management
Authors: Juye Kim
Date: 2020-12-15 00:00:00
Tasks: energy management, Management, Prediction, Time Series Analysis

Energy consumed in buildings takes significant portions of the total global energy usage. A large amount of building energy is used for heating, cooling, ventilation, and air-conditioning (HVAC). However, compared to its importance, building energy management systems nowadays are limited in controlling HVAC based on simple rule-based control (RBC) technologies. The ability to design systems that can efficiently manage HVAC can reduce energy usage and greenhouse gas emissions, and, all in all, it can help us to mitigate climate change. This paper proposes predictive time-series models of occupancy patterns using LSTM. Prediction signal for future room occupancy status on the next time span (e.g., next 30 minutes) can be directly used to operate HVAC. For example, based on the prediction and considering the time for cooling or heating, HVAC can be turned on before the room is being used (e.g., turn on 10 minutes earlier). Also, based on the next room empty prediction timing, HVAC can be turned off earlier, and it can help us increase the efficiency of HVAC while not decreasing comfort. We demonstrate our approach's capabilities using real-world energy data collected from multiple rooms of a university building. We show that LSTM's room occupancy prediction based HVAC control could save energy usage by 50% compared to conventional RBC based control.
Inference of Causal Effects when Control Variables are Unknown
Authors: Ludvig Hult, Dave Zachariah
Date: 2020-12-15 00:00:00
Tasks: valid

Conventional methods in causal effect inferencetypically rely on specifying a valid set of control variables. When this set is unknown or misspecified, inferences will be erroneous. We propose a method for inferring average causal effects when all potential confounders are observed, but thecontrol variables are unknown. When the data-generating process belongs to the class of acyclical linear structural causal models, we prove that themethod yields asymptotically valid confidence intervals. Our results build upon a smooth characterization of linear directed acyclic graphs. We verify the capability of the method to produce valid confidence intervals for average causal effects using synthetic data, even when the appropriate specification of control variables is unknown.
On simplification of Dual-Youla approach for closed-loop identification
Authors: Toshiharu Sugie, Ichiro Maruta
Date: 2020-12-15 00:00:00

The dual Youla method for closed loop identification is known to have several practically important merits. Namely, it provides an accurate plant model irrespective of noise models, and fits inherently to handle unstable plants by using coprime factorization. In addition, the method is empirically robust against the uncertainty of the controller knowledge. However, use of coprime factorization may cause a big barrier against industrial applications. This paper shows how to derive a simplified version of the method which identifies the plant itself without coprime factorization, while enjoying all the merits of the dual Youla method. This simplified version turns out to be identical to the stabilized prediction error method which was proposed by the authors recently. Detailed simulation results are given to demonstrate the above merits.
Towards open and expandable cognitive AI architectures for large-scale multi-agent human-robot collaborative learning
Authors: Georgios Th. Papadopoulos, Margherita Antona, Constantine Stephanidis
Date: 2020-12-15 00:00:00
Tasks: Federated Learning, Self-Learning

Learning from Demonstration (LfD) constitutes one of the most robust methodologies for constructing efficient cognitive robotic systems. Despite the large body of research works already reported, current key technological challenges include those of multi-agent learning and long-term autonomy. Towards this direction, a novel cognitive architecture for multi-agent LfD robotic learning is introduced, targeting to enable the reliable deployment of open, scalable and expandable robotic systems in large-scale and complex environments. In particular, the designed architecture capitalizes on the recent advances in the Artificial Intelligence (AI) field, by establishing a Federated Learning (FL)-based framework for incarnating a multi-human multi-robot collaborative learning environment. The fundamental conceptualization relies on employing multiple AI-empowered cognitive processes (implementing various robotic tasks) that operate at the edge nodes of a network of robotic platforms, while global AI models (underpinning the aforementioned robotic tasks) are collectively created and shared among the network, by elegantly combining information from a large number of human-robot interaction instances. Regarding pivotal novelties, the designed cognitive architecture a) introduces a new FL-based formalism that extends the conventional LfD learning paradigm to support large-scale multi-agent operational settings, b) elaborates previous FL-based self-learning robotic schemes so as to incorporate the human in the learning loop and c) consolidates the fundamental principles of FL with additional sophisticated AI-enabled learning methodologies for modelling the multi-level inter-dependencies among the robotic tasks. The applicability of the proposed framework is explained using an example of a real-world industrial case study for agile production-based Critical Raw Materials (CRM) recovery.
Machine Learning for automatic identification of new minor species
Authors: Frederic Schmidt, Guillaume Cruz Mermy, Justin Erwin, Severine Robert, Lori Neary, Ian R. Thomas, Frank Daerden, Bojan Ristic, Manish R. Patel, Giancarlo Bellucci, Jose-Juan Lopez-Moreno, Ann-Carine Vandaele
Date: 2020-12-15 00:00:00
Tasks: BIG-bench Machine Learning

One of the main difficulties to analyze modern spectroscopic datasets is due to the large amount of data. For example, in atmospheric transmittance spectroscopy, the solar occultation channel (SO) of the NOMAD instrument onboard the ESA ExoMars2016 satellite called Trace Gas Orbiter (TGO) had produced $\sim$10 millions of spectra in 20000 acquisition sequences since the beginning of the mission in April 2018 until 15 January 2020. Other datasets are even larger with $\sim$billions of spectra for OMEGA onboard Mars Express or CRISM onboard Mars Reconnaissance Orbiter. Usually, new lines are discovered after a long iterative process of model fitting and manual residual analysis. Here we propose a new method based on unsupervised machine learning, to automatically detect new minor species. Although precise quantification is out of scope, this tool can also be used to quickly summarize the dataset, by giving few endmembers ("source") and their abundances. We approximate the dataset non-linearity by a linear mixture of abundance and source spectra (endmembers). We used unsupervised source separation in form of non-negative matrix factorization to estimate those quantities. Several methods are tested on synthetic and simulation data. Our approach is dedicated to detect minor species spectra rather than precisely quantifying them. On synthetic example, this approach is able to detect chemical compounds present in form of 100 hidden spectra out of $10^4$, at 1.5 times the noise level. Results on simulated spectra of NOMAD-SO targeting CH$_{4}$ show that detection limits goes in the range of 100-500 ppt in favorable conditions. Results on real martian data from NOMAD-SO show that CO$_{2}$ and H$_{2}$O are present, as expected, but CH$_{4}$ is absent. Nevertheless, we confirm a set of new unexpected lines in the database, attributed by ACS instrument Team to the CO$_{2}$ magnetic dipole.
Machine Learning for MU-MIMO Receive Processing in OFDM Systems
Authors: Mathieu Goutay, Fayçal Ait Aoudia, Jakob Hoydis, Jean-Marie Gorce
Date: 2020-12-15 00:00:00
Tasks: BIG-bench Machine Learning

Machine learning (ML) starts to be widely used to enhance the performance of multi-user multiple-input multiple-output (MU-MIMO) receivers. However, it is still unclear if such methods are truly competitive with respect to conventional methods in realistic scenarios and under practical constraints. In addition to enabling accurate signal reconstruction on realistic channel models, MU-MIMO receive algorithms must allow for easy adaptation to a varying number of users without the need for retraining. In contrast to existing work, we propose an ML-enhanced MU-MIMO receiver that builds on top of a conventional linear minimum mean squared error (LMMSE) architecture. It preserves the interpretability and scalability of the LMMSE receiver, while improving its accuracy in two ways. First, convolutional neural networks (CNNs) are used to compute an approximation of the second-order statistics of the channel estimation error which are required for accurate equalization. Second, a CNN-based demapper jointly processes a large number of orthogonal frequency-division multiplexing (OFDM) symbols and subcarriers, which allows it to compute better log likelihood ratios (LLRs) by compensating for channel aging. The resulting architecture can be used in the up- and downlink and is trained in an end-to-end manner, removing the need for hard-to-get perfect channel state information (CSI) during the training phase. Simulation results demonstrate consistent performance improvements over the baseline which are especially pronounced in high mobility scenarios.
Fast-Convergent Dynamics for Distributed Allocation of Resources Over Switching Sparse Networks with Quantized Communication Links
Authors: Mohammadreza Doostmohammadian, Alireza Aghasi, Mohammad Pirani, Ehsan Nekouei, Usman A. Khan, Themistoklis Charalambous
Date: 2020-12-15 00:00:00
Tasks: Edge-computing

This paper proposes networked dynamics to solve resource allocation problems over time-varying multi-agent networks. The state of each agent represents the amount of used resources (or produced utilities) while the total amount of resources is fixed. The idea is to optimally allocate the resources among the group of agents by minimizing the overall cost function subject to fixed sum of resources. Each agents' information is restricted to its own state and cost function and those of its immediate in-neighbors. This is motivated by distributed applications such as mobile edge-computing, economic dispatch over smart grids, and multi-agent coverage control. This work provides a fast convergent solution (in comparison with linear dynamics) while considering relaxed network connectivity with quantized communication links. The proposed dynamics reaches optimal solution over switching (possibly disconnected) undirected networks as far as their union over some bounded non-overlapping time-intervals has a spanning-tree. We prove feasibility of the solution, uniqueness of the optimal state, and convergence to the optimal value under the proposed dynamics, where the analysis is applicable to similar 1st-order allocation dynamics with strongly sign-preserving nonlinearities, such as actuator saturation.
Scalable Verification of Quantized Neural Networks (Technical Report)
Authors: Thomas A. Henzinger, Mathias Lechner, Đorđe Žikelić
Date: 2020-12-15 00:00:00
Tasks: Computational Efficiency, Quantization

Formal verification of neural networks is an active topic of research, and recent advances have significantly increased the size of the networks that verification tools can handle. However, most methods are designed for verification of an idealized model of the actual network which works over real arithmetic and ignores rounding imprecisions. This idealization is in stark contrast to network quantization, which is a technique that trades numerical precision for computational efficiency and is, therefore, often applied in practice. Neglecting rounding errors of such low-bit quantized neural networks has been shown to lead to wrong conclusions about the network's correctness. Thus, the desired approach for verifying quantized neural networks would be one that takes these rounding errors into account. In this paper, we show that verifying the bit-exact implementation of quantized neural networks with bit-vector specifications is PSPACE-hard, even though verifying idealized real-valued networks and satisfiability of bit-vector specifications alone are each in NP. Furthermore, we explore several practical heuristics toward closing the complexity gap between idealized and bit-exact verification. In particular, we propose three techniques for making SMT-based verification of quantized neural networks more scalable. Our experiments demonstrate that our proposed methods allow a speedup of up to three orders of magnitude over existing approaches.
Active Learning for Deep Gaussian Process Surrogates
Authors: Annie Sauer, Robert B. Gramacy, David Higdon
Date: 2020-12-15 00:00:00
Tasks: Active Learning, Gaussian Processes, Uncertainty Quantification

Deep Gaussian processes (DGPs) are increasingly popular as predictive models in machine learning (ML) for their non-stationary flexibility and ability to cope with abrupt regime changes in training data. Here we explore DGPs as surrogates for computer simulation experiments whose response surfaces exhibit similar characteristics. In particular, we transport a DGP's automatic warping of the input space and full uncertainty quantification (UQ), via a novel elliptical slice sampling (ESS) Bayesian posterior inferential scheme, through to active learning (AL) strategies that distribute runs non-uniformly in the input space -- something an ordinary (stationary) GP could not do. Building up the design sequentially in this way allows smaller training sets, limiting both expensive evaluation of the simulator code and mitigating cubic costs of DGP inference. When training data sizes are kept small through careful acquisition, and with parsimonious layout of latent layers, the framework can be both effective and computationally tractable. Our methods are illustrated on simulation data and two real computer experiments of varying input dimensionality. We provide an open source implementation in the "deepgp" package on CRAN.