pwc-archive/papers-with-abstracts huggingface dataset:
Multi-Constitutive Neural Network for Large Deformation Poromechanics Problem
Authors: Qi Zhang, Yilin Chen, ZiYi Yang, Eric Darve
Date: 2020-10-11 00:00:00

In this paper, we study the problem of large-strain consolidation in poromechanics with deep neural networks (DNN). Given different material properties and different loading conditions, the goal is to predict pore pressure and settlement. We propose a novel method "multi-constitutive neural network" (MCNN) such that one model can solve several different constitutive laws. We introduce a one-hot encoding vector as an additional input vector, which is used to label the constitutive law we wish to solve. Then we build a DNN which takes $(\hat{X}, \hat{t})$ as input along with a constitutive law label and outputs the corresponding solution. It is the first time, to our knowledge, that we can evaluate multi-constitutive laws through only one training process while still obtaining good accuracies. We found that MCNN trained to solve multiple PDEs outperforms individual neural network solvers trained with PDE in some cases.
Successive Halving Top-k Operator
Authors: Michał Pietruszka, Łukasz Borchmann, Filip Graliński
Date: 2020-10-08 00:00:00

We propose a differentiable successive halving method of relaxing the top-k operator, rendering gradient-based optimization possible. The need to perform softmax iteratively on the entire vector of scores is avoided by using a tournament-style selection. As a result, a much better approximation of top-k with lower computational cost is achieved compared to the previous approach.
Modulation Pattern Detection Using Complex Convolutions in Deep Learning
Authors: Jakob Krzyston, Rajib Bhattacharjea, Andrew Stark
Date: 2020-10-14 00:00:00
Tasks: Deep Learning

Transceivers used for telecommunications transmit and receive specific modulation patterns that are represented as sequences of complex numbers. Classifying modulation patterns is challenging because noise and channel impairments affect the signals in complicated ways such that the received signal bears little resemblance to the transmitted signal. Although deep learning approaches have shown great promise over statistical methods in this problem space, deep learning frameworks continue to lag in support for complex-valued data. To address this gap, we study the implementation and use of complex convolutions in a series of convolutional neural network architectures. Replacement of data structure and convolution operations by their complex generalization in an architecture improves performance, with statistical significance, at recognizing modulation patterns in complex-valued signals with high SNR after being trained on low SNR signals. This suggests complex-valued convolutions enables networks to learn more meaningful representations. We investigate this hypothesis by comparing the features learned in each experiment by visualizing the inputs that results in one-hot modulation pattern classification for each network.
Federated Transfer Learning: concept and applications
Authors: Sudipan Saha, Tahir Ahmad
Date: 2020-09-26 00:00:00
Tasks: Federated Learning, Transfer Learning

Development of Artificial Intelligence (AI) is inherently tied to the development of data. However, in most industries data exists in form of isolated islands, with limited scope of sharing between different organizations. This is an hindrance to the further development of AI. Federated learning has emerged as a possible solution to this problem in the last few years without compromising user privacy. Among different variants of the federated learning, noteworthy is federated transfer learning (FTL) that allows knowledge to be transferred across domains that do not have many overlapping features and users. In this work we provide a comprehensive survey of the existing works on this topic. In more details, we study the background of FTL and its different existing applications. We further analyze FTL from privacy and machine learning perspective.
A semi-supervised autoencoder framework for joint generation and classification of breathing
Authors: Oscar Pastor-Serrano, Danny Lathouwers, Zoltán Perkó
Date: 2020-10-19 00:00:00
Tasks: Diagnostic, General Classification, Time Series, Time Series Analysis

One of the main problems with biomedical signals is the limited amount of patient-specific data and the significant amount of time needed to record the sufficient number of samples needed for diagnostic and treatment purposes. In this study, we present a framework to simultaneously generate and classify biomedical time series based on a modified Adversarial Autoencoder (AAE) algorithm and one-dimensional convolutions. Our work is based on breathing time series, with specific motivation to capture breathing motion during radiotherapy lung cancer treatments. First, we explore the potential in using the Variational Autoencoder (VAE) and AAE algorithms to model breathing from individual patients. We extend the AAE algorithm to allow joint semi-supervised classification and generation of different types of signals. To simplify the modeling task, we introduce a pre-processing and post-processing compressing algorithm that transforms the multi-dimensional time series into vectors containing time and position values, which are transformed back into time series through an additional neural network. By incorporating few labeled samples during training, our model outperforms other purely discriminative networks in classifying breathing baseline shift irregularities from a dataset completely different from the training set. To our knowledge, the presented framework is the first approach that unifies generation and classification within a single model for this type of biomedical data, enabling both computer aided diagnosis and augmentation of labeled samples within a single framework.
A marine radioisotope gamma-ray spectrum analysis method based on Monte Carlo simulation and MLP neural network
Authors: Wenhan Dai, Zhi Zeng, Daowei Dou, Hao Ma, Jianping Chen, Junli Li, HUI ZHANG
Date: 2020-10-24 00:00:00

The monitoring of Cs-137 in seawater using scintillation detector relies on the spectrum analysis method to extract the Cs-137 concentration. And when in poor statistic situation, the calculation result of the traditional net peak area (NPA) method has a large uncertainty. We present a machine learning based method to better analyze the gamma-ray spectrum with low Cs-137 concentration. We apply multilayer perceptron (MLP) to analyze the 662 keV full energy peak of Cs-137 in the seawater spectrum. And the MLP can be trained with a few measured background spectrums by combining the simulated Cs-137 signal with measured background spectrums. Thus, it can save the time of preparing and measuring the standard samples for generating the training dataset. To validate the MLP-based method, we use Geant4 and background gamma-ray spectrums measured by a seaborne monitoring device to generate an independent test dataset to test the result by our method and the traditional NPA method. We find that the MLP-based method achieves a root mean squared error of 0.159, 2.3 times lower than that of the traditional net peak area method, indicating the MLP-based method improves the precision of Cs-137 concentration calculation
Limitations of the recall capabilities in delay based reservoir computing systems
Authors: Felix Köster, Dominik Ehlert, Kathy Lüdge
Date: 2020-09-16 00:00:00

We analyze the memory capacity of a delay based reservoir computer with a Hopf normal form as nonlinearity and numerically compute the linear as well as the higher order recall capabilities. A possible physical realisation could be a laser with external cavity, for which the information is fed via electrical injection. A task independent quantification of the computational capability of the reservoir system is done via a complete orthonormal set of basis functions. Our results suggest that even for constant readout dimension the total memory capacity is dependent on the ratio between the information input period, also called the clock cycle, and the time delay in the system. Optimal performance is found for a time delay about 1.6 times the clock cycle
The De-democratization of AI: Deep Learning and the Compute Divide in Artificial Intelligence Research
Authors: Nur Ahmed, Muntasir Wahed
Date: 2020-10-22 00:00:00
Tasks: counterfactual, Fairness

Increasingly, modern Artificial Intelligence (AI) research has become more computationally intensive. However, a growing concern is that due to unequal access to computing power, only certain firms and elite universities have advantages in modern AI research. Using a novel dataset of 171394 papers from 57 prestigious computer science conferences, we document that firms, in particular, large technology firms and elite universities have increased participation in major AI conferences since deep learning's unanticipated rise in 2012. The effect is concentrated among elite universities, which are ranked 1-50 in the QS World University Rankings. Further, we find two strategies through which firms increased their presence in AI research: first, they have increased firm-only publications; and second, firms are collaborating primarily with elite universities. Consequently, this increased presence of firms and elite universities in AI research has crowded out mid-tier (QS ranked 201-300) and lower-tier (QS ranked 301-500) universities. To provide causal evidence that deep learning's unanticipated rise resulted in this divergence, we leverage the generalized synthetic control method, a data-driven counterfactual estimator. Using machine learning based text analysis methods, we provide additional evidence that the divergence between these two groups - large firms and non-elite universities - is driven by access to computing power or compute, which we term as the "compute divide". This compute divide between large firms and non-elite universities increases concerns around bias and fairness within AI technology, and presents an obstacle towards "democratizing" AI. These results suggest that a lack of access to specialized equipment such as compute can de-democratize knowledge production.
Event-Driven Learning of Systematic Behaviours in Stock Markets
Authors: Xianchao Wu
Date: 2020-10-23 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Event Extraction, Open Information Extraction

It is reported that financial news, especially financial events expressed in news, provide information to investors' long/short decisions and influence the movements of stock markets. Motivated by this, we leverage financial event streams to train a classification neural network that detects latent event-stock linkages and stock markets' systematic behaviours in the U.S. stock market. Our proposed pipeline includes (1) a combined event extraction method that utilizes Open Information Extraction and neural co-reference resolution, (2) a BERT/ALBERT enhanced representation of events, and (3) an extended hierarchical attention network that includes attentions on event, news and temporal levels. Our pipeline achieves significantly better accuracies and higher simulated annualized returns than state-of-the-art models when being applied to predicting Standard\&Poor 500, Dow Jones, Nasdaq indices and 10 individual stocks.
DeepFix: Fixing Common C Language Errors by Deep Learning
Authors: Rahul Gupta, Soham Pal, Aditya Kanade, Shirish Shevade
Date: 2017-02-04 00:00:00
Tasks: Deep Learning, Program Repair

The problem of automatically fixing programming errors is a very active research topic in software engineering. This is a challenging problem as fixing even a single error may require analysis of the entire program. In practice, a number of errors arise due to programmer’s inexperience with the programming language or lack of attention to detail. We call these common programming errors. These are analogous to grammatical errors in natural languages. Compilers detect such errors, but their error messages are usually inaccurate. In this work, we present an end-to-end solution, called DeepFix, that can fix multiple such errors in a program without relying on any external tool to locate or fix them. At the heart of DeepFix is a multi-layered sequence-to-sequence neural network with
attention which is trained to predict erroneous program locations along with the required correct statements. On a set of 6971 erroneous C programs written by students for 93 programming tasks, DeepFix could fix 1881 (27%) programs completely and 1338 (19%) programs partially.
ACTIVETHIEF: Model Extraction Using Active Learning and Unannotated Public Data
Authors: Soham Pal, Yash Gupta, Aditya Shukla, Aditya Kanade, Shirish Shevade, Vinod Ganapathy
Date: 2020-02-07 00:00:00
Tasks: Active Learning, BIG-bench Machine Learning, Model extraction

Machine learning models are increasingly being deployed in practice. Machine Learning as a Service (MLaaS) providers expose such models to queries by third-party developers through application programming interfaces (APIs). Prior work has developed model extraction attacks, in which an attacker extracts an approximation of an MLaaS model by making black-box queries to it. We design ACTIVETHIEF - a model extraction framework for deep neural networks that makes use of active learning techniques and unannotated public datasets to perform model extraction. It does not expect strong domain knowledge or access to annotated data on the part of the attacker. We demonstrate that (1) it is possible to use ACTIVETHIEF to extract deep classifiers trained on a variety of datasets from image and text domains, while querying the model with as few as 10-30% of samples from public datasets, (2) the resulting model exhibits a higher transferability success rate of adversarial examples than prior work, and (3) the attack evades detection by the state-of-the-art model extraction detection method, PRADA.
A dynamic factor model approach to incorporate Big Data in state space models for official statistics
Authors: Caterina Schiavoni, Franz Palm, Stephan Smeekes, Jan van den Brakel
Date: 2019-01-31 00:00:00
Tasks: State Space Models

In this paper we consider estimation of unobserved components in state space models using a dynamic factor approach to incorporate auxiliary information from high-dimensional data sources. We apply the methodology to unemployment estimation as done by Statistics Netherlands, who uses a multivariate state space model to produce monthly figures for the unemployment using series observed with the labour force survey (LFS). We extend the model by including auxiliary series of Google Trends about job-search and economic uncertainty, and claimant counts, partially observed at higher frequencies. Our factor model allows for nowcasting the variable of interest, providing reliable unemployment estimates in real-time before LFS data become available.
While Stability Lasts: A Stochastic Model of Non-Custodial Stablecoins
Authors: Ariah Klages-Mundt, Andreea Minca
Date: 2020-04-02 00:00:00

The `Black Thursday' crisis in cryptocurrency markets demonstrated deleveraging risks in over-collateralized non-custodial stablecoins. We develop a stochastic model that helps explain deleveraging crises in these over-collateralized systems. In our model, the stablecoin supply is decided by speculators who optimize the profitability of a leveraged position while incorporating the forward-looking cost of collateral liquidations, which involves the endogenous price of the stablecoin. We formally characterize regimes that are interpreted as stable and unstable for the stablecoin. We prove bounds on quadratic variation and the probability of large deviations in the stable domain and we demonstrate distinctly greater price variance in the unstable domain. We identify a deflationary deleveraging spiral by means of a submartingale. These deleveraging spirals, which resemble short squeezes, lead to faster collateral drawdown (and potential shortfalls) and are accompanied by higher price variance, as experienced on Black Thursday. We conclude by discussing non-custodial ways in which the issues raised in this paper can be mitigated.
What's There in the Dark
Authors: Sauradip Nag, Saptakatha Adak, Sukhendu Das
Date: 2019-09-24 00:00:00
Tasks: Scene Parsing, Semantic Segmentation

Scene  Parsing  is  an  important  cog  for  modern  autonomousdriving systems.   Most of the works in semantic segmenta-tion pertains to day-time scenes with favourable weather andillumination  conditions.   In  this  paper,  we  propose  a  noveldeep architecture, NiSeNet, that performs semantic segmen-tation of night scenes using a domain mapping approach ofsynthetic  to  real  data.   It  is  a  dual-channel  network,  wherewe designed a Real channel using DeepLabV3+ coupled withan MSE loss to preserve the spatial information.  In addition,we used an Adaptive channel reducing the domain gap be-tween  synthetic  and  real  night  images,  which  also  comple-ments  the  failures  of  Real  channel  output.   Apart  from  thedual channel, we introduced a novel fusion scheme to fuse theoutputs of two channels.  In addition to that, we compiled anew dataset Urban Night Driving Dataset (UNDD); it consistsof7125unlabelled day and night images; additionally, it has75night images with pixel-level annotations having classesequivalent to Cityscapes dataset.  We evaluated our approachon  the  Berkley  Deep  Drive  dataset,  the  challenging  Mapil-lary dataset and UNDD dataset to exhibit that the proposedmethod outperforms the state-of-the-art techniques in termsof accuracy and visual quality
MIX: A Joint Learning Framework for Detecting Both Clustered and Scattered Outliers in Mixed-Type Data
Authors: Hongzuo Xu, Yijie Wang, Yongjun Wang, Zhiyue Wu
Date: 2019-11-01 00:00:00
Tasks: Anomaly Detection, Outlier Detection

Mixed-type data are pervasive in real life, but very limited outlier detection methods are available for these data. Some existing methods handle mixed-type data by feature converting, whereas their performance is downgraded by information loss and noise caused by the transformation. Another kind of approaches separately evaluates outlierness in numerical and categorical features. However, they fail to adequately consider the behaviours of data objects in different feature spaces, often leading to suboptimal results. As for outlier form, both clustered outliers and scattered outliers are contained in many real-world data, but a number of outlier detectors are inherently restricted by their outlier definitions to simultaneously detect both of them. To address these issues, an unsupervised outlier detection method MIX is proposed. MIX constructs a joint learning framework to establish a cooperation mechanism to make separate outlier scoring constantly communicate and sufficiently grasp the behaviours of data objects in another feature space. Specifically, MIX iteratively performs outlier scoring in numerical and categorical space. Each outlier scoring phase can be iteratively and cooperatively enhanced by the prior knowledge given by another feature space. To target both clustered and scattered outliers, the outlier scoring phases capture the essential characteristic of outliers, i.e., evaluating outlierness via the deviation from the normal model. We show that MIX significantly outperforms eight state-of-the-art outlier detectors on twelve real-world datasets and obtains good scalability.
Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure
Authors: Ben Hutchinson, Andrew Smart, Alex Hanna, Emily Denton, Christina Greer, Oddur Kjartansson, Parker Barnes, Margaret Mitchell
Date: 2020-10-23 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making

Rising concern for the societal implications of artificial intelligence systems has inspired demands for greater transparency and accountability. However the datasets which empower machine learning are often used, shared and re-used with little visibility into the processes of deliberation which led to their creation. Which stakeholder groups had their perspectives included when the dataset was conceived? Which domain experts were consulted regarding how to model subgroups and other phenomena? How were questions of representational biases measured and addressed? Who labeled the data? In this paper, we introduce a rigorous framework for dataset development transparency which supports decision-making and accountability. The framework uses the cyclical, infrastructural and engineering nature of dataset development to draw on best practices from the software development lifecycle. Each stage of the data development lifecycle yields a set of documents that facilitate improved communication and decision-making, as well as drawing attention the value and necessity of careful data work. The proposed framework is intended to contribute to closing the accountability gap in artificial intelligence systems, by making visible the often overlooked work that goes into dataset creation.
Kernel Smoothing, Mean Shift, and Their Learning Theory with Directional Data
Authors: Yikun Zhang, Yen-Chi Chen
Date: 2020-10-23 00:00:00
Tasks: Astronomy, Clustering, Learning Theory

Directional data consist of observations distributed on a (hyper)sphere, and appear in many applied fields, such as astronomy, ecology, and environmental science. This paper studies both statistical and computational problems of kernel smoothing for directional data. We generalize the classical mean shift algorithm to directional data, which allows us to identify local modes of the directional kernel density estimator (KDE). The statistical convergence rates of the directional KDE and its derivatives are derived, and the problem of mode estimation is examined. We also prove the ascending property of the directional mean shift algorithm and investigate a general problem of gradient ascent on the unit hypersphere. To demonstrate the applicability of the algorithm, we evaluate it as a mode clustering method on both simulated and real-world data sets.
Zero-Shot Learning from scratch (ZFS): leveraging local compositional representations
Authors: Tristan Sylvain, Linda Petrini, R Devon Hjelm
Date: 2020-10-22 00:00:00
Tasks: General Classification, Representation Learning, zero-shot-classification, Zero-Shot Learning

Zero-shot classification is a generalization task where no instance from the target classes is seen during training. To allow for test-time transfer, each class is annotated with semantic information, commonly in the form of attributes or text descriptions. While classical zero-shot learning does not explicitly forbid using information from other datasets, the approaches that achieve the best absolute performance on image benchmarks rely on features extracted from encoders pretrained on Imagenet. This approach relies on hyper-optimized Imagenet-relevant parameters from the supervised classification setting, entangling important questions about the suitability of those parameters and how they were learned with more fundamental questions about representation learning and generalization. To remove these distractors, we propose a more challenging setting: Zero-Shot Learning from scratch (ZFS), which explicitly forbids the use of encoders fine-tuned on other datasets. Our analysis on this setting highlights the importance of local information, and compositional representations.