pwc-archive/papers-with-abstracts huggingface dataset:
Against Adversarial Learning: Naturally Distinguish Known and Unknown in Open Set Domain Adaptation
Authors: Sitong Mao, Xiao Shen, Fu-Lai Chung
Date: 2020-11-04 00:00:00
Tasks: Domain Adaptation

Open set domain adaptation refers to the scenario that the target domain contains categories that do not exist in the source domain. It is a more common situation in the reality compared with the typical closed set domain adaptation where the source domain and the target domain contain the same categories. The main difficulty of open set domain adaptation is that we need to distinguish which target data belongs to the unknown classes when machine learning models only have concepts about what they know. In this paper, we propose an "against adversarial learning" method that can distinguish unknown target data and known data naturally without setting any additional hyper parameters and the target data predicted to the known classes can be classified at the same time. Experimental results show that the proposed method can make significant improvement in performance compared with several state-of-the-art methods.
Causal Expectation-Maximisation
Authors: Marco Zaffalon, Alessandro Antonucci, Rafael Cabañas
Date: 2020-11-04 00:00:00
Tasks: Causal Inference, counterfactual, Counterfactual Inference

Structural causal models are the basic modelling unit in Pearl's causal theory; in principle they allow us to solve counterfactuals, which are at the top rung of the ladder of causation. But they often contain latent variables that limit their application to special settings. This appears to be a consequence of the fact, proven in this paper, that causal inference is NP-hard even in models characterised by polytree-shaped graphs. To deal with such a hardness, we introduce the causal EM algorithm. Its primary aim is to reconstruct the uncertainty about the latent variables from data about categorical manifest variables. Counterfactual inference is then addressed via standard algorithms for Bayesian networks. The result is a general method to approximately compute counterfactuals, be they identifiable or not (in which case we deliver bounds). We show empirically, as well as by deriving credible intervals, that the approximation we provide becomes accurate in a fair number of EM runs. These results lead us finally to argue that there appears to be an unnoticed limitation to the trending idea that counterfactual bounds can often be computed without knowledge of the structural equations.
Mixed Set Domain Adaptation
Authors: Sitong Mao, Keli Zhang, Fu-Lai Chung
Date: 2020-11-04 00:00:00
Tasks: Domain Adaptation

In the settings of conventional domain adaptation, categories of the source dataset are from the same domain (or domains for multi-source domain adaptation), which is not always true in reality. In this paper, we propose \textbf{\textit{Mixed Set Domain Adaptation} (MSDA)}. Under the settings of MSDA, different categories of the source dataset are not all collected from the same domain(s). For instance, category $1\sim k$ are collected from domain $\alpha$ while category $k+1\sim c$ are collected from domain $\beta$. Under such situation, domain adaptation performance will be further influenced because of the distribution discrepancy inside the source data. A feature element-wise weighting (FEW) method that can reduce distribution discrepancy between different categories is also proposed for MSDA. Experimental results and quality analysis show the significance of solving MSDA problem and the effectiveness of the proposed method.
Robust building footprint extraction from big multi-sensor data using deep competition network
Authors: Mehdi Khoshboresh-Masouleh, Mohammad R. Saradjian
Date: 2020-11-04 00:00:00
Tasks: Decoder, Quantization

Building footprint extraction (BFE) from multi-sensor data such as optical images and light detection and ranging (LiDAR) point clouds is widely used in various fields of remote sensing applications. However, it is still challenging research topic due to relatively inefficient building extraction techniques from variety of complex scenes in multi-sensor data. In this study, we develop and evaluate a deep competition network (DCN) that fuses very high spatial resolution optical remote sensing images with LiDAR data for robust BFE. DCN is a deep superpixelwise convolutional encoder-decoder architecture using the encoder vector quantization with classified structure. DCN consists of five encoding-decoding blocks with convolutional weights for robust binary representation (superpixel) learning. DCN is trained and tested in a big multi-sensor dataset obtained from the state of Indiana in the United States with multiple building scenes. Comparison results of the accuracy assessment showed that DCN has competitive BFE performance in comparison with other deep semantic binary segmentation architectures. Therefore, we conclude that the proposed model is a suitable solution to the robust BFE from big multi-sensor data.
Covariance Self-Attention Dual Path UNet for Rectal Tumor Segmentation
Authors: Haijun Gao, Bochuan Zheng, Dazhi Pan, Xiangyin Zeng
Date: 2020-11-04 00:00:00
Tasks: Segmentation, Tumor Segmentation

Deep learning algorithms are preferable for rectal tumor segmentation. However, it is still a challenge task to accurately segment and identify the locations and sizes of rectal tumors by using deep learning methods. To increase the capability of extracting enough feature information for rectal tumor segmentation, we propose a Covariance Self-Attention Dual Path UNet (CSA-DPUNet). The proposed network mainly includes two improvements on UNet: 1) modify UNet that has only one path structure to consist of two contracting path and two expansive paths (nam new network as DPUNet), which can help extract more feature information from CT images; 2) employ the criss-cross self-attention module into DPUNet, meanwhile, replace the original calculation method of correlation operation with covariance operation, which can further enhances the characterization ability of DPUNet and improves the segmentation accuracy of rectal tumors. Experiments illustrate that compared with the current state-of-the-art results, CSA-DPUNet brings 15.31%, 7.2%, 11.8%, and 9.5% improvement in Dice coefficient, P, R, F1, respectively, which demonstrates that our proposed CSA-DPUNet is effective for rectal tumor segmentation.
A Two-Stage Cascade Model with Variational Autoencoders and Attention Gates for MRI Brain Tumor Segmentation
Authors: Chenggang Lyu, Hai Shu
Date: 2020-11-04 00:00:00
Tasks: Brain Tumor Segmentation, Decoder, Segmentation, Tumor Segmentation

Automatic MRI brain tumor segmentation is of vital importance for the disease diagnosis, monitoring, and treatment planning. In this paper, we propose a two-stage encoder-decoder based model for brain tumor subregional segmentation. Variational autoencoder regularization is utilized in both stages to prevent the overfitting issue. The second-stage network adopts attention gates and is trained additionally using an expanded dataset formed by the first-stage outputs. On the BraTS 2020 validation dataset, the proposed method achieves the mean Dice score of 0.9041, 0.8350, and 0.7958, and Hausdorff distance (95%) of 4.953, 6.299, and 23.608 for the whole tumor, tumor core, and enhancing tumor, respectively. The corresponding results on the BraTS 2020 testing dataset are 0.8729, 0.8357, and 0.8205 for Dice score, and 11.4288, 19.9690, and 15.6711 for Hausdorff distance. The code is publicly available at https://github.com/shu-hai/two-stage-VAE-Attention-gate-BraTS2020.
Query Expansion System for the VoxCeleb Speaker Recognition Challenge 2020
Authors: Yu-Sen Cheng, Chun-Liang Shih, Tien-Hong Lo, Wen-Ting Tseng, Berlin Chen
Date: 2020-11-04 00:00:00
Tasks: Speaker Recognition, Speaker Verification

In this report, we describe our submission to the VoxCeleb Speaker Recognition Challenge (VoxSRC) 2020. Two approaches are adopted. One is to apply query expansion on speaker verification, which shows significant progress compared to baseline in the study. Another is to use Kaldi extract x-vector and to combine its Probabilistic Linear Discriminant Analysis (PLDA) score with ResNet score.
RetroXpert: Decompose Retrosynthesis Prediction like a Chemist
Authors: Chaochao Yan, Qianggang Ding, Peilin Zhao, Shuangjia Zheng, Jinyu Yang, Yang Yu, Junzhou Huang
Date: 2020-11-04 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Graph Neural Network, Prediction, Retrosynthesis

Retrosynthesis is the process of recursively decomposing target molecules into available building blocks. It plays an important role in solving problems in organic synthesis planning. To automate or assist in the retrosynthesis analysis, various retrosynthesis prediction algorithms have been proposed. However, most of them are cumbersome and lack interpretability about their predictions. In this paper, we devise a novel template-free algorithm for automatic retrosynthetic expansion inspired by how chemists approach retrosynthesis prediction. Our method disassembles retrosynthesis into two steps: i) identify the potential reaction center of the target molecule through a novel graph neural network and generate intermediate synthons, and ii) generate the reactants associated with synthons via a robust reactant generation model. While outperforming the state-of-the-art baselines by a significant margin, our model also provides chemically reasonable interpretation.
DR-Unet104 for Multimodal MRI brain tumor segmentation
Authors: Jordan Colman, Lei Zhang, Wenting Duan, Xujiong Ye
Date: 2020-11-04 00:00:00
Tasks: 3D Architecture, Brain Tumor Segmentation, Lesion Segmentation, Segmentation, Tumor Segmentation

In this paper we propose a 2D deep residual Unet with 104 convolutional layers (DR-Unet104) for lesion segmentation in brain MRIs. We make multiple additions to the Unet architecture, including adding the 'bottleneck' residual block to the Unet encoder and adding dropout after each convolution block stack. We verified the effect of introducing the regularisation of dropout with small rate (e.g. 0.2) on the architecture, and found a dropout of 0.2 improved the overall performance compared to no dropout, or a dropout of 0.5. We evaluated the proposed architecture as part of the Multimodal Brain Tumor Segmentation (BraTS) 2020 Challenge and compared our method to DeepLabV3+ with a ResNet-V2-152 backbone. We found that the DR-Unet104 achieved a mean dice score coefficient of 0.8862, 0.6756 and 0.6721 for validation data, whole tumor, enhancing tumor and tumor core respectively, an overall improvement on 0.8770, 0.65242 and 0.68134 achieved by DeepLabV3+. Our method produced a final mean DSC of 0.8673, 0.7514 and 0.7983 on whole tumor, enhancing tumor and tumor core on the challenge's testing data. We produced a competitive lesion segmentation architecture, despite only 2D convolutions, having the added benefit that it can be used on lower power computers than a 3D architecture. The source code and trained model for this work is openly available at https://github.com/jordan-colman/DR-Unet104.
Minimum Bayes Risk Training for End-to-End Speaker-Attributed ASR
Authors: Naoyuki Kanda, Zhong Meng, Liang Lu, Yashesh Gaur, Xiaofei Wang, Zhuo Chen, Takuya Yoshioka
Date: 2020-11-03 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Speaker Identification, speech-recognition, Speech Recognition

Recently, an end-to-end speaker-attributed automatic speech recognition (E2E SA-ASR) model was proposed as a joint model of speaker counting, speech recognition and speaker identification for monaural overlapped speech. In the previous study, the model parameters were trained based on the speaker-attributed maximum mutual information (SA-MMI) criterion, with which the joint posterior probability for multi-talker transcription and speaker identification are maximized over training data. Although SA-MMI training showed promising results for overlapped speech consisting of various numbers of speakers, the training criterion was not directly linked to the final evaluation metric, i.e., speaker-attributed word error rate (SA-WER). In this paper, we propose a speaker-attributed minimum Bayes risk (SA-MBR) training method where the parameters are trained to directly minimize the expected SA-WER over the training data. Experiments using the LibriSpeech corpus show that the proposed SA-MBR training reduces the SA-WER by 9.0 % relative compared with the SA-MMI-trained model.
Domain-independent generation and classification of behavior traces
Authors: Daniel Borrajo, Manuela Veloso
Date: 2020-11-03 00:00:00
Tasks: Classification, General Classification

Financial institutions mostly deal with people. Therefore, characterizing different kinds of human behavior can greatly help institutions for improving their relation with customers and with regulatory offices. In many of such interactions, humans have some internal goals, and execute some actions within the financial system that lead them to achieve their goals. In this paper, we tackle these tasks as a behavior-traces classification task. An observer agent tries to learn characterizing other agents by observing their behavior when taking actions in a given environment. The other agents can be of several types and the goal of the observer is to identify the type of the other agent given a trace of observations. We present CABBOT, a learning technique that allows the agent to perform on-line classification of the type of planning agent whose behavior is observing. In this work, the observer agent has partial and noisy observability of the environment (state and actions of the other agents). In order to evaluate the performance of the learning technique, we have generated a domain-independent goal-based simulator of agents. We present experiments in several (both financial and non-financial) domains with promising results.
Deep tree-ensembles for multi-output prediction
Authors: Felipe Kenji Nakano, Konstantinos Pliakos, Celine Vens
Date: 2020-11-03 00:00:00
Tasks: Classification, General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Multi-target regression, Prediction, regression, Representation Learning

Recently, deep neural networks have expanded the state-of-art in various scientific fields and provided solutions to long standing problems across multiple application domains. Nevertheless, they also suffer from weaknesses since their optimal performance depends on massive amounts of training data and the tuning of an extended number of parameters. As a countermeasure, some deep-forest methods have been recently proposed, as efficient and low-scale solutions. Despite that, these approaches simply employ label classification probabilities as induced features and primarily focus on traditional classification and regression tasks, leaving multi-output prediction under-explored. Moreover, recent work has demonstrated that tree-embeddings are highly representative, especially in structured output prediction. In this direction, we propose a novel deep tree-ensemble (DTE) model, where every layer enriches the original feature set with a representation learning component based on tree-embeddings. In this paper, we specifically focus on two structured output prediction tasks, namely multi-label classification and multi-target regression. We conducted experiments using multiple benchmark datasets and the obtained results confirm that our method provides superior results to state-of-the-art methods in both tasks.
Local SGD: Unified Theory and New Efficient Methods
Authors: Eduard Gorbunov, Filip Hanzely, Peter Richtárik
Date: 2020-11-03 00:00:00
Tasks: Federated Learning

We present a unified framework for analyzing local SGD methods in the convex and strongly convex regimes for distributed/federated training of supervised machine learning models. We recover several known methods as a special case of our general framework, including Local-SGD/FedAvg, SCAFFOLD, and several variants of SGD not originally designed for federated learning. Our framework covers both the identical and heterogeneous data settings, supports both random and deterministic number of local steps, and can work with a wide array of local stochastic gradient estimators, including shifted estimators which are able to adjust the fixed points of local iterations for faster convergence. As an application of our framework, we develop multiple novel FL optimizers which are superior to existing methods. In particular, we develop the first linearly converging local SGD method which does not require any data homogeneity or other strong assumptions.
Digital Twins: State of the Art Theory and Practice, Challenges, and Open Research Questions
Authors: Angira Sharma, Edward Kosasih, Jie Zhang, Alexandra Brintrup, Anisoara Calinescu
Date: 2020-11-02 00:00:00

Digital Twin was introduced over a decade ago, as an innovative all-encompassing tool, with perceived benefits including real-time monitoring, simulation and forecasting. However, the theoretical framework and practical implementations of digital twins (DT) are still far from this vision. Although successful implementations exist, sufficient implementation details are not publicly available, therefore it is difficult to assess their effectiveness, draw comparisons and jointly advance the DT methodology. This work explores the various DT features and current approaches, the shortcomings and reasons behind the delay in the implementation and adoption of digital twin. Advancements in machine learning, internet of things and big data have contributed hugely to the improvements in DT with regards to its real-time monitoring and forecasting properties. Despite this progress and individual company-based efforts, certain research gaps exist in the field, which have caused delay in the widespread adoption of this concept. We reviewed relevant works and identified that the major reasons for this delay are the lack of a universal reference framework, domain dependence, security concerns of shared data, reliance of digital twin on other technologies, and lack of quantitative metrics. We define the necessary components of a digital twin required for a universal reference framework, which also validate its uniqueness as a concept compared to similar concepts like simulation, autonomous systems, etc. This work further assesses the digital twin applications in different domains and the current state of machine learning and big data in it. It thus answers and identifies novel research questions, both of which will help to better understand and advance the theory and practice of digital twins.
Augmenting Organizational Decision-Making with Deep Learning Algorithms: Principles, Promises, and Challenges
Authors: Yash Raj Shrestha, Vaibhav Krishna, Georg von Krogh
Date: 2020-11-02 00:00:00
Tasks: Decision Making, Management

The current expansion of theory and research on artificial intelligence in management and organization studies has revitalized the theory and research on decision-making in organizations. In particular, recent advances in deep learning (DL) algorithms promise benefits for decision-making within organizations, such as assisting employees with information processing, thereby augment their analytical capabilities and perhaps help their transition to more creative work.
Dynamically Throttleable Neural Networks (TNN)
Authors: Hengyue Liu, Samyak Parajuli, Jesse Hostetler, Sek Chai, Bir Bhanu
Date: 2020-11-01 00:00:00

Conditional computation for Deep Neural Networks (DNNs) reduce overall computational load and improve model accuracy by running a subset of the network. In this work, we present a runtime throttleable neural network (TNN) that can adaptively self-regulate its own performance target and computing resources. We designed TNN with several properties that enable more flexibility for dynamic execution based on runtime context. TNNs are defined as throttleable modules gated with a separately trained controller that generates a single utilization control parameter. We validate our proposal on a number of experiments, including Convolution Neural Networks (CNNs such as VGG, ResNet, ResNeXt, DenseNet) using CiFAR-10 and ImageNet dataset, for object classification and recognition tasks. We also demonstrate the effectiveness of dynamic TNN execution on a 3D Convolustion Network (C3D) for a hand gesture task. Results show that TNN can maintain peak accuracy performance compared to vanilla solutions, while providing a graceful reduction in computational requirement, down to 74% reduction in latency and 52% energy savings.
Real-time parameter inference in reduced-order flame models with heteroscedastic Bayesian neural network ensembles
Authors: Ushnish Sengupta, Maximilian L. Croci, Matthew P. Juniper
Date: 2020-10-11 00:00:00
Tasks: parameter estimation

The estimation of model parameters with uncertainties from observed data is a ubiquitous inverse problem in science and engineering. In this paper, we suggest an inexpensive and easy to implement parameter estimation technique that uses a heteroscedastic Bayesian Neural Network trained using anchored ensembling. The heteroscedastic aleatoric error of the network models the irreducible uncertainty due to parameter degeneracies in our inverse problem, while the epistemic uncertainty of the Bayesian model captures uncertainties which may arise from an input observation's out-of-distribution nature. We use this tool to perform real-time parameter inference in a 6 parameter G-equation model of a ducted, premixed flame from observations of acoustically excited flames. We train our networks on a library of 2.1 million simulated flame videos. Results on the test dataset of simulated flames show that the network recovers flame model parameters, with the correlation coefficient between predicted and true parameters ranging from 0.97 to 0.99, and well-calibrated uncertainty estimates. The trained neural networks are then used to infer model parameters from real videos of a premixed Bunsen flame captured using a high-speed camera in our lab. Re-simulation using inferred parameters shows excellent agreement between the real and simulated flames. Compared to Ensemble Kalman Filter-based tools that have been proposed for this problem in the combustion literature, our neural network ensemble achieves better data-efficiency and our sub-millisecond inference times represent a savings on computational costs by several orders of magnitude. This allows us to calibrate our reduced-order flame model in real-time and predict the thermoacoustic instability behaviour of the flame more accurately.
Depth Self-Optimized Learning Toward Data Science
Authors: Ziqi Zhang
Date: 2020-11-02 00:00:00
Tasks: Reinforcement Learning (RL)

We propose a two-stage model called Depth Self-Optimized Learning (DSOL), which aims to realize ANN depth self-configuration, self-optimization as well as ANN training without manual intervention. In the first stage of DSOL, it will configure ANN of specific depth according to a specific dataset. In the second stage, DSOL will continuously optimize ANN based on Reinforcement Learning (RL). Finally, the optimal depth is returned to the first stage of DSOL for training, so that DSOL can configure the appropriate ANN depth and perform more reasonable optimization when processing similar datasets again. In the experiment, we ran DSOL on the Iris and Boston housing datasets, and the results showed that DSOL performed well. We have uploaded the experiment records and code to our Github.