pwc-archive/papers-with-abstracts huggingface dataset:
Tehran Stock Exchange Prediction Using Sentiment Analysis of Online Textual Opinions
Authors: Arezoo Hatefi Ghahfarrokhi, Mehrnoush Shamsfard
Date: 2019-08-30 00:00:00
Tasks: Sentiment Analysis

In this paper, we investigate the impact of the social media data in predicting the Tehran Stock Exchange (TSE) variables for the first time. We consider the closing price and daily return of three different stocks for this investigation. We collected our social media data from Sahamyab.com/stocktwits for about three months. To extract information from online comments, we propose a hybrid sentiment analysis approach that combines lexicon-based and learning-based methods. Since lexicons that are available for the Persian language are not practical for sentiment analysis in the stock market domain, we built a particular sentiment lexicon for this domain. After designing and calculating daily sentiment indices using the sentiment of the comments, we examine their impact on the baseline models that only use historical market data and propose new predictor models using multi regression analysis. In addition to the sentiments, we also examine the comments volume and the users' reliabilities. We conclude that the predictability of various stocks in TSE is different depending on their attributes. Moreover, we indicate that for predicting the closing price only comments volume and for predicting the daily return both the volume and the sentiment of the comments could be useful. We demonstrate that Users' Trust coefficients have different behaviors toward the three stocks.
Predicting digital asset market based on blockchain activity data
Authors: Zvezdin Besarabov, Todor Kolev
Date: 2018-10-15 00:00:00

Blockchain technology shows significant results and huge potential for serving as an interweaving fabric that goes through every industry and market, allowing decentralized and secure value exchange, thus connecting our civilization like never before. The standard approach for asset value predictions is based on market analysis with an LSTM neural network. Blockchain technologies, however, give us access to vast amounts of public data, such as the executed transactions and the account balance distribution. We explore whether analyzing this data with modern Deep Leaning techniques results in higher accuracies than the standard approach. During a series of experiments on the Ethereum blockchain, we achieved $4$ times error reduction with blockchain data than an LSTM approach with trade volume data. By utilizing blockchain account distribution histograms, spatial dataset modeling, and a Convolutional architecture, the error was reduced further by 26\%. The proposed methodologies are implemented in an open source cryptocurrency prediction framework, allowing them to be used in other analysis contexts.
Dependence Structure Estimation via Copula
Authors: Jian Ma, Zengqi Sun
Date: 2008-04-28 00:00:00

Dependence strucuture estimation is one of the important problems in machine learning domain and has many applications in different scientific areas. In this paper, a theoretical framework for such estimation based on copula and copula entropy -- the probabilistic theory of representation and measurement of statistical dependence, is proposed. Graphical models are considered as a special case of the copula framework. A method of the framework for estimating maximum spanning copula is proposed. Due to copula, the method is irrelevant to the properties of individual variables, insensitive to outlier and able to deal with non-Gaussianity. Experiments on both simulated data and real dataset demonstrated the effectiveness of the proposed method.
Video Representation Learning by Dense Predictive Coding
Authors: Tengda Han, Weidi Xie, Andrew Zisserman
Date: 2019-09-10 00:00:00
Tasks: Action Recognition, Representation Learning, Self-Supervised Action Recognition, Self-Supervised Action Recognition Linear, Self-Supervised Learning, Temporal Action Localization

The objective of this paper is self-supervised learning of spatio-temporal embeddings from video, suitable for human action recognition. We make three contributions: First, we introduce the Dense Predictive Coding (DPC) framework for self-supervised representation learning on videos. This learns a dense encoding of spatio-temporal blocks by recurrently predicting future representations; Second, we propose a curriculum training scheme to predict further into the future with progressively less temporal context. This encourages the model to only encode slowly varying spatial-temporal signals, therefore leading to semantic representations; Third, we evaluate the approach by first training the DPC model on the Kinetics-400 dataset with self-supervised learning, and then finetuning the representation on a downstream task, i.e. action recognition. With single stream (RGB only), DPC pretrained representations achieve state-of-the-art self-supervised performance on both UCF101(75.7% top1 acc) and HMDB51(35.7% top1 acc), outperforming all previous learning methods by a significant margin, and approaching the performance of a baseline pre-trained on ImageNet.
Towards Understanding the Importance of Shortcut Connections in Residual Networks
Authors: Tianyi Liu, Minshuo Chen, Mo Zhou, Simon S. Du, Enlu Zhou, Tuo Zhao
Date: 2019-09-10 00:00:00
Proceeding: NeurIPS 2019 12

Residual Network (ResNet) is undoubtedly a milestone in deep learning. ResNet is equipped with shortcut connections between layers, and exhibits efficient training using simple first order algorithms. Despite of the great empirical success, the reason behind is far from being well understood. In this paper, we study a two-layer non-overlapping convolutional ResNet. Training such a network requires solving a non-convex optimization problem with a spurious local optimum. We show, however, that gradient descent combined with proper normalization, avoids being trapped by the spurious local optimum, and converges to a global optimum in polynomial time, when the weight of the first layer is initialized at 0, and that of the second layer is initialized arbitrarily in a ball. Numerical experiments are provided to support our theory.
Deep Learning for Automated Classification and Characterization of Amorphous Materials
Authors: Kirk Swanson, Shubhendu Trivedi, Joshua Lequieu, Kyle Swanson, Risi Kondor
Date: 2019-09-10 00:00:00
Tasks: Classification, Deep Learning, General Classification

It is difficult to quantify structure-property relationships and to identify structural features of complex materials. The characterization of amorphous materials is especially challenging because their lack of long-range order makes it difficult to define structural metrics. In this work, we apply deep learning algorithms to accurately classify amorphous materials and characterize their structural features. Specifically, we show that convolutional neural networks and message passing neural networks can classify two-dimensional liquids and liquid-cooled glasses from molecular dynamics simulations with greater than 0.98 AUC, with no a priori assumptions about local particle relationships, even when the liquids and glasses are prepared at the same inherent structure energy. Furthermore, we demonstrate that message passing neural networks surpass convolutional neural networks in this context in both accuracy and interpretability. We extract a clear interpretation of how message passing neural networks evaluate liquid and glass structures by using a self-attention mechanism. Using this interpretation, we derive three novel structural metrics that accurately characterize glass formation. The methods presented here provide us with a procedure to identify important structural features in materials that could be missed by standard techniques and give us a unique insight into how these neural networks process data.
Meta-Learning with Implicit Gradients
Authors: Aravind Rajeswaran, Chelsea Finn, Sham Kakade, Sergey Levine
Date: 2019-09-10 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Few-Shot Image Classification, Few-Shot Learning, Meta-Learning

A core capability of intelligent systems is the ability to quickly learn new tasks by drawing on prior experience. Gradient (or optimization) based meta-learning has recently emerged as an effective approach for few-shot learning. In this formulation, meta-parameters are learned in the outer loop, while task-specific models are learned in the inner-loop, by using only a small amount of data from the current task. A key challenge in scaling these approaches is the need to differentiate through the inner loop learning process, which can impose considerable computational and memory burdens. By drawing upon implicit differentiation, we develop the implicit MAML algorithm, which depends only on the solution to the inner level optimization and not the path taken by the inner loop optimizer. This effectively decouples the meta-gradient computation from the choice of inner loop optimizer. As a result, our approach is agnostic to the choice of inner loop optimizer and can gracefully handle many gradient steps without vanishing gradients or memory constraints. Theoretically, we prove that implicit MAML can compute accurate meta-gradients with a memory footprint that is, up to small constant factors, no more than that which is required to compute a single inner loop gradient and at no overall increase in the total computational cost. Experimentally, we show that these benefits of implicit MAML translate into empirical gains on few-shot image recognition benchmarks.
Representation of Constituents in Neural Language Models: Coordination Phrase as a Case Study
Authors: Aixiu An, Peng Qian, Ethan Wilcox, Roger Levy
Date: 2019-09-10 00:00:00
Proceeding: IJCNLP 2019 11

Neural language models have achieved state-of-the-art performances on many NLP tasks, and recently have been shown to learn a number of hierarchically-sensitive syntactic dependencies between individual words. However, equally important for language processing is the ability to combine words into phrasal constituents, and use constituent-level features to drive downstream expectations. Here we investigate neural models' ability to represent constituent-level features, using coordinated noun phrases as a case study. We assess whether different neural language models trained on English and French represent phrase-level number and gender features, and use those features to drive downstream expectations. Our results suggest that models use a linear combination of NP constituent number to drive CoordNP/verb number agreement. This behavior is highly regular and even sensitive to local syntactic context, however it differs crucially from observed human behavior. Models have less success with gender agreement. Models trained on large corpora perform best, and there is no obvious advantage for models trained using explicit syntactic supervision.
Deep Hashing Learning for Visual and Semantic Retrieval of Remote Sensing Images
Authors: Weiwei Song, Shutao Li, Jon Atli Benediktsson
Date: 2019-09-10 00:00:00
Tasks: Deep Hashing, Image Retrieval, Retrieval, Semantic Retrieval

Driven by the urgent demand for managing remote sensing big data, large-scale remote sensing image retrieval (RSIR) attracts increasing attention in the remote sensing field. In general, existing retrieval methods can be regarded as visual-based retrieval approaches which search and return a set of similar images from a database to a given query image. Although retrieval methods have achieved great success, there is still a question that needs to be responded to: Can we obtain the accurate semantic labels of the returned similar images to further help analyzing and processing imagery? Inspired by the above question, in this paper, we redefine the image retrieval problem as visual and semantic retrieval of images. Specifically, we propose a novel deep hashing convolutional neural network (DHCNN) to simultaneously retrieve the similar images and classify their semantic labels in a unified framework. In more detail, a convolutional neural network (CNN) is used to extract high-dimensional deep features. Then, a hash layer is perfectly inserted into the network to transfer the deep features into compact hash codes. In addition, a fully connected layer with a softmax function is performed on hash layer to generate class distribution. Finally, a loss function is elaborately designed to simultaneously consider the label loss of each image and similarity loss of pairs of images. Experimental results on two remote sensing datasets demonstrate that the proposed method achieves the state-of-art retrieval and classification performance.
Discovery of Useful Questions as Auxiliary Tasks
Authors: Vivek Veeriah, Matteo Hessel, Zhongwen Xu, Richard Lewis, Janarthanan Rajendran, Junhyuk Oh, Hado van Hasselt, David Silver, Satinder Singh
Date: 2019-09-10 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Reinforcement Learning, Reinforcement Learning (RL)

Arguably, intelligent agents ought to be able to discover their own questions so that in learning answers for them they learn unanticipated useful knowledge and skills; this departs from the focus in much of machine learning on agents learning answers to externally defined questions. We present a novel method for a reinforcement learning (RL) agent to discover questions formulated as general value functions or GVFs, a fairly rich form of knowledge representation. Specifically, our method uses non-myopic meta-gradients to learn GVF-questions such that learning answers to them, as an auxiliary task, induces useful representations for the main task faced by the RL agent. We demonstrate that auxiliary tasks based on the discovered GVFs are sufficient, on their own, to build representations that support main task learning, and that they do so better than popular hand-designed auxiliary tasks from the literature. Furthermore, we show, in the context of Atari 2600 videogames, how such auxiliary tasks, meta-learned alongside the main task, can improve the data efficiency of an actor-critic agent.
Patient trajectory prediction in the Mimic-III dataset, challenges and pitfalls
Authors: Jose F. Rodrigues-Jr, Gabriel Spadon, Bruno Brandoli, Sihem Amer-Yahia
Date: 2019-09-10 00:00:00
Tasks: Prognosis, Trajectory Prediction

Automated medical prognosis has gained interest as artificial intelligence evolves and the potential for computer-aided medicine becomes evident. Nevertheless, it is challenging to design an effective system that, given a patient's medical history, is able to predict probable future conditions. Previous works, mostly carried out over private datasets, have tackled the problem by using artificial neural network architectures that cannot deal with low-cardinality datasets, or by means of non-generalizable inference approaches. We introduce a Deep Learning architecture whose design results from an intensive experimental process. The final architecture is based on two parallel Minimal Gated Recurrent Unit networks working in bi-directional manner, which was extensively tested with the open-access Mimic-III dataset. Our results demonstrate significant improvements in automated medical prognosis, as measured with Recall@k. We summarize our experience as a set of relevant insights for the design of Deep Learning architectures. Our work improves the performance of computer-aided medicine and can serve as a guide in designing artificial neural networks used in prediction tasks.
Prediction of Overall Survival of Brain Tumor Patients
Authors: Rupal Agravat, Mehul S. Raval
Date: 2019-09-10 00:00:00
Tasks: Brain Tumor Segmentation, Prediction, Prognosis, Tumor Segmentation

Automated brain tumor segmentation plays an important role in the diagnosis and prognosis of the patient. In addition, features from the tumorous brain help in predicting patients overall survival. The main focus of this paper is to segment tumor from BRATS 2018 benchmark dataset and use age, shape and volumetric features to predict overall survival of patients. The random forest classifier achieves overall survival accuracy of 59% on the test dataset and 67% on the dataset with resection status as gross total resection. The proposed approach uses fewer features but achieves better accuracy than state of the art methods.
Structure-Attentioned Memory Network for Monocular Depth Estimation
Authors: Jing Zhu, Yunxiao Shi, Mengwei Ren, Yi Fang, Kuo-Chin Lien, Junli Gu
Date: 2019-09-10 00:00:00
Tasks: Depth Estimation, Domain Adaptation, Monocular Depth Estimation

Monocular depth estimation is a challenging task that aims to predict a corresponding depth map from a given single RGB image. Recent deep learning models have been proposed to predict the depth from the image by learning the alignment of deep features between the RGB image and the depth domains. In this paper, we present a novel approach, named Structure-Attentioned Memory Network, to more effectively transfer domain features for monocular depth estimation by taking into account the common structure regularities (e.g., repetitive structure patterns, planar surfaces, symmetries) in domain adaptation. To this end, we introduce a new Structure-Oriented Memory (SOM) module to learn and memorize the structure-specific information between RGB image domain and the depth domain. More specifically, in the SOM module, we develop a Memorable Bank of Filters (MBF) unit to learn a set of filters that memorize the structure-aware image-depth residual pattern, and also an Attention Guided Controller (AGC) unit to control the filter selection in the MBF given image features queries. Given the query image feature, the trained SOM module is able to adaptively select the best customized filters for cross-domain feature transferring with an optimal structural disparity between image and depth. In summary, we focus on addressing this structure-specific domain adaption challenge by proposing a novel end-to-end multi-scale memorable network for monocular depth estimation. The experiments show that our proposed model demonstrates the superior performance compared to the existing supervised monocular depth estimation approaches on the challenging KITTI and NYU Depth V2 benchmarks.
Deep MR Brain Image Super-Resolution Using Spatio-Structural Priors
Authors: Venkateswararao Cherukuri, Tiantong Guo, Steve. J. Schiff, Vishal Monga
Date: 2019-09-10 00:00:00
Tasks: Image Enhancement, Image Super-Resolution, Super-Resolution

High resolution Magnetic Resonance (MR) images are desired for accurate diagnostics. In practice, image resolution is restricted by factors like hardware and processing constraints. Recently, deep learning methods have been shown to produce compelling state-of-the-art results for image enhancement/super-resolution. Paying particular attention to desired hi-resolution MR image structure, we propose a new regularized network that exploits image priors, namely a low-rank structure and a sharpness prior to enhance deep MR image super-resolution (SR). Our contributions are then incorporating these priors in an analytically tractable fashion \color{black} as well as towards a novel prior guided network architecture that accomplishes the super-resolution task. This is particularly challenging for the low rank prior since the rank is not a differentiable function of the image matrix(and hence the network parameters), an issue we address by pursuing differentiable approximations of the rank. Sharpness is emphasized by the variance of the Laplacian which we show can be implemented by a fixed feedback layer at the output of the network. As a key extension, we modify the fixed feedback (Laplacian) layer by learning a new set of training data driven filters that are optimized for enhanced sharpness. Experiments performed on publicly available MR brain image databases and comparisons against existing state-of-the-art methods show that the proposed prior guided network offers significant practical gains in terms of improved SNR/image quality measures. Because our priors are on output images, the proposed method is versatile and can be combined with a wide variety of existing network architectures to further enhance their performance.
Scalable Structure Learning of Continuous-Time Bayesian Networks from Incomplete Data
Authors: Dominik Linzner, Michael Schmidt, Heinz Koeppl
Date: 2019-09-10 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Time Series, Time Series Analysis

Continuous-time Bayesian Networks (CTBNs) represent a compact yet powerful framework for understanding multivariate time-series data. Given complete data, parameters and structure can be estimated efficiently in closed-form. However, if data is incomplete, the latent states of the CTBN have to be estimated by laboriously simulating the intractable dynamics of the assumed CTBN. This is a problem, especially for structure learning tasks, where this has to be done for each element of a super-exponentially growing set of possible structures. In order to circumvent this notorious bottleneck, we develop a novel gradient-based approach to structure learning. Instead of sampling and scoring all possible structures individually, we assume the generator of the CTBN to be composed as a mixture of generators stemming from different structures. In this framework, structure learning can be performed via a gradient-based optimization of mixture weights. We combine this approach with a new variational method that allows for a closed-form calculation of this mixture marginal likelihood. We show the scalability of our method by learning structures of previously inaccessible sizes from synthetic and real-world data.
BINOCULARS for Efficient, Nonmyopic Sequential Experimental Design
Authors: Shali Jiang, Henry Chai, Javier Gonzalez, Roman Garnett
Date: 2019-09-10 00:00:00
Proceeding: ICML 2020 1
Tasks: Bayesian Optimization, Experimental Design

Finite-horizon sequential experimental design (SED) arises naturally in many contexts, including hyperparameter tuning in machine learning among more traditional settings. Computing the optimal policy for such problems requires solving Bellman equations, which are generally intractable. Most existing work resorts to severely myopic approximations by limiting the decision horizon to only a single time-step, which can underweight exploration in favor of exploitation. We present BINOCULARS: Batch-Informed NOnmyopic Choices, Using Long-horizons for Adaptive, Rapid SED, a general framework for deriving efficient, nonmyopic approximations to the optimal experimental policy. Our key idea is simple and surprisingly effective: we first compute a one-step optimal batch of experiments, then select a single point from this batch to evaluate. We realize BINOCULARS for Bayesian optimization and Bayesian quadrature -- two notable SED problems with radically different objectives -- and demonstrate that BINOCULARS significantly outperforms myopic alternatives in real-world scenarios.
Differentiable Mask for Pruning Convolutional and Recurrent Networks
Authors: Ramchalam Kinattinkara Ramakrishnan, Eyyüb Sari, Vahid Partovi Nia
Date: 2019-09-10 00:00:00
Tasks: Model Compression, Multi-Task Learning, Neural Architecture Search

Pruning is one of the most effective model reduction techniques. Deep networks require massive computation and such models need to be compressed to bring them on edge devices. Most existing pruning techniques are focused on vision-based models like convolutional networks, while text-based models are still evolving. The emergence of multi-modal multi-task learning calls for a general method that works on vision and text architectures simultaneously. We introduce a \emph{differentiable mask}, that induces sparsity on various granularity to fill this gap. We apply our method successfully to prune weights, filters, subnetwork of a convolutional architecture, as well as nodes of a recurrent network.