pwc-archive/papers-with-abstracts huggingface dataset:
A Stock Trading System for a Medium Volatile Asset using Multi Layer Perceptron
Authors: Ivan Letteri, Giuseppe Della Penna, Giovanni De Gasperis, Abeer Dyoub
Date: 2022-01-17 00:00:00

Stock market forecasting is a lucrative field of interest with promising profits but not without its difficulties and for some people could be even causes of failure. Financial markets by their nature are complex, non-linear and chaotic, which implies that accurately predicting the prices of assets that are part of it becomes very complicated. In this paper we propose a stock trading system having as main core the feed-forward deep neural networks (DNN) to predict the price for the next 30 days of open market, of the shares issued by Abercrombie & Fitch Co. (ANF) in the stock market of the New York Stock Exchange (NYSE). The system we have elaborated calculates the most effective technical indicator, applying it to the predictions computed by the DNNs, for generating trades. The results showed an increase in values such as Expectancy Ratio of 2.112% of profitable trades with Sharpe, Sortino, and Calmar Ratios of 2.194, 3.340, and 12.403 respectively. As a verification, we adopted a backtracking simulation module in our system, which maps trades to actual test data consisting of the last 30 days of open market on the ANF asset. Overall, the results were promising bringing a total profit factor of 3.2% in just one month from a very modest budget of $100. This was possible because the system reduced the number of trades by choosing the most effective and efficient trades, saving on commissions and slippage costs.
Stock exchange shares ranking and binary-ternary compressive coding
Authors: Igor Nesiolovskiy
Date: 2021-10-20 00:00:00
Tasks: Time Series, Time Series Analysis

This paper proposes a method for ranking the investment attractiveness of exchange-traded stocks where investment risk is not related to the volatility indicator but instead is related to the indicator of compression of the time series of price changes. The article describes in detail the ranking algorithm, provides an example of ranking the shares of all companies included in the Dow Jones stock index. The paper additionally compares the results of ranking these stocks by volatility and compression and also shows the strengths of the second indicator, which is formed using the method of binary-ternary compression of historical financial data.
Aligned Molecular Clouds towards SS433 and L=348.5 degrees; Possible Evidence for Galactic "Vapor Trail" Created by Relativistic Jet
Authors: H. Yamamoto, S. Ito, S. Ishigami, M. Fujishita, T. Kawase, A. Kawamura, N. Mizuno, T. Onishi, A. Mizuno, N. M. McClure-Griffiths, Y. Fukui
Date: 2008-04-11 00:00:00

We have carried out a detailed analysis of the NANTEN 12CO(J=1-0) dataset in two large areas of ~25 square degrees towards SS433 (l~40 degree) and of ~18 square degrees towards l~348.5 degree, respectively. We have discovered two groups of remarkably aligned molecular clouds at |b|~1--5 degree in the two regions. In SS433, we have detected 10 clouds in total, which are well aligned nearly along the axis of the X-ray jet emanating from SS433. These clouds have similar line-of-sight velocities of 42--56 km s^-1 and the total projected length of the feature is ~300 pc, three times larger than that of the X-ray jet, at a distance of 3 kpc. Towards l~348.5 degree, we have detected four clouds named as MJG348.5 at line-of-sight velocities of -80 -- -95 km s^-1 in V_LSR, which also show alignment nearly perpendicular to the Galactic plane. The total length of the feature is ~400 pc at a kinematic distance of 6 kpc. In the both cases, the CO clouds are distributed at high galactic latitudes where such clouds are very rare. In addition, their alignments and coincidence in velocity should be even rarer, suggesting that they are physically associated. We tested a few possibilities to explain these clouds, including protostellar outflows, supershells, and interactions with energetic jets. Among them, a favorable scenario is that the interaction between relativistic jet and the interstellar medium induced the formation of molecular clouds over the last ~10^5-6 yrs. It is suggested that the timescale of the relativistic jet may be considerably larger, in the order of 10^5-6 yrs, than previously thought in SS433. The driving engine of the jet is obviously SS433 itself in SS433, although the engine is not yet identified in MJG348.5 among possible several candidates detected in the X-rays and TeV gamma rays.
Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanism
Authors: Siqi Miao, Miaoyuan Liu, Pan Li
Date: 2022-01-31 00:00:00
Tasks: Graph Learning

Interpretable graph learning is in need as many scientific applications depend on learning models to collect insights from graph-structured data. Previous works mostly focused on using post-hoc approaches to interpret pre-trained models (graph neural networks in particular). They argue against inherently interpretable models because the good interpretability of these models is often at the cost of their prediction accuracy. However, those post-hoc methods often fail to provide stable interpretation and may extract features that are spuriously correlated with the task. In this work, we address these issues by proposing Graph Stochastic Attention (GSAT). Derived from the information bottleneck principle, GSAT injects stochasticity to the attention weights to block the information from task-irrelevant graph components while learning stochasticity-reduced attention to select task-relevant subgraphs for interpretation. The selected subgraphs provably do not contain patterns that are spuriously correlated with the task under some assumptions. Extensive experiments on eight datasets show that GSAT outperforms the state-of-the-art methods by up to 20%$\uparrow$ in interpretation AUC and 5%$\uparrow$ in prediction accuracy. Our code is available at https://github.com/Graph-COM/GSAT.
Bayesian Optimization For Multi-Objective Mixed-Variable Problems
Authors: Haris Moazam Sheikh, Philip S. Marcus
Date: 2022-01-30 00:00:00
Tasks: Bayesian Optimization

Optimizing multiple, non-preferential objectives for mixed-variable, expensive black-box problems is important in many areas of engineering and science. The expensive, noisy, black-box nature of these problems makes them ideal candidates for Bayesian optimization (BO). Mixed-variable and multi-objective problems, however, are a challenge due to BO's underlying smooth Gaussian process surrogate model. Current multi-objective BO algorithms cannot deal with mixed-variable problems. We present MixMOBO, the first mixed-variable, multi-objective Bayesian optimization framework for such problems. Using MixMOBO, optimal Pareto-fronts for multi-objective, mixed-variable design spaces can be found efficiently while ensuring diverse solutions. The method is sufficiently flexible to incorporate different kernels and acquisition functions, including those that were developed for mixed-variable or multi-objective problems by other authors. We also present HedgeMO, a modified Hedge strategy that uses a portfolio of acquisition functions for multi-objective problems. We present a new acquisition function, SMC. Our results show that MixMOBO performs well against other mixed-variable algorithms on synthetic problems. We apply MixMOBO to the real-world design of an architected material and show that our optimal design, which was experimentally fabricated and validated, has a normalized strain energy density $10^4$ times greater than existing structures.
Simple Calibration via Geodesic Kernels
Authors: Jayanta Dey, Haoyin Xu, Ashwin De Silva, Joshua T. Vogelstein
Date: 2022-01-31 00:00:00
Tasks: Out-of-Distribution Detection, regression

Deep discriminative approaches, such as decision forests and deep neural networks, have recently found applications in many important real-world scenarios. However, deploying these learning algorithms in safety-critical applications raises concerns, particularly when it comes to ensuring calibration for both in-distribution and out-of-distribution regions. Many popular methods for in-distribution (ID) calibration, such as isotonic and Platt's sigmoidal regression, exhibit adequate ID calibration performance. However, these methods are not calibrated for the entire feature space, leading to overconfidence in the out-of-distribution (OOD) region. Existing OOD calibration methods generally exhibit poor ID calibration. In this paper, we jointly address the ID and OOD problems. We leveraged the fact that deep models learn to partition feature space into a union of polytopes, that is, flat-sided geometric objects. We introduce a geodesic distance to measure the distance between these polytopes and further distinguish samples within the same polytope using a Gaussian kernel. Our experiments on both tabular and vision benchmarks show that the proposed approaches, namely Kernel Density Forest (KDF) and Kernel Density Network (KDN), obtain well-calibrated posteriors for both ID and OOD samples, while mostly preserving the classification accuracy and extrapolating beyond the training data to handle OOD inputs appropriately.
Interconnect Parasitics and Partitioning in Fully-Analog In-Memory Computing Architectures
Authors: Md Hasibul Amin, Mohammed Elbtity, Ramtin Zand
Date: 2022-01-29 00:00:00
Tasks: CPU

Fully-analog in-memory computing (IMC) architectures that implement both matrix-vector multiplication and non-linear vector operations within the same memory array have shown promising performance benefits over conventional IMC systems due to the removal of energy-hungry signal conversion units. However, maintaining the computation in the analog domain for the entire deep neural network (DNN) comes with potential sensitivity to interconnect parasitics. Thus, in this paper, we investigate the effect of wire parasitic resistance and capacitance on the accuracy of DNN models deployed on fully-analog IMC architectures. Moreover, we propose a partitioning mechanism to alleviate the impact of the parasitic while keeping the computation in the analog domain through dividing large arrays into multiple partitions. The SPICE circuit simulation results for a 400 X 120 X 84 X 10 DNN model deployed on a fully-analog IMC circuit show that a 94.84% accuracy could be achieved for MNIST classification application with 16, 8, and 8 horizontal partitions, as well as 8, 8, and 1 vertical partitions for first, second, and third layers of the DNN, respectively, which is comparable to the ~97% accuracy realized by digital implementation on CPU. It is shown that accuracy benefits are achieved at the cost of higher power consumption due to the extra circuitry required for handling partitioning.
Fuzzy Segmentations of a String
Authors: Armen Kostanyan, Arevik Harmandayan
Date: 2022-01-31 00:00:00
Tasks: Clustering, Segmentation, Text Segmentation

This article discusses a particular case of the data clustering problem, where it is necessary to find groups of adjacent text segments of the appropriate length that match a fuzzy pattern represented as a sequence of fuzzy properties. To solve this problem, a heuristic algorithm for finding a sufficiently large number of solutions is proposed. The key idea of the proposed algorithm is the use of the prefix structure to track the process of mapping text segments to fuzzy properties. An important special case of the text segmentation problem is the fuzzy string matching problem, when adjacent text segments have unit length and, accordingly, the fuzzy pattern is a sequence of fuzzy properties of text characters. It is proven that the heuristic segmentation algorithm in this case finds all text segments that match the fuzzy pattern. Finally, we consider the problem of a best segmentation of the entire text based on a fuzzy pattern, which is solved using the dynamic programming method. Keywords: fuzzy clustering, fuzzy string matching, approximate string matching
Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learning
Authors: Denis Yarats, David Brandfonbrener, Hao liu, Michael Laskin, Pieter Abbeel, Alessandro Lazaric, Lerrel Pinto
Date: 2022-01-31 00:00:00
Tasks: Diversity, Offline RL, reinforcement-learning, Reinforcement Learning (RL)

Recent progress in deep learning has relied on access to large and diverse datasets. Such data-driven progress has been less evident in offline reinforcement learning (RL), because offline RL data is usually collected to optimize specific target tasks limiting the data's diversity. In this work, we propose Exploratory data for Offline RL (ExORL), a data-centric approach to offline RL. ExORL first generates data with unsupervised reward-free exploration, then relabels this data with a downstream reward before training a policy with offline RL. We find that exploratory data allows vanilla off-policy RL algorithms, without any offline-specific modifications, to outperform or match state-of-the-art offline RL algorithms on downstream tasks. Our findings suggest that data generation is as important as algorithmic advances for offline RL and hence requires careful consideration from the community. Code and data can be found at https://github.com/denisyarats/exorl .
NAS-Bench-Suite: NAS Evaluation is (Now) Surprisingly Easy
Authors: Yash Mehta, Colin White, Arber Zela, Arjun Krishnakumar, Guri Zabergja, Shakiba Moradian, Mahmoud Safari, Kaicheng Yu, Frank Hutter
Date: 2022-01-31 00:00:00
Proceeding: ICLR 2022 4
Tasks: AutoML, image-classification, Image Classification, Neural Architecture Search, object-detection, Object Detection, speech-recognition, Speech Recognition

The release of tabular benchmarks, such as NAS-Bench-101 and NAS-Bench-201, has significantly lowered the computational overhead for conducting scientific research in neural architecture search (NAS). Although they have been widely adopted and used to tune real-world NAS algorithms, these benchmarks are limited to small search spaces and focus solely on image classification. Recently, several new NAS benchmarks have been introduced that cover significantly larger search spaces over a wide range of tasks, including object detection, speech recognition, and natural language processing. However, substantial differences among these NAS benchmarks have so far prevented their widespread adoption, limiting researchers to using just a few benchmarks. In this work, we present an in-depth analysis of popular NAS algorithms and performance prediction methods across 25 different combinations of search spaces and datasets, finding that many conclusions drawn from a few NAS benchmarks do not generalize to other benchmarks. To help remedy this problem, we introduce NAS-Bench-Suite, a comprehensive and extensible collection of NAS benchmarks, accessible through a unified interface, created with the aim to facilitate reproducible, generalizable, and rapid NAS research. Our code is available at https://github.com/automl/naslib.
Signing the Supermask: Keep, Hide, Invert
Authors: Nils Koster, Oliver Grothe, Achim Rettinger
Date: 2022-01-31 00:00:00
Proceeding: ICLR 2022 4

The exponential growth in numbers of parameters of neural networks over the past years has been accompanied by an increase in performance across several fields. However, due to their sheer size, the networks not only became difficult to interpret but also problematic to train and use in real-world applications, since hardware requirements increased accordingly. Tackling both issues, we present a novel approach that either drops a neural network's initial weights or inverts their respective sign. Put simply, a network is trained by weight selection and inversion without changing their absolute values. Our contribution extends previous work on masking by additionally sign-inverting the initial weights and follows the findings of the Lottery Ticket Hypothesis. Through this extension and adaptations of initialization methods, we achieve a pruning rate of up to 99%, while still matching or exceeding the performance of various baseline and previous models. Our approach has two main advantages. First, and most notable, signed Supermask models drastically simplify a model's structure, while still performing well on given tasks. Second, by reducing the neural network to its very foundation, we gain insights into which weights matter for performance. The code is available on GitHub.
Optimizing LLVM Pass Sequences with Shackleton: A Linear Genetic Programming Framework
Authors: Hannah Peeler, Shuyue Stella Li, Andrew N. Sloss, Kenneth N. Reid, Yuan Yuan, Wolfgang Banzhaf
Date: 2022-01-31 00:00:00
Tasks: Evolutionary Algorithms

In this paper we introduce Shackleton as a generalized framework enabling the application of linear genetic programming -- a technique under the umbrella of evolutionary algorithms -- to a variety of use cases. We also explore here a novel application for this class of methods: optimizing sequences of LLVM optimization passes. The algorithm underpinning Shackleton is discussed, with an emphasis on the effects of different features unique to the framework when applied to LLVM pass sequences. Combined with analysis of different hyperparameter settings, we report the results on automatically optimizing pass sequences using Shackleton for two software applications at differing complexity levels. Finally, we reflect on the advantages and limitations of our current implementation and lay out a path for further improvements. These improvements aim to surpass hand-crafted solutions with an automatic discovery method for an optimal pass sequence.
Metrics for saliency map evaluation of deep learning explanation methods
Authors: Tristan Gomez, Thomas Fréour, Harold Mouchère
Date: 2022-01-31 00:00:00
Tasks: Deep Learning

Due to the black-box nature of deep learning models, there is a recent development of solutions for visual explanations of CNNs. Given the high cost of user studies, metrics are necessary to compare and evaluate these different methods. In this paper, we critically analyze the Deletion Area Under Curve (DAUC) and Insertion Area Under Curve (IAUC) metrics proposed by Petsiuk et al. (2018). These metrics were designed to evaluate the faithfulness of saliency maps generated by generic methods such as Grad-CAM or RISE. First, we show that the actual saliency score values given by the saliency map are ignored as only the ranking of the scores is taken into account. This shows that these metrics are insufficient by themselves, as the visual appearance of a saliency map can change significantly without the ranking of the scores being modified. Secondly, we argue that during the computation of DAUC and IAUC, the model is presented with images that are out of the training distribution which might lead to an unreliable behavior of the model being explained. To complement DAUC/IAUC, we propose new metrics that quantify the sparsity and the calibration of explanation methods, two previously unstudied properties. Finally, we give general remarks about the metrics studied in this paper and discuss how to evaluate them in a user study.
StRegA: Unsupervised Anomaly Detection in Brain MRIs using a Compact Context-encoding Variational Autoencoder
Authors: Soumick Chatterjee, Alessandro Sciarra, Max Dünnwald, Pavan Tummala, Shubham Kumar Agrawal, Aishwarya Jauhari, Aman Kalra, Steffen Oeltze-Jafra, Oliver Speck, Andreas Nürnberger
Date: 2022-01-31 00:00:00
Tasks: Anomaly Detection, Unsupervised Anomaly Detection

Expert interpretation of anatomical images of the human brain is the central part of neuro-radiology. Several machine learning-based techniques have been proposed to assist in the analysis process. However, the ML models typically need to be trained to perform a specific task, e.g., brain tumour segmentation or classification. Not only do the corresponding training data require laborious manual annotations, but a wide variety of abnormalities can be present in a human brain MRI - even more than one simultaneously, which renders representation of all possible anomalies very challenging. Hence, a possible solution is an unsupervised anomaly detection (UAD) system that can learn a data distribution from an unlabelled dataset of healthy subjects and then be applied to detect out of distribution samples. Such a technique can then be used to detect anomalies - lesions or abnormalities, for example, brain tumours, without explicitly training the model for that specific pathology. Several Variational Autoencoder (VAE) based techniques have been proposed in the past for this task. Even though they perform very well on controlled artificially simulated anomalies, many of them perform poorly while detecting anomalies in clinical data. This research proposes a compact version of the "context-encoding" VAE (ceVAE) model, combined with pre and post-processing steps, creating a UAD pipeline (StRegA), which is more robust on clinical data, and shows its applicability in detecting anomalies such as tumours in brain MRIs. The proposed pipeline achieved a Dice score of 0.642$\pm$0.101 while detecting tumours in T2w images of the BraTS dataset and 0.859$\pm$0.112 while detecting artificially induced anomalies, while the best performing baseline achieved 0.522$\pm$0.135 and 0.783$\pm$0.111, respectively.
Memory-Efficient Backpropagation through Large Linear Layers
Authors: Daniel Bershatsky, Aleksandr Mikhalev, Alexandr Katrutsa, Julia Gusak, Daniil Merkulov, Ivan Oseledets
Date: 2022-01-31 00:00:00
Tasks: Model Compression

In modern neural networks like Transformers, linear layers require significant memory to store activations during backward pass. This study proposes a memory reduction approach to perform backpropagation through linear layers. Since the gradients of linear layers are computed by matrix multiplications, we consider methods for randomized matrix multiplications and demonstrate that they require less memory with a moderate decrease of the test accuracy. Also, we investigate the variance of the gradient estimate induced by the randomized matrix multiplication. We compare this variance with the variance coming from gradient estimation based on the batch of samples. We demonstrate the benefits of the proposed method on the fine-tuning of the pre-trained RoBERTa model on GLUE tasks.
Few-Shot Backdoor Attacks on Visual Object Tracking
Authors: Yiming Li, Haoxiang Zhong, Xingjun Ma, Yong Jiang, Shu-Tao Xia
Date: 2022-01-31 00:00:00
Proceeding: ICLR 2022 4
Tasks: Autonomous Driving, Backdoor Attack, Object, Object Tracking, Visual Object Tracking

Visual object tracking (VOT) has been widely adopted in mission-critical applications, such as autonomous driving and intelligent surveillance systems. In current practice, third-party resources such as datasets, backbone networks, and training platforms are frequently used to train high-performance VOT models. Whilst these resources bring certain convenience, they also introduce new security threats into VOT models. In this paper, we reveal such a threat where an adversary can easily implant hidden backdoors into VOT models by tempering with the training process. Specifically, we propose a simple yet effective few-shot backdoor attack (FSBA) that optimizes two losses alternately: 1) a \emph{feature loss} defined in the hidden feature space, and 2) the standard \emph{tracking loss}. We show that, once the backdoor is embedded into the target model by our FSBA, it can trick the model to lose track of specific objects even when the \emph{trigger} only appears in one or a few frames. We examine our attack in both digital and physical-world settings and show that it can significantly degrade the performance of state-of-the-art VOT trackers. We also show that our attack is resistant to potential defenses, highlighting the vulnerability of VOT models to potential backdoor attacks.