pwc-archive/papers-with-abstracts huggingface dataset:
Prediction of financial time series using LSTM and data denoising methods
Authors: Qi Tang, Tongmei Fan, Ruchen Shi, Jingyan Huang, Yidan Ma
Date: 2021-03-05 00:00:00
Tasks: Denoising, Time Series, Time Series Analysis

In order to further overcome the difficulties of the existing models in dealing with the non-stationary and nonlinear characteristics of high-frequency financial time series data, especially its weak generalization ability, this paper proposes an ensemble method based on data denoising methods, including the wavelet transform (WT) and singular spectrum analysis (SSA), and long-term short-term memory neural network (LSTM) to build a data prediction model, The financial time series is decomposed and reconstructed by WT and SSA to denoise. Under the condition of denoising, the smooth sequence with effective information is reconstructed. The smoothing sequence is introduced into LSTM and the predicted value is obtained. With the Dow Jones industrial average index (DJIA) as the research object, the closing price of the DJIA every five minutes is divided into short-term (1 hour), medium-term (3 hours) and long-term (6 hours) respectively. . Based on root mean square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE) and absolute percentage error standard deviation (SDAPE), the experimental results show that in the short-term, medium-term and long-term, data denoising can greatly improve the accuracy and stability of the prediction, and can effectively improve the generalization ability of LSTM prediction model. As WT and SSA can extract useful information from the original sequence and avoid overfitting, the hybrid model can better grasp the sequence pattern of the closing price of the DJIA. And the WT-LSTM model is better than the benchmark LSTM model and SSA-LSTM model.
NPT-Loss: A Metric Loss with Implicit Mining for Face Recognition
Authors: Syed Safwan Khalid, Muhammad Awais, Chi-Ho Chan, ZhenHua Feng, Ammarah Farooq, Ali Akbari, Josef Kittler
Date: 2021-03-05 00:00:00
Tasks: Face Recognition, Triplet

Face recognition (FR) using deep convolutional neural networks (DCNNs) has seen remarkable success in recent years. One key ingredient of DCNN-based FR is the appropriate design of a loss function that ensures discrimination between various identities. The state-of-the-art (SOTA) solutions utilise normalised Softmax loss with additive and/or multiplicative margins. Despite being popular, these Softmax+margin based losses are not theoretically motivated and the effectiveness of a margin is justified only intuitively. In this work, we utilise an alternative framework that offers a more direct mechanism of achieving discrimination among the features of various identities. We propose a novel loss that is equivalent to a triplet loss with proxies and an implicit mechanism of hard-negative mining. We give theoretical justification that minimising the proposed loss ensures a minimum separability between all identities. The proposed loss is simple to implement and does not require heavy hyper-parameter tuning as in the SOTA solutions. We give empirical evidence that despite its simplicity, the proposed loss consistently achieves SOTA performance in various benchmarks for both high-resolution and low-resolution FR tasks.
Unsupervised Learning for Robust Fitting:A Reinforcement Learning Approach
Authors: Giang Truong, Huu Le, David Suter, Erchuan Zhang, Syed Zulqarnain Gilani
Date: 2021-03-05 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Robust model fitting is a core algorithm in a large number of computer vision applications. Solving this problem efficiently for datasets highly contaminated with outliers is, however, still challenging due to the underlying computational complexity. Recent literature has focused on learning-based algorithms. However, most approaches are supervised which require a large amount of labelled training data. In this paper, we introduce a novel unsupervised learning framework that learns to directly solve robust model fitting. Unlike other methods, our work is agnostic to the underlying input features, and can be easily generalized to a wide variety of LP-type problems with quasi-convex residuals. We empirically show that our method outperforms existing unsupervised learning approaches, and achieves competitive results compared to traditional methods on several important computer vision problems.
Stratified Sampling for Extreme Multi-Label Data
Authors: Maximillian Merrillees, Lan Du
Date: 2021-03-05 00:00:00
Tasks: Extreme Multi-Label Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

Extreme multi-label classification (XML) is becoming increasingly relevant in the era of big data. Yet, there is no method for effectively generating stratified partitions of XML datasets. Instead, researchers typically rely on provided test-train splits that, 1) aren't always representative of the entire dataset, and 2) are missing many of the labels. This can lead to poor generalization ability and unreliable performance estimates, as has been established in the binary and multi-class settings. As such, this paper presents a new and simple algorithm that can efficiently generate stratified partitions of XML datasets with millions of unique labels. We also examine the label distributions of prevailing benchmark splits, and investigate the issues that arise from using unrepresentative subsets of data for model development. The results highlight the difficulty of stratifying XML data, and demonstrate the importance of using stratified partitions for training and evaluation.
Causal Attention for Vision-Language Tasks
Authors: Xu Yang, Hanwang Zhang, GuoJun Qi, Jianfei Cai
Date: 2021-03-05 00:00:00
Proceeding: CVPR 2021 1

We present a novel attention mechanism: Causal Attention (CATT), to remove the ever-elusive confounding effect in existing attention-based vision-language models. This effect causes harmful bias that misleads the attention module to focus on the spurious correlations in training data, damaging the model generalization. As the confounder is unobserved in general, we use the front-door adjustment to realize the causal intervention, which does not require any knowledge on the confounder. Specifically, CATT is implemented as a combination of 1) In-Sample Attention (IS-ATT) and 2) Cross-Sample Attention (CS-ATT), where the latter forcibly brings other samples into every IS-ATT, mimicking the causal intervention. CATT abides by the Q-K-V convention and hence can replace any attention module such as top-down attention and self-attention in Transformers. CATT improves various popular attention-based vision-language models by considerable margins. In particular, we show that CATT has great potential in large-scale pre-training, e.g., it can promote the lighter LXMERT~\cite{tan2019lxmert}, which uses fewer data and less computational power, comparable to the heavier UNITER~\cite{chen2020uniter}. Code is published in \url{https://github.com/yangxuntu/catt}.
Moving from Cross-Project Defect Prediction to Heterogeneous Defect Prediction: A Partial Replication Study
Authors: Hadi Jahanshahi, Mucahit Cevik, Ayşe Başar
Date: 2021-03-05 00:00:00
Tasks: Prediction, Transfer Learning

Software defect prediction heavily relies on the metrics collected from software projects. Earlier studies often used machine learning techniques to build, validate, and improve bug prediction models using either a set of metrics collected within a project or across different projects. However, techniques applied and conclusions derived by those models are restricted by how identical those metrics are. Knowledge coming from those models will not be extensible to a target project if no sufficient overlapping metrics have been collected in the source projects. To explore the feasibility of transferring knowledge across projects without common labeled metrics, we systematically integrated Heterogeneous Defect Prediction (HDP) by replicating and validating the obtained results. Our main goal is to extend prior research and explore the feasibility of HDP and finally to compare its performance with that of its predecessor, Cross-Project Defect Prediction. We construct an HDP model on different publicly available datasets. Moreover, we propose a new ensemble voting approach in the HDP context to utilize the predictive power of multiple available datasets. The result of our experiment is comparable to that of the original study. However, we also explored the feasibility of HDP in real cases. Our results shed light on the infeasibility of many cases for the HDP algorithm due to its sensitivity to the parameter selection. In general, our analysis gives a deep insight into why and how to perform transfer learning from one domain to another, and in particular, provides a set of guidelines to help researchers and practitioners to disseminate knowledge to the defect prediction domain.
Adaptive Gaussian Fuzzy Classifier for Real-Time Emotion Recognition in Computer Games
Authors: Daniel Leite, Volnei Frigeri Jr., Rodrigo Medeiros
Date: 2021-03-05 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Emotion Recognition

Human emotion recognition has become a need for more realistic and interactive machines and computer systems. The greatest challenge is the availability of high-performance algorithms to effectively manage individual differences and nonstationarities in physiological data streams, i.e., algorithms that self-customize to a user with no subject-specific calibration data. We describe an evolving Gaussian Fuzzy Classifier (eGFC), which is supported by an online semi-supervised learning algorithm to recognize emotion patterns from electroencephalogram (EEG) data streams. We extract features from the Fourier spectrum of EEG data. The data are provided by 28 individuals playing the games 'Train Sim World', 'Unravel', 'Slender The Arrival', and 'Goat Simulator' - a public dataset. Different emotions prevail, namely, boredom, calmness, horror and joy. We analyze the effect of individual electrodes, time window lengths, and frequency bands on the accuracy of user-independent eGFCs. We conclude that both brain hemispheres may assist classification, especially electrodes on the frontal (Af3-Af4), occipital (O1-O2), and temporal (T7-T8) areas. We observe that patterns may be eventually found in any frequency band; however, the Alpha (8-13Hz), Delta (1-4Hz), and Theta (4-8Hz) bands, in this order, are the highest correlated with emotion classes. eGFC has shown to be effective for real-time learning of EEG data. It reaches a 72.2% accuracy using a variable rule base, 10-second windows, and 1.8ms/sample processing time in a highly-stochastic time-varying 4-class classification problem.
Environmental Sound Classification on the Edge: A Pipeline for Deep Acoustic Networks on Extremely Resource-Constrained Devices
Authors: Md Mohaimenuzzaman, Christoph Bergmeir, Ian Thomas West, Bernd Meyer
Date: 2021-03-05 00:00:00
Tasks: Audio Classification, Environmental Sound Classification, GPU, Model Compression, Quantization, Sound Classification

Significant efforts are being invested to bring state-of-the-art classification and recognition to edge devices with extreme resource constraints (memory, speed, and lack of GPU support). Here, we demonstrate the first deep network for acoustic recognition that is small, flexible and compression-friendly yet achieves state-of-the-art performance for raw audio classification. Rather than handcrafting a once-off solution, we present a generic pipeline that automatically converts a large deep convolutional network via compression and quantization into a network for resource-impoverished edge devices. After introducing ACDNet, which produces above state-of-the-art accuracy on ESC-10 (96.65%), ESC-50 (87.10%), UrbanSound8K (84.45%) and AudioEvent (92.57%), we describe the compression pipeline and show that it allows us to achieve 97.22% size reduction and 97.28% FLOP reduction while maintaining close to state-of-the-art accuracy 96.25%, 83.65%, 78.27% and 89.69% on these datasets. We describe a successful implementation on a standard off-the-shelf microcontroller and, beyond laboratory benchmarks, report successful tests on real-world datasets.
Pilot Investigation for a Comprehensive Taxonomy of Autonomous Entities
Authors: William Wagner, Anna Źakowska, Clement Aladi, Joseph Santhosh
Date: 2021-03-05 00:00:00
Tasks: Miscellaneous

This paper documents an exploratory pilot study to define the term Autonomous Entity, and any characteristics that are required to identify or classify an Autonomous Entity. Our solution builds on previous work with regard to philosophical and scientific classification methods but focuses on a novel Design Science Research Methodology (DSRM) and model to help identify those characteristics which make any autonomous entity similar or different from others. We have solved the problem of not having an existing term to define our lens by creating a new combinatorial term: "Riskyishness". We present a DSRM and instrument for initial investigation, as well as observational and statistical descriptions of their use in the real world to solicit domain expertise and statistical evidence. Further, we demonstrate a specific application of the methodology by creating a second artifact - a tool to score existing and future technologies based on Riskyishness. The first artifact also provides a technique to disentangle miscellaneous existing technologies or add dimensions to the tools to capture future additions and paradigm shifts.
IAFA: Instance-aware Feature Aggregation for 3D Object Detection from a Single Image
Authors: Dingfu Zhou, Xibin Song, Yuchao Dai, Junbo Yin, Feixiang Lu, Jin Fang, Miao Liao, Liangjun Zhang
Date: 2021-03-05 00:00:00
Tasks: 3D Object Detection, Autonomous Driving, Depth Estimation, Monocular 3D Object Detection, object-detection, Object Detection, vehicle detection

3D object detection from a single image is an important task in Autonomous Driving (AD), where various approaches have been proposed. However, the task is intrinsically ambiguous and challenging as single image depth estimation is already an ill-posed problem. In this paper, we propose an instance-aware approach to aggregate useful information for improving the accuracy of 3D object detection with the following contributions. First, an instance-aware feature aggregation (IAFA) module is proposed to collect local and global features for 3D bounding boxes regression. Second, we empirically find that the spatial attention module can be well learned by taking coarse-level instance annotations as a supervision signal. The proposed module has significantly boosted the performance of the baseline method on both 3D detection and 2D bird-eye's view of vehicle detection among all three categories. Third, our proposed method outperforms all single image-based approaches (even these methods trained with depth as auxiliary inputs) and achieves state-of-the-art 3D detection performance on the KITTI benchmark.
A Novel Framework for Threat Analysis of Machine Learning-based Smart Healthcare Systems
Authors: Nur Imtiazul Haque, Mohammad Ashiqur Rahman, Md Hasan Shahriar, Alvi Ataur Khalil, Selcuk Uluagac
Date: 2021-03-05 00:00:00
Tasks: BIG-bench Machine Learning

Smart healthcare systems (SHSs) are providing fast and efficient disease treatment leveraging wireless body sensor networks (WBSNs) and implantable medical devices (IMDs)-based internet of medical things (IoMT). In addition, IoMT-based SHSs are enabling automated medication, allowing communication among myriad healthcare sensor devices. However, adversaries can launch various attacks on the communication network and the hardware/firmware to introduce false data or cause data unavailability to the automatic medication system endangering the patient's life. In this paper, we propose SHChecker, a novel threat analysis framework that integrates machine learning and formal analysis capabilities to identify potential attacks and corresponding effects on an IoMT-based SHS. Our framework can provide us with all potential attack vectors, each representing a set of sensor measurements to be altered, for an SHS given a specific set of attack attributes, allowing us to realize the system's resiliency, thus the insight to enhance the robustness of the model. We implement SHChecker on a synthetic and a real dataset, which affirms that our framework can reveal potential attack vectors in an IoMT system. This is a novel effort to formally analyze supervised and unsupervised machine learning models for black-box SHS threat analysis.
Joint Network Topology Inference via Structured Fusion Regularization
Authors: Yanli Yuan, De Wen Soh, Xiao Yang, Kun Guo, Tony Q. S. Quek
Date: 2021-03-05 00:00:00
Tasks: Computational Efficiency

Joint network topology inference represents a canonical problem of jointly learning multiple graph Laplacian matrices from heterogeneous graph signals. In such a problem, a widely employed assumption is that of a simple common component shared among multiple networks. However, in practice, a more intricate topological pattern, comprising simultaneously of sparse, homogeneity and heterogeneity components, would exhibit in multiple networks. In this paper, we propose a general graph estimator based on a novel structured fusion regularization that enables us to jointly learn multiple graph Laplacian matrices with such complex topological patterns, and enjoys both high computational efficiency and rigorous theoretical guarantee. Moreover, in the proposed regularization term, the topological pattern among networks is characterized by a Gram matrix, endowing our graph estimator with the ability of flexible modelling different types of topological patterns by different choices of the Gram matrix. Computationally, the regularization term, coupling the parameters together, makes the formulated optimization problem intractable and thus, we develop a computationally-scalable algorithm based on the alternating direction method of multipliers (ADMM) to solve it efficiently. Theoretically, we provide a theoretical analysis of the proposed graph estimator, which establishes a non-asymptotic bound of the estimation error under the high-dimensional setting and reflects the effect of several key factors on the convergence rate of our algorithm. Finally, the superior performance of the proposed method is illustrated through simulated and real data examples.
Teachers Do More Than Teach: Compressing Image-to-Image Models
Authors: Qing Jin, Jian Ren, Oliver J. Woodford, Jiazhuo Wang, Geng Yuan, Yanzhi Wang, Sergey Tulyakov
Date: 2021-03-05 00:00:00
Proceeding: CVPR 2021 1
Tasks: Knowledge Distillation

Generative Adversarial Networks (GANs) have achieved huge success in generating high-fidelity images, however, they suffer from low efficiency due to tremendous computational cost and bulky memory usage. Recent efforts on compression GANs show noticeable progress in obtaining smaller generators by sacrificing image quality or involving a time-consuming searching process. In this work, we aim to address these issues by introducing a teacher network that provides a search space in which efficient network architectures can be found, in addition to performing knowledge distillation. First, we revisit the search space of generative models, introducing an inception-based residual block into generators. Second, to achieve target computation cost, we propose a one-step pruning algorithm that searches a student architecture from the teacher model and substantially reduces searching cost. It requires no l1 sparsity regularization and its associated hyper-parameters, simplifying the training procedure. Finally, we propose to distill knowledge through maximizing feature similarity between teacher and student via an index named Global Kernel Alignment (GKA). Our compressed networks achieve similar or even better image fidelity (FID, mIoU) than the original models with much-reduced computational cost, e.g., MACs. Code will be released at https://github.com/snap-research/CAT.
Unintended Effects on Adaptive Learning Rate for Training Neural Network with Output Scale Change
Authors: Ryuichi Kanoh, Mahito Sugiyama
Date: 2021-03-05 00:00:00

A multiplicative constant scaling factor is often applied to the model output to adjust the dynamics of neural network parameters. This has been used as one of the key interventions in an empirical study of lazy and active behavior. However, we show that the combination of such scaling and a commonly used adaptive learning rate optimizer strongly affects the training behavior of the neural network. This is problematic as it can cause \emph{unintended behavior} of neural networks, resulting in the misinterpretation of experimental results. Specifically, for some scaling settings, the effect of the adaptive learning rate disappears or is strongly influenced by the scaling factor. To avoid the unintended effect, we present a modification of an optimization algorithm and demonstrate remarkable differences between adaptive learning rate optimization and simple gradient descent, especially with a small ($<1.0$) scaling factor.
Unsupervised Motion Representation Enhanced Network for Action Recognition
Authors: Xiaohang Yang, Lingtong Kong, Jie Yang
Date: 2021-03-05 00:00:00
Tasks: Action Recognition, Optical Flow Estimation, Representation Learning, Video Understanding

Learning reliable motion representation between consecutive frames, such as optical flow, has proven to have great promotion to video understanding. However, the TV-L1 method, an effective optical flow solver, is time-consuming and expensive in storage for caching the extracted optical flow. To fill the gap, we propose UF-TSN, a novel end-to-end action recognition approach enhanced with an embedded lightweight unsupervised optical flow estimator. UF-TSN estimates motion cues from adjacent frames in a coarse-to-fine manner and focuses on small displacement for each level by extracting pyramid of feature and warping one to the other according to the estimated flow of the last level. Due to the lack of labeled motion for action datasets, we constrain the flow prediction with multi-scale photometric consistency and edge-aware smoothness. Compared with state-of-the-art unsupervised motion representation learning methods, our model achieves better accuracy while maintaining efficiency, which is competitive with some supervised or more complicated approaches.
Vicinal and categorical domain adaptation
Authors: Hui Tang, Kui Jia
Date: 2021-03-05 00:00:00
Tasks: Domain Adaptation, Unsupervised Domain Adaptation

Unsupervised domain adaptation aims to learn a task classifier that performs well on the unlabeled target domain, by utilizing the labeled source domain. Inspiring results have been acquired by learning domain-invariant deep features via domain-adversarial training. However, its parallel design of task and domain classifiers limits the ability to achieve a finer category-level domain alignment. To promote categorical domain adaptation (CatDA), based on a joint category-domain classifier, we propose novel losses of adversarial training at both domain and category levels. Since the joint classifier can be regarded as a concatenation of individual task classifiers respectively for the two domains, our design principle is to enforce consistency of category predictions between the two task classifiers. Moreover, we propose a concept of vicinal domains whose instances are produced by a convex combination of pairs of instances respectively from the two domains. Intuitively, alignment of the possibly infinite number of vicinal domains enhances that of original domains. We propose novel adversarial losses for vicinal domain adaptation (VicDA) based on CatDA, leading to Vicinal and Categorical Domain Adaptation (ViCatDA). We also propose Target Discriminative Structure Recovery (TDSR) to recover the intrinsic target discrimination damaged by adversarial feature alignment. We also analyze the principles underlying the ability of our key designs to align the joint distributions. Extensive experiments on several benchmark datasets demonstrate that we achieve the new state of the art.