pwc-archive/papers-with-abstracts huggingface dataset:
Credit scoring using neural networks and SURE posterior probability calibration
Authors: Matthieu Garcin, Samuel Stephan
Date: 2021-07-15 00:00:00
Tasks: regression

In this article we compare the performances of a logistic regression and a feed forward neural network for credit scoring purposes. Our results show that the logistic regression gives quite good results on the dataset and the neural network can improve a little the performance. We also consider different sets of features in order to assess their importance in terms of prediction accuracy. We found that temporal features (i.e. repeated measures over time) can be an important source of information resulting in an increase in the overall model accuracy. Finally, we introduce a new technique for the calibration of predicted probabilities based on Stein's unbiased risk estimate (SURE). This calibration technique can be applied to very general calibration functions. In particular, we detail this method for the sigmoid function as well as for the Kumaraswamy function, which includes the identity as a particular case. We show that stacking the SURE calibration technique with the classical Platt method can improve the calibration of predicted probabilities.
Market risk factors analysis for an international mining company. Multi-dimensional, heavy-tailed-based modelling
Authors: Łukasz Bielak, Aleksandra Grzesiek, Joanna Janczura, Agnieszka Wyłomańska
Date: 2021-07-15 00:00:00
Tasks: Management

Mining companies to properly manage their operations and be ready to make business decisions, are required to analyze potential scenarios for main market risk factors. The most important risk factors for KGHM, one of the biggest companies active in the metals and mining industry, are the price of copper (Cu), traded in US dollars, and the Polish zloty (PLN) exchange rate (USDPLN). The main scope of the paper is to understand the mid- and long-term dynamics of these two risk factors. For a mining company it might help to properly evaluate potential downside market risk and optimise hedging instruments. From the market risk management perspective, it is also important to analyze the dynamics of these two factors combined with the price of copper in Polish zloty (Cu in PLN), which jointly drive the revenues, cash flows, and financial results of the company. Based on the relation between analyzed risk factors and distribution analysis, we propose to use two-dimensional vector autoregressive (VAR) model with the $\alpha-$stable distribution. The non-homogeneity of the data is reflected in two identified regimes: first - corresponding to the 2008 crisis and second - to the stable market situation. As a natural implication of the model fitted to market assets, we derive the dynamics of the copper price in PLN, which is not a traded asset but is crucial for the KGHM company risk exposure. A comparative study is performed to demonstrate the effect of including dependencies of the assets and the implications of the regime change. Since for various international companies, risk factors are given rather in the national than the market currency, the approach is universal and can be used in different market contexts, like mining or oil companies, but also other commodities involved in the global trading system.
Finding Significant Features for Few-Shot Learning using Dimensionality Reduction
Authors: Mauricio Mendez-Ruiz, Ivan Garcia Jorge Gonzalez-Zapata, Gilberto Ochoa-Ruiz, Andres Mendez-Vazquez
Date: 2021-07-06 00:00:00
Tasks: Dimensionality Reduction, Few-Shot Learning, Meta-Learning, Metric Learning

Few-shot learning is a relatively new technique that specializes in problems where we have little amounts of data. The goal of these methods is to classify categories that have not been seen before with just a handful of samples. Recent approaches, such as metric learning, adopt the meta-learning strategy in which we have episodic tasks conformed by support (training) data and query (test) data. Metric learning methods have demonstrated that simple models can achieve good performance by learning a similarity function to compare the support and the query data. However, the feature space learned by a given metric learning approach may not exploit the information given by a specific few-shot task. In this work, we explore the use of dimension reduction techniques as a way to find task-significant features helping to make better predictions. We measure the performance of the reduced features by assigning a score based on the intra-class and inter-class distance, and selecting a feature reduction method in which instances of different classes are far away and instances of the same class are close. This module helps to improve the accuracy performance by allowing the similarity function, given by the metric learning method, to have more discriminative features for the classification. Our method outperforms the metric learning baselines in the miniImageNet dataset by around 2% in accuracy performance.
Confidence Conditioned Knowledge Distillation
Authors: Sourav Mishra, Suresh Sundaram
Date: 2021-07-06 00:00:00
Tasks: Knowledge Distillation

In this paper, a novel confidence conditioned knowledge distillation (CCKD) scheme for transferring the knowledge from a teacher model to a student model is proposed. Existing state-of-the-art methods employ fixed loss functions for this purpose and ignore the different levels of information that need to be transferred for different samples. In addition to that, these methods are also inefficient in terms of data usage. CCKD addresses these issues by leveraging the confidence assigned by the teacher model to the correct class to devise sample-specific loss functions (CCKD-L formulation) and targets (CCKD-T formulation). Further, CCKD improves the data efficiency by employing self-regulation to stop those samples from participating in the distillation process on which the student model learns faster. Empirical evaluations on several benchmark datasets show that CCKD methods achieve at least as much generalization performance levels as other state-of-the-art methods while being data efficient in the process. Student models trained through CCKD methods do not retain most of the misclassifications commited by the teacher model on the training set. Distillation through CCKD methods improves the resilience of the student models against adversarial attacks compared to the conventional KD method. Experiments show at least 3% increase in performance against adversarial attacks for the MNIST and the Fashion MNIST datasets, and at least 6% increase for the CIFAR10 dataset.
Low-Rank Temporal Attention-Augmented Bilinear Network for financial time-series forecasting
Authors: Mostafa Shabani, Alexandros Iosifidis
Date: 2021-07-05 00:00:00
Tasks: Prediction, Time Series, Time Series Analysis, Time Series Forecasting

Financial market analysis, especially the prediction of movements of stock prices, is a challenging problem. The nature of financial time-series data, being non-stationary and nonlinear, is the main cause of these challenges. Deep learning models have led to significant performance improvements in many problems coming from different domains, including prediction problems of financial time-series data. Although the prediction performance is the main goal of such models, dealing with ultra high-frequency data sets restrictions in terms of the number of model parameters and its inference speed. The Temporal Attention-Augmented Bilinear network was recently proposed as an efficient and high-performing model for Limit Order Book time-series forecasting. In this paper, we propose a low-rank tensor approximation of the model to further reduce the number of trainable parameters and increase its speed.
Free-Text Keystroke Dynamics for User Authentication
Authors: Jianwei Li, Han-Chih Chang, Mark Stamp
Date: 2021-07-01 00:00:00
Tasks: Feature Engineering

In this research, we consider the problem of verifying user identity based on keystroke dynamics obtained from free-text. We employ a novel feature engineering method that generates image-like transition matrices. For this image-like feature, a convolution neural network (CNN) with cutout achieves the best results. A hybrid model consisting of a CNN and a recurrent neural network (RNN) is also shown to outperform previous research in this field.
Mitigating Memorization in Sample Selection for Learning with Noisy Labels
Authors: Kyeongbo Kong, Junggi Lee, Youngchul Kwak, Young-Rae Cho, Seong-Eun Kim, Woo-Jin Song
Date: 2021-07-08 00:00:00
Tasks: Learning with noisy labels, Memorization

Because deep learning is vulnerable to noisy labels, sample selection techniques, which train networks with only clean labeled data, have attracted a great attention. However, if the labels are dominantly corrupted by few classes, these noisy samples are called dominant-noisy-labeled samples, the network also learns dominant-noisy-labeled samples rapidly via content-aware optimization. In this study, we propose a compelling criteria to penalize dominant-noisy-labeled samples intensively through class-wise penalty labels. By averaging prediction confidences for the each observed label, we obtain suitable penalty labels that have high values if the labels are largely corrupted by some classes. Experiments were performed using benchmarks (CIFAR-10, CIFAR-100, Tiny-ImageNet) and real-world datasets (ANIMAL-10N, Clothing1M) to evaluate the proposed criteria in various scenarios with different noise rates. Using the proposed sample selection, the learning process of the network becomes significantly robust to noisy labels compared to existing methods in several noise types.
Classifying Component Function in Product Assemblies with Graph Neural Networks
Authors: Vincenzo Ferrero, Kaveh Hassani, Daniele Grandi, Bryony DuPont
Date: 2021-07-08 00:00:00
Tasks: Graph Neural Network

Function is defined as the ensemble of tasks that enable the product to complete the designed purpose. Functional tools, such as functional modeling, offer decision guidance in the early phase of product design, where explicit design decisions are yet to be made. Function-based design data is often sparse and grounded in individual interpretation. As such, function-based design tools can benefit from automatic function classification to increase data fidelity and provide function representation models that enable function-based intelligent design agents. Function-based design data is commonly stored in manually generated design repositories. These design repositories are a collection of expert knowledge and interpretations of function in product design bounded by function-flow and component taxonomies. In this work, we represent a structured taxonomy-based design repository as assembly-flow graphs, then leverage a graph neural network (GNN) model to perform automatic function classification. We support automated function classification by learning from repository data to establish the ground truth of component function assignment. Experimental results show that our GNN model achieves a micro-average F${_1}$-score of 0.832 for tier 1 (broad), 0.756 for tier 2, and 0.783 for tier 3 (specific) functions. Given the imbalance of data features, the results are encouraging. Our efforts in this paper can be a starting point for more sophisticated applications in knowledge-based CAD systems and Design-for-X consideration in function-based design.
NVCell: Standard Cell Layout in Advanced Technology Nodes with Reinforcement Learning
Authors: Haoxing Ren, Matthew Fojtik, Brucek Khailany
Date: 2021-07-09 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

High quality standard cell layout automation in advanced technology nodes is still challenging in the industry today because of complex design rules. In this paper we introduce an automatic standard cell layout generator called NVCell that can generate layouts with equal or smaller area for over 90% of single row cells in an industry standard cell library on an advanced technology node. NVCell leverages reinforcement learning (RL) to fix design rule violations during routing and to generate efficient placements.
Backprop-Free Reinforcement Learning with Active Neural Generative Coding
Authors: Alexander Ororbia, Ankur Mali
Date: 2021-07-10 00:00:00
Tasks: Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In humans, perceptual awareness facilitates the fast recognition and extraction of information from sensory input. This awareness largely depends on how the human agent interacts with the environment. In this work, we propose active neural generative coding, a computational framework for learning action-driven generative models without backpropagation of errors (backprop) in dynamic environments. Specifically, we develop an intelligent agent that operates even with sparse rewards, drawing inspiration from the cognitive theory of planning as inference. We demonstrate on several control problems, in the online learning setting, that our proposed modeling framework performs competitively with deep Q-learning models. The robust performance of our agent offers promising evidence that a backprop-free approach for neural inference and learning can drive goal-directed behavior.
Probabilistic analysis of solar cell optical performance using Gaussian processes
Authors: Rahul Jaiswal, Manel Martínez-Ramón, Tito Busani
Date: 2021-06-26 00:00:00
Tasks: Gaussian Processes, Prediction

This work investigates application of different machine learning based prediction methodologies to estimate the performance of silicon based textured cells. Concept of confidence bound regions is introduced and advantages of this concept are discussed in detail. Results show that reflection profiles and depth dependent optical generation profiles can be accurately estimated using Gaussian processes with exact knowledge of uncertainty in the prediction values.It is also shown that cell design parameters can be estimated for a desired performance metric.
Inferring the Structure of Ordinary Differential Equations
Authors: Juliane Weilbach, Sebastian Gerwinn, Christian Weilbach, Melih Kandemir
Date: 2021-07-05 00:00:00
Tasks: regression, Symbolic Regression

Understanding physical phenomena oftentimes means understanding the underlying dynamical system that governs observational measurements. While accurate prediction can be achieved with black box systems, they often lack interpretability and are less amenable for further expert investigation. Alternatively, the dynamics can be analysed via symbolic regression. In this paper, we extend the approach by (Udrescu et al., 2020) called AIFeynman to the dynamic setting to perform symbolic regression on ODE systems based on observations from the resulting trajectories. We compare this extension to state-of-the-art approaches for symbolic regression empirically on several dynamical systems for which the ground truth equations of increasing complexity are available. Although the proposed approach performs best on this benchmark, we observed difficulties of all the compared symbolic regression approaches on more complex systems, such as Cart-Pole.
A Fixed Version of Quadratic Program in Gradient Episodic Memory
Authors: Wei Zhou, Yiying Li
Date: 2021-07-07 00:00:00
Tasks: Continual Learning

Gradient Episodic Memory is indeed a novel method for continual learning, which solves new problems quickly without forgetting previously acquired knowledge. However, in the process of studying the paper, we found there were some problems in the proof of the dual problem of Quadratic Program, so here we give our fixed version for this problem.
Machine Learning-Based Analysis of Free-Text Keystroke Dynamics
Authors: Han-Chih Chang, Jianwei Li, Mark Stamp
Date: 2021-07-01 00:00:00
Tasks: BIG-bench Machine Learning

The development of active and passive biometric authentication and identification technology plays an increasingly important role in cybersecurity. Keystroke dynamics can be used to analyze the way that a user types based on various keyboard input. Previous work has shown that user authentication and classification can be achieved based on keystroke dynamics. In this research, we consider the problem of user classification based on keystroke dynamics features collected from free-text. We implement and analyze a novel a deep learning model that combines a convolutional neural network (CNN) and a gated recurrent unit (GRU). We optimize the resulting model and consider several relevant related problems. Our model is competitive with the best results obtained in previous comparable research.
Convolutional Neural Bandit for Visual-aware Recommendation
Authors: Yikun Ban, Jingrui He
Date: 2021-07-02 00:00:00

Online recommendation/advertising is ubiquitous in web business. Image displaying is considered as one of the most commonly used formats to interact with customers. Contextual multi-armed bandit has shown success in the application of advertising to solve the exploration-exploitation dilemma existing in the recommendation procedure. Inspired by the visual-aware recommendation, in this paper, we propose a contextual bandit algorithm, where the convolutional neural network (CNN) is utilized to learn the reward function along with an upper confidence bound (UCB) for exploration. We also prove a near-optimal regret bound $\tilde{\mathcal{O}}(\sqrt{T})$ when the network is over-parameterized, and establish strong connections with convolutional neural tangent kernel (CNTK). Finally, we evaluate the empirical performance of the proposed algorithm and show that it outperforms other state-of-the-art UCB-based bandit algorithms on real-world image data sets.
Deep Match to Rank Model for Personalized Click-Through Rate Prediction
Authors: Ze Lye, Yu Dong, Chengfu Huo, Weijun Ren
Date: 2020-04-01 00:00:00
Proceeding: Proceedings of the AAAI Conference on Artificial Intelligence 2020 4
Tasks: Click-Through Rate Prediction, Collaborative Filtering, Prediction, Recommendation Systems

Click-through rate (CTR) prediction is a core task in the field of recommender system and many other applications. For CTR prediction model, personalization is the key to improve the performance and enhance the user experience. Recently, several models are proposed to extract user interest from user behavior data which reflects user's personalized preference implicitly. However, existing works in the field of CTR prediction mainly focus on user representation and pay less attention on representing the relevance between user and item, which directly measures the intensity of user's preference on target item. Motivated by this, we propose a novel model named Deep Match to Rank (DMR) which combines the thought of collaborative filtering in matching methods for the ranking task in CTR prediction. In DMR, we design User-to-Item Network and Item-to-Item Network to represent the relevance in two forms. In User-to-Item Network, we represent the relevance between user and item by inner product of the corresponding representation in the embedding space. Meanwhile, an auxiliary match network is presented to supervise the training and push larger inner product to represent higher relevance. In Item-to-Item Network, we first calculate the item-to-item similarities between user interacted items and target item by attention mechanism, and then sum up the similarities to obtain another form of user-to-item relevance. We conduct extensive experiments on both public and industrial datasets to validate the effectiveness of our model, which outperforms the state-of-art models significantly.
Phonetic Spelling Algorithm Implementations for R
Authors: James P. Howard II
Date: 2020-10-07 00:00:00
Proceeding: Journal of Statistical Software 2020 10
Tasks: Record linking

The phonics package provides several functions for indexing words by their English language pronunciation. Over nearly one hundred years, many different algorithms have been developed to support word and name indexing. From Soundex, developed in the early 20th century and predating the digital computer, through to modern digital phonetic algorithms like Phonex, the phonics package provides support for more than a dozen methods. Together, these provide phonetic algorithms appropriate for use in name indexing and name matching across a variety of English language use cases.
Corruption and Wealth: Unveiling a national prosperity syndrome in Europe
Authors: Juan C. Correa, Klaus Jaffe
Date: 2015-11-05 00:00:00

Data mining revealed a cluster of economic, psychological, social and cultural indicators that in combination predicted corruption and wealth of European nations. This prosperity syndrome of self-reliant citizens, efficient division of labor, a sophisticated scientific community, and respect for the law, was clearly distinct from that of poor countries that had a diffuse relationship between high corruption perception, low GDP/capita, high social inequality, low scientific development, reliance on family and friends, and languages with many words for guilt. This suggests that there are many ways for a nation to be poor, but few ones to become rich, supporting the existence of synergistic interactions between the components in the prosperity syndrome favoring economic growth. No single feature was responsible for national prosperity. Focusing on synergies rather than on single features should improve our understanding of the transition from poverty and corruption to prosperity in European nations and elsewhere.
Perspective transformation for accurate detection of 3D bounding boxes of vehicles in traffic surveillance
Authors: Viktor Kocur
Date: 2019-02-08 00:00:00
Proceeding: 24th Computer Vision Winter Workshop 2019 2
Tasks: Vehicle Speed Estimation

Detection and tracking of vehicles captured by traffic surveillance cameras is a key component of intelligent traffic systems. In this paper a novel method of detecting 3D bounding boxes of vehicles is presented. Using the known geometry of the surveilled scene, we propose an algorithm to construct a perspective transformation. The transformation enables us to simplify the problem of detecting 3D bounding boxes to detecting 2D bounding boxes with one additional parameter. We can therefore utilize modified 2D object detectors based on deep convolutional networks to detect 3D bounding boxes of vehicles. Known 3D bounding boxes of vehicles can be utilized to improve results on tasks such as fine-grained vehicle classification or vehicle re-identification. We test the accuracy of our detector by comparing the accuracy of speed measurement on the BrnoCompSpeed dataset with the existing state of the art method. Our method decreases the mean error in speed measurement by 22 % (1.10 km/h to 0.86 km/h) and the median error in speed measurement by 33 % (0.97 km/h to 0.65 km/h mean), while also increasing the recall (83.3 % to 89.3 %).
Convolutional Neural Network Hyperparameters optimization for Facial Emotion Recognition
Authors: Adrian Vulpe-Grigorași, Ovidiu Grigore
Date: 2021-03-25 00:00:00
Proceeding: 12th International Symposium on Advanced Topics in Electrical Engineering (ATEE) 2021 3
Tasks: Emotion Recognition, Facial Emotion Recognition, Facial Expression Recognition (FER)

This paper presents a method of optimizing the hyperparameters of a convolutional neural network in order to increase accuracy in the context of facial emotion recognition. The optimal hyperparameters of the network were determined by generating and training models based on Random Search algorithm applied on a search space defined by discrete values of hyperparameters. The best model resulted was trained and evaluated using FER2013 database, obtaining an accuracy of 72.16%.