pwc-archive/papers-with-abstracts huggingface dataset:
The Impact of Feature Selection and Transformation on Machine Learning Methods in Determining the Credit Scoring
Authors: Oguz Koc, Omur Ugur, A. Sevtap Kestel
Date: 2023-03-09 00:00:00
Tasks: feature selection

Banks utilize credit scoring as an important indicator of financial strength and eligibility for credit. Scoring models aim to assign statistical odds or probabilities for predicting if there is a risk of nonpayment in relation to many other factors which may be involved in. This paper aims to illustrate the beneficial use of the eight machine learning (ML) methods (Support Vector Machine, Gaussian Naive Bayes, Decision Trees, Random Forest, XGBoost, K-Nearest Neighbors, Multi-layer Perceptron Neural Networks) and Logistic Regression in finding the default risk as well as the features contributing to it. An extensive comparison is made in three aspects: (i) which ML models with and without its own wrapper feature selection performs the best; (ii) how feature selection combined with appropriate data scaling method influences the performance; (iii) which of the most successful combination (algorithm, feature selection, and scaling) delivers the best validation indicators such as accuracy rate, Type I and II errors and AUC. An open-access credit scoring default risk data sets on German and Australian cases are taken into account, for which we determine the best method, scaling, and features contributing to default risk best and compare our findings with the literature ones in related. We illustrate the positive contribution of the selection method and scaling on the performance indicators compared to the existing literature.
A fixed point approach for computing actuarially fair Pareto optimal risk-sharing rules
Authors: Fallou Niakh
Date: 2023-03-09 00:00:00
Tasks: Fairness

Risk-sharing is one way to pool risks without the need for a third party. To ensure the attractiveness of such a system, the rule should be accepted and understood by all participants. A desirable risk-sharing rule should fulfill actuarial fairness and Pareto optimality while being easy to compute. This paper establishes a one-to-one correspondence between an actuarially fair Pareto optimal (AFPO) risk-sharing rule and a fixed point of a specific function. A fast numerical method for computing these risk-sharing rules is also derived. As a result, we are able to compute AFPO risk-sharing rules for a large number of heterogeneous participants in this framework.
Restoration based Generative Models
Authors: Jaemoo Choi, Yesom Park, Myungjoo Kang
Date: 2023-02-20 00:00:00
Tasks: Denoising, Image Restoration

Denoising diffusion models (DDMs) have recently attracted increasing attention by showing impressive synthesis quality. DDMs are built on a diffusion process that pushes data to the noise distribution and the models learn to denoise. In this paper, we establish the interpretation of DDMs in terms of image restoration (IR). Integrating IR literature allows us to use an alternative objective and diverse forward processes, not confining to the diffusion process. By imposing prior knowledge on the loss function grounded on MAP-based estimation, we eliminate the need for the expensive sampling of DDMs. Also, we propose a multi-scale training, which improves the performance compared to the diffusion process, by taking advantage of the flexibility of the forward process. Experimental results demonstrate that our model improves the quality and efficiency of both training and inference. Furthermore, we show the applicability of our model to inverse problems. We believe that our framework paves the way for designing a new type of flexible general generative model.
Natural Gradient Methods: Perspectives, Efficient-Scalable Approximations, and Analysis
Authors: Rajesh Shrestha
Date: 2023-03-06 00:00:00

Natural Gradient Descent, a second-degree optimization method motivated by the information geometry, makes use of the Fisher Information Matrix instead of the Hessian which is typically used. However, in many cases, the Fisher Information Matrix is equivalent to the Generalized Gauss-Newton Method, that both approximate the Hessian. It is an appealing method to be used as an alternative to stochastic gradient descent, potentially leading to faster convergence. However, being a second-order method makes it infeasible to be used directly in problems with a huge number of parameters and data. This is evident from the community of deep learning sticking with the stochastic gradient descent method since the beginning. In this paper, we look at the different perspectives on the natural gradient method, study the current developments on its efficient-scalable empirical approximations, and finally examine their performance with extensive experiments.
A self-interpretable module for deep image classification on small data
Authors: Biagio La Rosa, Roberto Capobianco, Daniele Nardi
Date: 2022-08-05 00:00:00
Proceeding: Applied Intelligence 2022 8
Tasks: image-classification, Image Classification

Deep neural networks are the driving force of the recent explosion of machine learning applications in everyday life. However, they usually require a lot of training data to work well, and they act as black-boxes, making predictions without any explanation about them. This paper presents Memory Wrap, a module (i.e, a set of layers) that can be added to deep learning models to improve their performance and interpretability in settings where few data are available. Memory Wrap adopts a sparse content-attention mechanism between the input and some memories of past training samples. We show that adding Memory Wrap to standard deep neural networks improves their performance when they learn from a limited set of data, and allows them to reach comparable performance when they learn from the full dataset. We discuss how the analysis of its structure and content-attention weights helps to get insights about its decision process and makes their predictions more interpretable, compared to the same networks without Memory Wrap. We test our approach on image classification tasks using several networks on three different datasets, namely CIFAR10, SVHN, and CINIC10.
Prototype-based Interpretable Graph Neural Networks
Authors: Alessio Ragno, Biagio La Rosa, Roberto Capobianco
Date: 2022-11-16 00:00:00
Proceeding: IEEE Transactions on Artificial Intelligence 2022 11
Tasks: Explainable Models, Graph Neural Network, Node Classification

Graph neural networks have proved to be a key tool for dealing with many problems and domains such as chemistry, natural language processing and social networks. While the structure of the layers is simple, it is difficult to identify the patterns learned by the graph neural network. Several works propose post-hoc methods to explain graph predictions, but few of them try to generate interpretable models. Conversely, the topic of the interpretable models is highly investigated in image recognition. Given the similarity between image and graph domains, we analyze the adaptability of prototype-based neural networks for graph and node classification. In particular, we investigate the use of two interpretable networks, ProtoPNet and TesNet, in the graph domain. We show that the adapted networks manage to reach better or higher accuracy scores than their respective black-box models and comparable performances with state-of-the-art self-explainable models. Showing how to extract ProtoPNet and TesNet explanations on graph neural networks, we further study how to obtain global and local explanations for the trained models. We then evaluate the explanations of the interpretable models by comparing them with post-hoc approaches and self-explainable models. Our findings show that the application of TesNet and ProtoPNet to the graph domain produces qualitative predictions while improving their reliability and transparency.
State of the Art of Visual Analytics for eXplainable Deep Learning
Authors: Biagio La Rosa, Graziano Blasilli, Romain Bourqui, David Auber, Giuseppe Santucci, Roberto Capobianco, Enrico Bertini, Romain Giot, Marco Angelini
Date: 2023-02-06 00:00:00
Proceeding: Computer Graphics Forum 2023 2
Tasks: Deep Learning, Survey

The use and creation of machine-learning-based solutions to solve problems or reduce their computational costs are becoming increasingly widespread in many domains. Deep Learning plays a large part in this growth. However, it has drawbacks such as a lack of explainability and behaving as a black-box model. During the last few years, Visual Analytics has provided several proposals to cope with these drawbacks, supporting the emerging eXplainable Deep Learning field. This survey aims to (i) systematically report the contributions of Visual Analytics for eXplainable Deep Learning; (ii) spot gaps and challenges; (iii) serve as an anthology of visual analytical solutions ready to be exploited and put into operation by the Deep Learning community (architects, trainers and end users) and (iv) prove the degree of maturity, ease of integration and results for specific domains. The survey concludes by identifying future research challenges and bridging activities that are helpful to strengthen the role of Visual Analytics as effective support for eXplainable Deep Learning and to foster the adoption of Visual Analytics solutions in the eXplainable Deep Learning community. An interactive explorable version of this survey is available online at https://aware-diag-sapienza.github.io/VA4XDL.
A New Multiple Source Domain Adaptation Fault Diagnosis Method between Different Rotating Machines
Authors: un Zhu, Nan Chen, Member, IEEE, and Changqing Shen
Date: 2020-09-03 00:00:00
Proceeding: TRANSACTIONS ON INDUSTRIAL INFORMA TICS 2020 9
Tasks: Domain Adaptation, Fault Diagnosis, Transfer Learning

Fault diagnosis based on data-driven meth-
ods are widely investigated when enough supervised sam-
ples of the target machine are available to build a reliable
model. However, the labeled samples in practical operated
machine are usually scarce and difficult to collect. If the
model is built based on the sufficient labeled samples from
different source machines, the diagnosis performance will
degenerate owing to the domain discrepancy. To solve this
issue, transfer learning is proposed by leveraging knowl-
edge learned from source domain to target domain. While
transfer learning methods for fault diagnosis have been
actively studied, most of them focus on learning from a
single source. Since the labeled samples can come from
multiple domains, more general diagnosis knowledge can
be learned, which is beneficial to the prediction for the
target domain. Therefore, a new transfer learning approach
based on multi-source domain adaptation is proposed. A
multi-adversarial learning strategy is utilized for obtaining
feature representations, which are invariant to the multiple
domain shifts and discriminative for the learning goal at
the same time. Extensive experimental analysis on four
different bearing datasets is performed to illustrate the
effectiveness and advantage of the proposed method.
Cross-Domain Open-Set Machinery Fault Diagnosis Based on Adversarial Network With Multiple Auxiliary Classifiers
Authors: un Zhu, Cheng-Geng Huang
Date: 2021-12-28 00:00:00
Proceeding: TRANSACTIONS ON INDUSTRIAL INFORMA TICS 2021 12
Tasks: Domain Adaptation, Fault Diagnosis, Transfer Learning

Cross-domain fault diagnosis methods based
on transfer learning attempt to leverage knowledge from a
domain with sufficient labeled samples to a different but
related domain with few or even nonlabeled samples. These
methods have been widely investigated in the past years.
Notwithstanding the efficacy, most existing approaches as-
sume that the label spaces of training and testing data
are the same. However, this assumption is not practical in
actual applications because new fault category usually hap-
pens in the testing stage. A cross-domain open-set transfer
diagnosis method is presented in this article to manage
the aforementioned problem. Domain adversarial model is
employed to discriminate known from unknown target in-
stances. Moreover, multiple auxiliary classifiers introduce
a weighting module to evaluate the distinguishing domain
knowledge to provide target instances with representative
weights. The new adversarial domain adaptation network
with diverse supplementary classifiers can effectively iden-
tify the unknown and known fault categories in the target
domain and bridge the domain shift between the common
fault category of the source and target domain. Experi-
ments on two bearing datasets show the effectiveness and
advantage of the proposed method.
ATPL: Mutually enhanced adversarial training and pseudo labeling for unsupervised domain adaptation
Authors: Changan Yi, Haotian Chen, Yonghui Xu, Yong liu, Lei Jiang, Haishu Tan
Date: 2022-08-17 00:00:00
Proceeding: Knowledge-Based Systems 2022 8
Tasks: Domain Adaptation, Unsupervised Domain Adaptation

Unsupervised domain adaptation aims to transfer knowledge from a labeled source domain to a related but unlabeled target domain. Most existing approaches either adversarially reduce the domain shift or use pseudo-labels to provide category information during adaptation. However, an adversarial training method may sacrifice the discriminability of the target data, since no category information is available. Moreover, a pseudo labeling method is difficult to produce high-confidence samples, since the classifier is often source-trained and there exists the domain discrepancy. Thus, it may have a negative influence on learning target representations. A potential solution is to make them compensate each other to simultaneously guarantee the feature transferability and discriminability, which are the two key criteria of feature representations in domain adaptation. In this paper, we propose a novel method named ATPL, which mutually promotes Adversarial Training and Pseudo Labeling for unsupervised domain adaptation. ATPL can produce high-confidence pseudo-labels by adversarial training. Accordingly, ATPL will use the pseudo-labeled information to improve the adversarial training process, which can guarantee the feature transferability by generating adversarial data to fill in the domain gap. Those pseudo-labels can also boost the feature discriminability. Extensive experiments on real datasets demonstrate that the proposed ATPL method outperforms state-of-the-art unsupervised domain adaptation methods.
Prediction of reservoir key parameters in ‘sweet spot’ on the basis of particle swarm optimization to TCN-LSTM network
Authors: Fengcai Huo
Date: 2022-11-05 00:00:00
Proceeding: 2022 2022 11
Tasks: Prediction

In oil reservoirs, the sweet spot is found that the well could be positioned quickly and accurately, the drilling rate and the oil-gas production are increased, development cost is reduced. Among them, sorting, granularity and porosity are important factors to evaluate whether the exploration area is a sweet spot. A low permeability oil reservoir is taken as the research object, this paper mainly focuses on the prediction of the above evaluation parameters. In order to solve this problem, this paper proposed a new deep learning hybrid model. The model is constructed based on temporal convolutional network (TCN) and long short-term memory network (LSTM). Firstly, the influence of logging parameters on the prediction of evaluation indexes is analyzed. Secondly, the model is used to predict the screened sequence data. In the process of model construction, particle swarm optimization (PSO) is used to optimize the global hyperparameters, and finally the prediction model is obtained. The model is compared with TCN algorithm, traditional machine learning and empirical formula. This model improves the prediction accuracy of reservoir evaluation parameters in low permeability oilfield.
JPEG-resistant Adversarial Images
Authors: Richard Shin, Dawn Song
Date: 2017-12-08 00:00:00
Proceeding: NIPS 2017 Workshop on Machine Learning and Computer Security 2017 12

Several papers have explored the use of JPEG compression as a defense against adversarial images. In this work, we show that we can generate adversarial images which survive JPEG compression, by including a differentiable approximation to JPEG in the target model. By ensembling multiple target models employing varying levels of compression, we generate adversarial images with up to 691× greater success rate than the baseline method on a model using JPEG as defense.
CFR-ICL: Cascade-Forward Refinement with Iterative Click Loss for Interactive Image Segmentation
Authors: Shoukun Sun, Min Xian, Fei Xu, Luca Capriotti, Tiankai Yao
Date: 2023-03-09 00:00:00
Tasks: Image Augmentation, Image Segmentation, Interactive Segmentation, Segmentation, Semantic Segmentation

The click-based interactive segmentation aims to extract the object of interest from an image with the guidance of user clicks. Recent work has achieved great overall performance by employing feedback from the output. However, in most state-of-the-art approaches, 1) the inference stage involves inflexible heuristic rules and requires a separate refinement model, and 2) the number of user clicks and model performance cannot be balanced. To address the challenges, we propose a click-based and mask-guided interactive image segmentation framework containing three novel components: Cascade-Forward Refinement (CFR), Iterative Click Loss (ICL), and SUEM image augmentation. The CFR offers a unified inference framework to generate segmentation results in a coarse-to-fine manner. The proposed ICL allows model training to improve segmentation and reduce user interactions simultaneously. The proposed SUEM augmentation is a comprehensive way to create large and diverse training sets for interactive image segmentation. Extensive experiments demonstrate the state-of-the-art performance of the proposed approach on five public datasets. Remarkably, our model reduces by 33.2\%, and 15.5\% the number of clicks required to surpass an IoU of 0.95 in the previous state-of-the-art approach on the Berkeley and DAVIS sets, respectively.
Enhancing MAP-Elites with Multiple Parallel Evolution Strategies
Authors: Manon Flageat, Bryan Lim, Antoine Cully
Date: 2023-03-10 00:00:00
Tasks: GPU

With the development of fast and massively parallel evaluations in many domains, Quality-Diversity (QD) algorithms, that already proved promising in a large range of applications, have seen their potential multiplied. However, we have yet to understand how to best use a large number of evaluations as using them for random variations alone is not always effective. High-dimensional search spaces are a typical situation where random variations struggle to effectively search. Another situation is uncertain settings where solutions can appear better than they truly are and naively evaluating more solutions might mislead QD algorithms. In this work, we propose MAP-Elites-Multi-ES (MEMES), a novel QD algorithm based on Evolution Strategies (ES) designed to exploit fast parallel evaluations more effectively. MEMES maintains multiple (up to 100) simultaneous ES processes, each with its own independent objective and reset mechanism designed for QD optimisation, all on just a single GPU. We show that MEMES outperforms both gradient-based and mutation-based QD algorithms on black-box optimisation and QD-Reinforcement-Learning tasks, demonstrating its benefit across domains. Additionally, our approach outperforms sampling-based QD methods in uncertain domains when given the same evaluation budget. Overall, MEMES generates reproducible solutions that are high-performing and diverse through large-scale ES optimisation on easily accessible hardware.
Rewarding Chatbots for Real-World Engagement with Millions of Users
Authors: Robert Irvine, Douglas Boubert, Vyas Raina, Adian Liusie, Ziyi Zhu, Vineet Mudupalli, Aliaksei Korshuk, Zongyi Liu, Fritz Cremer, Valentin Assassi, Christie-Carol Beauchamp, Xiaoding Lu, Thomas Rialan, William Beauchamp
Date: 2023-03-10 00:00:00
Tasks: Chatbot, Language Modelling

The emergence of pretrained large language models has led to the deployment of a range of social chatbots for chitchat. Although these chatbots demonstrate language ability and fluency, they are not guaranteed to be engaging and can struggle to retain users. This work investigates the development of social chatbots that prioritize user engagement to enhance retention, specifically examining the use of human feedback to efficiently develop highly engaging chatbots. The proposed approach uses automatic pseudo-labels collected from user interactions to train a reward model that can be used to reject low-scoring sample responses generated by the chatbot model at inference time. Intuitive evaluation metrics, such as mean conversation length (MCL), are introduced as proxies to measure the level of engagement of deployed chatbots. A/B testing on groups of 10,000 new daily chatbot users on the Chai Research platform shows that this approach increases the MCL by up to 70%, which translates to a more than 30% increase in user retention for a GPT-J 6B model. Future work aims to use the reward model to realise a data fly-wheel, where the latest user conversations can be used to alternately fine-tune the language model and the reward model.
Self-supervised Training Sample Difficulty Balancing for Local Descriptor Learning
Authors: Jiahan Zhang, Dayong Tian
Date: 2023-03-10 00:00:00
Tasks: Retrieval

In the case of an imbalance between positive and negative samples, hard negative mining strategies have been shown to help models learn more subtle differences between positive and negative samples, thus improving recognition performance. However, if too strict mining strategies are promoted in the dataset, there may be a risk of introducing false negative samples. Meanwhile, the implementation of the mining strategy disrupts the difficulty distribution of samples in the real dataset, which may cause the model to over-fit these difficult samples. Therefore, in this paper, we investigate how to trade off the difficulty of the mined samples in order to obtain and exploit high-quality negative samples, and try to solve the problem in terms of both the loss function and the training strategy. The proposed balance loss provides an effective discriminant for the quality of negative samples by combining a self-supervised approach to the loss function, and uses a dynamic gradient modulation strategy to achieve finer gradient adjustment for samples of different difficulties. The proposed annealing training strategy then constrains the difficulty of the samples drawn from negative sample mining to provide data sources with different difficulty distributions for the loss function, and uses samples of decreasing difficulty to train the model. Extensive experiments show that our new descriptors outperform previous state-of-the-art descriptors for patch validation, matching, and retrieval tasks.
On the Fusion Strategies for Federated Decision Making
Authors: Mert Kayaalp, Yunus Inan, Visa Koivunen, Emre Telatar, Ali H. Sayed
Date: 2023-03-10 00:00:00
Tasks: Decision Making

We consider the problem of information aggregation in federated decision making, where a group of agents collaborate to infer the underlying state of nature without sharing their private data with the central processor or each other. We analyze the non-Bayesian social learning strategy in which agents incorporate their individual observations into their opinions (i.e., soft-decisions) with Bayes rule, and the central processor aggregates these opinions by arithmetic or geometric averaging. Building on our previous work, we establish that both pooling strategies result in asymptotic normality characterization of the system, which, for instance, can be utilized to derive approximate expressions for the error probability. We verify the theoretical findings with simulations and compare both strategies.