Reviewerly/exHarmony huggingface dataset:
Exploring CycleGAN for Bias Reduction in Gender Classification: Generative Modelling for Diversifying Data Augmentation

Deep learning algorithms have become more prevalent in real world applications. With these developments, bias is observed in the predictions made by these algorithms. One of the reasons for this is the algorithm’s capture of bias existing in the data set being used. This paper investigates the influence of using generative adversarial networks (GANs) as a gender-to-gender data pre-processing step on the bias and accuracy measured for a VGG-16 gender classification model. A cyclic generative adversarial network (CycleGAN) is trained on the Adience data set to perform the gender-to-gender data augmentation. This architecture allows for an unpaired domain mapping and results in two generators that double the training images generating a male for every female and vice versa. The VGG-16 gender classification model uses training data to produce an accuracy that indicates its performance. In addition, the model’s fairness is calculated using demographic parity and equalized odds to indicate its bias. The evaluation of the results provided by the proposed methodology in this paper shows that the accuracy decreases when Cycle-GAN pre-processing is applied. In addition, the bias also decreases, especially when measured on an imbalanced data set. However, the decrease in bias needs to be more significant to change our evaluation of the model from unfair to fair, showing the proposed methodology to be effective but insufficient to remove bias from the data set.
Uncertainty-Guided Test-Time Training for Face Forgery Detection

Face forgery detection is becoming increasingly important in computer vision as facial manipulation technologies cause serious concerns. Recent works have resorted to the frequency domain to develop face forgery detectors and achieved better generalization achievements. However, there are still unignorable problems: a) the role of frequency is not always sufficiently effective and generalized to different forgery technologies; and b) the network trained on public datasets is unable to effectively quantify its uncertainty. To address the generalization issue, we design a Dynamic Dual-spectrum Interaction Network (DDIN) that allows test-time training with uncertainty guidance. RGB and frequency features are first interacted in multi-level by using a Frequency-guided Attention Module (FAM). Then these multi-modal features are merged with a Dynamic Fusion Module (DFM). Moreover, we further exploit uncertain perturbations as guidance during the test-time training phase. The network can dynamically fuse the features with quality discrepancies, thus improving the generalization of forgery detection. Comprehensive evaluations of several benchmark databases corroborate the superior generalization performance of DDIN.
Diffusion Init: Stronger Initialisation of Decision-Based Black-Box Attacks for Visual Object Tracking

Adversarial attacks have emerged in the field of visual object tracking to mislead the tracker and result in its failure. Black-box attacks in particular have attracted increasing attention for their affinity with real-world applications. In the paradigm of decision-based black-box attacks, the magnitude of perturbation is gradually amplified, while the optimisation direction is predefined by an initial adversarial sample. Considering the pivotal role played by the initial adversarial sample in determining the success of an attack, we utilise the noise generated from the reverse process of a diffusion model as a better attacking direction. On the one hand, the diffusion model generates Gaussian noise, which formulate global information interaction, with a comprehensive impact on Transformer-based trackers. On the other hand, the diffusion model pays more attention to the target region during the inverse process, resulting in a more powerful perturbation of the target object. Our method, which is widely applicable, has been validated on a range of trackers using several benchmarking datasets. It is shown to deliver more extensive tracking performance degradation, compared to other state-of-the-art methods. We also investigate different approaches to the problem of generating the initial adversarial sample, confirming the effectiveness and rationality of our proposed diffusion initialisation method.
Sarcasm Detection in News Headlines Using Evidential Deep Learning-Based LSTM and GRU

Sarcasm has become quite inter-related with the day to day life of all. In news robust sarcasm is often used to grab the attention of the viewers. This research aims to detect sarcasm using Evidential deep learning. This technique uses uncertainty estimations for identifying the sentiments from news headlines dataset. Also, LSTM and GRU have been used with Evidential deep learning approach. The purpose of using LSTM is that it can classify texts from headlines in order to analysis the sentiments. Moreover, we have used GRU which is an recurrent neural networks (RNN) and it effectively models sequential data. The architecture of the GRU network is ideally suited for identifying dependencies and extended contextual relationships within news headings. Overall, our proposed model uses Evidential deep learning based LSTM and GRU to identify the sentiments of robust sarcasms from news headlines.
SimCLR-Inception: An Image Representation Learning and Recognition Model for Robot Vision

Effective feature extraction is a key component in image recognition for robot vision. This paper presents an improved contrastive learning-based image feature extraction and classification model, termed SimCLR-Inception, to realize effective and accurate image recognition. By using the SimCLR, this model generates positive and negative image samples from unlabeled data through image augmentation and then minimizes the contrastive loss function to learn the image representations by exploring more underlying structure information. Furthermore, this proposed model uses the Inception V3 model to classify the image representations for improving recognition accuracy. The SimCLR-Inception model is compared with four representative image recognition models, including LeNet, VGG16, Inception V3, and EfficientNet V2 on a real-world Multi-class Weather (MW) data set. We use four representative metrics: accuracy, precision, recall, and F1-Score, to verify the performance of different models for image recognition. We show that the presented SimCLR-Inception model achieves all the successful runs and gives almost the best results. The accuracy is at least $$4\%$$ improved by the Inception V3 model. It suggests that this model would work better for robot vision.
A Class Incremental Learning Algorithm for a Compact-Sized Probabilistic Neural Network and Its Empirical Comparison with Multilayered Perceptron Neural Networks

It is well known that class incremental learning using deep learning is difficult to achieve, since deep learning approaches inherently suffer from the catastrophic forgetting in the training mode. In contrast, a probabilistic neural network is capable of performing classification tasks based upon a set of local spaces, each composed of a training pattern, and thereby class incremental learning can be robustly performed. In this paper, we propose a class incremental learning method by exploiting the property of a probabilistic neural network, while reducing effectively the number of the training patterns to be stored within the hidden layer, and compare the performance of the class incremental learning tasks obtained using a multilayered perceptron model with that using a probabilistic neural network. Simulation results using seven publicly available datasets show that both the classification accuracies of an original probabilistic neural network and the proposed incremental learning method are 2.59 to 26.58 times higher than that of the deep learning in class incremental learning. Moreover, we observed that the class incremental learning performed using a probabilistic neural network exhibited a robust performance compared to the deep neural networks with iCaRL. In addition, it was observed that the proposed learning method was able to reduce effectively the number of the units in the hidden layer, while with the decrease in accuracy by only 1.77% to 7.06%, compared to the original one.
EfficientSRFace: An Efficient Network with Super-Resolution Enhancement for Accurate Face Detection

In face detection, low-resolution faces, such as numerous small faces of a human group in a crowded scene, are common in dense face prediction tasks. They usually contain limited visual clues and make small faces less distinguishable from the other small objects, which poses great challenge to accurate face detection. Although deep convolutional neural network has significantly promoted the research on face detection recently, current deep face detectors rarely take into account low-resolution faces and are still vulnerable to the real-world scenarios where massive amount of low-resolution faces exist. Consequently, they usually achieve degraded performance for low-resolution face detection. In order to alleviate this problem, we develop an efficient detector termed EfficientSRFace by introducing a feature-level super-resolution reconstruction network for enhancing the feature representation capability of the model. This module plays an auxiliary role in the training process, and can be removed during the inference without increasing the inference time. Extensive experiments on public benchmarking datasets, such as FDDB and WIDER Face, show that the embedded image super-resolution module can significantly improve the detection accuracy at the cost of a small amount of additional parameters and computational overhead, while helping our model achieve competitive performance compared with the state-of-the-arts.
Dual Branch Fusion Network for Pathological Image Classification with Extreme Different Image Size

Pathological image classification is a hot research topic in computer vision. Accurate classification of pathological images plays a vital role in disease diagnosis and prognosis prediction. However, the size of pathological images varies greatly, which brings great challenges to AI-assisted diagnosis. Convolutional Neural Network (CNN) has been widely used in pathological image classification tasks and achieved good results. However, it is difficult for CNN to handle images that are too large in size, it needs to compress excessively large pathological images, which will bring the loss of image information. To address these issues, we proposed a two-branch fusion model, named BiFusionNet, which combines CNN and Graph Neural Network (GNN). In the CNN branch, we employ the Densenet201 network to extract the feature of pathological images. For the GNN branch, we used PNA as the graph neural network, and the pathological images were processed into the graph data structure, and then input into the three-layer graph neural network layer to extract the graph feature. Finally, the convolutional feature and the graph feature were fused to realize a more accurate classification of pathological images. In order to make the two branches complement each other, Focal loss was added to the GNN branch based on Cross Entropy Loss, and the samples, which were difficult to classify, were set to higher weights. Our model achieved the supreme classification performance of 67.03% ± 2.04% on breast cancer dataset BRACS, and the best classification performance was 97.33% ± 1.25% in the CRA dataset of rectal cancer.
Anime Sketch Colourization Using Enhanced Pix2pix GAN

Coloring manga sketches is challenging due to the wide gap between the meaning of color and sketches. Unlike Western comics, Japanese manga sketch does not provide color, rather they are in binary form using a screening technique. These styles differ in how regions are filled, but converting between them is a difficult task that is typically done manually. This paper aims to develop a method for coloring Japanese manga composed of sketches and dialogue. Coloring is considered a challenging and time-consuming task for creators. The proposed method proposes using a pix2pix GAN (Generative Adversarial Network) to color a manga sketch when given as input. Additionally, an alternative approach is suggested, which involves modifying the generator loss function to improve the performance. This paper aims to automate the process of coloring a single sketch, with the potential to extend the research in the future to apply the colors consistently across multiple manga strips.
Will XAI Provide Real Explanation or Just a Plausible Rationalization?

This paper discusses the analogies between the mainstream theory of human mind and the two broad paradigms that are employed when building artificial intelligence systems. Then it ponders the idea how those analogies could be utilized in building a truly explainable artificial intelligence (AI) applications. The core part is devoted to the problem of unwanted rationalization that could disguise the true reasons lying behind the decisions of the explainable AI systems.
Efficient Shapley Values Calculation for Transformer Explainability

Class activation mapping (CAM) methods have achieved great model explainability performance for CNNs. However, these methods do not perform so well for Transformers, whose architectures are fundamentally different from CNNs. Instead, gradient-weighted attention visualization methods, with effective consideration for the self-attention and skip-connection, achieve very promising explainability for Transformers. These methods compute gradients by back-propagation to achieve class-specific and accurate explainability. In this work, to further increase the accuracy and efficiency in Transformer explainability, we propose a novel method which is both class-specific and gradient-free. The token importance is calculated using Shapley value method, which has a solid base on game theory but is conventionally very computational expensive to use in practice. To calculate the Shapley value accurately and efficiently for each token, we decouple the self-attention from the information flow in Transformers and freeze other unrelated values. In this way, we construct a linear version of Transformer so that the Shapley values can be calculated conveniently. Using Shapley values for explainability, our method not only improves the explainability further but also becomes class-specific without using gradients, surpassing other gradient-based methods in both accuracy and efficiency. Furthermore, we show that explainability methods for CNNs and Transformers can be bridged under the 1st-order Taylor expansion of our method, resulting in (1) a significant explainability improvement for a modified GradCAM method in Transformers and (2) new insights into understanding the existing gradient-based attention visualization methods. Extensive experiments show that our method is superior compared to state-of-the-arts methods. Our code will be made available.
Heatmap Overlay Using Neutral Body Model for Visualizing the Measured Gaze Distributions of Observers

We propose a method for visualizing where an observer’s gaze focuses on a subject in a still image using a neutral human body model. Generally, two-dimensional (2D) heatmaps are superimposed on still images to visualize an observer’s gaze distribution, which indicates where an observer looks when observing a subject. To investigate gaze distributions, eye-tracking researchers need a method to directly compare the 2D heatmaps because body pose and shape differ among subjects. Thus, a comparison of the gaze distributions using the 2D heatmaps is time-consuming if there is no acceptable method to handle the body pose and shape variations. Instead, our visualization method superimposes a three-dimensional (3D) heatmap representing the gaze distribution on the surface of a neutral human body, which has a fixed pose and shape for all subjects to visualize the locations at which an observer’s gaze focuses. Experimental results show that our visualization method allows eye-tracking researchers to compare gaze distributions more directly than the conventional visualization method using 2D heatmaps on still images.
Toward Defensive Letter Design

A major approach for defending against adversarial attacks aims at controlling only image classifiers to be more resilient, and it does not care about visual objects, such as pandas and cars, in images. This means that visual objects themselves cannot take any defensive actions, and they are still vulnerable to adversarial attacks. In contrast, letters are artificial symbols, and we can freely control their appearance unless losing their readability. In other words, we can make the letters more defensive to the attacks. This paper poses three research questions related to the adversarial vulnerability of letter images: (1) How defensive are the letters against adversarial attacks? (2) Can we estimate how defensive a given letter image is before attacks? (3) Can we control the letter images to be more defensive against adversarial attacks? For answering the first and second questions, we measure the defensibility of letters by employing Iterative Fast Gradient Sign Method (I-FGSM) and then build a deep regression model for estimating the defensibility of each letter image. We also propose a two-step method based on a generative adversarial network (GAN) for generating character images with higher defensibility, which solves the third research question.
Character Structure Analysis by Adding and Pruning Neural Networks in Handwritten Kanji Recognition

In recent years, there has been a growing need for techniques that enable users to explain the basis for decisions made by neural networks in image recognition problems. While many conventional methods have focused on presenting the spatial basis for pattern recognition judgments by identifying the regions in the image that significantly influence the recognition results, our proposed method focuses on presenting a structural basis for recognizing handwritten Kanji characters. This is accomplished by pruning neural networks to acquire detectors for simple patterns that are commonly found in Kanji characters. During the process of sequentially adding these detectors, we also apply pruning to the network connecting the detectors, thereby aiming to precisely acquire simple hierarchical connections among the detectors as a structural recognition process in pattern recognition. We successfully applied this method to simple handwritten Kanji images, and achieved Kanji recognition by combining detectors for simple patterns without significantly affecting the recognition rate.
Scene Text Detection with Box Supervision and Level Set Evolution

For arbitrarily-shaped scene text detection, most existing methods require expensive polygon-level annotations for supervised training. In order to reduce the cost in data annotation, we propose a novel bounding box supervised scene text detection method, which needs training images only labeled in rectangular boxes. The detection model integrates the classical level set model with deep neural network. It consists a backbone network, a text proposal network, and a text segmentation network. For weakly-supervised training of the segmentation network using box supervision, the proposed method iteratively learns a series of level sets through a Chan-Vese energy based loss function. The segmentation network is trained by minimizing the fully differentiable level set energy function wherein the text instance boundary is iteratively updated. Further, both the input image and its deep features are employed in the level set energy function to improve the convergence. The proposed method can be trained in weakly-supervised or mixed-supervised manner. Extensive experiments on five benchmarks (ICDAR2015, C-SVT, CTW1500, Total-Text, and ICDAR-ArT) show that our mixed-supervised model can achieve competitive detection performance.
SSC-$$l_0$$: Sparse Subspace Clustering with the $$l_0$$ Inequality Constraint

Self-expression learning methods often obtain a coefficient matrix to measure the similarity between pairs of samples. However, directly using all points to represent a fixed sample in a class under the self-expression framework may not be ideal, as points from other classes participate in the representing process. To alleviate this issue, this study attempts to achieve representation learning between points only coming from the same class. In practice, it is easier for data points from the same class to represent each other than that from different classes. So, when reconstructing a point, if the number of non-zero elements in the coefficient vector is limited, a model is more likely to select data points from the class where the reconstructed point lies to complete the reconstruction work. Based on this idea, we propose Sparse Subspace Clustering with the $$l_0$$ inequality constraint (SSC- $$l_0$$ ). In SSC- $$l_0$$ , the $$l_0$$ inequality constraint determines the maximum number of non-zero elements in the coefficient vector, which helps SSC- $$l_0$$ to conduct representation learning among the points in the same class. After introducing the simplex constraint to ensure the translation invariance of the model, an optimization method concerning $$l_0$$ inequality constraint is formed to solve the proposed SSC- $$l_0$$ , and its convergence is theoretically analyzed. Extensive experiments on well-known datasets demonstrate the superiority of SSC- $$l_0$$ compared to several state-of-the-art methods.
A Novel Convolutional Neural Network with Large Kernel for Classification of Crash Test Photos

Since variation of car’s types, direction classification of crash test photos for multiple type of vehicles is a big challenge to office automation. Moreover, because of the similarity of images before and after crash test, semantic identification of these two classes is also difficult. Inspired by recent advances in large kernel CNNs, in this paper, we introduce a 31 × 31 extra-large convolutional kernel to gain more effective receptive field and makes the model more powerful for semantic segmentation. To meet the requirement to generate testing report, totally 14 classes were applied in our experiment. The accuracy of 97.1% was obtained on a self-build photo dataset.
Underwater Image Restoration Based on Light Attenuation Prior and Scene Depth Fusion Model

Underwater images often suffer from blurry details, color distortion, and low contrast due to light absorption and scattering in water. Existing restoration technologies use a fixed attenuation coefficient value, which fails to account for the uncertainty of the water body and leads to suboptimal restoration results. To address these issues, we propose a scene depth fusion model that considers underwater light attenuation to obtain a more accurate attenuation coefficient for image restoration. Our method employs the quadtree decom-position method and a depth map to estimate the background light. We then fuse and refine the depth map, compute the attenuation coefficient of the water medium for a more precise transmission map, and apply a reversed underwater imaging model to restore the image. Experiments demonstrate that our method effectively enhances the details and colors of underwater images while improving the contrast. Moreover, our method outperforms several state-of-the-art methods in terms of both accuracy and quality, showing its superior performance.