Reviewerly/exHarmony huggingface dataset:
Seq-UPS: Sequential Uncertainty-aware Pseudo-label Selection for Semi-Supervised Text Recognition

This paper looks at semi-supervised learning (SSL) for image-based text recognition. One of the most popular SSL approaches is pseudo-labeling (PL). PL approaches assign labels to unlabeled data before re-training the model with a combination of labeled and pseudo-labeled data. However, PL methods are severely degraded by noise and are prone to over-fitting to noisy labels, due to the inclusion of erroneous high confidence pseudo-labels generated from poorly calibrated models, thus, rendering threshold-based selection ineffective. Moreover, the combinatorial complexity of the hypothesis space and the error accumulation due to multiple incorrect autoregressive steps posit pseudo-labeling challenging for sequence models. To this end, we propose a pseudo-label generation and an uncertainty-based data selection framework for semi-supervised text recognition. We first use Beam-Search inference to yield highly probable hypotheses to assign pseudo-labels to the unlabelled examples. Then we adopt an ensemble of models, sampled by applying dropout, to obtain a robust estimate of the uncertainty associated with the prediction, considering both the character-level and word-level predictive distribution to select good quality pseudo-labels. Extensive experiments on several benchmark handwriting and scene-text datasets show that our method outperforms the baseline approaches and the previous state-of- the-art semi-supervised text-recognition methods.
Mutual Learning for Long-Tailed Recognition

Deep neural networks perform well in artificially- balanced datasets, but real-world data often has a long-tailed distribution. Recent studies have focused on developing unbiased classifiers to improve tail class performance. Despite the efforts to learn a fine classifier, we cannot guarantee a solid performance if the representations are of poor quality. However, learning high-quality representations in a long-tailed setting is difficult because the features of tail classes easily overfit the training dataset. In this work, we propose a mutual learning framework that generates high- quality representations in long-tailed settings by exchanging information between networks. We show that the proposed method can improve representation quality and establish a new state-of-the-art record on several long-tailed recognition benchmark datasets, including CIFAR100-LT, ImageNet-LT, and iNaturalist 2018.
Cross-identity Video Motion Retargeting with Joint Transformation and Synthesis

In this paper, we propose a novel dual-branch Transformation-Synthesis network (TS-Net), for video motion retargeting. Given one subject video and one driving video, TS-Net can produce a new plausible video with the subject appearance of the subject video and motion pattern of the driving video. TS-Net consists of a warp-based transformation branch and a warp-free synthesis branch. The novel design of dual branches combines the strengths of deformation-grid-based transformation and warp-free generation for better identity preservation and robustness to occlusion in the synthesized videos. A mask-aware similarity module is further introduced to the transformation branch to reduce computational overhead. Experimental results on face and dance datasets show that TS-Net achieves better performance in video motion retargeting than several state-of-the-art models as well as its single-branch variants. Our code is available at https://github.com/nihaomiao/WACV23_TSNet.
Arbitrary Style Guidance for Enhanced Diffusion-Based Text-to-Image Generation

Diffusion-based text-to-image generation models like GLIDE and DALLE-2 have gained wide success recently for their superior performance in turning complex text inputs into images of high quality and wide diversity. In particular, they are proven to be very powerful in creating graphic arts of various formats and styles. Although current models supported specifying style formats like oil painting or pencil drawing, fine-grained style features like color distributions and brush strokes are hard to specify as they are randomly picked from a conditional distribution based on the given text input. Here we propose a novel style guidance method to support generating images using arbitrary style guided by a reference image. The generation method does not require a separate style transfer model to generate desired styles while maintaining image quality in generated content as controlled by the text input. Additionally, the guidance method can be applied without a style reference, denoted as self style guidance, to generate images of more diverse styles. Comprehensive experiments prove that the proposed method remains robust and effective in a wide range of conditions, including diverse graphic art forms, image content types and diffusion models.
High-Quality RGB-D Reconstruction via Multi-View Uncalibrated Photometric Stereo and Gradient-SDF

Fine-detailed reconstructions are in high demand in many applications. However, most of the existing RGB-D reconstruction methods rely on pre-calculated accurate camera poses to recover the detailed surface geometry, where the representation of a surface needs to be adapted when optimizing different quantities. In this paper, we present a novel multi-view RGB-D based reconstruction method that tackles camera pose, lighting, albedo, and surface normal estimation via the utilization of a gradient signed distance field (gradient-SDF). The proposed method formulates the image rendering process using specific physically-based model(s) and optimizes the surface’s quantities on the actual surface using its volumetric representation, as opposed to other works which estimate surface quantities only near the actual surface. To validate our method, we investigate two physically-based image formation models for natural light and point light source applications. The experimental results on synthetic and real-world datasets demonstrate that the proposed method can recover high-quality geometry of the surface more faithfully than the state-of-the-art and further improves the accuracy of estimated camera poses <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Improving Multi-fidelity Optimization with a Recurring Learning Rate for Hyperparameter Tuning

Despite the evolution of Convolutional Neural Networks (CNNs), their performance is surprisingly dependent on the choice of hyperparameters. However, it remains challenging to efficiently explore large hyperparameter search space due to the long training times of modern CNNs. Multi-fidelity optimization enables the exploration of more hyperparameter configurations given budget by early termination of unpromising configurations. However, it often results in selecting a sub-optimal configuration as training with the high-performing configuration typically converges slowly in an early phase. In this paper, we propose Multi-fidelity Optimization with a Recurring Learning rate (MORL) which incorporates CNNs’ optimization process into multi-fidelity optimization. MORL alleviates the problem of slow-starter and achieves a more precise low-fidelity approximation. Our comprehensive experiments on general image classification, transfer learning, and semi-supervised learning demonstrate the effectiveness of MORL over other multi-fidelity optimization methods such as Successive Halving Algorithm (SHA) and Hyperband. Furthermore, it achieves significant performance improvements over hand-tuned hyperparameter configuration within a practical budget.
Automated Artificial Intelligence Empowered White Blood Cells Classification燤odel

White blood cells (WBC) or leukocytes are a vital component of the blood which forms the immune system, which is accountable to fight foreign elements.The WBC images can be exposed to different data analysis approaches which categorize different kinds of WBC.Conventionally, laboratory tests are carried out to determine the kind of WBC which is erroneous and time consuming.Recently, deep learning (DL) models can be employed for automated investigation of WBC images in short duration.Therefore, this paper introduces an Aquila Optimizer with Transfer Learning based Automated White Blood Cells Classification (AOTL-WBCC) technique.The presented AOTL-WBCC model executes data normalization and data augmentation process (rotation and zooming) at the initial stage.In addition, the residual network (ResNet) approach was used for feature extraction in which the initial hyperparameter values of the ResNet model are tuned by the use of AO algorithm.Finally, Bayesian neural network (BNN) classification technique has been implied for the identification of WBC images into distinct classes.The experimental validation of the AOTL-WBCC methodology is performed with the help of Kaggle dataset.The experimental results found that the AOTL-WBCC model has outperformed other techniques which are based on image processing and manual feature engineering approaches under different dimensions.
Adaptive Dynamic Dipper Throated Optimization for Feature Selection in Medical Data

The rapid population growth results in a crucial problem in the early detection of diseases in medical research.Among all the cancers unveiled, breast cancer is considered the second most severe cancer.Consequently, an exponential rising in death cases incurred by breast cancer is expected due to the rapid population growth and the lack of resources required for performing medical diagnoses.Utilizing recent advances in machine learning could help medical staff in diagnosing diseases as they offer effective, reliable, and rapid responses, which could help in decreasing the death risk.In this paper, we propose a new algorithm for feature selection based on a hybrid between powerful and recently emerged optimizers, namely, guided whale and dipper throated optimizers.The proposed algorithm is evaluated using four publicly available breast cancer datasets.The evaluation results show the effectiveness of the proposed approach from the accuracy and speed perspectives.To prove the superiority of the proposed algorithm, a set of competing feature 1884 CMC, 2023, vol.75, no.1 selection algorithms were incorporated into the conducted experiments.In addition, a group of statistical analysis experiments was conducted to emphasize the superiority and stability of the proposed algorithm.The best-achieved breast cancer prediction average accuracy based on the proposed algorithm is 99.453%.This result is achieved in an average time of 3.6725 s, the best result among all the competing approaches utilized in the experiments.
Detection of Worker’s Safety Helmet and Mask and Identification of Worker Using Deeplearning

This paper proposes a method for detecting a helmet for the safety of workers from risk factors and a mask worn indoors and verifying a worker's identity while wearing a helmet and mask for security.The proposed method consists of a part for detecting the worker's helmet and mask and a part for verifying the worker's identity.An algorithm for helmet and mask detection is generated by transfer learning of Yolov5's s-model and m-model.Both models are trained by changing the learning rate, batch size, and epoch.The model with the best performance is selected as the model for detecting masks and helmets.At a learning rate of 0.001, a batch size of 32, and an epoch of 200, the s-model showed the best performance with a mAP of 0.954, and this was selected as an optimal model.The worker's identification algorithm consists of a facial feature extraction part and a classifier part for the worker's identification.The algorithm for facial feature extraction is generated by transfer learning of Facenet, and SVM is used as the classifier for identification.The proposed method makes trained models using two datasets, a masked face dataset with only a masked face, and a mixed face dataset with both a masked face and an unmasked face.And the model with the best performance among the trained models was selected as the optimal model for identification when using a mask.As a result of the experiment, the model by transfer learning of Facenet and SVM using a mixed face dataset showed the best performance.When the optimal model was tested with a mixed dataset, it showed an accuracy of 95.4%.Also, the proposed model was evaluated as data from 500 images of taking 10 people with a mobile phone.The results showed that the helmet and mask were detected well and identification was also good.
FreeREA: Training-Free Evolution-based Architecture Search

In the last decade, most research in Machine Learning contributed to the improvement of existing models, with the aim of increasing the performance of neural networks for the solution of a variety of different tasks. However, such advancements often come at the cost of an increase of model memory and computational requirements. This represents a significant limitation for the deployability of research output in realistic settings, where the cost, the energy consumption, and the complexity of the framework play a crucial role. To solve this issue, the designer should search for models that maximise the performance while limiting its footprint. Typical approaches to reach this goal rely either on manual procedures, which cannot guarantee the optimality of the final design, or upon Neural Architecture Search algorithms to automatise the process, at the expenses of extremely high computational time. This paper provides a solution for the fast identification of a neural network that maximises the model accuracy while preserving size and computational constraints typical of tiny devices. Our approach, named FreeREA, is a custom cell-based evolution NAS algorithm that exploits an optimised combination of training-free metrics to rank architectures during the search, thus without need of model training. Our experiments, carried out on the common benchmarks NAS-Bench-101 and NATS-Bench, demonstrate that i) FreeREA is a fast, efficient, and effective search method for models automatic design; ii) it outperforms State of the Art training-based and training-free techniques in all the datasets and benchmarks considered, and iii) it can easily generalise to constrained scenarios, representing a competitive solution for fast Neural Architecture Search in generic constrained applications. The code is available at \url{https://github.com/NiccoloCavagnero/FreeREA}.
One-Class Arabic Signature Verification: A Progressive Fusion of Optimal Features

Signature verification is regarded as the most beneficial behavioral characteristic-based biometric feature in security and fraud protection.It is also a popular biometric authentication technology in forensic and commercial transactions due to its various advantages, including noninvasiveness, user-friendliness, and social and legal acceptability.According to the literature, extensive research has been conducted on signature verification systems in a variety of languages, including English, Hindi, Bangla, and Chinese.However, the Arabic Offline Signature Verification (OSV) system is still a challenging issue that has not been investigated as much by researchers due to the Arabic script being distinguished by changing letter shapes, diacritics, ligatures, and overlapping, making verification more difficult.Recently, signature verification systems have shown promising results for recognizing signatures that are genuine or forgeries; however, performance on skilled forgery detection is still unsatisfactory.Most existing methods require many learning samples to improve verification accuracy, which is a major drawback because the number of available signature samples is often limited in the practical application of signature verification systems.This study addresses these issues by presenting an OSV system based on multifeature fusion and discriminant feature selection using a genetic algorithm (GA).In contrast to existing methods, which use multiclass learning approaches, this study uses a oneclass learning strategy to address imbalanced signature data in the practical application of a signature verification system.The proposed approach is tested on three signature databases (SID-Arabic handwriting signatures, CEDAR (Center of Excellence for Document Analysis and Recognition), and UTSIG (University of Tehran Persian Signature), and experimental results show that the proposed system outperforms existing systems in terms of reducing the False Acceptance Rate (FAR), False Rejection Rate (FRR), and Equal Error Rate (ERR).The proposed system achieved 5% improvement.220
Visualizing Global Explanations of Point Cloud DNNs

So far, few researchers have targeted the explainability of point cloud neural networks. Part of the explainability methods are not directly applicable to those networks due to the structural specifics. In this work, we show that Activation Maximization (AM) with traditional pixel-wise regularizations fails to generate human-perceptible global explanations for point cloud networks. We propose new generative model-based AM approaches to clearly outline the global explanations and enhance their comprehensibility. Additionally, we propose a composite evaluation metric to address the limitations of existing evaluating methods, which simultaneously takes into account activation value, diversity and perceptibility. Extensive experiments demonstrate that our generative-based AM approaches outperform regularization-based ones both qualitatively and quantitatively. To the best of our knowledge, this is the first work investigating global explainability of point cloud networks. Our code is available at: https://github.com/Explain3D/PointCloudAM.
Image-free Domain Generalization via CLIP for 3D Hand Pose Estimation

RGB-based 3D hand pose estimation has been successful for decades thanks to large-scale databases and deep learning. However, the hand pose estimation network does not operate well for hand pose images whose characteristics are far different from the training data. This is caused by various factors such as illuminations, camera angles, diverse backgrounds in the input images, etc. Many existing methods tried to solve it by supplying additional large-scale unconstrained/target domain images to augment data space; however collecting such large-scale images takes a lot of labors. In this paper, we present a simple image-free domain generalization approach for the hand pose estimation framework that uses only source domain data. We try to manipulate the image features of the hand pose estimation network by adding the features from text descriptions using the CLIP (Contrastive Language-Image Pretraining) model. The manipulated image features are then exploited to train the hand pose estimation network via the contrastive learning framework. In experiments with STB and RHD datasets, our algorithm shows improved performance over the state-of-the-art domain generalization approaches.
Continuous Mobile User Authentication Using a Hybrid CNN-Bi-LSTM Approach

Internet of Things (IoT) devices incorporate a large amount of data in several fields, including those of medicine, business, and engineering.User authentication is paramount in the IoT era to assure connected devices' security.However, traditional authentication methods and conventional biometrics-based authentication approaches such as face recognition, fingerprints, and password are vulnerable to various attacks, including smudge attacks, heat attacks, and shoulder surfing attacks.Behavioral biometrics is introduced by the powerful sensing capabilities of IoT devices such as smart wearables and smartphones, enabling continuous authentication.Artificial Intelligence (AI)-based approaches introduce a bright future in refining large amounts of homogeneous biometric data to provide innovative user authentication solutions.This paper presents a new continuous passive authentication approach capable of learning the signatures of IoT users utilizing smartphone sensors such as a gyroscope, magnetometer, and accelerometer to recognize users by their physical activities.This approach integrates the convolutional neural network (CNN) and recurrent neural network (RNN) models to learn signatures of human activities from different users.A series of experiments are conducted using the MotionSense dataset to validate the effectiveness of the proposed method.Our technique offers a competitive verification accuracy equal to 98.4%.We compared the proposed method with several conventional machine learning and CNN models and found that our proposed model achieves higher identification accuracy than the recently developed verification systems.The high accuracy achieved by the proposed method proves its effectiveness in recognizing IoT users passively through their physical activity patterns.
Enhanced Clustering Based OSN Privacy Preservation to Ensure k-Anonymity, t-Closeness, l-Diversity, and Balanced Privacy Utility

Online Social Networks (OSN) sites allow end-users to share a great deal of information, which may also contain sensitive information, that may be subject to commercial or non-commercial privacy attacks.As a result, guaranteeing various levels of privacy is critical while publishing data by OSNs.The clustering-based solutions proved an effective mechanism to achieve the privacy notions in OSNs.But fixed clustering limits the performance and scalability.Data utility degrades with increased privacy, so balancing the privacy utility trade-off is an open research issue.The research has proposed a novel privacy preservation model using the enhanced clustering mechanism to overcome this issue.The proposed model includes phases like pre-processing, enhanced clustering, and ensuring privacy preservation.The enhanced clustering algorithm is the second phase where authors modified the existing fixed k-means clustering using the threshold approach.The threshold value is determined based on the supplied OSN data of edges, nodes, and user attributes.Clusters are k-anonymized with multiple graph properties by a novel one-pass algorithm.After achieving the k-anonymity of clusters, optimization was performed to achieve all privacy models, such as k-anonymity, t-closeness, and l-diversity.The proposed privacy framework achieves privacy of all three network components, i.e., link, node, and user attributes, with improved utility.The authors compare the proposed technique to underlying methods using OSN Yelp and Facebook datasets.The proposed approach outperformed the underlying state of art methods for Degree of Anonymization, computational efficiency, and information loss.
Pruning-Guided Curriculum Learning for Semi-Supervised Semantic Segmentation

This study focuses on improving the quality of pseudolabeling in the context of semi-supervised semantic segmentation. Previous studies have adopted confidence thresholding to reduce erroneous predictions in pseudo-labeled data and to enhance their qualities. However, numerous pseudolabels with high confidence scores exist in the early training stages even though their predictions are incorrect, and this ambiguity limits confidence thresholding substantially. In this paper, we present a novel method to resolve the ambiguity of confidence scores with the guidance of network pruning. A recent finding showed that network pruning severely impairs the network generalization ability on samples that are not yet well learned or represented. Inspired by this finding, we refine the confidence scores by reflecting the extent to which the predictions are affected by pruning. Furthermore, we adopted a curriculum learning strategy for the confidence score, which enables the network to learn gradually from easy to hard samples. This approach resolves the ambiguity by suppressing the learning of noisy pseudolabels, the confidence scores of which are difficult to trust owing to insufficient training in the early stages. Extensive experiments on various benchmarks demonstrate the superiority of our framework over state-of-the-art alternatives.