Reviewerly/exHarmony huggingface dataset:
LoSAC: An Efficient Local Stochastic Average Control Method for Federated Optimization

Federated optimization (FedOpt), which targets at collaboratively training a learning model across a large number of distributed clients, is vital for federated learning. The primary concerns in FedOpt can be attributed to the model divergence and communication efficiency, which significantly affect the performance. In this article, we propose a new method, i.e., LoSAC, to learn from heterogeneous distributed data more efficiently. Its key algorithmic insight is to locally update the estimate for the global full gradient after each regular local model update. Thus, LoSAC can keep clients’ information refreshed in a more compact way. In particular, we have studied the convergence result for LoSAC. Besides, the bonus of LoSAC is the ability to defend the information leakage from the recent technique Deep Leakage Gradients (DLG). Finally, experiments have verified the superiority of LoSAC comparing with state-of-the-art FedOpt algorithms. Specifically, LoSAC significantly improves communication efficiency by more than 100% on average, mitigates the model divergence problem, and equips with the defense ability against DLG.
Breast Cancer Detection Using Convolutional Neural Network with EfficientNet Architecture

Breast cancer is the second most deadly disease in Indonesia after lung cancer. This is caused due to a lack of public awareness of this disease. Histopathological examination is one of the methods of detecting breast cancer which is performed manually by a pathologist. However, the diagnostic process of histopathological examination takes a long time and allows a shift in diagnosis. Therefore, technologies are needed to help pathologists to get a diagnosis quickly and accurately. One of them is deep learning. In this research, the deep learning-based system model was developed using the Convolutional Neural Network (CNN) with EfficientNet architecture to help radiologists to get better accuracy and performance in detecting breast cancer. The system performance was tested on 1361 histopathology images consisting of two classes, which are normal and cancer with data ratios for train, validation, and test are 75%, 10%, and 15%. By using the proposed method, the system is able to classify images with the best accuracy rate of 94.26%, loss of 39.52%, precision of 94%, and recall of 94%. This result was achieved on a system model with number parameters, which are image size 64x64 pixels, Adam optimizer, learning rate 0.001, batch size 32, and epoch 50.
An Integrated Unsupervised and Supervised Learning Technique for Interference Analysis in Vehicular Communications

In vehicular communications, various sensors and technologies such as radars, cameras, and LiDARs coexist and often share the same radio frequency spectrum. This results in a complex and dynamic radio environment with significant mutual interference, which poses an immense challenge to achieve reliable communication and performance. Intelligent techniques based on machine learning and artificial intelligence have been found as promising solutions to tackle this challenge. In this paper, we propose a hybrid technique that integrates an unsupervised clustering method with a Convolutional Neural Network (CNN) based system that detects the interfering packets by processing the received signal spectrogram. We then compare the prediction accuracy performance of the proposed system under different Signal-to-Noise-Ratio conditions and with different training mechanisms. Our results suggest that the proposed system can perform reliably under a wide range of SNR conditions.
Patterns and Predictions: Generative Adversarial Networks for Neighborhood Generation

Urban climate patterns affect the quality of life of growing urban populations. Studying microclimate patterns, particularly relating to heat, is key to protecting urban residents. The morphology of urban neighborhoods affects local weather patterns, and the development of new neighborhoods could potentially affect future weather. Given the complexity of these relationships, machine learning is a perfect candidate for analyzing the data. This study leverages an adversarial network, containing two competing models, to predict future neighborhood possibilities given the land cover in the area. The model has been trained on data from Los Angeles, California, with the images divided into residential, commercial, and mixed neighborhoods. These divisions allow for patterns and predictions to be analyzed on a neighborhood-specific level, addressing the effects of building distribution on localized weather patterns. Once these predictions have been made, they can be fed into existing models and the impact on climate can be examined.
Utilizing Object Capabilities to Improve Web Application Security

Nowadays, more and more applications are built with web technologies like HTML, CSS, and JavaScript, which are then executed in browsers. The web is utilized as an operating system independent application platform. With this change, authorization models change and no longer depend on operating system accounts and underlying access controls and file permissions. Instead, these accounts are now implemented in the applications themselves, including all of the protective measures and security controls that are required for this. Because of the inherent complexity, flaws in the authorization logic are among the most common security vulnerabilities in web applications. Most applications are built on the concept of the Access-Control List (ACLs), a security model that decides who can access what object. Object Capabilities, transferable rights to perform operations on specific objects, have been proposed as an alternative to ACLs, since they are not susceptible to certain attacks prevalent for ACLs. While their use has been investigated for various domains, like smart contracts, they have not been widely applied for web applications. In this paper, we therefore present a general overview of the capability based authorization model and adapt those approaches for use in web applications. Based on a prototype implementation, we show the possibilities of Object Capabilities to enhance security, but also provide insights on existing pitfalls and problems in porting such models to the web domain.
D-ESRGAN: A Dual-Encoder GAN with Residual CNN and Vision Transformer for Iris Image Super-Resolution

Iris images captured in less-constrained environments, especially at long distances often suffer from the interference of low resolution, resulting in the loss of much valid iris texture information for iris recognition. In this paper, we propose a dual-encoder super-resolution generative adversarial network (D-ESRGAN) for compensating texture lost of the raw image meanwhile maintaining the newly generated textures more natural. Specifically, the proposed D-ESRGAN not only integrates the residual CNN encoder to extract local features, but also employs an emerging vision transformer encoder to capture global associative information. The local and global features from two encoders are further fused for the subsequent reconstruction of high-resolution features. During the training, we develop a three-stage strategy to alleviate the problem that generative adversarial networks are prone to collapse. Moreover, to boost the iris recognition performance, we introduce a triplet loss to push away the distance of super-resolved iris images with different IDs, and pull the distance of super-resolved iris images with the same ID much closer. Experimental results on the public CASIA-Iris-distance and CASIA-Iris-M1 datasets show that D-ESRGAN archives better performance than state-of-the-art baselines in terms of both super-resolution image quality metrics and iris recognition metric.
Lightweight Periocular Recognition through Low-bit Quantization

Deep learning-based systems for periocular recognition make use of the high recognition performance of neural networks, which, however, is accompanied by high computational costs and memory footprints. This can lead to deployability problems, especially in mobile devices and embedded systems. Few previous works strived towards building lighter models, however, while still depending on floating-point numbers associated with higher computational cost and memory footprint. In this paper, we propose to adapt model quantization for periocular recognition. This, within the proposed scheme, leads to reducing the memory footprint of periocular recognition network by up to five folds while maintaining high recognition performance. We present a comprehensive analysis over three backbones and diverse experimental protocols to stress the consistency of our conclusions, along with a comparison with a wide set of baselines that prove the optimal trade-off between performance and model size achieved by our proposed solution. The code and pre-trained models have been made available at https://github.com/jankolf/ijcb-periocular-quantization.
Ollivier-Ricci Curvature for Head Pose Estimation from a Single Image

Head pose estimation is not only a crucial challenge for many real-world applications, such as driver attention detection analysis, but it represents an interesting strategy to support biometric frameworks as well. This paper aims to estimate head pose from a single image by applying notions of network curvature. In the real world, many complex networks have groups of nodes that are well connected to each other with significant functional roles. Similarly, the interactions of facial landmarks can be represented as complex dynamic systems modeled by weighted graphs. The functionality of such a system is therefore intrinsically linked to the topology and geometry of the underlying graph. In this work, using the geometric notion of Ollivier-Ricci curvature (ORC) on weighted graphs as input to the XGBoost regression model, we show that the intrinsic geometric basis of ORC offers a natural approach to discovering underlying common structure within a pool of poses. Experiments on the BIWI, AFLW2000 and Pointing '04 datasets show that the ORC_XGB method performs well compared to state-of-the-art methods, both landmark-based and image-only.
Online Learning Facial Expression Detection using Simplified AlexNet Deep Learning Architecture: Image Data Samples Comparison Experiment

Online learning is one of the methods for students to interact with their teachers. However, interactivity aspects in online education should be evaluated. The method to automate the detection of students' enthusiasm when participating in online learning is still not much found. In this paper, we define the facial expressions that could show when students participate in online learning: bored, confused, and excited/delighted. We built a classifier system for these online learning face expressions. We collected a small dataset (for around 480 images) containing face images with these three facial expressions to be trained for the deep learning model. We proposed simplified AlexNet architecture as a deep learning model for a relatively small dataset, that is, AlexNet architecture with lower parameters (such as filters and the number of neurons in the fully connected layer). Image preprocessing methods such as resizing and augmentation are applied for the experiment. We also compare the model performance, given a distinct (three kinds) number of training data (480 samples, 1,500 samples, and 3,000 samples). When our proposed model is trained with these three kinds of training samples amount, the model performance consecutively has an average accuracy of 78.69%, 84.53%, and 90.80% in 5-Fold Cross Validation. Using ANOVA Test, we get a p-value of 0.00073, which is statistically significant to reject the null hypothesis that training samples do not affect our proposed model's accuracy.
PDVN: A Patch-based Dual-view Network for Face Liveness Detection using Light Field Focal Stack

Light Field Focal Stack (LFFS) can be efficiently rendered from a light field (LF) image captured by plenoptic cameras. Differences in the 3D surface and texture of biometric samples are internally reflected in the defocus blur and local patterns between the rendered slices of LFFS. This unique property makes LFFS quite appropriate to differentiate presentation attack instruments (PAIs) from bona fide samples. A patch-based dual-view network (PDVN) is proposed in this paper to leverage the merits of LFFS for face presentation attack detection (PAD). First, original LFFS data are divided into various local patches along spatial dimensions, which distracts the model from learning the useless facial semantics and greatly relieve the problem of insufficient samples. The strategy of dual-view branches is innovatively proposed, wherein the original view and microscopic view can simultaneously contribute to liveness detection. Separable 3D convolution on the focal dimension is verified to be more effective than vanilla 3D convolution for extracting discriminative features from LFFS data. The voting mechanism on predictions of patch LFFS samples further strengthens the robustness of the proposed framework. PDVN is compared with other face PAD methods on IST LLFFSD dataset and achieves perfect performance, i.e., ACER drops to 0.
A Dataless FaceSwap Detection Approach Using Synthetic Images

Face swapping technology used to create “Deepfakes” has advanced significantly over the past few years and now enables us to create realistic facial manipulations. Current deep learning algorithms to detect deepfakes have shown promising results, however, they require large amounts of training data, and as we show they are biased towards a particular ethnicity. We propose a deepfake detection methodology that eliminates the need for any real data by making use of synthetically generated data using Style-GAN3. This not only performs at par with the traditional training methodology of using real data but it shows better generalization capabilities when finetuned with a small amount of real data. Furthermore, this also reduces biases created by facial image datasets that might have sparse data from particular ethnicities. To promote reproducibility the code base has been made publicly available <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> https://github.com/anubhav1997/youneednodataset
Contrastive Domain Adaptation: A Self-Supervised Learning Framework for sEMG-Based Gesture Recognition

Gesture recognition using surface electromyography (sEMG) shows its great potential in the field of human-computer interaction (HCI). Previous works achieve relatively good performance based on the assumption of invariant statistic distribution. However, the practical application effect is unsatisfactory due to the problem of domain shift. Existing approaches need plenty of labeled sEMG samples from target scenarios for calibration, which is burdensome for experimenters and users. In this work, we present a contrastive self-supervised learning framework (ConSSL) for sEMG-based gesture recognition to realize domain adaptation in target domains. After pretraining on a bunch of unlabeled samples, only a small number of labeled samples are needed for calibration and domain adaptation. Experimental results indicate that the proposed framework out-performs other approaches even <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$if\leq 50\%$</tex> labeled samples in target scenarios are available and achieves the state-of-the-art.
Face Regions Impact Recognition Accuracy Differently Across Demographics

Variation in face recognition accuracy across demographic groups has attracted attention from news media, civil liberties advocates and academic researchers. The problem is challenging, in that both the impostor distribution (matches across different people) and the genuine distribution (matches across same people) may vary across demographic groups. Simple answers such as balancing the number of subjects and images in the training data do not have a substantial impact on demographic accuracy disparities. We present the first investigation into whether parts of the face - such as eyes, nose, mouth - show the same accuracy differences across demographic groups as are seen with matching the whole face. We show that matching focused on different parts of the face may result in opposite accuracy differences across demographics. For example, using the eye region for face matching results in Caucasian males having a better impostor distribution (lower similarity scores) than Caucasian females, but using the nose regionfor face matching results in Caucasian females having a better impostor distribution. We also show that it is possible to select face region(s) that effectively minimize the difference in the impostor or genuine distributions across at least some demographics. Our results suggest that a new pathway to reducing accuracy disparity across demographic groups may be to weight the parts of the face differently in matching.
SketchCLIP: Text-based Attribute Manipulation for Face Sketch Synthesis

This paper proposes a method of modifying the face sketch with text descriptions. Face sketch is widely used in the criminal field and digital entertainment field. Forensic painters usually draw face sketches based on descriptions provided by witnesses or clients. However, drawing a face sketch often takes lots of time and effort. Existing face sketch synthesis studies have not considered text-based sketch manipulation, and we find that applying text-driven editing methods on natural images directly to face sketches causes severe distortion of generated results. Therefore, this paper proposes a novel text-based attribute manipulation method for face sketch synthesis, named SketchCLIP. Our approach adopts text-driven attribute manipulation by using the powerful Contrastive Language-Image Pre-Training (CLIP) model, which not only conforms to the current drawing process of face sketches but also does not require tedious manual operations and allows for more diverse modifications. Besides, we design an intra-modality fine-tuning module to eliminate distortion and improve the quality of the modified face sketch. Through extensive comparison experiments on public face sketch datasets, our method is demonstrated to be very excellent in the effectiveness of the face sketch processing and the quality of modified results.
In-group and Out-group Performance Bias in Facial Retouching Detection

Accuracy alone is not sufficient to establish the efficacy of an AI algorithm-issues of demographic bias are an important area of concern. Demographic bias in face recognition algorithms has attracted more attention from the re-search community to date, but bias can also be a problem for face image analysis algorithms, such as detection of manipulated face images. In this paper, we investigate performance of humans and algorithms at detecting retouched face images of subjects from different origin (America, India, China) and gender groups. To be representative of the state of retouching detection, we use eight different algorithms from the literature. In addition to overall human accuracy, differences across origin and gender of the human performing the task are analyzed. We observe different bias patterns, such as algorithms show higher in-group accuracy than out-group, while the extent of retouching and familiarity drives differences in detection accuracy for humans. This is the first work to analyze and compare bias exhibited by humans and algorithms in similar tasks of detecting retouched face images.
Which One is Better? Self-supervised Temporal Coherence Learning for Skeleton Based Action Recognition

Recently, researchers have achieved significant results in the skeleton based action recognition task. To better model the skeleton sequences, existing methods learned the feature representations in the self-supervised setting by solving pretext tasks, such as predicting the order of a shuffled skeleton sequence or verifying whether a given skeleton sequence is shuffled or not. However, these pretext tasks are either too challenging or too easy for the encoder to obtain a proper skeleton representation for action recognition. Therefore, we propose a novel self-pretraining pretext task, Which One Is Better (WOIB), to identify which one is more temporally coherent, given two shuffled skeleton sequences. Experiments on the NTU RGB+D, NTU RGB+D 120, and Kinetics-Skeleton datasets with different network architectures show significant improvements in recognition accuracy, demonstrating that such a well-designed pretext task is general and able to drive the encoder to learn more discriminative representations.
A Heuristic Approach towards Continuous Implicit Authentication

Smartphones nowadays handle large amounts of sensitive user information, since users exchange undisclosed information on an everyday basis. This generates the need for more effective authentication mechanisms, deviating from the traditional ones. In this direction, many research approaches are targeted towards continuous implicit authentication, on the basis of modelling the constant interaction of the user with the device. These approaches yield promising results, however certain improvements can be made by exploiting the sequential order of the predictions and the known performance metrics. In this work, we propose a heuristics algorithm, which, given a series of predictions from any continuous implicit authentication model, can ex-ploit the sequential order in order to fix any false predictions and improve the accuracy of the smartphone security system. Preliminary evaluation on several axes indicates that our approach can effectively improve any CIA model and achieve significantly better results.
Efficacy, Efficiency and Effectiveness of SMarty-based Software Product Line Inspection Techniques: a Controlled Quasi-Experiment

Software quality is impacted by several factors, including the quality of the artifacts generated during the development process. Especially for Software Product Lines (SPL), as artifacts are reused for various specific products, a defect can be spread out over an SPL. Our research group has previously created two inspection techniques for UML-based SPLs: a checklist-based, named SMartyCheck, and a perspective-based, named SMartyPerspective. Seeking to understand the efficiency, efficacy, and effectiveness of such techniques, we carried out a controlled quasi-experiment with 16 participants from the Software Engineering area. It aimed at inspecting feature diagrams and use case, class, component, and sequence diagrams designed using the SMarty approach for UML-based variability support. We also considered ad hoc inspections in such a study. The results of this experiment provide incipient evidence of no statistical difference among the compared techniques for efficiency, efficacy, and effectiveness.
Benchmark Generation with VEVOS: A Coverage Analysis of Evolution Scenarios in Variant-Rich Systems

Clone-and-own development is a simple and flexible approach to realize multi-variant software systems in practice but typically provokes costly challenges in maintaining a continuously evolving set of variants. Therefore, managed clone-and-own development is key to efficiently mitigate these problems. While supporting techniques have been proposed in the literature, hardly any of them have been evaluated in a realistic setting due to a substantial lack of publicly available clone-and-own projects which could be used as experimental subjects. Recently, we presented the benchmark generation framework VEVOS for simulating clone-and-own development. However, it is yet unclear to which extent VEVOS can cover key scenarios for evaluating evolving variant-rich systems. This paper examines to what extent benchmarks created by VEVOS satisfy evaluation requirements for evolution scenarios demanded by the community. In addition, we report on our own experiences when employing VEVOS within six studies and elaborate on necessary extensions we implemented into VEVOS.