Reviewerly/exHarmony huggingface dataset:
Space-Transform Margin Loss with Mixup for Long-Tailed Visual Recognition

In the real world, naturally collected data often exhibits a long-tailed distribution, where the head classes have a larger number of samples compared to the tail classes. This long-tailed data distribution often introduces a bias in classification results, leading to incorrect classifications that harm the tail classes. Mixup is a simple but effective data augmentation method that transforms data into a new shrinking space, resulting in a regularization effect that is beneficial for classification. Therefore, many researchers consider using Mixup to promote the performance of long-tailed learning. However, these methods do not consider the special space transformation of data caused by Mixup in long-tail learning. In this paper, we present the Space-Transform Margin (STM) loss function, which offers a novel approach to dynamically adjusting the margin between classes by leveraging the shrinking strength introduced by Mixup. In this way, the margin of data can adapt to the special space transformation of Mixup. In the experiments, our solution achieves state-of-the-art performance on benchmark datasets, including CIFAR10-LT, CIFAR100-LT, ImageNet-LT, and iNaturalist 2018.
C-volution: A Hybrid Operator for Visual Recognition

Convolution is a fundamental building block of modern neural networks, playing a critical role in the success of deep learning for vision tasks. However, convolutional neural networks exhibit limited spatial context due to their local receptive field, which also neglects global/long-term dependent relations. To this end, we propose a lightweight hybrid structure operator, called C-volution. The operator utilizes a multi-branch architecture to extract spatial and channel information from input data separately, enabling the network to capture abstract features while preserving important spatial information. In addition, summarize context information in a larger spatial range by generating dynamic kernels to strengthen the spatial contextual aggregation capability, overcoming the difficulty of long-term interactions in convolutions. This paper validates the efficacy of our operator through extensive experiments on ImageNet classification, COCO detection and segmentation, and the results have demonstrated the proposed C-volution when paired with ResNet50 achieves an outstanding boost in performance on visual tasks(+2.0% top-1 accuracy, +3.1% box mAP, and +2.0% mask mAP) while having low parameters (i.e., CedNet50@16.3M Params).
Improving Detection of DeepFakes through Facial Region Analysis in Images

In the evolving landscape of digital media, the discipline of media forensics, which encompasses the critical examination and authentication of digital images, videos, and audio recordings, has emerged as an area of paramount importance. This heightened significance is predominantly attributed to the burgeoning concerns surrounding the proliferation of DeepFakes, which are highly realistic and manipulated media content, often created using advanced artificial intelligence techniques. Such developments necessitate a profound understanding and advancement in media forensics to ensure the integrity of digital media in various domains. Current research endeavours are primarily directed towards addressing a common challenge observed in DeepFake datasets, which pertains to the issue of overfitting. Many suggested remedies centre around the application of data augmentation methods, with a frequently adopted strategy being the incorporation of random erasure or cutout. This method entails the random removal of sections from an image to introduce diversity and mitigate overfitting. Generating disparities between the altered and unaltered images serves to inhibit the model from excessively adapting itself to individual samples, thus leading to more favourable results. Nonetheless, the stochastic nature of this approach may inadvertently obscure facial regions that harbour vital information necessary for DeepFake detection. Due to the lack of guidelines on specific regions for cutout, most studies use a randomised approach. However, in recent research, face landmarks have been integrated to designate specific facial areas for removal, even though the selection remains somewhat random. Therefore, there is a need to acquire a more comprehensive insight into facial features and identify which regions hold more crucial data for the identification of DeepFakes. In this study, the investigation delves into the data conveyed by various facial components through the excision of distinct facial regions during the training of the model. The goal is to offer valuable insights to enhance forthcoming face removal techniques within DeepFake datasets, fostering a deeper comprehension among researchers and advancing the realm of DeepFake detection. Our study presents a novel method that uses face cutout techniques to improve understanding of key facial features crucial in DeepFake detection. Moreover, the method combats overfitting in DeepFake datasets by generating diverse images with these techniques, thereby enhancing model robustness. The developed methodology is validated against publicly available datasets like FF++ and Celeb-DFv2. Both face cutout groups surpassed the Baseline, indicating cutouts improve DeepFake detection. Face Cutout Group 2 excelled, with 91% accuracy on Celeb-DF and 86% on the compound dataset, suggesting external facial features’ significance in detection. The study found that eyes are most impactful and the nose is least in model performance. Future research could explore the augmentation policy’s effect on video-based DeepFake detection.
NCMatch: Semi-supervised Learning with Noisy Labels via Noisy Sample Filter and Contrastive Learning

Semi-supervised learning (SSL) has been widely studied in recent years, which aims to improve the performance of supervised learning by utilizing unlabeled data. However, the presence of noisy labels on labeled data is an inevitable consequence of either limited expertise in the labeling process or inadvertent labeling errors. While these noisy labels can have a detrimental impact on the performance of the model, leading to decreased accuracy and reliability. In this paper, we introduce the paradigm of Semi-Supervised Learning with Noisy Labels (SSLNL), which aims to address the challenges of semi-supervised classification when confronted with limited labeled data and the presence of label noise. To address the challenges of SSLNL, we propose the Noisy Samples Filter (NSF) module, which effectively filters out noisy samples by leveraging class agreement. Additionally, we propose the Semi-Supervised Contrastive Learning (SSCL) module, which harnesses the power of high-confidence pseudo-labels generated by a semi-supervised model to enhance the extraction of robust features through contrastive learning. Extensive experiments on CIFAR-10, CIFAR-100, and SVHN datasets demonstrate that our proposed method, outperforms state-of-the-art methods in addressing the problems of SSL and SSLNL, thus validating the effectiveness of our solution.
A Quantum-Based Attention Mechanism in Scene Text Detection

Attention mechanisms have provided benefits in very many visual tasks, e.g. image classification, object detection, semantic segmentation. However, few attention modules have been proposed specifically for scene text detection. We propose an attention mechanism based on Quantum-State-based Mapping (QSM) that enhances channel and spatial attention, introduces higher-order representations, and mixes contextual information. Our approach includes two attention modules: Quantum-based Convolutional Attention Module (QCAM), a plug-and-play module applicable to pre-trained text detection models; Adaptive Channel Information Transfer Module (ACTM), which replaces feature pyramids and complex networks of DBNet++ with a 35.9% reduction in FLOPs. In CNN-based methods, our QCAM achieves state-of-the-art performance on three benchmarks. Remarkably, when compared to the Transformer-based methods such as FSG, our QCAM remains competitive in F-measure on all benchmarks. Notably, QCAM has a 29.5% reduction in parameters compared to FSG, resulting in a balance between detection accuracy and efficiency. ACTM significantly improves F-measure over DBNet++ on three benchmarks, providing an alternative to feature pyramids in scene text detection. The codes, models and training logs are available at https://github.com/yws-wxs/QCAM .
Multi-image 3D Face Reconstruction via an Adaptive Aggregation Network

Image-based 3D face reconstruction suffers from inherent drawbacks of incomplete visible regions and interference from occlusion or lighting. One solution is to utilize multiple face images for collecting sufficient knowledges. Nevertheless, most existing methods typically do not make full use of information among different images since they roughly fuse the results of individual reconstructed face for multi-image 3D face modeling, thus may ignore the intrinsic relations within various images. To tackle this problem, we propose a framework named Adaptive Aggregation Network (ADANet) to investigate the subtle correlations among multiple images for 3D face reconstruction. Specifically, we devise an Aggregation Module that can adaptively establish both the in-face and cross-face relationships by exploiting the local- and long-range dependencies among visible facial regions of multiple images, thus can effectively extract complementary aggregation face features in the multi-image scenario. Furthermore, we incorporate contour-aware information to promote the boundary consistency of 3D face model. The seamless combination of these novel designs forms a more accurate and robust multi-image 3D face reconstruction scheme. Extensive experiments demonstrate the effectiveness of our proposed ADANet.
An ANN-Guided Approach to Task-Free Continual Learning with Spiking Neural Networks

Task-Free Continual Learning (TFCL) poses a formidable challenge in lifelong learning, as it operates without task-specific information. Leveraging spiking neural networks (SNNs) for TFCL is particularly intriguing due to their promising results in low-energy applications. However, existing research has predominantly focused on employing SNNs for solving single-task classification problems. In this work, our goal is to utilize ANN to guide SNN in addressing catastrophic forgetting and model compression issues, while treating SNNs as the basic network of the model. We introduce AGT-SNN (ANN-Guided TFCL for Spiking Neural Networks), a novel framework that empowers SNNs to engage in lifelong learning without relying on task-specific information. We conceptualize the learning process of the model as a multiplayer game, involving participants in the roles of players and referees. Our model’s fundamental components comprise player-referee pairs, where the player module adopts a SNN-based Variational Autoencoder (VAE) and the referee module employs a ANN-based Generative Adversarial Network (GAN). To dynamically expand the number of components, we propose an innovative method called Adversarial Similarity Expansion (ASE). ASE evaluates the performance of the current player against previously learned players without accessing any task-specific information. Additionally, we propose a innovative pruning strategy that selectively removes redundant components while preserving the diversity of knowledge, thereby reducing the model’s complexity. Through comprehensive experimental validation, we demonstrate that our proposed framework enables SNNs to achieve exceptional performance while maintaining an appropriate network size.
Staged Transformer Network with Color Harmonization for Image Outpainting

Image outpainting aims at generating new looking-realistic content beyond the original boundaries for a given image patch. Existing image outpainting methods tend to generate images with erroneous structures and unnatural colors when extrapolating the sub-image all-side. To solve this problem, we propose a Transformer-based staged image outpainting network. Specifically, we restructure the encoder-decoder architecture by adding hierarchical cross attention to the connection in each layer. We propose a staged expanding module that splits the extrapolation into vertical and horizontal steps so that the generated images can have consistent contextual information and similar texture. A color harmonization module that adjusts both local and global color information is also presented to make color transitions more natural. Our experiments prove that the proposed method outperforms the advanced methods on multiple datasets.
A Data Hierarchical Encryption Scheme Based on Attribute Hiding under Multiple Authorization Centers

The data hierarchical Ciphertext-Policy Attribute-Based Encryption (CP-ABE) scheme implements multiple hierarchical data encryption of a single access policy, which reduces the computation and storage overhead. However, existing data hierarchical CP-ABE schemes have some problems, such as the leakage of personal privacy information through access policies or user attributes in plaintext form, and these schemes grant enough privileges to a single authorization center. If the authorization center is untrusted or attacked, keys can be used to illegally access data, which is the key escrow problem. To solve these problems, we propose an Attribute Hiding and Multiple Authorization Centers-based Data Hierarchical Encryption Scheme (AH-MAC-DHE). Firstly, we propose an Attribute Convergence Hiding Mechanism (ACHM). This mechanism solves the problem of personal privacy information leakage by hiding access policies and user attributes. Secondly, we design Privilege-Dispersed Multiple Authorization Centers (PD-MAC). PD-MAC solves the problem of key escrow by dispersing the privileges of the single authorization center to the user authorization center and attribute authorization center. Finally, we prove that AH-MAC-DHE is secure under the decisional q-parallel Bilinear Diffie-Hellman Exponent (BDHE) assumption, which also satisfies anti-collusion and privacy security. The experimental results indicate that compared with existing schemes, AH-MAC-DHE performs well.
Privacy‐preserving in the smart healthcare system using steganography and chaotic functions based on DNA

Abstract The smart healthcare system is one of the Internet of Things‐based applications that are increasingly used nowadays. One of the requirements of this system is privacy‐preserving, which should protect the disclosure of the patient record contents. In the present paper, a combination of chaotic functions and a new DNA‐based steganography method, and image blocking are suggested to protect patients' privacy‐preserving. First, the image is blocked, and the initial key for the chaotic function will be gained by using the centers of the blocks. Then the data and image are transformed into DNA sequences. Finally, one of the randomly chosen DNA strands of data is XOR with the center of one randomly selected block and will be placed in one of the block pixels. The failure to send the initial key separately to generate random numbers, as well as the random selection of secret data bits and image pixels for steganography has increased the security of the proposed method. The simulation results not only indicate the quality of the stego‐image but also show better performance of the proposed method than the existing methods.
Design and analyses of web scraping on burstable virtual machines

Summary Web scraping is a widely used technique for decision‐making, collecting, and structuring public data from the internet. As the volume of data continues to grow, the need for more efficient methods of data extraction becomes crucial. This article introduces a novel web scraping framework that utilizes Burstable virtual machines (VMs) on Amazon Web Services with the objective of reducing the monetary cost of execution while ensuring compliance with service level agreements (SLAs). To achieve this, the framework utilizes a combination of fixed and temporary Burstable VMs in a mixed cluster, which can be elastically scaled up to fulfill the SLA and scaled down to minimize monetary costs. Two strategies for handling VM allocation are proposed and evaluated: (i) a queue and SLA‐based strategy that employs queue size information and SLA criteria to determine the required number of VMs for the current scraping requests, and (ii) a credit‐based strategy that incorporates information about Burstable VM credits to effectively manage instance creation and termination. Experimental tests show that the proposed framework meets the defined SLA while achieving cost reductions of up to 74% compared to an approach that executes on fixed‐size clusters of Burstable instances.
ImageNet classification with Raspberry Pis: federated learning algorithms of local classifiers

Purpose This study aims to propose novel edge device-tailored federated learning algorithms of local classifiers (stochastic gradient descent, support vector machines), namely, FL- l SGD and FL- l SVM. These algorithms are designed to address the challenge of large-scale ImageNet classification. Design/methodology/approach The authors’ FL- l SGD and FL- l SVM trains in a parallel and incremental manner to build an ensemble local classifier on Raspberry Pis without requiring data exchange. The algorithms load small data blocks of the local training subset stored on the Raspberry Pi sequentially to train the local classifiers. The data block is split into k partitions using the k- means algorithm, and models are trained in parallel on each data partition to enable local data classification. Findings Empirical test results on the ImageNet data set show that the authors’ FL- l SGD and FL- l SVM algorithms with 4 Raspberry Pis (Quad core Cortex-A72, ARM v8, 64-bit SoC @ 1.5GHz, 4GB RAM) are faster than the state-of-the-art LIBLINEAR algorithm run on a PC (Intel(R) Core i7-4790 CPU, 3.6 GHz, 4 cores, 32GB RAM). Originality/value Efficiently addressing the challenge of large-scale ImageNet classification, the authors’ novel federated learning algorithms of local classifiers have been tailored to work on the Raspberry Pi. These algorithms can handle 1,281,167 images and 1,000 classes effectively.
Customer Service Ethics in the Metaverse: A Value-Sensitive Design Science Approach

The aim of this paper is to develop recommendations for the value-sensitive design of social virtual reality (VR) applications with respect to the interaction between customers and customer service representatives, considering the value-sensitive design approach. Based on a systematic literature search, 125 relevant hits could be found, which especially assigns the values of openness, security, and liability to problems and categorizes them into six problem areas. These values of openness, security, and liability and problem fields form the requirements for a value-sensitive design of social VR applications. Based on these requirements, preliminary design principles were derived in two evaluation cycles, each assessed using an online survey (Survey 1 with 221 participants and Survey 2 with 310 participants) and presented in the form of instantiation. One main finding of the paper can help counteract discrimination in social VR applications within the interaction between customers and customer service representatives.
Perception and Decision-Making for Multi‑Modal Interaction Based on Fuzzy Theory in the Dynamic Environment

The projection-augmented reality assembly assistance system, which supports natural multi-modal human–computer interaction, has the potential to improve interaction efficiency. However, the effectiveness of the interaction method is compromised by the interference caused by the dynamically changing industrial environment. To address this issue, this study proposed a multi-model interaction model based on fuzzy theory and developed an environment perception decision-making interaction system. This system continuously monitors real-time environment data, evaluates interaction performance, and visually recommends the optimal method to assist users in completing their tasks. A comparative experiment was conducted between the proposed system and a non-environment perception interaction system. The collected data, including interaction success rate, task time, system usability, and user experience, were analyzed. The results demonstrated that the proposed system effectively enhances the adaptability and comfort of computer systems in complex environments and improves communication efficiency between humans and machines.
Unravelling the digital competence of students in physiotherapy education through the European digital competence framework

Purpose To describe the distribution and extent of the physiotherapy students’ digital competence, using the European Digital Competence Framework (DigComp).
Rhythm Research in Interactive System Design: A Literature Review

Rhythm can create a sense of structure or order, making it easier for users to understand and navigate interactive systems, thereby making user engagement more enjoyable and motivating. Recently, designers are increasingly interested in integrating rhythm into the design of interactive systems. However, the existing literature on this topic is fragmented, making it difficult for designers to find comprehensive guidance. This paper presents the first literature review of rhythm research for designing an interactive system based on the PRISMA method. Through an analysis of 59 papers based on the critical review method (e.g., research through design), we identify six research genres and provide a map to help future researchers locate their research position. Because there is currently no clear classification framework to guide designers in integrating rhythm into interactive systems, we propose a classification framework that includes six design qualities: three human rhythm-driven qualities and three computer rhythm-driven qualities based on the literature review. This classification framework provides a foundation for designers to regulate rhythm in interactive systems. In addition, we discuss emerging issues related to rhythm genres and rhythm classification framework and design qualities. We reflect on the effectiveness of our proposed framework by revisiting previous works on designing interactive systems, and provide potential directions for future research in this area. Overall, this paper aims to provide comprehensive guidance for designers who wish to integrate rhythm into the design of interactive systems, and to advance the understanding of the role of rhythm in interactive system design.