Reviewerly/exHarmony huggingface dataset:
Federated Deep Multi-View Clustering with Global Self-Supervision

Federated multi-view clustering has the potential to learn a global clustering model from data distributed across multiple devices. In this setting, label information is unknown and data privacy must be preserved, leading to two major challenges. First, views on different clients often have feature heterogeneity, and mining their complementary cluster information is not trivial. Second, the storage and usage of data from multiple clients in a distributed environment can lead to incompleteness of multi-view data. To address these challenges, we propose a novel federated deep multi-view clustering method that can mine complementary cluster structures from multiple clients, while dealing with data incompleteness and privacy concerns. Specifically, in the server environment, we propose sample alignment and data extension techniques to explore the complementary cluster structures of multiple views. The server then distributes global prototypes and global pseudo-labels to each client as global self-supervised information. In the client environment, multiple clients use the global self-supervised information and deep autoencoders to learn view-specific cluster assignments and embedded features, which are then uploaded to the server for refining the global self-supervised information. Finally, the results of our extensive experiments demonstrate that our proposed method exhibits superior performance in addressing the challenges of incomplete multi-view data in distributed environments.
NPF-200: A Multi-Modal Eye Fixation Dataset and Method for Non-Photorealistic Videos

Non-photorealistic videos are in demand with the wave of the metaverse, but lack of sufficient research studies. This work aims to take a step forward to understand how humans perceive non-photorealistic videos with eye fixation (i.e., saliency detection), which is critical for enhancing media production, artistic design, and game user experience. To fill in the gap of missing a suitable dataset for this research line, we present NPF-200, the first large-scale multi-modal dataset of purely non-photorealistic videos with eye fixations. Our dataset has three characteristics: 1) it contains soundtracks that are essential according to vision and psychological studies; 2) it includes diverse semantic content and videos are of high-quality; 3) it has rich motions across and within videos. We conduct a series of analyses to gain deeper insights into this task and compare several state-of-the-art methods to explore the gap between natural images and non-photorealistic data. Additionally, as the human attention system tends to extract visual and audio features with different frequencies, we propose a universal frequency-aware multi-modal non-photorealistic saliency detection model called NPSNet, demonstrating the state-of-the-art performance of our task. The results uncover strengths and weaknesses of multi-modal network design and multi-domain training, opening up promising directions for future works. Our dataset and code can be found at https://github.com/Yangziyu/NPF200
SiFDetectCracker: An Adversarial Attack Against Fake Voice Detection Based on Speaker-Irrelative Features

Voice is a vital medium for transmitting information. The advancement of speech synthesis technology has resulted in high-quality synthesized voices indistinguishable from human ears. These fake voices have been widely used in natural Deepfake production and other malicious activities, raising serious concerns regarding security and privacy. To deal with this situation, there have been many studies working on detecting fake voices and reporting excellent performance. However, is the story really over? In this paper, we propose SiFDetectCracker, a black-box adversarial attack framework based on Speaker-Irrelative Features (SiFs) against fake voice detection. We select background noise and mute parts before and after the speaker's voice as the primary attack features. By modifying these features in synthesized speech, the fake speech detector will make a misjudgment. Experiments show that SiFDetectCracker achieved a success rate of more than 80% in bypassing existing state-of-the-art fake voice detection systems. We also conducted several experiments to evaluate our attack approach's transferability and activation factor.
Application-layer Characterization and Traffic Analysis for Encrypted QUIC Transport Protocol

Quick UDP Internet Connection (QUIC) is an emerging end-to-end encrypted, transport-layer protocol, which has been increasingly adopted by popular web services to improve communication security and quality of experience (QoE) towards end-users. However, this tendency makes the traffic analysis more challenging, given the limited information in the QUIC packet header and full encryption on the payload. To address this challenge, a novel rule-based approach is proposed to estimate the application-level traffic attributes without decrypting QUIC packets. Based on the size, timing, and direction information, our proposed algorithm analyzes the associated network traffic to infer the identity of each HTTP request and response pair, as well as the multiplexing feature in each QUIC connection. The inferred HTTP attributes can be used to evaluate the QoE of application-layer services and identify the service categories for traffic classification in the encrypted QUIC connections.
Inter-Temporal Reward Decisions with Strategic Ethical Hackers

A skyrocketing increase in cyber attacks significantly elevates the importance of secure software development. Companies launch various bug-bounty programs to reward ethical hackers for identifying potential vulnerabilities in their systems before malicious hackers can exploit them. One of the most difficult decisions in bug-bounty programs is how to reward ethical hackers appropriately. This paper develops a model of an intertemporal reward strategy with endogenous e-hacker behaviors. We formulate a novel game model to characterize the interactions between a software vendor and multiple ethical hackers. The impacts of the ethical hackers’ strategic bug-hoarding behaviors and their competition on the program’s performance are evaluated. We demonstrate the effectiveness of the dynamic reward mechanism in attracting ethical hackers and encouraging early bug reports. The optimal levels of rewards and timing of reward change are discussed. We show that ignoring the ethical hackers’ strategic behaviors might lead to either too little incentive to attract ethical hackers or too much incentive which motivates them to hoard bugs for higher rewards.
Security Analysis of Trust on the Controller in the Matter Protocol Specification

Matter is an open-source connectivity standard allowing smart home IoT (Internet of Things) devices from different vendors to interoperate with one another. A controller <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> in a Matter system commissions new devices into the Matter fabric. The device needs to present a credential called Device Attestation Certificate (DAC), indicating that it is from a trusted vendor and has gone through the necessary testing to be compliant with the Matter standard. However, the controller is not required to prove to the device that it is from a trustworthy vendor. We verified through experimentation that anyone can create a Matter controller that can commission a commercial Matter device. We analyze the security implication of this design choice in Matter, and present a few scenarios where a malicious controller can exert harm to an otherwise healthy Matter ecosystem. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Throughout this paper the term “controller” refers to the controller component in the Matter protocol specification. It does not refer to any specific controller from any vendor.
Homomorphic Comparison Method Based on Dynamically Polynomial Composite Approximating Sign Function

Performing comparison operations between two numbers is one of the most commonly used operations in some applications. However, performing non-arithmetic operations, such as comparison, on data encrypted with fully homomorphic encryption that only supports addition and multiplication operations, has always been a challenging task. Recently, Cheon et al.(Asiacrypt 2020) proposed a new comparison method in homomorphic encryption using composite polynomial approximation of the sign function, and proved that their method has optimal asymptotic complexity. In this paper, we propose a new comparison method using dynamic polynomial composition to approximate the sign function. We analyze the optimality of the depth consumption and computational complexity of this approximation method. Additionally, we extend this comparison algorithm to multiple applications, such as Min/Max algorithm and k-way sorting network algorithm. Our numerical analysis shows that for computing the approximate comparison of any two 32-bit integers encrypted by HEAAN, our homomorphic comparison algorithm (for $\epsilon=2^{-16},\alpha=2^{-32}$) requires 0.83ms in amortized running time. This reduces the running time by about 25% compared to the previous optimal method.
SecDINT: Preventing Data-oriented Attacks via Intel SGX Escorted Data Integrity

Data-oriented attacks with the intent to corrupt critical memory data without violating Control-flow Integrity (CFI) pose significant threats to legitimate program execution. Existing mitigations predominantly rely on software-based memory safety measures to ensure critical data integrity, a solution often associated with elevated performance overhead and susceptibility to intricate attack techniques. In this paper, we present a CPU level data integrity design, named Intel SGX Escorted Data Integrity (SecDINT), to automatically protect sensitive variables against data-oriented attacks. Our design can achieve the data integrity of sensitive variables via SGX enforced isolation in binaries. We evaluate SecDINT on real-world applications. The results reveal that SecDINT can effectively identify sensitive variables, enforce data integrity, and prevent data-oriented attacks. Comparative analysis with existing software-based strategies (e.g., 103% runtime overhead in Data-flow Integrity, 116% in SoftBound with CETS), showcased SecDINT’s remarkable capability in drastically reducing overhead to as low as 20.1%.
Foreground/Background-Masked Interaction Learning for Spatio-temporal Action Detection

Spatio-temporal Action Detection (SAD) aims to recognize the multi-class actions, and meanwhile locate their spatio-temporal occurrence in untrimmed videos. Besides relying on the inherent inter-actor interactions, most previous SAD approaches model actor interactions between multi-actors and the whole frames or special parts (e.g., objects/hands). However, such approaches are relatively graceless by 1) roughly treating all various actors to equivalently interact with frames/parts or by 2) sumptuously borrowing multiple costly detectors to acquire the special parts. To solve the above dilemma, we propose a novel Foreground/Background-masked Interaction Learning (dubbed as FBI Learning) framework to learn the multi-actor features by attentively interacting with the hands-down foreground and background frames. Specifically, we first design a new Mask-guided Cross Attention (MCA) mechanism that calculates the masked cross-attentions to capture the compact relations between the actors and foreground/background regions. Next, we present a new Actor-guided Feature Aggregation (AFA) scheme that integrates foreground- and background-interacted actor features with the learnable actor-based weights. Finally, we construct a long-term feature bank that associates temporal context information to facilitate action classification. Extensive experiments are conducted on commonly available UCF101-24, MultiSports, and AVA v2.1/v2.2 datasets, which illustrate the competitive performance of FBI Learning against the state-of-the-art methods.
A Stealthy Inference Attack on Split Learning with a Split-Fuse Defensive Measure

The privacy vulnerabilities and communication inefficiencies of federated learning have motivated the development of the split learning architecture. Google’s recent Federated Reconstruction architecture combines federated and split learning architectures into a unified design, aiming to improve communication and computation scalability. While split learning aims to protect the privacy of clients’ data, recent work has revealed its vulnerability to malicious adversaries through model poisoning. We aim to investigate the privacy aspect of split learning, as an independent architecture or the significant component of federated reconstruction architecture and expose its shortcomings. Different from the existing literature, we illustrate that an honest-but-curious adversary can infer the private properties of clients’ data without model poisoning or manipulation. We demonstrate the practicality of the property inference attack against split learning using various datasets. To reduce information leakages and protect clients’ privacy, we propose Bundle-Net architecture as a privacy-preserving distributed learning mechanism and assess its effectiveness in thwarting inference attacks.
Self-Contrastive Graph Diffusion Network

Augmentation techniques and sampling strategies are crucial in contrastive learning, but in most existing works, augmentation techniques require careful design, and their sampling strategies can only capture a small amount of intrinsic supervision information. Additionally, the existing methods require complex designs to obtain two different representations of the data. To overcome these limitations, we propose a novel framework called the Self-Contrastive Graph Diffusion Network (SCGDN). Our framework consists of two main components: the Attentional Module (AttM) and the Diffusion Module (DiFM). AttM aggregates higher-order structure and feature information to get an excellent embedding, while DiFM balances the state of each node in the graph through Laplacian diffusion learning and allows the cooperative evolution of adjacency and feature information in the graph. Unlike existing methodologies, SCGDN is an augmentation-free approach that avoids "sampling bias" and semantic drift, without the need for pre-training. We conduct a high-quality sampling of samples based on structure and feature information. If two nodes are neighbors, they are considered positive samples of each other. If two disconnected nodes are also unrelated on kNN graph, they are considered negative samples for each other. The contrastive objective reasonably uses our proposed sampling strategies, and the redundancy reduction term minimizes redundant information in the embedding and can well retain more discriminative information. In this novel framework, the graph self-contrastive learning paradigm gives expression to a powerful force. The results manifest that SCGDN can consistently generate out performance over both the contrastive methods and the classical methods. The source code is available at https://github.com/kunzhan/SCGDN.
Taking a Part for the Whole: An Archetype-agnostic Framework for Voice-Face Association

Voice-face association is generally specialized as a cross-modal cognitive matching problem, and recent attention has been paid on the feasibility of devising the computational mechanisms for recognizing such associations. Existing works are commonly resorting to the combination of contrastive learning and classification-based loss to correlate the heterogeneous datas. Nevertheless, the reliance on typical features of each category, known as archetypes, derived from the combination suffer from the weak invariance of modality-specific features within the same identity, which might induce a cross-modal joint feature space with calibration deviations. To tackle these problems, this paper presents an efficient Archetype-agnostic framework for reliable voice-face association. First, an Archetype-agnostic Subspace Merging (AaSM) method is carefully designed to perform feature calibration which can well get rid of the archetype dependence to facilitate the mutual perception of datas. Further, an efficient Bilateral Connection Re-gauging scheme is proposed to quantitatively screen and calibrate the biased datas, namely loose pairs that deviate from joint feature space. Besides, an Instance Equilibrium strategy is dynamically derived to optimize the training process on loose data pairs and significantly improve the data utilization. Through the joint exploitation of the above, the proposed framework can well associate the voice-face data to benefit various kinds of cross-modal cognitive tasks. Extensive experiments verify the superiorities of the proposed voice-face association framework and show its competitive performances with the state-of-the-arts.
PowToon software’s impact on EFL students’ writing of narrative text

The development of technology brings challenges for teachers to be more innovative and creative in delivering materials. Teachers need to explore currently medium for teaching. To assist teachers and students in overcoming their difficulties, PowToon Software was chosen. The goal of this study was to find out whether PowToon as a learning media was effective in improving the narrative text writing ability of grade XI students at SMA Negeri 1 Pegandon in the 2023/2024 academic year. This study used a quasi-experimental method. The sample consisted of grade XI 1 students as the control group and XI 2 as the experimental group. The control group and experimental group completed the pre-test and post-test. Data analysis in the study used SPSS. The results showed a significance improvement in students' writing achievement. This was evidenced by the significance value of 0.003. The value was smaller than 0.05 and this means that H0 was rejected and H1 was accepted. It can be summarized that the use of PowToon Software as a learning media was effective to improve students' narrative text writing ability.
Possibilities and features of the elementary cognitive and educational functions development

The paper discusses research aimed at studying the possibilities of the development of elementary cognitive functions. Various methods of development and their impact on children in the learning process are considered.
Performing decision-making tasks through dynamics of recurrent neural networks trained with reinforcement learning

In this work, recurrent neural networks are considered as functional models that perform target tasks inspired by cognitive neuroscience experiments. These networks are trained with reinforcement learning methods, after that structure and dynamics mechanisms underlying their behaviors are studied. We use two versions—with and without a context signal—of the cognitive perceptual decision-making task. Population and single-neuron dynamics are studied resulting in stimulus processing as well as successful completing the target tasks. Functionally specialized neurons as well as cluster structure of the trained networks are found and investigated. Similarities and differences between the model neural networks and biological prototypes are discussed.
Characterisation of Data Augmentation Techniques Using Visualisation

Data Augmentation (DA) alleviates the data-space limitations by generating new instances for training machine learning (ML) models. However, in practice, the effectiveness of the DA techniques adopted in varying downstream tasks lacks analytical explainability, mainly due to its domain-specificity nature. This paper describes a new ideation of characterising DA techniques from a visualisation perspective by explaining feature-space characteristics of augmented and original data in the high-dimensional space. This is accomplished by assessing the feasibility of using principal component analysis (PCA) for capturing the distinctiveness inherited by DA techniques. In a problem setting of COVID-19 CT detection, the proximity analysis suggests that computing PCA using low-dimensional features can contribute significant analytical values for the characterisation of DA techniques. The comparison of the clustering performances using data instances from the image and feature domains further highlights the significance of abstract features for more effective visual analysis and interpretation. Furthermore, the non-trivial connection between feature inputs and image outputs demonstrated in the connectivity analysis also indicates the significance of visualisation in analysing ML decisions, particularly with high-level visual explanations.
A Hybrid Classification Approach For Artificial Speech Detection

The emergence of voice biometrics has revolutionized user authentication methods by delivering enhanced security and convenience, steadily replacing less secure authentication techniques. However, the automatic speaker verification (ASV) systems remained vulnerable to spoofing attacks, especially artificial speech attacks. This is because artificial speech can be generated rapidly and in abundance using state-of-the-art speech synthesis and voice conversion algorithms. Recently, a one-class learning-based countermeasure known as the AIR-ASVspoof was introduced for artificial speech detection, which has proven to be the best-performing non-fusion system. As AIR-ASVspoof focuses solely on the bona fide class as the target in artificial speech detection, its practicality in a real-world environment may be restricted. The reason for this limitation lies in the fact that bona fide speech can be recorded in different environments, thereby increasing the risk of encountering false negatives. Hence, this paper proposes a hybrid approach to mitigate the issue, named Hybrid AIR-ASVspoof (HAIR-ASVspoof), by hybridizing end-to-end learning and classic machine learning. In this approach, deep features will be extracted from the dataset using the trained AIR-ASVspoof model and classified using a classic machine learning model such as Support Vector Machine (SVM). The objectives are: (i) to extract deep features using the AIR-ASVspoof; and (ii) to compare the performance of the AIR-ASVspoof and the proposed hybrid approach. An experiment was conducted to find the appropriate backend classifier for the proposed hybrid approach. Experimental results showed that the proposed approach outperformed the original AIR-ASVspoof, with an Equal Error Rate (EER) of 0.57%.
A Smartphone-Based IoT System to Monitor Driving Behaviors by Using a Machine Learning Classifier

Traffic accidents cause great casualties and property losses all over the world. Thus, traffic safety is one of the very important topics in smart cities. One of the major causes of traffic accidents is bad driving behaviors. This paper proposes a smartphone-based IoT (Internet of Things) system to monitor driving behaviors by using a machine learning classifier. In this system, a smartphone is put into a car near the driver. A cloud server gets data from accelerators and GPS (Global Positioning System) sensors in the smartphone through a 4G/5G network. In the cloud server, an SVM (Support Vector Machine) classifier and a GPS speed-detecting algorithm are installed to identify abnormal driving behaviors and the instantaneous speed of the car. A user performance report is generated at the end of a journey. The report shows abnormal driving behaviors during the journey and information when the car is over the speed limit. We carried out some experiments and we found that the system can identify different abnormal driving behaviors and driving speed properly. Moreover, the system generated user performance reports properly.