Reviewerly/exHarmony huggingface dataset:
Retrieving GNN Architecture for Collaborative Filtering

Graph Neural Networks (GNNs) have been widely used in Collaborative Filtering (CF). However, when given a new recommendation scenario, the current options are either selecting from existing GNN architectures or employing Neural Architecture Search (NAS) to obtain a well-performing GNN model, both of which are expensive in terms of human expertise or computational resources.To address the problem, in this work,we propose a novel neural retrieval approach, dubbed RGCF, to search a well-performing architecture for GNN-based CF rapidly when handling new scenarios. Specifically, we design the neural retrieval approach based on meta-learning by developing two-level meta-features, ranking loss, and task-level data augmentation, and in a retrieval paradigm, RGCF can directly return a well-performing architecture given a new dataset (query), thus being efficient inherently. Experimental results on two mainstream tasks, i.e., rating prediction and item ranking, show that RGCF outperforms all models either by human-designed or NAS on two new datasets in terms of effectiveness and efficiency. Particularly, the efficiency improvement is significant, taking as an example that RGCF is 61.7-206.3x faster than a typical reinforcement learning based NAS approach on the two new datasets. Code and data are available at https://github.com/BUPT-GAMMA/RGCF.
Differentially Private Deep Q-Learning for Pattern Privacy Preservation in MEC Offloading

Mobile edge computing (MEC) is a promising paradigm to meet the quality of service (QoS) requirements of latency-sensitive IoT applications.However, attackers may eavesdrop on the offloading decisions to infer the edge server's (ES's) queue information and users' usage patterns, thereby incurring the pattern privacy (PP) issue.Therefore, we propose an offloading strategy which jointly minimizes the latency, ES's energy consumption, and task dropping rate, while preserving PP.Firstly, we formulate the dynamic computation offloading procedure as a Markov decision process (MDP).Next, we develop a Differential Privacy Deep Q-learning based Offloading (DP-DQO) algorithm to solve this problem while addressing the PP issue by injecting noise into the generated offloading decisions.This is achieved by modifying the deep Q-network (DQN) with a Function-output Gaussian process mechanism.We provide a theoretical privacy guarantee and a utility guarantee (learning error bound) for the DP-DQO algorithm and finally, conduct simulations to evaluate the performance of our proposed algorithm by comparing it with greedy and DQN-based algorithms.
Stale TLS Certificates: Investigating Precarious Third-Party Access to Valid TLS Keys

Certificate authorities enable TLS server authentication by generating certificates that attest to the mapping between a domain name and a cryptographic keypair, for up to 398 days. This static, name-to-key caching mechanism belies a complex reality: a tangle of dynamic infrastructure involving domains, servers, cryptographic keys, etc. When any of these operations changes, the authentication information in a certificate becomes stale and no longer accurately reflects reality. In this work, we examine the broader phenomenon of certificate invalidation events and discover three classes of security-relevant events that enable a third-party to impersonate a domain outside of their control. Longitudinal measurement of these precarious scenarios reveals that they affect over 15K new domains per day, on average. Unfortunately, modern certificate revocation provides little recourse, so we examine the potential impact of reducing certificate lifetimes (cache duration): shortening the current 398-day limit to 90 days yields a 75% decrease in precarious access to valid TLS keys.
Real-Time Vehicle Detection for Surveillance of River Dredging Areas Using Convolutional Neural Networks

The presence of illegal activities such as illegitimate mining and sand theft in river dredging areas leads to economic losses. However, manual monitoring is expensive and time-consuming. Therefore, automated surveillance systems are preferred to mitigate such activities, as they are accurate and available at all times. In order to monitor river dredging areas, two essential steps for surveillance are vehicle detection and license plate recognition. Most current frameworks for vehicle detection employ plain feed-forward Convolutional Neural Networks (CNNs) as backbone architectures. However, these are scale-sensitive and cannot handle variations in vehicles' scales in consecutive video frames. To address these issues, Scale Invariant Hybrid Convolutional Neural Network (SIH-CNN) architecture is proposed for real-time vehicle detection in this study. The publicly available benchmark UA-DETRAC is used to validate the performance of the proposed architecture. Results show that the proposed SIH-CNN model achieved a mean average precision (mAP) of 77.76% on the UA-DETRAC benchmark, which is 3.94% higher than the baseline detector with real-time performance of 48.4 frames per seconds.
Class-constrained t-SNE: Combining Data Features and Class Probabilities

Data features and class probabilities are two main perspectives when, e.g., evaluating model results and identifying problematic items. Class probabilities represent the likelihood that each instance belongs to a particular class, which can be produced by probabilistic classifiers or even human labeling with uncertainty. Since both perspectives are multi-dimensional data, dimensionality reduction (DR) techniques are commonly used to extract informative characteristics from them. However, existing methods either focus solely on the data feature perspective or rely on class probability estimates to guide the DR process. In contrast to previous work where separate views are linked to conduct the analysis, we propose a novel approach, class-constrained t-SNE, that combines data features and class probabilities in the same DR result. Specifically, we combine them by balancing two corresponding components in a cost function to optimize the positions of data points and iconic representation of classes – class landmarks. Furthermore, an interactive user-adjustable parameter balances these two components so that users can focus on the weighted perspectives of interest and also empowers a smooth visual transition between varying perspectives to preserve the mental map. We illustrate its application potential in model evaluation and visual-interactive labeling. A comparative analysis is performed to evaluate the DR results
Research on the intelligent teaching mode of college piano classroom based on big data technology

Abstract In this paper, firstly, under the perspective of big data, the innovation research of piano intelligent teaching mode and the application strategy of intelligent piano teaching mode under big data thinking are explained, respectively. Secondly, in music recognition in a college piano classroom, the piano music signal needs to be sampled, quantized and pre-processed to avoid the frequency bias caused by the low-frequency components of the piano music signal. Then, based on the big data technology, the piano intelligent teaching note recognition system is constructed by using the improved time domain algorithm and the improved time-frequency domain combination algorithm, and the recognition of note time value in piano intelligent teaching and the piano intelligent teaching note recognition system are studied and analyzed. The results show that the improved time-frequency domain combination is more effective in responding to the time-value information of piano music signals in recognition simulation. The note recognition system studied in this project has seen a recognition rate increase of almost 5% in system performance. This paper studies the application of intelligent piano teaching in professional learning in universities, which can lead to the development and popularity of intelligent teaching methods in piano classrooms.
JPEG Column: 94th JPEG Meeting

The 94 th JPEG meeting was held online from 17 to 21 January 2022. A major milestone has been reached at this meeting with the release of the final call for proposals under the JPEG AI project. This standard aims at the joint standardization of the first image coding standard based on machine learning by the IEC, ISO and ITU, offering a single stream, compact compressed domain representation, targeting both human visualization with significant compression efficiency improvement over image coding standards in common use at equivalent subjective quality and effective performance for image processing and computer vision tasks.
YOLO-IR-Free: An Improved Algorithm for Real-Time Detection of Vehicles in Infrared Images

In the field of object detection algorithms, the task of infrared vehicle detection holds significant importance. By utilizing infrared sensors, this approach detects the thermal radiation emitted by vehicles, enabling robust vehicle detection even during nighttime or adverse weather conditions, thus enhancing traffic safety and the efficiency of intelligent driving systems. Current techniques for infrared vehicle detection encounter difficulties in handling low contrast, detecting small objects, and ensuring real-time performance. In the domain of lightweight object detection algorithms, certain existing methodologies face challenges in effectively balancing detection speed and accuracy for this specific task. In order to address this quandary, this paper presents an improved algorithm, called YOLO-IR-Free, an anchor-free approach based on improved attention mechanism YOLOv7 algorithm for real-time detection of infrared vehicles, to tackle these issues. We introduce a new attention mechanism and network module to effectively capture subtle textures and low-contrast features in infrared images. The use of an anchor-free detection head instead of an anchor-based detection head is employed to enhance detection speed. Experimental results demonstrate that YOLO-IR-Free outperforms other methods in terms of accuracy, recall rate, and average precision scores, while maintaining good real-time performance.
AttentionViz: A Global View of Transformer Attention

Transformer models are revolutionizing machine learning, but their inner workings remain mysterious. In this work, we present a new visualization technique designed to help researchers understand the self-attention mechanism in transformers that allows these models to learn rich, contextual relationships between elements of a sequence. The main idea behind our method is to visualize a joint embedding of the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">query</i> and key vectors used by transformer models to compute attention. Unlike previous attention visualization techniques, our approach enables the analysis of global patterns across multiple input sequences. We create an interactive visualization tool, AttentionViz (demo: http://attentionviz.com), based on these joint query-key embeddings, and use it to study attention mechanisms in both language and vision transformers. We demonstrate the utility of our approach in improving model understanding and offering new insights about query-key interactions through several application scenarios and expert feedback
Zero-Shot Image Retrieval with Human Feedback

Composed image retrieval extends traditional content-based image retrieval (CBIR) combining a query image with additional descriptive text to express user intent and specify supplementary requests related to the visual attributes of the query image. This approach holds significant potential for e-commerce applications, such as interactive multimodal searches and chatbots. In our demo, we present an interactive composed image retrieval system based on the SEARLE approach, which tackles this task in a zero-shot manner efficiently and effectively. The demo allows users to perform image retrieval iteratively refining the results using textual feedback.
The quest for the reliability of machine learning models in binary classification on tabular data

Abstract In this paper we explore the reliability of contexts of machine learning (ML) models. There are several evaluation procedures commonly used to validate a model (precision, F1 Score and others); However, these procedures are not linked to the evaluation of learning itself, but only to the number of correct answers presented by the model. This characteristic makes it impossible to assess whether a model was able to learn through elements that make sense of the context in which it is inserted. Therefore, the model could achieves good results in the training stage but poor results when the model needs to be generalized. When there are many different models that achieve similar performance, the model that presented the highest number of hits in training does not mean that this model is the best. Therefore, we created a methodology based on Item Response Theory that allows us to identify whether an ML context is unreliable, providing an extra and different validation for ML models.
DJAN: Deep Joint Adaptation Network for Wildlife Image Recognition

Wildlife recognition is of utmost importance for monitoring and preserving biodiversity. In recent years, deep-learning-based methods for wildlife image recognition have exhibited remarkable performance on specific datasets and are becoming a mainstream research direction. However, wildlife image recognition tasks face the challenge of weak generalization in open environments. In this paper, a Deep Joint Adaptation Network (DJAN) for wildlife image recognition is proposed to deal with the above issue by taking a transfer learning paradigm into consideration. To alleviate the distribution discrepancy between the known dataset and the target task dataset while enhancing the transferability of the model’s generated features, we introduce a correlation alignment constraint and a strategy of conditional adversarial training, which enhance the capability of individual domain adaptation modules. In addition, a transformer unit is utilized to capture the long-range relationships between the local and global feature representations, which facilitates better understanding of the overall structure and relationships within the image. The proposed approach is evaluated on a wildlife dataset; a series of experimental results testify that the DJAN model yields state-of-the-art results, and, compared to the best results obtained by the baseline methods, the average accuracy of identifying the eleven wildlife species improves by 3.6 percentage points.
Detection of Ponzi scheme on Ethereum using machine learning algorithms

Abstract Security threats posed by Ponzi schemes present a considerably higher risk compared to many other online crimes. These fraudulent online businesses, including Ponzi schemes, have witnessed rapid growth and emerged as major threats in societies like Nigeria, particularly due to the high poverty rate. Many individuals have fallen victim to these scams, resulting in significant financial losses. Despite efforts to detect Ponzi schemes using various methods, including machine learning (ML), current techniques still face challenges, such as deficient datasets, reliance on transaction records, and limited accuracy. To address the negative impact of Ponzi schemes, this paper proposes a novel approach focusing on detecting Ponzi schemes on Ethereum using ML algorithms like random forest (RF), neural network (NN), and K-nearest neighbor (KNN). Over 20,000 datasets related to Ethereum transaction networks were gathered from Kaggle and preprocessed for training the ML models. After evaluating and comparing the three models, RF demonstrated the best performance with an accuracy of 0.94, a class-score of 0.8833, and an overall-score of 0.96667. Comparative evaluations with previous models indicate that our model achieves high accuracy. Moreover, this innovative work successfully detects key fraud features within the Ponzi scheme dataset, reducing the number of features from 70 to only 10 while maintaining a high level of accuracy. The main strength of this proposed method lies in its ability to detect clever Ponzi schemes from their inception, offering valuable insights to combat these financial threats effectively.
MCW: A Generalizable Deepfake Detection Method for Few-Shot Learning

With the development of deepfake technology, deepfake detection has received widespread attention. Although some deepfake forensics techniques have been proposed, they are still very difficult to implement in real-world scenarios. This is due to the differences in different deepfake technologies and the compression or editing of videos during the propagation process. Considering the issue of sample imbalance with few-shot scenarios in deepfake detection, we propose a multi-feature channel domain-weighted framework based on meta-learning (MCW). In order to obtain outstanding detection performance of a cross-database, the proposed framework improves a meta-learning network in two ways: it enhances the model’s feature extraction ability for detecting targets by combining the RGB domain and frequency domain information of the image and enhances the model’s generalization ability for detecting targets by assigning meta weights to channels on the feature map. The proposed MCW framework solves the problems of poor detection performance and insufficient data compression resistance of the algorithm for samples generated by unknown algorithms. The experiment was set in a zero-shot scenario and few-shot scenario, simulating the deepfake detection environment in real situations. We selected nine detection algorithms as comparative algorithms. The experimental results show that the MCW framework outperforms other algorithms in cross-algorithm detection and cross-dataset detection. The MCW framework demonstrates its ability to generalize and resist compression with low-quality training images and across different generation algorithm scenarios, and it has better fine-tuning potential in few-shot learning scenarios.
A Harris Hawk Optimization with chaotic map based image encryption for multimedia application

Image security plays a vital role in communication networks. Despite advancements in encryption, securing protective image data remains a computationally challenging problem, requiring the use of a secure environment to secure data transmitted from the device through the network. Encryption is essential for safeguarding essential data against unauthorized access or malfunction, especially for images. The contribution of the proposed model is to provide an analytical hybrid Harris Hawk Optimization (HHO) with a chaotic map approach which can be used to improve the overall performance of standard encryption techniques in image-based encrypted communication. Previously, the algorithm computes numerous chaotic Logistic map and an encrypted images, where the pending plain image calculates the session key for the map’s input parameters. Afterwards, encrypted images are presented as hawks and made to work well with HHO. The optimized ciphered image is expressed as a fitness function using correlation coefficients related to nearby pixels. When compared to existing algorithms, the proposed NPCR (99%), UACI (33%), and entropy (7.97) demonstrate superior performance. The proposed hybrid approach outperforms traditional algorithms in terms of security while preserving image quality.
Conversational Search Based on Utterance-Mask-Passage Post-training

Conversational Search (CS) aims to retrieve relevant passages from multiple documents based on the questions given by the user in conversations. Since users ask questions and get answers step by step through multiple rounds of conversations, conversation history is usually utilized to enhance retrieval accuracy. Existing methods rely too heavily on conversation history to retrieve while ignoring the semantics of the current question. However, the conversation history is not fully relevant to the current question, which includes some document information irrelevant to the current question. It is challenging to extract utterances information relevant to the current question from the conversation history to facilitate retrieval without breaking semantic coherence. We propose the reranker based on the Utterance-Mask-Passage (UtMP) post-training method to address this challenge, which includes three training tasks: passage relevance classification, utterance correlation classification, and context mask. Our method decomposes complex conversation history into short contexts, learns fine-grained semantic associations between utterances and document passages through three training tasks based on multi-task learning, and further learns correlations between conversation history and document passages based on contrastive learning. On the MultiDoc2Dial dataset, our results are 1.1% and 1.2% higher than the SOTA on the Recall@1 and MRR@10 metrics, respectively, which verifies the improvement of our method on retrieval performance. Extensive experiments show that our method helps deal with conversation histories with multiple documents information.
Beyond Prediction Similarity: ShapGAP for Evaluating Faithful Surrogate Models in XAI

Abstract The growing importance of Explainable Artificial Intelligence (XAI) has highlighted the need to understand the decision-making processes of black-box models. Surrogation, emulating a black-box model (BB) with a white-box model (WB), is crucial in applications where BBs are unavailable due to security or practical concerns. Traditional fidelity measures only evaluate the similarity of the final predictions, which can lead to a significant limitation: considering a WB faithful even when it has the same prediction as the BB but with a completely different rationale. Addressing this limitation is crucial to develop Trustworthy AI practical applications beyond XAI. To address this issue, we introduce ShapGAP, a novel metric that assesses the faithfulness of surrogate models by comparing their reasoning paths, using SHAP explanations as a proxy. We validate the effectiveness of ShapGAP by applying it to real-world datasets from healthcare and finance domains, comparing its performance against traditional fidelity measures. Our results show that ShapGAP enables better understanding and trust in XAI systems, revealing the potential dangers of relying on models with high task accuracy but unfaithful explanations. ShapGAP serves as a valuable tool for identifying faithful surrogate models, paving the way for more reliable and Trustworthy AI applications.
A PTV-based feature-point matching algorithm for binocular stereo photogrammetry

Abstract The procedure of feature matching is one of the most important components in binocular or multi-ocular stereoscopic photogrammetry. In this paper, a feature-point matching algorithm based on the technique of particle tracking velocimetry is proposed for the scenario of measuring complex surface morphology by dense-point three-dimensional reconstruction. The core idea is to mix the epipolar-line constraint of line-of-sight (LOS) with the measure of a global similarity pairing and estimate the depth of each feature point in an iterative way. Experimental test is conducted to verify the algorithm performance by measuring the surface topology of a wave-like model. The result demonstrates that the feature-point matching algorithm is superior to traditional LOS method in terms of accuracy and robustness. Moreover, replacing the first module of coarse matching in the proposed algorithm by LOS will save the computational cost significantly without sacrificing the measurement accuracy.