Reviewerly/exHarmony huggingface dataset:
Pedestrian Attribute Recognition Based on Multimodal Transformer

Pedestrian attribute recognition (PAR) is susceptible to variable shooting angles, lighting, and occlusions. Improving recognition accuracy to suit its application in various complex scenarios is one of the most important tasks. In this paper, based on the Image-Text Multimodal Transformer, the intra-modal and inter-modal correlations are learned from pedestrian images and attribute labels. The applicability of six different multimodal fusion frameworks for attribute recognition is explored. The impact of different frameworks’ fused feature division methods on recognition accuracy is compared and analyzed. The comparative experiments verify the robustness and efficiency of the Early Concatenate framework, which has achieved multiple best metric scores on the two major public PAR datasets, PA100k and RAP. This paper not only proposes a new Transformer-based high-accuracy multimodal network, but also provides feasible ideas and directions for further research on PAR. The comparative discussion based on various multimodal frameworks also provides a perspective that can be learned for other multimodal tasks.
Sparse Graph Hashing with Spectral Regression

Learning-based hashing has received increasing research attention due to its promising efficiency for large-scale similarity search. However, most existing manifold-based hashing methods cannot capture the intrinsic structure and discriminative information of image samples. In this paper, we propose a new learning-based hashing method, namely, Sparse Graph Hashing with Spectral Regression (SGHSR), for approximate nearest neighbor search. We first propose a sparse graph model to learn the real-valued codes which can not only preserves the manifold structure of the data, but also adaptively selects sparse and discriminative features. Then, we use a spectral regression to convert the real-valued codes into high-quality binary codes such that the information loss between the original space and the Hamming space can be well minimized. Extensive experimental results on three widely used image databases demonstrate that our SGHSR method outperforms the state-of-the-art unsupervised manifold-based hashing methods.
A Dual-Path Approach for Gaze Following in Fisheye Meeting Scenes

Gaze following plays a crucial role in scene comprehension tasks, as it captures users’ visual information from their facial and eye movements, thereby predicting their gaze positions. This technique finds its application in various domains such as human-computer interaction and medical diagnosis. In the domain of multi-party meeting scenes, some studies have utilized fisheye cameras to capture the entire meeting scene. In this work, we focus on gaze following methods that utilize fisheye images for meeting scenes and collect the GazeMeeting dataset that contains 31,915 fisheye samples. We also propose a dual-path feature fusing model for gaze following, which fuses the learned features in the planar and spherical domains by introducing spherical convolutions. The dual-pathway model can learn the distortion information of different positions from scene images, achieving a normalized L2 distance of 0.0657 on our self-built GazeMeeting dataset. This result represents a 22.80% improvement over the current state-of-the-art methods. Additionally, our proposed model achieves a normalized L2 distance of 0.1326 on GazeFollow dataset, outperforming the current state-of-the-art methods by 3.35%.
DLUT: Decoupled Learning-Based Unsupervised Tracker

Unsupervised learning has shown immense potential in object tracking, where accurate classification and regression are crucial for unsupervised trackers. However, the classification and regression branches of most unsupervised trackers calculate object similarities by sharing cross-correlation modules. This leads to high coupling between different branches, thus hindering the network performance. To address the above issue, we propose a Decoupled Learning-based Unsupervised Tracker (DLUT). Specifically, we separate the training pipelines of different branches to unlock their inherent learning potential so that different branches can fully explore the focused feature regions of interest. Furthermore, we design independent adaptive decoupling-correlation modules according to the characteristics of each branch to obtain more discriminative and easily locatable feature response maps. Finally, to suppress the noise interference brought by unsupervised pseudo-label training and highlight the foreground object, we propose a novel suppression-ranking-based unsupervised training strategy. Extensive experiments demonstrate that our DLUT outperforms state-of-the-art unsupervised trackers.
Discussion on the Strategy of Improving Class Efficiency in Physics Teaching of Senior High School

In the field of education, the classroom is the main place for students to learn knowledge, cultivate skills and train thinking, and improving classroom efficiency is of vital significance for students’ learning and future development. As a subject that requires high levels of abstract thinking and experimental skills, the importance of classroom teaching efficiency in high school physics is particularly prominent. This paper discusses how to improve the efficiency of high school physics classroom teaching in view of the disadvantages of high school physics teaching, aiming to provide some useful references for middle school physics teachers.
360-VIO: A Robust Visual–Inertial Odometry Using a 360° Camera

Visual–inertial odometry (VIO) is an important component for robots working in industrial environments to obtain accurate and robust pose estimation. In this article, we introduce a novel VIO algorithm designed specifically for 360° cameras, exploiting their wide field of view. We build our algorithm based on an EKF-based filtering framework and propose a novel measurement model for 360° camera leveraging the reprojection error on the tangent plane of the spherical surface. By this, our 360-VIO effectively mitigates errors caused by the image distortion and fully exploit the omnidirectional observation, resulting in superior accuracy and robustness toward illumination changes and fast camera movements. To the best of our knowledge, no public 360-based visual–inertial dataset is currently available. To address this gap, we have constructed a novel 360° camera dataset under different challenging environments to evaluate the performance of our proposed algorithm. The results of our experiments demonstrate the impressive robustness and accuracy of our method compared with other state-of-the-art VIO methods across diverse and challenging environments. To facilitate future research on 360-VIO, we will release the method, self-collected dataset and the calibration tool at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Gatsby23/360-VIO</uri> .
DeepIA: An Interpretability Analysis based Test Data Generation Method for DNN

Recently, deep neural networks (DNN) have been widely applied in various fields, such as image classification, even replace humans to make decisions in some specific tasks. However, like traditional software, DNNs inevitably contain defects. If defective DNN models are applied in safety-critical fields, such as autonomous driving and medical diagnosis, it may cause disastrous consequences. Therefore, effective testing methods are urgently needed to improve the reliability of DNNs. The existing DNN testing methods typically generate test data by either globally modifying the original data or taking adversarial approaches. The generated test data typically struggle to simultaneously achieve good performance in both the degree of difference from the original data and the Error-inducing Success Rate (ESR) with respect to the target DNN model. Moreover, the perturbation-based methods are difficult to be understood by humans. To address the above issue, this paper proposes DeepIA, an interpretability analysis based test data generation method for DNN. DeepIA analyzes the interpretability of decision-making behaviors for DNN. According to the interpretability analysis results, the original training data is split into different regions to evaluate their influences on decision-making results of the DNN. After that, the most significant regions of the original test data are transformed to generate new test data. Experimental results show that the interpretability method effectively enhances the misleading ability of DeepIA for the DNN model under test. Compared with DeepTest and DeepSearch, DeepIA can generate test data with minor permutations and greater ESR.
Leveraging Conditional Statement to Generate Acceptance Tests Automatically via Traceable Sequence Generation

In software development, testing is critical in guaranteeing software quality, with test case design at the core of the testing phase. However, generating effective test cases requires deep expertise and significant time and effort. Therefore, much prior research has turned to methods of Natural Language Processing, utilizing generative deep learning methods to automate test case generation. These earlier studies, however, have largely ignored the crucial role of conditional statements within software requirements - a factor we believe is indispensable for generating high-quality test cases. To bridge this gap, we introduce a pioneering approach for automatically deriving antecedents and consequents from requirements, termed as Traceable Sequence Generation (TSG). The TSG generates conditional statements first and then generates corresponding test cases by constructing a Cause-Effect-Graph. To verify the effectiveness of TSG, we constructed a requirement-to-test-case dataset, called Code Test Case Eval (CTCE). The dataset also includes annotated conditional statements for each segment of the requirement text, so we can utilize them to improve test case generation easily. Our experimental results indicate that TSG notably surpasses traditional and NLP-based methods, excelling in conditional statement extraction and generating high-coverage test cases.
Accelerating Multi-Agent DDPG on CPU-FPGA Heterogeneous Platform

Multi-Agent Reinforcement Learning (MARL) is a key technology in artificial intelligence applications such as robotics, surveillance, energy systems, etc. Multi-Agent Deep Deterministic Policy Gradient (MADDPG) is a state-of-the-art MARL algorithm that has been widely adopted and considered a popular baseline for novel MARL algorithms. However, existing implementations of MADDPG on CPU and CPU-GPU platforms do not exploit fine-grained parallelism between cooperative agents and handle inter-agent communication sequentially, leading to sub-optimal throughput performance in MADDPG training. In this work, we develop the first high-throughput MADDPG accelerator on a CPU-FPGA heterogeneous platform. Specifically, we develop dedicated hardware modules that enable parallel training of each agent's internal Deep Neural Networks (DNNs) and support low-latency inter-agent communication using an on-chip agent interconnection network. Our experimental results show that the speed performance of agent neural network training improves by a factor of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$3.6\times-24.3\times$</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$1.5\times-29.5\times$</tex> compared with state-of-the-art CPU and CPU-GPU implementations. Our design achieves up to a <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$1.99\times$</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$1.93\times$</tex> improvement in overall system throughput compared with CPU and CPU-GPU implementations, respectively.
Revealing Ongoing Sensor Attacks in Industrial Control System Via Setpoint Modification

A variety of Intrusion Detection Systems (IDSs) for Industrial Control Systems have been proposed to detect attacks and alert operators. Passive and active detection schemes are characterised by whether or not they interact with the process under control, though both categories of approach have limitations relating to either known correlations in the process data or the use of explicit system modelling. We propose setpoint modification as a strategy to address those limitations. The approach superimposes Gaussian noises on setpoint values, which aids in revealing latent correlations between setpoints and measurements, thereby allowing machine learning-based IDSs to learn them during training and verify during inference. We show that by applying the approach to a linear system with PID control, statistical tests can be configured such that the distortion power of sensor attacks is nullified. Building on this foundation, we further adapt passive IDSs for active discovery of sensor attacks in a process-agnostic fashion. The proposed strategy is evaluated using a nonlinear and simulated industrial benchmark, affirming that the approach enhances intrusion detection performance when the specific sensor under consideration is targeted whilst incurring marginal cost. Finally, we explore changing setpoints concurrently when the attacker could manipulate an arbitrary sensor, which also boosts detection performance and motivates the exploration of setpoint selection.
G-MAP: A Graph Neural Network-Based Framework for Memory Access Prediction

Memory access prediction is a crucial problem in data prefetchers, as it helps us improve memory performance and reduce latency in computing systems. Existing works model the problem as a sequence prediction problem. This can be limited in its ability to capture complex patterns and dependencies in memory access behavior. In recent years, Graph Neural Networks (GNNs) have emerged as a promising technique for modeling and predicting complex relationships in graph-structured data. In this paper, we introduce G-MAP, a novel Graph Neural Network-based framework for Memory Access Prediction. First, we propose Me <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$m2Graph$</tex> , a novel approach mapping a memory access sequence to a graph representation, capturing both the spatial and temporal locality in the sequence. Second, we implement various GNNs for G-MAP, including Graph Convolutional Network (GCN), Gated Graph Sequence Neural Network (GG-NN), and Graph Attention Network (GAT). Those models take the graph generated from Mem2Graph as input and predict future memory address jumps (deltas). We evaluate the effectiveness of G-MAP using the SPEC 2006 benchmark. G-MAP using GG-NN shows the highest performance among all models, achieving 0.7526 F1-Score on the average, which is 10.77% higher than the Multi-Layer Perceptron baseline.
Accelerating GNN-Based SAR Automatic Target Recognition on HBM-Enabled FPGA

Synthetic Aperture Radar (SAR) automatic target recognition (ATR) is a key technique for remote-sensing image recognition. In real-world applications, massive SAR images are captured by airplanes or satellites, requiring high-throughput and low-latency processing. Recently, Graph Neural Networks (GNNs) have shown superior performance for SAR ATR in terms of accuracy and computational complexity. In this paper, we accelerate GNN-based SAR ATR on an FPGA. In the proposed design, we develop a customized data path and memory organization to execute various computation kernels of GNNs, including feature aggregation and feature transformation. We exploit the high bandwidth memory (HBM) of the FPGA to speed up data loading and store intermediate results. We employ the splitting kernel technique to improve the routability and frequency of the design on FPGA. We implement the proposed design using High-level Synthesis (HLS) on a state-of-the-art data-center FPGA board, the AMDlXilinx Alveo U280. Compared with implementations on state-of-the-art CPUs (GPUs), our FPGA implementation achieves a 5.2x (l.57x) lower latency, a lOx (3.3x) higher throughput, and is 36.2x (7.35x) more energy efficient.
Data Consumption and User Experience in Video Lecture Sessions via Mobile Telephony Network

The feasibility of remote teaching as an alternative to face-to-face classes during the period of social isolation caused by the Covid-19 pandemic encountered many difficulties in accessing the Internet in remote regions, and even in urban areas of the country. This work presents a study of Internet access capabilities using the cell phone network for video-class sessions in environments similar to those that could be used by students in remote areas and with weak cell phone signals, in particular indigenous communities. The work seeks to answer three research questions related to data consumption, monetary cost and quality of user experience (QoE) per video lesson session. Measurement experiments with real devices show that the cost per video session is proportional to the QoE of the users, as well as to the conditions of the wireless access network. The results show an 80% reduction in cost per video session while maintaining the median QoE on the Mean Opinion Score (MOS) scale.
Ensemble Transfer Learning for Time Series Forecasting: a Sensitivity Analysis Framework for a Shallow Neural Network

Transfer learning applied to time-series forecasting is a current topic of high interest to the machine learning community. This paper addresses the need for empirical studies, as identified in recent survey papers that advocate the necessity of practical guidelines for TL approaches and method design for time series forecasting. This paper's main contributions include proposing a comprehensive sensitivity analysis framework and methodology for TL schemes, starting with the introduction of four novel TL performance metrics. Our experiments validate the usability of the methodology and offer insightful information using an MLP shallow network as a use-case scenario. The results of the experiments demonstrate the advantages of using ensemble techniques for time series forecasting over using single models in transfer learning. The experiments also offer empirical insights into various parameters that impact the transfer learning gain while raising the question of network dimensioning requirements when designing a neural network for transfer learning. The project aims to culminate in the design of a pre-trained model that can benefit practitioners in time-series forecasting.
DEM Generation From Mono-Multispectral Images via Uncalibrated Epipolar Rectification and 3D Registration

Generation of Digital Elevation Models (DEM) typically requires fine image matching between set of images acquired with a wide-baseline and appropriate B/H ratio. Wide baseline stereo imaging is an expensive process since it requires two or more acquisitions at the cost of nominal mono-imaging operations. Due to the wide-baseline, angular separation in views is required, which results in increased occlusion problems leading to holes in the generated DEM. Multiview images taken with wide baselines also have lower overlap area compared to images taken from narrow-baseline systems. Moreover, while making orthorectified maps, large tilts need to be corrected using processing techniques to obtain a top-down view. To alleviate these issues, we propose a novel DEM generation pipeline that utilizes the narrow-baseline information between bands of a single multispectral image. Our method can generate sub-pixel accurate disparity maps while tackling spectral information change. Final DEM is generated from the disparity by using reference Ground Control Points (GCPs) and an iterative control point registration procedure. We demonstrate the effectiveness of our DEM generation pipeline on Resourcesat-2 LISS- IV multispectral image with a BIH ratio as low as 0.018. Our results show that sub pixel disparity of the order of 1/50th of a pixel can be computed. A visual and quantitative comparison of the generated DEM with the wide-baseline DEM from Cartosat-l shows that the elevation information is estimated with good overall agreement even for a hilly region. Our technique makes it possible to extract elevation information from mono-multispectral images with unusually low B/H ratios, potentially enabling the generation of DEMs from multispectral images that would not normall be considered viable for this purpose.
A Multiple Indicator Credit Ranking Scheme Based on Notary Mechanism

Blockchain has penetrated multiple fields such as finance, healthcare, supply chain, and intelligent transportation. However, the value exchange between different independent blockchain systems limits the expansion of blockchain. Cross-chain technology can achieve the transfer of digital assets between blockchain networks, and the notary mechanism is a typical cross-chain technology that can achieve value exchange between different blockchains. However, it requires trust in specific notaries. To address these issues, this paper proposes a multi-indicator credit ranking scheme based on notaries to improve the reliability of notaries and ensure the security of value exchange between different blockchains. To tackle the problem of single and incomplete inherent value evaluation indicators in traditional notary mechanisms, a multi-indicator evaluation is adopted to ensure the fairness of notary node credit evaluation. To address the issue of inaccurate comprehensive credit evaluation of notaries, a comprehensive evaluation and ranking of notary nodes are conducted based on their historical transaction information to improve the accuracy of their credit evaluation. The experimental results show that using multiple indicators to evaluate the inherent value of notary nodes can comprehensively evaluate the scheme and accurately screen out high-credit nodes to act as notaries, improving the security of cross-chain asset transactions.
Towards Explainable Machine Learning: The Effectiveness of Reservoir Computing in Wireless Receive Processing

Deep learning has seen a rapid adoption in a variety of wireless communications applications, including at the physical layer. While it has delivered impressive performance in tasks such as channel equalization and receive processing/symbol detection, it leaves much to be desired when it comes to explaining this superior performance. In this work, we investigate the specific task of channel equalization by applying a popular learning-based technique known as Reservoir Computing (RC), which has shown superior performance compared to conventional methods and other learning-based approaches. Specifically, we apply the echo state network (ESN) as a channel equalizer and provide a first principles-based signal processing understanding of its operation. With this groundwork, we incorporate the available domain knowledge in the form of the statistics of the wireless channel directly into the weights of the ESN model. This paves the way for optimized initialization of the ESN model weights, which are traditionally untrained and randomly initialized. Finally, we show the improvement in receive processing/symbol detection performance with this optimized initialization through simulations. This is a first step towards explainable machine learning (XML) and assigning practical model interpretability that can be utilized together with the available domain knowledge to improve performance and enhance detection reliability.