Reviewerly/exHarmony huggingface dataset:
Fast Key Points Detection and Matching for Tree-Structured Images

This paper offers a new authentication algorithm based on image matching of nano-resolution visual identifiers with tree-shaped patterns. The algorithm includes image-to-tree conversion by greedy extraction of the fractal pattern skeleton along with a custom-built graph matching algorithm that is robust against imaging artifacts such as scaling, rotation, scratch, and illumination change. The proposed algorithm is applicable to a variety of tree-structured image matching, but our focus is on dendrites, recently-developed visual identifiers. Dendrites are entropy rich and unclonable with existing 2D and 3D printers due to their natural randomness, nano-resolution granularity, and 3D facets, making them an appropriate choice for security applications such as supply chain trace and tracking. The proposed algorithm improves upon graph matching with standard image descriptors. It also improves [1], which faces various problems when deploying in real-world applications. For instance, image inconsistency due to the camera sensor noise may cause unexpected feature extraction leading to inaccurate tree conversion and authentication failure. Also, previous tree extraction algorithms are prohibitively slow hindering their scalability to large systems. In this paper, we fix the current issues of [1] and accelerate the key points extraction up to 10-times faster by implementing a new skeleton extraction method, a new key points searching algorithm, as well as an optimized key point matching algorithm. Using minimum enclosing circle and center points, make the algorithm robust to the choice of pattern shape. We show that our algorithms outperform standard key descriptors such as SIFT, FAST, and ORB in terms of accuracy (with a margin of 6% to 10%), while utilizing much fewer key points (about 20% to 80% reduction). In contrast to [1] our algorithm handles general graphs with loop connections, therefore is applicable to a wider range of applications such as transportation map analysis, fingerprints, and retina vessel imaging. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Lightweight YOLOv4 Algorithm for Remote Sensing Image Detection

Remote sensing images have the characteristics of complex backgrounds, high resolution, and small targets. Although the existing object detection algorithms can improve the detection accuracy, there are generally problems such as a large number of model parameters, high computational cost, and poor real-time performance. Aiming at the above problems, this paper designs a lightweight object detection algorithm GSC-YOLO based on YOLOv4 to achieve fast and accurate detection of remote sensing images. First, Ghostnet is used as the feature extraction network of GSC-YOLO to reduce the number of parameters and improve the detection speed; Secondly, the improved shuffle attention mechanism is introduced in the prediction head to make the model pay attention to important information and improve the detection accuracy; Finally, the Confidence Propagation Cluster algorithm CP-Cluster is used to post-process the prediction frame to improve the object recognition. Taking the preprocessed DOTA dataset as the experimental object, the experimental results show that the GSC-YOLO algorithm has a detection accuracy of 93.44%, a detection speed of 58 frames per second, and a model size of 43.65MB. Compared with the remote sensing image object detection algorithm based on YOLOv4, the detection accuracy is increased by 3.93%, the detection speed is increased by 1.87 times, and the model size is reduced by 5.62 times, which is more suitable for deployment on devices with limited resources.
Enhancing Colorectal Cancer Histological Image Classification Using Transfer Learning and ResNet50 CNN Model

Medical image analysis is crucial in healthcare research. The convolutional neural network (CNN) has great potential in improving the precision and speed of medical diagnosis. In medical diagnostics, CNNs have displayed promising results, indicating their capability to enhance the accuracy and efficiency of the diagnostic process, accurately classifying complex medical images remains challenging. Colorectal cancer, a significant cause of global mortality, emphasizes the need for early detection and diagnosis to ensure successful treatment. We develop a new method combining transfer learning and a ResNet50 CNN model with the Adam optimizer to increase the accuracy in the classification of the histopathology images of colorectal cancer. The experimental results demonstrated outstanding performance with an accuracy of 99.99% in training and an accuracy of 99.77% in validation which were excellent performance on widely recognized evaluation metrics. In conclusion, the proposed method surpasses other related studies using CNN models for histopathology image classification. It provides a practical solution to further improve the classification performance of colorectal cancer histopathology images. The study result shows the efficacy of transfer learning in the analysis of medical images. Moreover, the proposed approach outperforms existing methods in medical image analysis, underscoring its potential to empower medical professionals in enhancing diagnostic capabilities and making more informed clinical decisions for patients.
Watermarks for Generative Adversarial Network Based on Steganographic Invisible Backdoor

Model watermarking has become an important solution to protect the intellectual property right (IPR) of deep neural networks (DNN) models. However, there are few researches on the IPR protection of generative adversarial networks (GAN), which are widely used to generate photorealistic images. In the current backdoor-based watermarking method for GAN models, the trigger pattern of the watermark is easy to be detected and invalidated by the adversary, which may fail to achieve IPR protection. To address this drawback, we propose a new GAN model watermarking method, where an invisible backdoor based on steganography is injected into the target GAN model as a watermark. Experimentally, the proposed method effectively trades off the performance of GAN on original task and the robustness of watermarking for removal attacks. Moreover, the generated triggers can effectively resist being detected by attackers.
End-To-End Part-Level Action Parsing With Transformer

The divide-and-conquer strategy, which interprets part-level action parsing as a detect-then-parsing pipeline, has been widely used and become a general tool for part-level action understanding. However, existing methods that derive from the strategy usually suffer from either strong dependence on prior detection or high computational complexity. In this paper, we present the first fully end-to-end part-level action parsing framework with transformers, termed PATR. Unlike existing methods, our method regards part-level action parsing as a hierarchical set prediction problem and unifies person detection, body part detection, and action state recognition into one model. In PATR, predefined learnable representations, including general instance representations and general part representations, are guided to adaptively attend to the image features that are relevant to target body parts. Then, conditioning on corresponding learnable representations, attended image features are hierarchically decoded into corresponding semantics (i.e., person location, body part location, and action states for each body part). In this way, PATR relies on characteristics of body parts, instead of prior predictions like bounding boxes, to parse action states, thus removing the strong dependence between sub-tasks and eliminating the computational burdens caused by the multi-stage paradigm. Extensive experiments conducted on challenging Kinetic-TPS indicate that our method achieves very competitive results. In particular, our model outperforms all state-of-the-art part-level action parsing approaches by a margin, reaching around 3.8±2.0% Acc <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</inf> higher than previous methods. These findings indicate the potential of PATR to serve as a new baseline for part-level action parsing methods in the future. Our code and models are publicly available. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Guided Focal Stack Refinement Network for Light Field Salient Object Detection

Light field salient object detection (SOD) is an emerging research direction attributed to the richness of light field data. However, most existing methods lack effective handling of focal stacks, therefore making the latter involved in a lot of interfering information and degrade the performance of SOD. To address this limitation, we propose to utilize multi-modal features to refine focal stacks in a guided manner, resulting in a novel guided focal stack refinement network called GFRNet. To this end, we propose a guided refinement and fusion module (GRFM) to refine focal stacks and aggregate multi-modal features. In GRFM, all-in-focus (AiF) and depth modalities are utilized to refine focal stacks separately, leading to two novel sub-modules for different modalities, namely AiF-based refinement module (ARM) and depth-based refinement module (DRM). Such refinement modules enhance structural and positional information of salient objects in focal stacks, and are able to improve SOD accuracy. Experimental results on four benchmark datasets demonstrate the superiority of our GFRNet model against 12 state-of-the-art models.
Fine-grained Learning for Visible-Infrared Person Re-identification

Visible-Infrared Person Re-identification aims to retrieve specific identities from different modalities. In order to relieve the modality discrepancy, previous works mainly concentrate on aligning the distribution of high-level features, while disregarding the exploration of fine-grained information. In this paper, we propose a novel Fine-grained Information Exploration Network (FIENet) to implement discriminative representation, further alleviating the modality discrepancy. Firstly, we propose a Progressive Feature Aggregation Module (PFAM) to progressively aggregate mid-level features, and a Multi-Perception Interaction Module (MPIM) to achieve the interaction with diverse perceptions. Additionally, combined with PFAM and MPIM, more fine-grained information can be extracted, which is beneficial for FIENet to focus on discriminative human parts in both modalities effectively. Secondly, in terms of the feature center, we introduce an Identity-Guided Center Loss (IGCL) to supervise identity representation with intra-identity and inter-identity information. Finally, extensive experiments are conducted to demonstrate that our method achieves state-of-the-art performance.
Designing of Prompts for Hate Speech Recognition with In-Context Learning

In-context learning is a recent paradigm in natural language understanding, where a pretrained large language model (LLM) directly performs a new task without any update to its parameters by taking a test instance, new task description and a few training examples (e.g. input-label pairs) as its input. However, performance has been shown to strongly depend on the task description and selected training examples (both together termed as prompts here). In this paper, we use GPT-3 as the LLM, hate speech recognition as the new task, and we investigate how to design effective prompts for better performance. Our preliminary experimental results show that: (1) substantial number of input-labels pairs are necessary for good performance (2) informative task descriptions can further boost performance by ingesting our prior knowledge as inference guidance.
A Cross-direction Task Decoupling Network for Small Logo Detection

Logo detection plays an integral role in many applications. However, handling small logos is still difficult since they occupy too few pixels in the image, which burdens the extraction of discriminative features. The aggregation of small logos also brings a great challenge to the classification and localization of logos. To solve these problems, we creatively propose Cross-direction Task Decoupling Network (CTDNet) for small logo detection. We first introduce Cross-direction Feature Pyramid (CFP) to realize cross-direction feature fusion by adopting horizontal transmission and vertical transmission. In addition, Multi-frequency Task Decoupling Head (MTDH) decouples the classification and localization tasks into two branches. A multi-frequency attention convolution branch is designed to achieve more accurate regression by combining discrete cosine transform and convolution creatively. Comprehensive experiments on four logo datasets demonstrate the effectiveness and efficiency of the proposed method.
A Self-Training Framework Based on Multi-Scale Attention Fusion for Weakly Supervised Semantic Segmentation

Weakly supervised semantic segmentation (WSSS) based on image-level labels is challenging since it is hard to obtain complete semantic regions. To address this issue, we propose a self-training method that utilizes fused multi-scale class-aware attention maps. Our observation is that attention maps of different scales contain rich complementary information, especially for large and small objects. Therefore, we collect information from attention maps of different scales and obtain multi-scale attention maps. We then apply denoising and reactivation strategies to enhance the potential regions and reduce noisy areas. Finally, we use the refined attention maps to retrain the network. Experiments showthat our method enables the model to extract rich semantic information from multi-scale images and achieves 72.4% mIou scores on both the PASCAL VOC 2012 validation and test sets. The code is available at https://bupt-ai-cz.github.io/SMAF.
SASFormer: Transformers for Sparsely Annotated Semantic Segmentation

Semantic segmentation based on sparse annotation has advanced in recent years. It labels only part of each object in the image, leaving the remainder unlabeled. Most of the existing approaches are time-consuming and often necessitate a multi-stage training strategy. In this work, we propose a simple yet effective sparse annotated semantic segmentation framework based on segformer, dubbed SASFormer, that achieves remarkable performance. Specifically, the framework first generates hierarchical patch attention maps, which are then multiplied by the network predictions to produce correlated regions separated by valid labels. Besides, we also introduce the affinity loss to ensure consistency between the features of correlation results and network predictions. Extensive experiments showcase that our proposed approach is superior to existing methods and achieves cutting-edge performance. The source code is available at https://github.com/su-hui-zz/SASFormer.
Fine-grained Domain Adaptive Crowd Counting via Point-derived Segmentation

Due to domain shift, a large performance drop is usually observed when a trained crowd counting model is deployed in the wild. While existing domain-adaptive crowd counting methods achieve promising results, they typically regard each crowd image as a whole and reduce domain discrepancies in a holistic manner, thus limiting further improvement of domain adaptation performance. To this end, we propose to untangle domain-invariant crowd and domain-specific background from crowd images and design a fine-grained domain adaption method for crowd counting. Specifically, to disentangle crowd from background, we propose to learn crowd segmentation from point-level crowd counting annotations in a weakly-supervised manner. Based on the derived segmentation, we design a crowd-aware domain adaptation mechanism consisting of two crowd-aware adaptation modules, i.e., Crowd Region Transfer (CRT) and Crowd Density Alignment (CDA). The CRT module is designed to guide crowd features transfer across domains beyond background distractions. The CDA module dedicates to regularising target-domain crowd density generation by its own crowd density distribution. Our method outperforms previous approaches consistently in the widely-used adaptation scenarios.
Simulating Turing Machine in Augmented Reality

We present an easy-use interactive user interface for simulating Automaton and Turing machine in Augmented Reality (AR). We design a C-like programming language to describe the automaton and Turing machine, visualize their data structures, and simulate their running process in AR given to the user-specified input string or tape. Our Automaton and Turing machine simulator support three different modes, including Finite Automata (FA), Pushdown Automata (PDA), and Turing Machine (TM). We have deployed our AR Turing machine simulator on HoloLens2, users can interact with the simulator through AR buttons at the same time they can edit the scripts on the desktop where there is a USB cable connecting the desktop and HoloLens2 such that the Turing machine simulator can be updated simultaneously as users update their scripts. This mechanism can enable the user to efficiently design the Turing machine through our interactive interface in augmented reality.
Weakly Supervised Video Anomaly Detection Based on Cross-Batch Clustering Guidance

Weakly supervised video anomaly detection (WSVAD) is a challenging task since only video-level labels are available for training. In previous studies, the discriminative power of the learned features is not strong enough, and the data imbalance resulting from the mini-batch training strategy is ignored. To address these two issues, we propose a novel WSVAD method based on cross-batch clustering guidance. To enhance the discriminative power of features, we propose a batch clustering based loss to encourage a clustering branch to generate distinct normal and abnormal clusters based on a batch of data. Meanwhile, we design a cross-batch learning strategy by introducing clustering results from previous minibatches to reduce the impact of data imbalance. In addition, we propose to generate more accurate segment-level anomaly scores based on batch clustering guidance to further improve the performance of WSVAD. Extensive experiments on two public datasets demonstrate the effectiveness of our approach.
EXPANSE: A Continual and Progressive Learning System for Deep Transfer Learning

Deep transfer learning (DTL) techniques attempt to tackle the limitations of deep learning, the dependency on extensive training data and the training costs by reusing obtained knowledge from source data for target data. However, the current DTL techniques suffer from either catastrophic forgetting dilemma (losing the previously obtained knowledge) or overly biased pre-trained models (harder to adapt to target data) in fine-tuning pre-trained models or freezing a part of the pre-trained model, respectively. We propose a new continual/progressive learning approach for deep transfer learning to tackle these limitations. We extend the pre-trained model by expanding pre-trained layers (adding new nodes to each layer) in the model instead of only adding new layers. Hence the method is named EXPANSE. Our experimental results confirm that we can tackle distant source and target data using this technique. At the same time, the final model is still valid on the source data, achieving a promising deep continual learning approach. Moreover, we offer a new way of training deep learning models inspired by the human education system. We termed this two-step training: learning basics first, then adding complexities and uncertainties. The evaluation implies that the two-step training extracts more meaningful features and a finer basin on the error surface since it can achieve better accuracy compared to regular training.
A Content-based Viewport Prediction Framework for 360° Video Using Personalized Federated Learning and Fusion Techniques

Viewport prediction is a key enabler for 360° video streaming over wireless networks. To improve the prediction accuracy, a common approach is to use a content-based viewport prediction model. Saliency detection based on traditional convolutional neural networks (CNNs) suffers from distortion due to equirectangular projection. Also, the viewers may have their own viewing behavior and are not willing to share their historical head movement with others. To address the aforementioned issues, in this paper, we first develop a saliency detection model using a spherical CNN (SPCNN). Then, we train the viewers’ head movement prediction model using personalized federated learning (PFL). Finally, we propose a content-based viewport prediction framework by integrating the video saliency map and the head orientation map of each viewer using fusion techniques. The experimental results show that our proposed framework provides higher average accuracy and precision when compared with three state-of-the-art algorithms from the literature.
Towards Specification Completion for Systems with Emergent Behavior based on DevOps

Software systems may experience multiple emergent behaviors during their operation time. These emergent system behaviors occur when system engineers develop their system under the closed-world assumption, but this assumption is not met during its operation. This means that system engineers work on the basis that they have complete knowledge of the system and its environment during its design, when the system specification that has been created is actually incomplete. In this paper, an observation of an emergent behavior is considered to be a solid proof that the system model specification is still incomplete. A conceptual framework is proposed to harness the emergent behavior and complete the system specification that is provided during the its design. The framework consists of two parts, system development and system operations. It is built on a model-driven approach in order to provide a clear definition of the emergent behavior and a concrete development scheme. The framework exploits the DevOps paradigm as a successful paradigm to achieve the ultimate goal of developing complete system models through the continuous specification completion based on the observed emergent behavior. The goal of this framework is to help develop high-quality and reliable emergent systems based on the specification derived from the emergent behavior that occurred at run time.
DD-GCN: Directed Diffusion Graph Convolutional Network for Skeleton-based Human Action Recognition

Graph Convolutional Networks (GCNs) have been widely used in skeleton-based human action recognition. In GCN-based methods, the spatio-temporal graph is fundamental for capturing motion patterns. However, existing approaches ignore the physical dependency and synchronized spatio-temporal correlations between joints, which limits the representation capability of GCNs. To solve these problems, we construct the directed diffusion graph for action modeling and introduce the activity partition strategy to optimize the weight sharing mechanism of graph convolution kernels. In addition, we present the spatio-temporal synchronization encoder to embed synchronized spatio-temporal semantics. Finally, we propose Directed Diffusion Graph Convolutional Network (DD-GCN) for action recognition, and the experiments on three public datasets: NTU-RGB+D, NTU-RGB+D 120, and NW-UCLA, demonstrate the state-of-the-art performance of our method.