Reviewerly/exHarmony huggingface dataset:
Hierarchical visual-semantic interaction for scene text recognition

Proper interaction between visual and semantic features is crucial to obtain a powerful feature representation for scene text recognition (STR). The existing interaction methods usually treat visual and semantic features as distinct tokens and use transformers to learn contextual information and prior language knowledge, and they achieve promising performance for STR task. However, there still remain several issues needed to be further addressed such as the imbalance number and mis-alignment between visual and semantic features, and the necessarily of stacking transformers to a progressive improvement in accuracy. To this aim, this paper proposes a novel interaction manner namely hierarchical visual-semantic interaction (HVSI) which contains three novel modules including a hierarchical visual-semantic interaction module, fusion module, and visual-semantic alignment module. The hierarchical visual-semantic interaction module employs multiple visual-semantic interaction blocks in various scales to enhance the representation power of visual features and semantic features. To better exploit multi-scale visual and semantic features, the fusion module is introduced to fuse multiple semantic features based on attention mechanisms. Furthermore, our HVSI presents a simple plug-in block named visual-semantic alignment module to alleviate mis-alignment of semantic features by mapping them into a unified semantic space, which helps improve the performance of HVSI. Extensive experiments on multiple benchmarks including English and Chinese text recognition datasets show that our method obtains state-of-the-art or competitive performances.
GNaN: A natural neighbor search algorithm based on universal gravitation

The natural neighbor (NaN) method and its search algorithm (NaN-Searching) are widely used in many fields, including pattern recognition and image processing. NaN-Searching fundamentally overcomes the problem of the conventional nearest neighbor algorithm in selecting parameters for datasets with arbitrary shapes and achieves good results. However, this algorithm uses the conventional distance metric as the neighbor judgment criterion, which cannot accurately reflect the overall structure of the dataset in the process of neighbor search. Inspired by Newton's law of universal gravitation, we propose a NaN search algorithm based on universal gravitation (GNaN-Searching). Our algorithm calculates gravitation using the structural features of data points in the dataset, it utilizes the gravitation between data as the neighbor judgment criterion, and inherits the no-parameter and dynamic neighborhood characteristics of the NaN search algorithm. Experimental results show that the natural neighborhood graph obtained by our method has a high performance in the representation of manifold data. We also applied the new method to clustering and outlier detection and achieved satisfactory results.
Frequency-aware feature aggregation network with dual-task consistency for RGB-T salient object detection

RGB-Thermal salient object detection (SOD) aims to merge two spectral images to segment visually appealing objects. Current methods primarily extract salient object information in the pixel perspective. However, biological and psychological research indicates notable frequency sensitivity of the human visual system (HVS). The high-frequency (HF) and low-frequency (LF) information in images are processed by different neural channels, which has been overlooked in SOD. In this study, we argue that the objective of RGB-T SOD is not only to enhance feature representation in the pixel-aware but also to emulate human visual mechanisms. To our best knowledge, we explore RGB-T SOD from the frequency perspective for the first time. Specifically, we first present a frequency-aware multi-spectral feature aggregation module (FMFA) to exploit the separability and complementarity of frequency-aware features, generating and making full use of LF and HF cues. FMFA improves the feature representation of RGB-T from the frequency perspective and provides stronger frequency cues for boundary auxiliary tasks. Then, we develop an HF-guided signed distance map prediction module (HF-SDM) with dual-task consistency to effectively alleviate the coarse mask caused by blur boundary. HF-SDM employs the geometric relationship of objects which boosts the interaction between salient regions and boundaries. As a result, the model can gain sharper boundaries for salient objects. Finally, we propose a frequency-aware feature aggregation network (FFANet) incorporated with dual-task learning. Extensive experiments on RGB-T SOD datasets demonstrate that our proposed method outperforms other state-of-the-art methods. Ablation studies and visualizations further verify the effectiveness and interpretability of our method.
Optimal Fuzzy Intensification System for Contrast Distorted Medical Images

Contrast enhancement plays a significant role in medical imaging technology for clinical diagnosis and observing the patient's health conditions. However, medical images may suffer from different degradations, such as poor visibility and low contrast, which are significant challenges for the automatic diagnosis of diseases. This article develops an ameliorated fuzzy transformation model to significantly enhance the contrast by adjusting the imbalanced intensity levels of medical images. The proposed optimal fuzzy intensification system (OFIS) utilizes the entire dynamic range by expanding the segmented intensity level of the input image. The fuzzy expansion process can enrich the diagnostic details of the processed image. The moth flame optimization is incorporated to optimize the gamma parameter to improve the clear visibility by adjusting the local contrast of the dark regions. Experiments on benchmark datasets reveal that the proposed method has superior performance in enhancing contrast and visibility of the local details of abnormalities in medical images compared with existing methods.
An integrated approach for magnification independent breast cancer classification

Breast cancer (BC) is the second most common cancer worldwide, and its diagnosis and treatment rely heavily on histopathology image analysis. Moreover, continuous evaluation of deep learning techniques in medical imaging modalities is essential for improving early diagnosis. The Break-His dataset is the de facto standard for such analysis. Researchers have actively explored various approaches to classifying these images into binary, four, and eight based on the magnification-independent (MI) and magnification-dependent (MD). However, existing processes necessitate separate models for each classification based on the magnification factor and entail training multiple models. It leads to increased training time and model complexity. Moreover, model performance in eight-class is relatively low compared to other classifications. This paper introduces a modified Swin-Transformer V2 architecture for multi-labeled and eight-class classification on BC histopathology images. Further, the eight-class output was reconfigured to operate in an integrated approach, allowing the model to predict binary, four, and eight classes simultaneously for both MI and MD. On the Break-His dataset, the proposed model achieved an accuracy of 98.27%, 97.95%, 98.97%, and 99.16% for eight-class, malignant, benign, and binary classification tasks, respectively. Additionally, the multi-labeled approach demonstrated a performance improvement in the eight-class category compared to current leading models, achieving an enhancement of up to 5%, and the model was trained in just 2 h. Overall, the proposed model can improve the prognosis of breast cancer patients by leading to more accurate diagnosis and treatment. The implemented code can be accessed at https://github.com/kbnarayanavit/integrated-approach.
Explainable deep learning for attack intelligence and combating cyber-physical attacks

Cyber-physical control loops comprising sensors, actuators and controllers pose the most valued and critical part of the industrial Internet of Things (IIoT) as it regulates the state of the physical process, such as water treatment or gas flow. Thus, any malicious activities could lead to physical damage, affecting human safety. Cyber-physical attacks against the physical process are difficult to detect using existing threats and attack intelligence due to the (1) lack of such intelligence for the physical process and operational technology systems and (2) such attacks affect the process parameters and states. Artificial Intelligence (AI)-based attack intelligence is required. This study proposes an attack intelligence framework for identifying cyber–physical attacks and extracting attack intelligence. We propose an attribution module for attack identification using various machine and deep learning algorithms. We also utilize Explainable AI (XAI) to improve the explainability of the attack attribution module and extract attack intelligence. Our proposed framework is evaluated and tested using a gas pipeline dataset as a use case. We demonstrate that the proposed framework improves the understanding of attacks and provides attack rules, assisting security analysts in securing critical physical processes.
Light-modulated graphene-based φ <sub>0</sub> Josephson junction and −φ <sub>0</sub> to φ <sub>0</sub> transition

We investigate the chiral edge states-induced Josephson current–phase relation in a graphene-based Josephson junction modulated by the off-resonant circularly polarized light and the staggered sublattice potential. By solving the Bogoliubov–de Gennes equation, a φ 0 Josephson junction is induced in the coaction of the off-resonant circularly polarized light and the staggered sublattice potential, which arises from the fact that the center of-mass wave vector of Cooper pair becomes finite and the opposite center of-mass wave vector to compensate is lacking in the nonsuperconducting region. Interestingly, when the direction of polarization of light is changed, − φ 0 to φ 0 transition generates, which generalizes the concept of traditional 0– π transition. Our findings provide a purely optical way to manipulate a phase-controllable Josephson device and guidelines for future experiments to confirm the presence of graphene-based φ 0 Josephson junction.
Receptive Field-Based All-Optical Spiking Neural Network for Image Processing

We report on a novel structure of a receptive field (RF)-based multi-layer all-optical neural network using a micropillar laser with a saturable absorber (SA) for image processing. From the perspective of biological vision, the realization of image processing based on the RF provides the biological rationality for the machine vision implemented by the spiking neural network (SNN). By exploiting the fast physical mechanisms of gain and absorption in the SA laser, the photonic spike-timing-dependent plasticity (STDP) curves are achieved to train the weights. Here, the source image pixels are mapped into the temporal information of spike trains injected into the neural network through the temporal coding method called time-to-first-spike encoding. Different source images are processed and tested by the proposed photonic SNN. Simulation results show that our proposed system can process not only simple binary images but also complex color images under the adjustment of STDP rules. When considering the robustness, we demonstrate the tolerance of the image segmentation to the time jitter. These results indicate that our proposed photonic SNN can achieve high-resolution processing of complex source images. Additionally, the time-multiplexing technique can be further adopted to simplify the RF structure, which is expected to reduce the complexity of the whole system, thus facilitating physical applications. Our work offers the prospect for a high-speed photonic spiking platform for image processing.
Domain Adaptation for In-Air to Underwater Image Enhancement via Deep Learning

Underwater image enhancement has drawn a lot of attention due to its significance in ocean exploration. Since it is extremely hard to obtain a degraded image and its clear version in the same scene, most deep learning-based methods utilize synthetic datasets for training. However, synthetic datasets often fail to describe the natural appearance and show poor capabilities of generalization to real scenes. In this study, we present an in-air to underwater image enhancement (IUIE) framework that overcomes the limitations of underwater synthetic datasets. Specifically, the presented framework consists of a color correction step and a domain adaptation step. In the color correction step, we propose a color correction mechanism by utilizing the white balance algorithm to eliminate color bias and produce a more natural appearance. In the domain adaptation step, the network can reduce the domain gap between the in-air domain and the underwater domain and obtain a better generalization in real scenes. To accommodate spatially varying textures and edges, we propose a residual adaptive module in the domain adaptation network. Experimental results on the test data indicate that our IUIE framework not only obtains more remarkable UIQM and BRISQUE scores than the previous methods do but also achieves superior results in visual perception.
Visualization of object-oriented software in a city metaphor: Comprehending the implemented variability and its technical debt

While many large-scale software systems intensively implement variability to reuse software and speed up development, they often do not document it, hampering its comprehension. This is especially the case for variability-rich object-oriented (OO) systems that heavily rely on existing OO mechanisms (i.e., inheritance, overloading and some patterns) to implement it in a single codebase. With no traceability information, the variability is buried in the codebase, hampering its identification, analysis, and understanding. While variability management becomes increasingly difficult over the system evolution, the implementation mechanisms also bring additional complexity to the codebase, which eventually leads to technical debt, threatening even more the software quality. In this article, we report on the design and evaluation of an extensible visualization, VariCity, that exhibits zones of high density of OO variability implementations. It relies on the city metaphor to represent the classes of the system as buildings whose dimensions are used to show variability metrics inherent to the implementation classes. They are linked together through streets depicting usage relationships and grouping in neighborhoods classes using each other. The extensibility of VariCity is demonstrated with VariMetrics, which highlights OO quality metrics on the buildings, revealing quality-critical classes concentrating variability implementations. We evaluate the visualization capacity to reveal zones concentrating variability implementations and being quality-critical by applying it to multiple variability-intensive open source software systems. We also report on a controlled experiment comparing the gain brought by the visualization to the use of an IDE.
Role Engine Implementation for a Continuous and Collaborative Multirobot System

In situations involving teams of diverse robots, assigning appropriate roles to each robot and evaluating their performance is crucial. These roles define the specific characteristics of a robot within a given context. The stream of actions exhibited by a robot based on its assigned role are referred to as the process role. Our research addresses the depiction of process roles using a multivariate probabilistic function. The main aim of this study is to develop a role engine for collaborative multirobot systems and optimize the behavior of the robots. The role engine is designed to assign suitable roles to each robot, generate approximately optimal process roles, update them on time, and identify instances of robot malfunction or trigger replanning when necessary. The environment considered is dynamic, involving obstacles and other agents. The role engine operates hybrid, with central initiation and decentralized action, and assigns unlabeled roles to agents. We employ the Gaussian process (GP) inference method to optimize process roles based on local constraints and constraints related to other agents. Furthermore, we propose an innovative approach that utilizes the environment’s skeleton to address initialization and feasibility evaluation challenges. We successfully demonstrated the proposed approach’s feasibility, and efficiency through simulation studies and real-world experiments involving diverse mobile robots.
Privacy-Preserving Realization of Fuzzy Clustering and Fuzzy Modeling Through Vertical Federated Learning

In this study, we elaborate on a realization of fuzzy clustering and the construction of fuzzy rule-based models on the basis of vertically partitioned datasets in a privacy-preserving federated learning approach. The main focus of the overall design process is to construct a family of information granules (clusters) and the corresponding fuzzy rules in the presence of a collection of vertically partitioned datasets without compromising data privacy. These datasets are composed of the same data but are described by different features, and due to security considerations, data cannot be shared. The vertical federated fuzzy clustering can be realized as an iterative optimization process composed of successive cycles: 1) computation (update) of the prototypes and partition matrices performed on the basis of local datasets and 2) an integration of the local sources of knowledge carried out on a central coordinator-server. The update of the partition matrices can be completed using a distance-based or gradient-based approach. The communication of findings between local clients and the coordinator-server is realized through exchanging partition matrices, which are more general than numeric data and can avoid leakage of data privacy. Fuzzy models are optimized in a similar manner through exchanging the gradients of the performance index computed with respect to the parameters between the clients and the global coordinator. The proposed mechanism exhibits significant originality since the realization of fuzzy modeling in a vertical federated learning environment has not been studied. Experimental studies show that the proposed federated clustering and fuzzy model design could effectively reveal the structure of the entire dataset and achieve high performance compared with the results obtained in a centralized manner.
Optical voice security scheme for anticounterfeiting

In this work, we propose an optical voice security scheme for anticounterfeiting that uses voice watermark recording by an optical setup of parallel phase shifting digital holography (PPSDH) and an approach of converting the recorded voice into a suitable 2D voice watermark. The converted voice watermark is further encrypted in the optical domain in order to provide enhanced security in terms of physical keys for watermarking and other applications. The proposed scheme makes it difficult for counterfeiting in order to claim the ownership of voice or image data in case of watermarking. We present the experimental results of voice watermark recorded by optical microphone based on PPSDH and simulation results to verify the proposed optical security scheme for watermarking of optically recorded data. Best of our knowledge, this is the first proposal towards voice watermark generation in optical domain and the first demonstration of it for watermarking.
Ensemble of ghost convolution block with nested transformer encoder for dense object recognition

The Technological advancement and innovation is happening at rapid pace and within the ambit of computerized era, the recognition models have displayed exceptionally outstanding performance in object detection but still the technology relies on ‘Dense-packed object detection”. This object detection remains one of the greatest challenges of the present situation due to redundant feature map calculation complexity, diverse shapes, and alignment of the object in various directions. This paper suggests a GCB (Ghost Convolution Block) based on nested-transformer encoder block in the feature refinement network to overcome these intricacies. The GCB effectively alleviates the redundant feature maps calculation complexity by using DW (Depth-Wise separable) convolution operation. Whereas, the nested-transformer encoder block extracts in-depth information from diverse shaped objects and misaligned objects based on the MHSA (Multi-Head-Self-Attention) mechanism's query, key, and value parameter information. We propose to perform quantitative evaluations on the VOC, GWHD (Global Wheat Head Detection), and SKU-110K data sets and carried out an ablation study by using the YOLOv5 model with GCB and GCBTR (Ghost Convolution Block-based Transformer) modules. We compared this model with conventional other models, our model achieved 84.2% mAP, 80% precision, 78.1% recall, and 79% F1-score on VOC, 81.8% mAP, 91.6% precision, 73.1% recall, and 81.3% F1-score on SKU-110K and also achieved 95.7% mAP, 94.4% precision, 90.2% recall on GWHD datasets. Thus, obtained reliable results prove that the suggested model has shown superior performance when compared with other existing models as well as the YOLOv5 model in detecting dense objects.
An objective quality indicator for evaluating filtering of microscopic video in CASA system

Computer-Aided Sperm Analysis (CASA) has received great attention in the diagnosis of male reproductive health for its objectivity and accuracy. But its accuracy may be greatly affected by algorithms used in microscopic video preprocessing, especially algorithms for microscopic videos filtering. Owing to the Peak Signal to Noise Ratio (PSNR) and Structural Similarity (SSIM) indicators are not suitable for microscopic images, objective indicators for evaluating filtering of microscopic videos are scarce at present, leading to filtering algorithms are chosen arbitrarily. In addition, it is critical to establish quality control indicators for CASA from the perspective of image processing, but they are rarely mentioned for the reason that algorithms adopted in image processing aren’t public. To reduce influences of filtering on diagnostic results, it is imperative to establish an objective quality control indicator for evaluating filtering of microscopic video from the perspective of image preprocessing. The principle of formatting sperm tracking trajectory and the rules of grading trajectory continuity were introduced firstly. Then, impacts of filtering algorithms on the continuity of sperm trajectory were investigated and the relation between the continuity of sperm trajectory and the fluctuation of sperm segmentation thresholds was further explored. Finally, an objective indicator for appraising the quality of filtering for microscopic video was established based on SADTM (Sum of Absolute Difference between Threshold and the Mean). The proposed indicator can objectively evaluate the quality of microscopic video filtering and provide a guideline for algorithm selection, laying foundation for quality control for CASA.