Reviewerly/exHarmony huggingface dataset:
Hybrid unsupervised paradigm based deformable image fusion for 4D CT lung image modality

Deformable image registration plays a critical role in various clinical applications (e.g., image fusion, atlas creation, and tumors targeting). In radiation therapy, especially in the context of fast registration of computed tomography (CT) lung image modalities, it is used to determine the geometric transformation by relating the anatomic points in two images. The main challenge lies in effectively addressing the nonlinear large and small deformation between the inspiration and expiration phases. In this work, we propose an unsupervised hybrid paradigm-based registration network (HPRN) for the registration of 4D CT lung images without relying on ground truth data. The proposed HPRN exhibits effective learning of multi-scale and multi-resolution features, leading to the computation of a more accurate Deformation Vector Field (DVF). Furthermore, we incorporate the regularization, image similarity and Jacobian determinant loss functions, which results in improving capability in dealing with complex large and small deformations. We evaluate the effectiveness of the proposed model on the publicly accessible DIRLab 4DCT lung image dataset, which shows the effectiveness of the proposed framework by achieving better Target Registration Error (2.04±1.42mm) compared to other state-of-the-art unsupervised image registration algorithms.
A versatile device for implementing the optical quantum gates in multiple degrees of freedom

Optical quantum logic gates are the basic elements for building optical quantum computer. Usually, to implement different logic gates, different optical elements and frameworks are needed. Here, we propose a versatile device that a modified Sagnac interferometer composed of a spatial light modulator (SLM) for implementing multiple optical gates within a classical light framework. We demonstrate that the device can manipulate the polarization and full-field spatial modes of photons to perform multi-qubit logic operations. And versatile conversions between different logic gates are achieved by simply altering the hologram imposed on SLM. Our device shows a good performance in exploring the photon's full degrees of freedom, which makes it an appropriate candidate for integrated optical computing.
Variational Quantum Linear Solver-based Combination Rules in Dempster–Shafer Theory

Dempster–Shafer Theory (DST), as a method of handling uncertain information, is widely used in decision-making and information fusion. But the issue of exponential computational complexity limits its real-time application. Since quantum computers with its natural parallel computing capability can theoretically achieve speedup, an HHL-based algorithm for implementing DST operations on quantum circuits has been proposed. However, the algorithm is impossible to be implemented with high accuracy due to the limited performance of quantum computers in the current noisy intermediate-scale quantum (NISQ) era. In response to that, we utilize an up-to-date scheme Variational Quantum Linear Solver (VQLS) to conduct DST operations by solving the linear system. Besides, VQLS further reveals the structural consistency of DST with quantum computation, which enables faster circuit configuration and computation. In this paper, we first realize the transformation in belief functions by VQLS and then extends it to a methods of implementing combination rules. Simulated under a realistic classification task, the feasibility and accuracy of the proposed VQLS-based method are verified. The new VQLS-based method proposed is able to achieve combination rules at a lower error level with fewer quantum resources, which is more suitable for the NISQ era.
Privacy-Preserving Neural Networks for Smart Manufacturing

Abstract The rapid advance in sensing technology has expedited data-driven innovation in manufacturing by enabling the collection of large amounts of data from factories. Big data provides an unprecedented opportunity for smart decision-making in the manufacturing process. However, big data also attracts cyberattacks and makes manufacturing systems vulnerable due to the inherent value of sensitive information. The increasing integration of artificial intelligence (AI) within smart factories also exposes manufacturing equipment susceptible to cyber threats, posing a critical risk to the integrity of smart manufacturing systems. Cyberattacks targeting manufacturing data can result in considerable financial losses and severe business disruption. Therefore, there is an urgent need to develop AI models that incorporate privacy-preserving methods to protect sensitive information implicit in the models against model inversion attacks. Hence, this paper presents the development of a new approach called mosaic neuron perturbation (MNP) to preserve latent information in the framework of the AI model, ensuring differential privacy requirements while mitigating the risk of model inversion attacks. MNP is flexible to implement into AI models, balancing the trade-off between model performance and robustness against cyberattacks while being highly scalable for large-scale computing. Experimental results, based on real-world manufacturing data collected from the computer numerical control (CNC) turning process, demonstrate that the proposed method significantly improves the ability to prevent inversion attacks while maintaining high prediction performance. The MNP method shows strong potential for making manufacturing systems both smart and secure by addressing the risk of data breaches while preserving the quality of AI models.
Hierarchical graph augmented stacked autoencoders for multi-view representation learning

With recent success of deep neural networks, stacked autoencoder networks have received a lot of attention for robust unsupervised representation learning. However, recent autoencoder methods cannot make full use of multi-view information and thus fail to further improve many real-world applications by exploring the geometric structures of multi-view data. To address the above-mentioned issue, we introduce hierarchical graph augmented stacked autoencoders (HGSAE) for unsupervised multi-view representation learning. Specifically, a hierarchical graph structure is first adapted to stacked autoencoders to learn view-specific representations, aiming to preserve the geometric information of multi-view data through local and non-local graph regularizations. A general or common representation can then be learned by reconstructing each single view using fully connected neural networks. By doing this, the proposed method not only preserves the geometric information in multi-view data but also automatically balances the complementarity/consistency among different views. Extensive experiments on six popular unsupervised representation learning datasets demonstrate the effectiveness of our method when compared with recent state-of-the-art autoencoder methods.
SSL-Net: Sparse semantic learning for identifying reliable correspondences

Feature matching aims to identify reliable correspondences between two sets of given initial feature points, which is of considerable importance to photogrammetry and computer vision. In this study, we propose an innovative sparse semantic learning-based network, named SSL-Net, for feature matching. Specifically, SSL-Net includes a novel sparsity constraint (SC) block, which builds a sparse graph for sparse semantic learning. The SC block adopts a region-to-whole learning strategy to measure the confidence of nodes in the sparse graph. It helps the sparse graph preserve the semantic information of positive influence while rejecting unnecessary ones, thereby suppressing the negative influence of incorrect correspondences. In addition, SSL-Net also includes a channel-spatial attention feature gathering block, which gathers features along the spatial direction and channel dimension of correspondences. To mitigate the existence of label ambiguity, we incorporate the accommodation factor into the loss function of SSL-Net for feature matching. As a result, our network outperforms the state-of-the-art method by a considerable margin. Notably, SSL-Net achieves a 9.05% improvement under an error threshold of 5° over the state-of-the-art method for the relative pose estimation task on the YFCC100M dataset. Our code will be available at https://github.com/guobaoxiao/SSL-Net.
Beyond <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" altimg="si135.svg" display="inline" id="d1e3713"><mml:mi>k</mml:mi></mml:math>-Means++: Towards better cluster exploration with geometrical information

Although k-means and its variants are known for their remarkable efficiency, they suffer from a strong dependence on the prior knowledge of K and the assumption of a circle-like pattern, which can result in the algorithms dividing the input space instead of discovering non-predetermined data patterns. Thus, we propose beyond k-means++ that infers and utilizes explicit clusters by emphasizing local geometrical information for better cluster exploration. To avoid the K dependence, a novel framework of iterative division and aggregation (IDA) over k-means++ is presented. It begins with any K≥1, then increases and reduces K along with the procedure of clusters' division and aggregation, respectively. To break through the circle-like pattern limitation, we introduce a reasonability checking strategy (RCS) for cluster division. Given local geometrical information, RCS achieves arbitrary cluster shape support by rejecting edge patterns with distinguished convergence direction and merging adjacent clusters with pseudo-edge patterns. Furthermore, we design an edge shrinkage strategy (ESS). Taking edge patterns as the cluster prototype, it benefits accuracy by effectively avoiding representability reduction due to irregular distribution. To compensate for the loss of efficiency, a near maximin and random sampling algorithm is suggested for large-scale data with high dimensionality. Experimental results confirm that beyond k-means++ is featured by handling arbitrary cluster shapes with remarkable accuracy.
Adaptive local Principal Component Analysis improves the clustering of high-dimensional data

In local Principal Component Analysis (PCA), a distribution is approximated by multiple units, each representing a local region by a hyper-ellipsoid obtained through PCA. We present an extension for local PCA which adaptively adjusts both the learning rate of each unit and the potential function which guides the competition between the local units. Our local PCA method is an online neural network method where unit centers and shapes are modified after the presentation of each data point. For several benchmark distributions, we demonstrate that our method improves the overall quality of clustering, especially for high-dimensional distributions where many conventional methods do not perform satisfactorily. Our online method is also well suited for the processing of streaming data: The two adaptive mechanisms lead to a quick reorganization of the clustering when the underlying distribution changes.
Holistic Dynamic Frequency Transformer for image fusion and exposure correction

The correction of exposure-related issues is a pivotal component in enhancing the quality of images, offering substantial implications for various computer vision tasks. Historically, most methodologies have predominantly utilized spatial domain recovery, offering limited consideration to the potentialities of the frequency domain. Additionally, there has been a lack of a unified perspective towards low-light enhancement, exposure correction, and multi-exposure fusion, complicating and impeding the optimization of image processing. In response to these challenges, this paper proposes a novel methodology that leverages the frequency domain to improve and unify the handling of exposure correction tasks. Our method introduces Holistic Frequency Attention and Dynamic Frequency Feed-Forward Network, which replace conventional correlation computation in the spatial-domain. They form a foundational building block that facilitates a U-shaped Holistic Dynamic Frequency Transformer as a filter to extract global information and dynamically select important frequency bands for image restoration. Complementing this, we employ a Laplacian pyramid to decompose images into distinct frequency bands, followed by multiple restorers, each tuned to recover specific frequency-band information. The pyramid fusion allows a more detailed and nuanced image restoration process. Ultimately, our structure unifies the three tasks of low-light enhancement, exposure correction, and multi-exposure fusion, enabling comprehensive treatment of all classical exposure errors. Benchmarking on mainstream datasets for these tasks, our proposed method achieves state-of-the-art results, paving the way for more sophisticated and unified solutions in exposure correction.
Learning Foreground Information Bottleneck for few-shot semantic segmentation

Few-shot semantic segmentation aims to segment unseen classes with only a few annotated samples, which has great values for the real-world application in the wild. However, since the target class is treated as the background in the training, the network tends to extract much irrelevant nuisance factors, which results in the feature undermining problem for the target class. Consequently, it is difficult to produce an accurate segmentation map. To address this problem, in this paper, we apply the information bottleneck theory to few-shot semantic segmentation and propose the Foreground Information Bottleneck (FIB) module. Based on the support information, FIB module filters out the irrelevant information and promotes the foreground-related feature paradigms. Meanwhile, to solve the intractable mutual information and enable the end-to-end optimization of FIB module, we derive the Foreground Information Bottleneck Loss (FIBLoss) according to the inherent attribute of few-shot segmentation. Moreover, since there exists severe noise interference in the wild, we design a Target Information Refinement (TIR) block to further exploit discriminative cues of foreground. TIR block calculates the pairwise interaction and exploits the detailed information of the foreground object, which is beneficial to the feature refinement. Extensive experiments on two challenging datasets reflect the proposed FIB module significantly improves the performance of few-shot segmentation and delivers the state-of-the-art results.
EHR-KnowGen: Knowledge-enhanced multimodal learning for disease diagnosis generation

Electronic health records (EHRs) contain diverse patient information, including medical notes, clinical events, and laboratory test results. Integrating this multimodal data can improve disease diagnoses using deep learning models. However, effectively combining different modalities for diagnosis remains challenging. Previous approaches, such as attention mechanisms and contrastive learning, have attempted to address this but do not fully integrate the modalities into a unified feature space. This paper presents EHR-KnowGen, a multimodal learning model enhanced with external domain knowledge, for improved disease diagnosis generation from diverse patient information in EHRs. Unlike previous approaches, our model integrates different modalities into a unified feature space with soft prompts learning and leverages large language models (LLMs) to generate disease diagnoses. By incorporating external domain knowledge from different levels of granularity, we enhance the extraction and fusion of multimodal information, resulting in more accurate diagnosis generation. Experimental results on real-world EHR datasets demonstrate the superiority of our generative model over comparative methods, providing explainable evidence to enhance the understanding of diagnosis results.
A Swarm Intelligence Assisted IoT-Based Activity Recognition System for Basketball Rookies

Recent years have witnessed many applications of wearable sensor technology and machine learning on smart sports training, e.g., basketball. For rookie players, the fundamental skills are very important and can be improved using a scientific auxiliary training system, the core of which is human activity recognition (HAR) technique. For basketball players, one complete action is usually performed in a moment with significant dynamics and high-frequency spikes or noise. So, extracting meaningful features from intricate sensor signals is one of the important prerequisites. Besides, to improve the recognition accuracy, it is challenging to empirically determine an ideal feature subset out of the extracted high-dimensional features for particular types of basketball activities, which is essentially an NP-hard optimization problem. To address the above issues, we propose a smart activity recognition system hybridizing an IoT-based edge-cloud system and a swarm intelligence-based feature selection algorithm, named Adaptive Binary Particle Swarm Optimization (ABPSO). The devices can automatically collect and process basketball players' action signals. A comprehensive feature set with 300 features that can cover and capture the characteristics of different activities is established. Then, ABPSO with global search ability is able to identify the optimal feature subset combined with classifiers for accurate activity recognition. Thanks to ABPSO, the size of machine learning model and the amount of transmitted data is largely reduced, which makes it well compatible for IoT-based applications. Experiments performed on nine human subjects demonstrate that ABPSO can achieve accuracy of 97.26% for five fundamental activities on average, which outperforms five state-of-the-art FS algorithms, two classic filter-based FS methods and one deep learning based method.
Sharing-Net: Lightweight feedforward network for skeleton-based action recognition based on information sharing mechanism

With the development of metaverse, augmented reality and human–robot teleoperation, action recognition plays an increasingly important role. In this work, we propose a Lightweight Feedforward Cross-channel Information Sharing Network (Sharing-Net) for action recognition. A multi-feature input module is constructed, which includes Cartesian Motion features, Global Joint Distances (GJD), and Global Joint Angles (GJA). The three types of features can tackle the problems of velocity differentiation, viewpoint diversification and object-distance variation, respectively. In order to take full use of the restricted parameters caused by the lightweight structure to enhance the accuracy under the premise of guaranteeing high speed, a multi-feature cross-channel information sharing mechanism is proposed. Dynamic nonlinear composite mapping between feature channels uses cross-channel residual blocks to share data information and establish coupling relationships. Extensive experiments on 3 public datasets and a self-built dataset verify the effectiveness of proposed methods. Compared with the state-of-the-art (SOAT) methods, Sharing-Net achieves the best accuracy with high speed on JHMDB and SHREC and performs superior balance of accuracy and computational cost on NTU RGB+D.
An analysis of ensemble pruning methods under the explanation of Random Forest

“Black box” models created by modern machine learning techniques are typically hard to interpret. Thus, the necessity of explainable artificial intelligence (XAI) has grown for understanding the rationale behind those models and converting them into white boxes. Random Forest is a black box model essential in various domains due to its flexibility, ease of use, and remarkable predictive performance. One method for explaining a Random Forest is transforming it into a self-explainable Decision Tree using Forest-Based Tree (FBT) algorithm. It basically consists of three main phases, pruning, conjunction set generation, and Decision Tree construction. In this paper, we examine six state-of-the-art pruning approaches and analyze their effect on FBT performance through pruned FBT (PFBT) in order to minimize its computational complexity. This would make it appropriate for forests and datasets of any size. They are assessed on 30 datasets, and the results show that UMEP and Hybrid pruning methods can be effectively used in the pruning stage of the PFBT algorithm in terms of pruning time and predictive performance. However, the AUC-Greedy method achieves good performance with small-size datasets.
High-precision phase shift method for heavy-load reference mirrors based on nano-precision grating sensor monitoring

In large-aperture interferometry, achieving high-precision phase shifting for heavy-load reference mirrors is challenging. This study introduces a method based on nano-precision grating sensor monitoring. The technique utilizes spatial three-point synchronous driving by piezoelectric ceramics, combined with flexible hinges, to enable nanoscale phase shifting of heavy-load reference mirrors. Additionally, it integrates in-situ PID closed-loop drive monitoring at these three points, using nano-precision grating sensors and piezoelectric ceramics. To ensure nanoscale resolved spatial translation, the mirrors are supported by air-bearings that counteract gravity, coupled with a phase shift error calibration model. The final realization of high-precision and high-stability mechanical phase shifting in large-aperture interferometry circumvents the principle defects of existing large-aperture wavelength-tuned phase shifting. The experiments show that the established mechanical phase shifting system with heavy-load reference mirror has a resolution of 1.5 nm, a frequency response of 116 Hz, and a synchronization accuracy of 2° for the three phase shifting quantities, and the method provides a new method and means of phase shifting for the construction of large-aperture phase shifting interferometer.
Temporal Action Segmentation: An Analysis of Modern Techniques

Temporal action segmentation (TAS) in videos aims at densely identifying video frames in minutes-long videos with multiple action classes. As a long-range video understanding task, researchers have developed an extended collection of methods and examined their performance using various benchmarks. Despite the rapid growth of TAS techniques in recent years, no systematic survey has been conducted in these sectors. This survey analyzes and summarizes the most significant contributions and trends. In particular, we first examine the task definition, common benchmarks, types of supervision, and prevalent evaluation measures. In addition, we systematically investigate two essential techniques of this topic, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , frame representation and temporal modeling, which have been studied extensively in the literature. We then conduct a thorough review of existing TAS works categorized by their levels of supervision and conclude our survey by identifying and emphasizing several research gaps. In addition, we have curated a list of TAS resources, which is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/nus-cvml/awesome-temporal-action-segmentation</uri> .