Reviewerly/exHarmony huggingface dataset:
Ensemble of Unsupervised Learned Image Representations Based On Variational Autoencoders for Lung Adenocarcinoma Subtype Differentiation

Lung adenocarcinoma is a type of non-small cell lung cancer that accounts for about 40% of all lung cancers, which is divided into different molecular and histological subtypes associated with particular prognosis and treatment. Pathologists stratify for diagnosis mainly by its histo-morphological visual features and patterns, which tends to be challenging because of the nature of lung tissue, a mixture of histologically complex patterns and not having a specialized grading system. Here, an unsupervised computational approach based on an ensemble of tissue-specialized variational autoencoders, which were trained per histopathology subtype, to build an unsupervised embedded tissue-image representation. This representation was used to train a Random Forest classifier of three lung adenocarcinoma histology subtypes (lepidic, papillary and solid), and a 2D-visually interpretable projection from the learned embedded representation. Experimental results achieve an average F-score of 0.72 ± 0.05 in the test dataset and a well-separated 2D visual mapping of tissue subtypes.
Rescon-Net: Transfer Learning Based Nested Model for Cerebrovascular Segmentation with Aneurysm

In the field of cerebral vascular segmentation, existing models have achieved excellent results in normal subjects. However, such models may not be satisfactory in cerebral vessels with aneurysms or malformation in emergency cases, particularly, when the vessels are small, thus resulting in poor segmentation results. We propose a transfer learning strategy based pre-trained model termed ReSCon-Net, using publicly available dataset for cerebrovascular segmentation with aneurysms or malformation in emergency cases. In details, the transfer learning strategy takes advantage of pre-trained models using a large volume of dataset from normal subjects and provides a solution for relatively small clinical datasets of patients. This can be achieved by fine-tuning the parameters in the last layers of the proposed model termed ReSCon-Net, which consists of three blocks: ResMul, DeRes, and REAM. The first two blocks aim to address inadequate extraction of weak vessels by using different convolution sizes to achieve multiscale features; REAM block can enhance the vessel edges by obtaining the probability of a voxel weight corresponding to reverse.
MDT-Net: Multi-Domain Transfer by Perceptual Supervision for Unpaired Images in OCT Scan

Deep learning models tend to underperform in the presence of domain shifts. Domain transfer has recently emerged as a promising approach wherein images exhibiting a domain shift are transformed into other domains for augmentation or adaptation. However, with the absence of paired and annotated images, models merely learned by adversarial loss and cycle consistency loss could result in poor consistency of anatomy structures during the translation. Additionally, the complexity of learning multi-domain transfer could significantly increase with the number of target domains and source images. In this paper, we propose a multi-domain transfer network, named MDT-Net, to address the limitations above through perceptual supervision. Specifically, our model consists of a single encoder-decoder network and multiple domain-specific transfer modules to disentangle feature representations of the anatomy content and domain variance. Owing to this architecture, the model could significantly reduce the complexity when the translation is conducted among multiple domains. To demonstrate the performance of our method, we evaluate our model qualitatively and quantitatively on RETOUCH, an OCT dataset comprising scans from three different scanner devices (domains). Furthermore, we take the transfer results as additional training data for fluid segmentation to prove the advantage of our model indirectly, i.e., in the task of data adaptation and augmentation. Experimental results show that our method could bring universal improvement in these segmentation tasks, which demonstrates the effectiveness and efficiency of MDT-Net in multi-domain transfer.
Deep Learning Method for Probabilistic Particle Detection and Tracking in Fluorescence Microscopy Images

Analyzing particles in fluorescence microscopy images is important to obtain insights into viral and cellular processes. We introduce a deep learning method for probabilistic detection and tracking of fluorescent particles. For particle detection, we integrate temporal information for regressing a density map and determine sub-pixel particle positions. Detections close to particles are rewarded during training and highly nonlinear direct regression of positions is avoided. For tracking, we introduce a fully Bayesian neural network that emulates classical Bayesian filtering and exploits both aleatoric and epistemic uncertainty. The method considers uncertainty information of individual particle detections. Experiments based on the Particle Tracking Challenge data show that the proposed method outperforms previous methods. We also applied the method to fluorescence microscopy images of hepatitis C virus proteins.
Training Cross-Modality Cerebrovascular Segmentation Networks With Paired Images

Automatic cerebral vessel segmentation from Computed Tomography Angiography (CTA) and Magnetic Resonance Angiography (MRA) is an important task in clinical diagnosis of cranial vascular diseases. With the success of deep convolution neural network, cerebrovascular segmentation has made great progress. But in most of previous work, training the segmentation network in different modalities usually requires a large amount of data to be labeled for each modality, which is expensive and time-consuming. In this paper, we proposed a framework to realize cross-modality cerebrovascular segmentation between CTA and MRA with paired data. We first train the source domain segmentation network, through which the initial segmentation results of the target domain images can be obtained. Then pseudo-label of the target domain is generated by registering the initial segmentation of the target domain with the paired source domain label, and it is used to train the target domain segmentation network. Experiments demonstrate the effectiveness of our proposed framework in cross-modality segmentation.
Self-Supervised High-Dimentional Magnetic Resonance Image Denoising Using Super-Resolved Single Noisy Image

Denoising of magnetic resonance image (MRI) is a critical step in MRI image processing and analysis. With the advantage of not requiring paired noisy-clean images for training, self-supervised denoising methods are emerging as competitive alternatives to supervised denoising methods in MRI denoising. However, current self-supervised image denoising methods are not effective enough for MRI. In this work, we propose Noise2SR-M (N2SR-M), a self supervised denoising method for MR images, which is more efficient for high-dimensional MR images. N2SR-M is designed for training with paired noisy data of different sizes divided from a single high-dimensional noisy input image. Our N2SR-M model is able to utilize the redundant information from the additional image dimension to generate noisy image pairs for the denoising task. With the combination of additional dimension constraint and the effectiveness of SR method based training image pair generation, our model is more efficient for denoising high-dimensional MR images. The quantitative and qualitative improvements in blood oxygenation level dependent (BOLD) imaging denoising task demonstrate that N2SR-M successfully restores detailed image contents and removes tiny structural noise and artifacts from noise-corrupted high-dimensional MRI. Moreover, the denoised BOLD image also induces more efficient R2* image computation.
Multi-Scale Structural-aware Exposure Correction for Endoscopic Imaging

Endoscopy is the most widely imaging technique used for the diagnosis of cancerous lesions in hollow organs. However, endoscopic images are often affected by illumination artefacts: image parts may be over- or underexposed according to the light source pose and the tissue orientation. These artifacts have a strong negative impact on the performance of computer vision or artificial intelligence based diagnosis tools. Although endoscopic image enhancement methods are greatly required, little effort has been devoted to over- and underexposure enhancement in real-time. This contribution presents an extension to the objective function of LMSPEC, a method originally introduced to enhance images from natural scenes. It is used here for the exposure correction in endoscopic imaging and the preservation of structural information. To the best of our knowledge, this contribution is the first one that addresses the enhancement of both types of endoscopic artefacts (under- and over-exposure) using deep learning methods. Tested on the Endo4IE dataset, the proposed implementation has yielded a significant improvement over LMSPEC reaching a SSIM increase of 4.40% and 4.21% for over- and underexposed images, respectively.
Transformer-Based Multimodal Fusion for Survival Prediction by Integrating Whole Slide Images, Clinical, and Genomic Data

Survival prediction using whole slide images (WSIs) is a complex and difficult task, as handling gigapixel WSI directly is computationally impossible. In the past few years, people have worked out multiple instance learning (MIL) strategies to deal with WSIs, i.e., splitting WSI into many patches (instances) and aggregating features across patches. Moreover, to better predict the survival outcome of patients, different modalities have been explored, among which gene features are used the most frequently. In this paper, we explore a graph-based strategy to handle WSIs and investigate a transformer-based strategy to combine different modalities for survival prediction. Moreover, clinical data was also adopted and different encoding manners of clinical information were explored. Experiments on two public datasets from The Cancer Genome Atlas (TCGA) demonstrate the effectiveness of the proposed graph-transformer framework for survival prediction.
SC-GAN: Structure Consistent GAN for Modality Transfer with FFT and Multi-Scale Perception

The quality of the cornea endothelial microscopy image is critical for clinical analysis. Although the noncontact specular microscope is more user-friendly than the contact confocal microscope, the imaging quality of the specular microscope is lower. The modality transfer is a promising solution for image quality enhancement. This paper proposes a Structure Consistent Generative Adversarial Network (SC-GAN) to transfer the imaging style from the specular microscope to the confocal microscope. Specifically, we use the Fourier frequency domain consistency to preserve cell structure and propose a multi-scale perception discriminator to improve model robustness under cell size variation. Experiment results prove the effectiveness of our method.
Evidence Dictionary Network Using Multiple Instance Contrastive Learning for Explainable Pathological Image Analysis

In this study, we aim to construct a pathological image diagnosis support system that can perform accurate diagnoses and enable users to easily interpret the basis for its decisions. To this end, we propose an explainable neural network that can provide the basis for its decisions from a dictionary constructed by the network itself. The network explains the basis by retrieving images from its dictionary similar to local regions in an input image. The network learns histological patterns such as pathological findings that contributed to the decisions and constructs its dictionary from the representative patterns via multiple instance learning (MIL) and contrastive learning. MIL enables the network to learn histological patterns without pixel-wise annotations, whereas contrastive learning makes the dictionary items consistent for interpretability. We perform pathological whole-slide image classification using a lung biopsy dataset, and the results show that interpretable dictionary items could be learned. Further, based on the results obtained, we establish that our method is more diagnostically accurate than the existing methods.
Background Subtraction Approach to Unsupervised Cell Segmentation: Toward Excluding Spurious Features in Degraded Cytology Slides

Automation and augmentation of the cytological diagnosis by data-driven techniques require a large number of microscopic slide images. The slide images with different imaging and storage conditions may contain latent image features that are spuriously correlated with cell classes. This paper focuses on the fact that image features other than cells can induce inference in deep learning of cytology images. To address this problem without the cost of cell annotation, this paper proposes a novel image processing method that subtracts from the whole image the background pattern discolored by the degradation of the slides. In the proposed method, a deep neural network model learns to separate sparse cells from the background in an unsupervised manner. To demonstrate the capability of the proposed method to exclude the background features, it is experimentally shown that it achieves cell detection with a comparable performance to that of supervised methods.
Fusion Contours as a Source of Segmentation Training Data: A Simulation Study

Consensus contours are often used to reduce annotator error in segmentation data. Here, we investigate the use of multi-annotator labels, in simulated data, for training auto-segmentation models. We compare convolutional neural networks (CNNs) trained on all observer annotations, STAPLE and majority voting estimates. We generate annotation sets by simulating observer delineations with varying noise and bias. By altering the quantity of observers and their relative biases, we investigate the impact of bias on CNN performance.We find that models trained on STAPLE contours are significantly worse when presented with biased annotations. CNNs trained on all annotations performed the best and were able to implicitly account for biased annotations in the training set.
A Knowledge Distillation Framework for Multi-Organ Segmentation of Medaka Fish in Tomographic Image

Morphological atlases are an important tool in organismal studies, and modern high-throughput Computed Tomography (CT) facilities can produce hundreds of full-body high-resolution volumetric images of organisms. However, creating an atlas from these volumes requires accurate organ segmentation. In the last decade, machine learning approaches have achieved incredible results in image segmentation tasks, but they require large amounts of annotated data for training. In this paper, we propose a self-training framework for multi-organ segmentation in tomographic images of Medaka fish. We utilize the pseudo-labeled data from a pretrained Teacher model and adopt a Quality Classifier to refine the pseudo-labeled data. Then, we introduce a pixel-wise knowledge distillation method to prevent overfitting to the pseudo-labeled data and improve the segmentation performance. The experimental results demonstrate that our method improves mean Intersection over Union (IoU) by 5.9% on the full dataset and enables keeping the quality while using three times less markup.
Quantitative Evaluation of the Response to Neo-Adjuvant Chemotherapy based on Non-Rigid Registration of CEM

This paper introduces a new intensity-compensated longitudinal non-rigid registration method for Contrast Enhanced Mammography monitoring neoadjuvant chemotherapy (NAC). The registered NAC image subtraction, called residual fields, allows to discriminate between different lesion responses to treatment using features extraction. The approach registers low-energy 2D images of the exams acquired before and after the chemotherapy. The measured motion is then applied to the corresponding dual-energy recombined images. Consequently, the difference in registered images allows identifying local density and iodine uptake changes, especially in the lesion area. The registration converged for all 51 patients with 208 image pairs. Finally, the residual fields can be used to extract clinically relevant features used to assess the response of the treatment. The lesion response classification model, evaluated with the AUC and F-score, performed better when taking into account the residual features (AUC: 0.92 vs 0.79).
Prediction of ICU Readmission Using LightGBM Classifier

Decision to discharge a patient from an intensive care unit (ICU) is difficult, with a risk of prolonging unnecessarily the stay and a risk of ICU readmission, which is associated with adverse outcomes. We propose to use light gradient boosting (lightGBM) classifier to build decision-making systems to predict which patients are most likely to experience ICU readmission within the first three days. The classifier was developed and tested using MIMICIII database. We extracted many clinical data from the electronic health records (EHR) stored in MIMICIII database. Then, several statistical, temporal and spectral features were extracted. In addition, some feature selection methods were used to select the most informative data. The performance of our LightGBM classifier is superior to the previously tested machine learning (ML) classifiers, as evidenced by an area under the curve (AUC) of 78.6%. The development of our prediction model for ICU readmission offers a more accurate approach to identifying patients at high risk, potentially reducing readmission rates and improving patient outcomes.
Div-Attention: A Plug-and-Play Module for 3D Medical Image Segmentation

In medical image segmentation, fusion of multi-scale feature maps using skip-connection is proven to be an effective technique for performance boosting. However, one feature map is sufficient to make a prediction about the segmentation result, and there exists divergence between predictions made by feature maps of different scales. To better utilize feature maps of different scales, in this paper, we propose a plug-and-play module named div-attention. This module can be applied to any deep neural networks with skip-connections. It is placed at the junction between adjacent paths of skip-connections to re-segment the area where divergence exists and to fuse results for segmentation performance improvement, which means that this module is divergence-aware and is proposed for better utilizing this disagreement between predictions made by feature maps of diverse scales within one model. Experiment results show that this plug-and-play module can significantly boost segmentation performance in different deep neural networks and in diverse CT segmentation tasks.
Contrastive Learning of Equivariant Image Representations for Multimodal Deformable Registration

We propose a method for multimodal deformable image registration which combines a powerful deep learning approach to generate CoMIRs, dense image-like representations of multimodal image pairs, with INSPIRE, a robust framework for monomodal deformable image registration. We introduce new equivariance constraints to improve the consistency of CoMIRs under deformation. We evaluate the method on three publicly available multimodal datasets: remote sensing, histological, and cytological. The proposed method demonstrates general applicability and consistently outperforms reference registration tools elastix and VoxelMorph. We share source code of the proposed method and complete experimental setup as open-source at: https://github.com/MIDA-group/CoMIR_INSPIRE.
Evidence-Empowered Transfer Learning for Alzheimer’s Disease

Transfer learning has been widely utilized to mitigate the data scarcity problem in the field of Alzheimer’s disease (AD). Conventional transfer learning relies on re-using models trained on AD-irrelevant tasks such as natural image classification. However, it often leads to negative transfer due to the discrepancy between the non-medical source and target medical domains. To address this, we present evidence-empowered transfer learning for AD diagnosis. Unlike conventional approaches, we leverage an AD-relevant auxiliary task, namely morphological change prediction, without requiring additional MRI data. In this auxiliary task, the diagnosis model learns the evidential and transferable knowledge from morphological features in MRI scans. Experimental results demonstrate that our framework is not only effective in improving detection performance regardless of model capacity, but also more data-efficient and faithful.
Robust X-ray Image Stitching Algorithm Based on Refining Matching Results of Feature Descriptors

Panoramic X-ray image provides a convenient way for some orthopedic clinical diagnosis and preoperative planning, while the full body cannot be captured in a single X-ray scan. The classic refining algorithm RANSAC failed to identify the outliers (mismatched points) according to the low signal-to-noise ratio of the matching results of vanilla feature descriptors. The methods based on deep learning based are also difficult to carry out due to the lack of training data pairs clinically. This paper proposes a robust algorithm for X-ray image stitching based on refining the matching results of vanilla feature descriptors. The algorithm transforms the image stitching task into the clique problem in the graph theory owing to the scale consistency of the X-ray images, and the dynamic programming algorithm is used to accelerate the searching process for the maximal clique. 60 clinical X-ray image pairs from different parts of the human body are randomly collected to evaluate our proposed algorithm. The experiment results show the strong robustness throughout different stitching tasks of X-ray images and achieve the best performance among the frequently-used stitching software.
Scale-Aware Multi-Instance Learning for Early Prognosis of Subjects at Risk of Developing Hepatocellular Carcinoma

Hepatocellular carcinoma (HCC) is a type of primary liver cancer which can potentially lead to the subject’s death. The subject usually experiences the symptoms when the HCC has grown significantly and needs advanced treatment such as surgery and extensive chemotherapy. Thus it would be beneficial to classify subjects at risk of developing HCC based on non-invasive CT scans. In this work, we propose a novel method for pre-cancer stage classification. To our knowledge, we are the first to propose a Deep Learning based method to classify pre-cancer subjects based on CT images. Our hypothesis is that the subjects at risk of developing HCC may have grown different scales/levels of visual cues on the CT scans. We propose a scale-aware method to facilitate the prediction of a free of HCC subject to develop HCC without additional annotation cost. We show the efficacy of our method on a dataset of 60 subjects. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
SAA-NET: A Medical Image Segmentation Framework Based On Stream-Across Attention

Deep convolutional networks are widely applied in multiple tasks in medical image analysis. Although most convolutional neural network (CNN)-based methods have strong feature representation ability, they are hindered by lousy encoding of long-range interaction information. To alleviate this problem, we propose a novel U-shaped image segmentation network with the Stream-Across Attention module, termed as SAA-Net. Specifically, SAA module contains two types of attention branches, the multi-scale spatial and channel attention branches. Our spatial attention branch attempts to model long-range dependencies and capture local information at the same time by using kernels with different sizes. Channel attention branch can enhance channel information by selectively aggregating the features among all channel maps. In addition, to avoid negative impacts of network deepening, we propose DenseMLP to learn multi-level semantic features. Experimental results show that SAA-Net is competitive in contrast to current state-of-the-art (SOTA) methods on two public datasets.