Reviewerly/exHarmony huggingface dataset:
Entity Relation Aware Graph Neural Ranking for Biomedical Information Retrieval

The performance of biomedical information retrieval greatly depends on biomedical knowledge; however the knowledge of available medical knowledge base is often incomplete and out-of-dated. To solve the problem that incomplete knowledge bases cannot provide the medical knowledge required for biomedical information retrieval, the paper proposes an Entity Relation Aware Graph Neural Ranking model (ERAGNR), aiming to fully leverage the internal knowledge of the document to alleviate the problem caused by incomplete external knowledge bases. ERAGNR mines the relationships between biomedical entities in the document through entity relation extraction and combines them with external knowledge. It increases the semantic association and reduces the semantic gap between the query and the document. The method first constructs a knowledge-query graph and a document-entity graph, and then fuses the two graphs to obtain a knowledge-query-document-entity graph. In a multi-task learning framework that combines text retrieval and relation extraction tasks, ERAGNR employs a shared text encoder and a graph neural network. This enables ERAGNR to learn semantic matching patterns between queries and documents and recognize relationships between entities in the documents. As a result, the model can capture semantic matching signals between entity relationships in the context and queries. The experimental results show that ERAGNR outperforms the state-of-the-art models. Through biomedical relation extraction task, the model can learn the ability to capture the context of the entity relations in the document, so that the model can more accurately match the semantics between the query and the document.
E-HMFNet: A Knowledge-Enhanced Hierarchical Molecular Representation Fusion Network for Drug Recommendation

Combinatorial drug recommendation involves recommending appropriate drug combinations for patients based on their complex health conditions, which is an essential task for AI in healthcare. However, existing approaches have several limitations. Firstly, they fail to fully utilize important information such as the hierarchical structure of drug molecules, patient visit history, and prior medical knowledge. Secondly, they ignore the inherent associations between these pieces of information and only encode one or two of them in isolation, leading to sub-optimal results. To address these issues, we propose KE-HMFNet, which leverages patient visit history, hierarchical molecular representation of drugs, and prior medical knowledge, and explicitly models their inherent association to make medication recommendations that are both effective and safe. Specifically, we develop a patient-guided fusion mechanism to make the hierarchical molecular representation disease-relevant and substructure-aware. Additionally, we design a knowledge-enhanced medication relation representation module to capture the inherent relation between drugs based on the patient’s condition. Extensive experiments on the MIMIC-III dataset demonstrate that our approach achieves new state-of-the-art performance <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Synergistically Learning Class-specific Tokens for Multi-class Whole Slide Image Classification

The application of transformer architecture in analyzing whole slide images (WSIs) has become increasingly popular due to its remarkable ability to learn complex associations. Nevertheless, a significant drawback emerges in the multiclass analysis of WSIs. The majority of the transformer-based methods available currently rely primarily on a single, class-agnostic token. This approach might not ideally capture the subtleties of class-discriminative information. To address this challenge, we present an innovative approach tailored for multi-class WSI analysis that harnesses the power of class-specific tokens. Central to our method is a novel attention mechanism designed to foster a synergistic learning relationship between patch and class tokens, enhancing the granularity of information captured and ensuring a more comprehensive representation of the WSI. Complementing this, we introduce a dynamic class-centric training strategy designed to optimize token representation learning, ensuring each token is informatively aligned with its corresponding class. Through extensive experimentation on three challenging multi-class WSI analysis datasets, our method consistently demonstrates superior performance, underscoring its potential as a robust solution for multi-class WSI analysis tasks.
CPSS-Net: A Cross-pseudo Semi-supervised Network for Liver Vessel Segmentation from CTA Images

Accurate segmentation of liver vessel from CTA images is often challenging due to the limited availability of labeled data. In the field of medical image segmentation, semi-supervised learning has garnered significant attention as it utilizes unlabeled data to enhance the training of segmentation models. In this paper, we propose a novel cross-pseudo semi-supervised network (CPSS-Net) based on nnU-Net. The CPSS-Net contains three innovative components: 1) An probability prediction (PP) module is designed to generate probability maps for both labeled and unlabeled datasets, capturing model uncertainty through parallel nnU-Net; 2) A double pseudo-label (DPL) module is used to convert the predicted probability maps into double soft pseudo-labels using an adaptive sharpening function; 3) A cross pseudo-supervised (CPS) module is introduced to learn the mutual consistency of double pseudo-labels. Test experiments on both public and private datasets show that our method achieved a Dice score of 0.67 and a sensitivity score of 0.69, surpassing the segmentation accuracy of existing related methods.
Employing Nudge Theory and Persuasive Principles with Explainable AI in Clinical Decision Support

Persuasive XAI, also known as Persuasive Explainable Artificial Intelligence, refers to the combination of two concepts: explainable artificial intelligence (XAI) and persuasive technology. Persuasive XAI can be applied in decision-support systems in the healthcare domain in various ways to enhance the perception of patients about the outcomes, encourage adherence to treatment plans, and improve overall patient or clinician engagement. In this paper, we aim to analyze state-of-the-art persuasion strategies used in XAI and discuss how to apply them in Clinical Decision Support Systems (CDSS) to support clinicians and patients in making healthcare-related decisions.
Addressing Sparse Annotation: a Novel Semantic Energy Loss for Tumor Cell Detection from Histopathologic Images

Tumor cell detection plays a vital role in immunohistochemistry (IHC) quantitative analysis. While recent remarkable developments in fully-supervised deep learning have greatly contributed to the efficiency of this task, the necessity for manually annotating all cells of specific detection types remains impractical. Obviously, if we directly use full supervision to train these datasets, it can cause error in loss calculation due to the misclassification of unannotated cells as background. To address this issue, we observe that although some cells are omitted during the annotation process, these unannotated cells have a significant feature similarity with the annotated ones. Leveraging this characteristic, we propose a novel calibrated loss named Semantic Energy Loss (SEL). Specifically, our SEL automatically adjusts the loss to be lower for unannotated regions with similar semantic to the labeled ones, while penalizing regions with lager semantic difference. Besides, to prevent all regions from having similar semantics during training, we propose Stretched Feature Loss (SFL) that widen the semantic distance. We evaluate our method on two different IHC datasets and achieve significant performance improvements in both sparse and exhaustive annotation scenarios. Furthermore, we also validate that our method holds significant potential for detecting multiple types of cells. Our code is available at here.
SRPA: ScribbleMatch and Reliable-Guided Pixel Alignment for Scribble-Supervised Medical Image Segmentation

Medical image segmentation is a critical task in the field of medical image analysis. Recently, there has been increasing attention on scribble-supervised medical image segmentation due to its simplicity for label generation. However, the performance of a scribble-based task highly relies on the quality of learning from inconsistent annotations and the classification of pixels in high-entropy regions. In this paper, we propose a novel framework called SRPA that combines both ScribbleMatch and Reliable-Guided Pixel Alignment to enhance the performance of the scribble-based task. The ScribbleMatch technique utilizes the pseudo label incorporated from two different weakly perturbed views of the same image to supervise a strongly perturbed view, which assists in boosting the quality of the shape information learning as the scribble-based task lacks the prototypes to consistently capture shape prior to model training. The Reliable-Guided Pixel Alignment technique employs reliable pixels selected by contrasting two weakly perturbed views, which serve as the standard for blurred pixels in strongly perturbed images to maximally align. This ensures the reliable classification of high-entropy pixels. Our method is evaluated on the public ACDC and MSCMRseg datasets, and the results demonstrate that our approach surpasses current scribble-supervised segmentation methods. Code will be available at https://github.com/RheinSXY/SRPA.
A Privacy Preserving Computer-aided Medical Diagnosis Framework with Outsourced Model

Computer-aided diagnosis plays an increasingly important role in modern medical activities, relying largely on the deployment of medical machine learning models. Protecting the security of model parameters is crucial for model providers. However, the current schemes for protecting model parameters are mostly interactive. This interactive nature makes it difficult to support offline deployment of models and flexible authorization of prediction results, thus hindering the widespread application of computer-aided diagnosis. To address these limitations, we propose a new computer-aided medical diagnosis framework by designing a new identity-based inner product functional proxy re-encryption (IB-IPFPRE) scheme. Our framework supports private deployment of medical diagnostic models without compromising model parameters. It also enables access control of prediction results based on user identity. Compared to existing privacy-preserving prediction techniques, our framework significantly reduces communication overhead and does not require the model owner to be online in real-time. Furthermore, our scheme enables flexible delegation of prediction results, allowing users to authorize the sharing of prediction results with other entities as needed. We conducted extensive experiments for logistic regression on three medical datasets. The experiments demonstrate that our scheme achieved 40% to 7× performance improvement in LAN environment and 13× to 15× improvement in WAN environment, and did not require any communication overhead during the privacy preserving prediction phase.
M2-DIA: Enhancing Diagnostic Capabilities in Imbalanced Disease Data using Multimodal Diagnostic Ensemble Framework

The ADS (Automatic Diagnosis System) has garnered increasing attention for its potential to aid clinicians and enhance healthcare accessibility for patients. These systems are designed to identify a wide range of diseases, including those that may be overlooked by human doctors. However, the diagnostic data used in ADS suffer from disease imbalances due to statistical and medical rare diseases, limiting the scope of diagnosable conditions and finally reducing the accuracy of disease identification. Previous research attempted to address this imbalance using knowledge injection networks during the training phase. However, this approach relies on a training process makes it challenging to diagnose diseases absent from the training data, even with the use of knowledge data. To address this problem, we introduce Multimodal Diagnostic Ensemble Framework called M2-DIA which employs multiple representations—Text, Statistics, and One-hot vectors—derived from a patient’s medical condition. This multi-faceted approach improves its diagnostic capabilities, especially for diseases that are not present in training data. We evaluated our framework on the public dataset with multiple test datasets to verify diagnostic ability from various angles and showed our approach surpasses existing works.
A Transformer-based Method for Recommending the Continuing Care Retirement Communities

A continuing care retirement community (CCRC) establishs large-scale, full-function and high-quality chain communities of medical and senior caring which combined physical and psycological feathures of the older adults by meeting multiple types of care to echo different levels of care needs for the older adults. In this paper, we propose a Transformer-based method for recommending the CCRCs. Our proposed Transformer-based method involves a 3-stages embedding layer and a self-attention layer. The first embedding stage is based on the care preference similarities of the older adults in the CCRCs. The second embedding stage is based on the promotion interaction preference similarities of the older adults out of the CCRCs. The thrid embedding stage is based on the social relationship similarities of the older adults. In self-attention layer, we multiply the length of query vector and that of key vector with the scaling factor of original Transformer to make the training process more stable.
MGFuseSeg: Attention-Guided Multi-Granularity Fusion for Medical Image Segmentation

Convolutional Neural Networks (CNNs) have been widely used in medical image segmentation to efficiently develop computer-aided diagnosis systems. Due to the locality of convolutional operations, they can be used to extract fine-grained features, but with limitations in building global context and long-range spatial relationships. Recently, shifted window-based multi-layer perceptron (Swin-MLP) methods have demonstrated the ability to learn coarse-grained spatial features in a fixed-size window, but they are not well suited to dense-prediction tasks, such as medical image segmentation. To harmonize the strengths and mitigate the weaknesses of CNNs and SwinMLP in extracting features of varying granularity, we proposed two novel granularity fusion modules that use coarse-grained features to guide the fusion of fine-grained features based on the attention mechanism. Specifically, the first fusion module, named as BGFuse (Block Granularity Fuse), could fuse various scale block-grained features from Swin-MLP. The second fusion module, termed as LGFuse (Local Granularity Fuse), could fuse semantic coarse granularity information into fine granularity features. Equipped with these two fusion modules, we present a new attention-guided encoder-decoder network architecture (termed MGFuseSeg) for medical image segmentation. Without bells and whistles, the proposed MGFuseSeg significantly boosts the performance on three challenging segmentation benchmarks including Synapse, ACDC, and ISIC. Codes are available at https://github.com/apple1986/MGFuseSeg
Counterfactual-Integrated Gradients: Counterfactual Feature Attribution for Medical Records

The consideration of methods for causal inference, such as Average Treatment Effects in medical analytics, offers valuable insights into the effects of interventions. However, assessing causal effects through intervention has inherent limitations, as it is not feasible to evaluate a patient in two states simultaneously. To overcome this, A/B testing with control and experiment groups is often utilized to analyze the average effects of interventions. However, A/B testing is often limited to relatively small cohorts. Counterfactual explanations, an Explainable AI (XAI) technique, provide a way to determine how changes to an instance can impact associated predictions, thus evaluating an individual in hypothetical states using existing data without the need for physical intervention. In this context, this work introduces a new counterfactual explanation technique and proposes metrics and a theoretical analysis to evaluate its properties, helping to advance the understanding and application of counterfactual explanations in XAI.
RHViT: A Robust Hierarchical Transformer for 3D Multimodal Brain Tumor Segmentation Using Biased Masked Image Modeling Pre-training

Accurate brain tumor segmentation in medical image analysis is crucial for diagnosis and treatment planning. While computer-aided methods have shown promise, several challenges persist. Most existing methods struggle with smaller tumors, treating all regions uniformly. Additionally, they lack robustness when dealing with data corruption and handling missing modalities, common in clinical settings. In this paper, we present a robust hierarchical vision transformer (RHViT) for 3D multimodal brain tumor segmentation, employing an encoder-decoder structure. Our approach combines 3D convolutions and self-attention, offering efficient and effective training. 3D convolutions help capture local information and generate hierarchical features, improving tumor segmentation accuracy. To enhance robustness, we pre-train the encoder using masked image modeling (MIM). This pre-training equips the model to handle data corruption, resulting in improved segmentation even in challenging scenarios. Furthermore, we introduce a novel biased masking strategy during MIM to focus the model's attention on tumor regions. This facilitates better tumor representations and effective fusion of multimodal features. Importantly, our biased masking technique strengthens the model's resilience when dealing with incomplete multimodal data during testing, making it a practical choice. Extensive experiments confirm the superiority of our model over existing approaches.
Semi-MedSeq: Semi-supervised Semantic Segmentation for Medical Image Sequences

In clinical practice, medical imaging techniques include 2D video-based examinations that capture sequential scans, and 3D volumetric imaging that forms a comprehensive 3D representation from a stack of 2D slices. The medical image sequences produced by the above techniques provide valuable spatio-temporal characteristics for analysis and segmentation, but the annotation of image sequences is extremely time-consuming and labor-intensive. To exploit the coherence and address the scarcity of labeled data, we propose a novel semi-supervised semantic segmentation framework for medical image sequences, which consists of a conditional network and a denoising network. Specifically, we embed a Sequential Feature Reconstruction module into both networks. This module reconstructs the target frame from contiguous frames and captures their shared visual features. Guided by the context-enhancing information from the conditioning network, the denoising network suppresses background noise via a Diffusion-based Noise Elimination module. Extensive experiments are conducted on 2D and 3D tasks, including cardiac segmentation, polyp segmentation, placenta vessel segmentation and abdomen multi-organ segmentation. The results show our method is superior to existing semi-supervised methods and exhibits advantages over fully-supervised medical image segmentation methods with only 1/2 labeled data, validating its effectiveness and generalization ability.
A Novel Diffusion-Model-Based Bone Scan Image Inpainting Algorithm

Inpainting degraded region (e.g., due to patient wearing jewelry) in bone scan images is effective for improve image quality, robustness of machine analysis, and accuracy of disease diagnosis. In this paper, a novel diffusion-model-based bone image inpainting model is proposed, which recover missing regions in bone scan images with a pre-trained denoising diffusion probabilistic models(DDPM). Experimental results show that the average PSNR and SSIM values are better than the existing methods. the improvement in disease diagnosis was validated by changes in the accuracy of the classification network.The proposed algorithm presents a promising solution for a better method to inpaint bone scan image missing region.
MLLCD: A Meta Learning-based Method for Lung Cancer Diagnosis Using Histopathology Images

Lung cancer is a leading cause of death. An accurate early lung cancer diagnosis can improve a patient’s survival chances. Histopathological images are essential for cancer diagnosis. With the development of deep learning in the past decade, many scholars have used deep learning to learn the features of histopathological images and achieve lung cancer classification. However, deep learning requires a large quantity of annotated data to train the model to achieve a good classification effect, and collecting many annotated pathological images is time-consuming and expensive. Faced with the scarcity of pathological data, we present a meta-learning method for lung cancer diagnosis (called MLLCD). In detail, the MLLCD works in three steps. First, we preprocess all data using the bilinear interpolation method and then design the base learner which units a convolutional neural network(CNN) and transformer to distill local features and global features of pathology images with different resolutions. Finally, we train and update the base learner with a model-agnostic meta-learning (MAML) algorithm. Clinical Proteomic Tumor Analysis Consortium (CPTAC) cancer patient data demonstrate that our proposed model achieves the receiver operating characteristic (ROC) values of 0.94 for lung cancer diagnosis.
CSIFNet: Deep multimodal network based on cellular spatial information fusion for survival prediction

Using multimodal data to construct models for cancer survival prediction is essential for the prognosis of patients. Most of existing methods do not consider the cellular spatial organization of tumors at the single-cell level. In this work, we present CSIFNet, a deep multimodal network based on cellular spatial information fusion. The multimodal data used includes genomics data, clinical variables data and cellular spatial data. Specifically, we construct a spatial information fusion module (SIFM) to learn the local interactions between tumor cells and the tumor microenvironment to obtain a more comprehensive high-level representation of tumor cellular spatial information. Experimental results show that our proposed method has better performance than the state-of-the-art survival prediction methods. The source code is available at https://github.com/heyhola/CSIFNet.
Weakly Supervised Myeloma Cells Segmentation based on Point Annotation

Multiple Myeloma (MM) is a growing global health concern, and early diagnosis is crucial for effective treatment. Efforts are underway to produce digital pathology tools with human-level intelligence that are efficient, scalable, accessible, and cost-effective. Microscopic images have high resolution, where cells are enormous and dense. Therefore, the annotation process is time-consuming and complex for tasks such as segmentation due to pixel-level marking. In this paper, we design an end-to-end weakly supervised myeloma cell segmentation framework based on point annotation. It can achieve accurate cell segmentation comparable to fully supervised methods while reducing the need for manual annotation, greatly shortening annotation time. Experimental results demonstrate that our method achieves 98% of its fully-supervised performance with only 10 annotated random points per instance, and outperforms the fully-supervised Mask RCNN.