Reviewerly/exHarmony huggingface dataset:
Class-specific Prompts in Vision Transformer for Continual Learning of New Diseases

Current intelligent diagnosis systems are often trained to diagnose a small number of diseases and lack the ability of continually learning new disease knowledge. To have such continual learning ability, the deployed intelligent system needs to be continually updated based on training data of only new diseases, without accessing old data of previously learned diseases due to privacy concerns and challenges in data sharing across medical centers. In this study, a novel and effective prompt learning strategy is proposed to help a pretrained and fixed Vision Transformer (ViT) continually learn new diseases. In particular, a set of unique prompts for each disease are effectively learned such that discriminative disease features can be well extracted from the fixed ViT under the instructions of prompts during feature extraction, even though the ViT feature extractor is pretrained in the natural image domain. Extensive empirical evaluations on two medical image datasets and one natural image dataset demonstrate the superior performance of the proposed method. The source code is available at https://github.com/zhaoedf/CSPrompt.
Enhancing Medical Report Generation in Multi-Slice Fusion Scenarios

Automatic medical report generation has garnered considerable research attention owing to its practical significance in alleviating the workload burden on radiologists. Despite the promise shown by existing methods, limitations still persist due to their tendency to overlook the multi-view or high-resolution characteristics of medical images, which contain a wealth of rich and diverse information. To tackle this issue, we propose a Multi-Slice Fusion report generation framework (ReFuGen). In detail, ReFuGen utilizes a Multi-slice Feature Extractor to extract features from multiple image slices. Then, we introduce the Exemplary Slice Amplifier, which adaptively identifies important slices, applies weighting based on their contribution, and generates enhanced features by integrating spatial features from all slices. Next, the Adaptive Receptive Field Seq-Enhancer is proposed to enhance feature interactions across different channels by adaptively using a series of 1D convolutions. Lastly, we introduce the Text Knowledge Integrator, which integrates textual knowledge to address the issue of sparse features in medical images. Experimentally, we outperform state-of-the-art methods on three widely used benchmark datasets: IU X-Ray, MIMIC-CXR and FFA-IR.
Selecting Heterogeneous Features Based on Unified Density-Guided Neighborhood Relation for Complex Biomedical Data Analysis

Biomedical big data are usually high dimensional and collected in the form of a continuous influx of new features. Online Feature Selection (OFS) is a promising way to manage and analyze such data, as OFS circumvents the huge computation cost brought by simultaneously considering all the features, and can also dynamically maintain a distribution-fitting feature subset on the fly. However, almost all the OFS solutions are based on a naive premise that all features are of the same type, overlooking the fact that real biomedical data set usually consists of heterogeneous numerical and categorical features. This paper therefore proposes a new approach to Online Heterogeneous Feature Selection (OHFS), which dynamically maintains a feature subset that maximizes the number of neighborhood sets where all the objects within each neighborhood set are of the same class. To appropriately partition the objects into neighborhood sets, a density-guided relation is proposed, which adaptively forms non-overlapping neighborhood sets by detecting spatially compact objects. A unified density measure is also presented to avoid information loss in processing heterogeneous features. It turns out that the proposed approach features parameter- free, interpretability, and efficiency. It is capable of maintaining a concise feature subset while receiving any type of feature. Extensive experimental evaluations demonstrate its superiority.
Exploiting Behavior Multiplicity for Social Recommendation Via Graph Neural Networks

In recent years, researchers have attempted to use multi-behavior interaction information and social information to alleviate the data sparsity problem in collaborative filtering. However, existing research studies social recommendation and multi-behavior recommendation separately, making them insufficient in capturing heterogeneous coordination signals from both users and items due to the neglect of cross-user, cross-item interdependent knowledge. In this work, we propose Multi-Behavior Social Recommendation based Graph Neural Networks (MBSR-GNN), which incorporates social information and item knowledge to learn user preferences and item representations. We design a multi-layer cross-domain heterogeneous graph neural network to aggregate feature embeddings of users and items from different types of behavioral interactions, social connections, and item dependencies. Extensive experiments on three real-world datasets demonstrate that MBSR-GNN achieves consistent improvements over multiple state-of-the-arts. Specifically, compared to the best baseline, it achieves a 12.23% improvement in HR@10 and a 22.34% improvement in NDCG@10.
Facial Emotion Recognition using Landmark coordinate features

This study introduces a novel Facial Emotion Recognition (FER) algorithm designed for uncontrolled, or "in the wild," environments. Traditional image-based FER systems face challenges in these settings due to variations in pose, occlusion, lighting, and skin tones. Our approach overcomes these limitations by extracting and utilizing 3D facial landmarks instead of relying directly on image-based features. We employ the MediaPipe FaceMesh model to extract 478 normalized facial landmarks from the FER+ dataset, which includes images labeled with eight emotions. These landmarks define 2556 face tessellations, serving as embedding features for a transformer-based network. The algorithm's unique aspect lies in its ability to normalize images across different conditions and cameras, using a consistent set of features derived from 3D landmarks. This normalization enables the integration of diverse FER datasets for training, enhancing the algorithm’s applicability across various devices. Achieving 73.7% accuracy on the FER+ dataset, the algorithm demonstrates significant promise in emotion classification. Its adaptive attention mechanism focuses on well-represented landmarks, accounting for the face's angle and aspects, rather than solely on the emotion itself. This study marks a significant advancement in FER technology, offering a robust solution for emotion recognition in real-world and varied conditions.
Topic-BiGRU-U-Net for Document-level Relation Extraction from Biomedical Literature

Document-level biomedical relation extraction refers to extract relationship facts from unstructured biomedical literature. Due to the fact that many relationship facts span multiple sentences and involve complex interactions between entities, it requires models with strong logical reasoning capabilities. However, previous methods have had limitations in terms of logical reasoning, such as lack logical reasoning and the entities merely based on a single granularity. Although other efforts are based on multiple granularity, including mention pairs and entity pairs, the inference mechanism is weak. In this paper, we propose a module to enhance the model’s logical reasoning capabilities from four kinds of granularity (mention, entity, mention pairs, and entity pairs). Specifically, we encode the entities and mentions separately using BiGRU to capture contextual features. Simultaneously, we employ a U-Net network to model entity pairs and mention pairs, and enhance inter-sentence reasoning abilities. Additionally, to support the model’s reasoning capabilities, we introduce topic nodes into the traditional heterogeneous graph to fully extract document information. Experimental results on CDR and GDA datasets in the biomedical field demonstrate the outstanding performance of the proposed model.
Research on Common Structure of Motion Data

The movement process of organisms contains a large amount of information, which leads to different structure of motion data in different applications. How to construct a data structure to contain more common information of motion is a topic that researchers have been exploring. It is essential to set a series of criteria for the motion data structure to judge whether it can be a good representation of campaign information and fast and efficient readed/writen and calculated by computers before construct it. In this paper, we proposes 12 construction criteria for motion data structures through analysing the main current application scenarios of motion data, the way computer process data and the requirements of deep learning on data. We design a common structure of motion data CSMD based on these 12 criteria, which can be adapted to almost all common application. Our method is 78% faster than traditional methods. We also build a toolkit to process common motion data including C3D, ASF/AMC, BVH, CSMD, etc.
Use Predictive Learning Model to Tackle Data Breaches in Healthcare Domain

Healthcare data breaches are a growing problem that seriously threatens patient privacy, the reputation and trustworthiness of both public and private healthcare organizations. The aim of this paper is to elucidate the severity of healthcare data breaches, their potential impact on patient privacy and healthcare organizations, providing a predictive data breaches transformer conceptual architecture that can monitoring users actions that can result in possible system security violation and consequently in data breaches. At this regard, we introduce the description of a concept architecture for implementing a predictive data breaches transformer highlighting weakness and strengthens, and in the same time how the adoption of a predictive system can significantly limit the risk of the onset of possible data breaches by making the operator more aware in carrying out his activity in the processing of each type of data including personal health data.
DCA: Densely Cross-scale Attention Network for Anatomically-plausible Medical Image Segmentation

Deep learning models applied to medical image segmentation have achieved remarkable performance across various tasks. Despite their high accuracy, these models may produce predictions that clinicians deem anatomically implausible. This limitation arises due to the inherent anatomical variability and indistinct boundaries present in clinical segmentation targets, impeding the effectiveness of existing research efforts. To overcome these challenges, we propose an innovative architectural framework known as Densely Cross-scale Attention (DCA) network. This framework efficiently incorporates multi-scale local and global attention dependencies using dense connections, resulting in anatomically-plausible segmentations. DCA comprises three new modules: 1) a multi-level feature aggregation module that merges multi-scale local features to acquire feature representations with diverse granularities, thereby compensating for information loss caused by gradient dispersion; 2) a cross-scale attention module that facilitates the modeling of long-range dependencies across scales while effectively diminishing task-irrelevant noise; and 3) a multi-scale self-attention module that captures the multi-scale global spatial relationships of single-scale local features, further enhancing the model’s resilience in addressing complex lesions. The proposed efficient dense connections offer a versatile and potent methodology for modeling long-range dependencies across scales in any segmentation network. The efficacy of our approach is demonstrated through experiments conducted on two clinically relevant datasets, substantiating its ability to achieve superior segmentation accuracy and anatomical plausibility.
Decoupling MIL Transformer-based Network for Weakly Supervised Polyp Detection

Colonoscopy has emerged as a crucial examination for early colorectal cancer (CRC) diagnosis. Early detection of polyps can significantly enhance the survival rate of colorectal cancer. Most recent weakly supervised methods for detecting polyps are based on multiple instance learning (MIL), which employs labeled training data at the video-level (bag-level) to identify polyps at the frame-level (instance-level). However, existing methods often use the same features without considering the differences of video and snippet. Video classification usually focuses more on global features, while snippet classification relies more on leveraging multi-granularity detail information. This paper proposes decoupling the MIL network into the feature encoder and instance decoder. Furthermore, we introduce a novel Snippet-wise Cross Fusion Attention (SCA) that captures rich temporal context semantic features for instance classification. Additionally, our approach incorporates a parameter-efficient finetuning architecture called convolutional adapters, which aims to enhance the training process stability and improve the model's performance. Experimental results demonstrate consistent improvements over state-of-the-art methods on a newly introduced large-scale colonoscopy video dataset by a considerable 7.9% AUC and 1.16% AP. Our code and dataset will be made publicly available at: https://github.com/kanydao/Decoupling-MIL.
Coarse-to-fine Knowledge Graph Domain Adaptation based on Distantly-supervised Iterative Training

The knowledge graph (KG) is a highly needed basis to support the high-fidelity and high-interpretability modeling of various tasks in healthcare artificial intelligence. In this work, we focus on constructing an oncology knowledge graph that will be used in downstream cancer research and solution development. Modern supervised learning for knowledge graph construction requires a large amount of manually labeled data, which makes the process time-consuming and labor-intensive. Although there exists multiple research on named entity recognition and relation extraction based on distantly supervised learning, constructing a domain-specific knowledge graph from large collections of textual data without manual annotations is still an urgent problem to be solved. In response, we propose an integrated framework for adapting and re-learning knowledge graphs from a general domain (biomedical in our case) to a fine-defined domain (oncology). In this framework, we apply distant-supervision on cross-domain knowledge graph adaptation. Consequently, no manual data annotation is required to train the model. We introduce a novel iterative training strategy to facilitate the discovery of domain-specific named entities and triplets. Experimental results indicate that the proposed framework can perform domain adaptation and construction of knowledge graphs efficiently.
Trans-UNeter: A new Decoder of TransUNet for Medical Image Segmentation

Recently, how to integrate convolutional neural networks and transformers into a U-Net-like encoder-decoder structure has drawn growing interest in medical image segmentation, as transformer is more efficient in capturing longrange relations. Following this line of research, TransUNet is one representative work. However, it still insufficiently explores the rich relations of features from the encoder layer and the decoder layer with just a simple concatenation, which weakens their effectiveness to some extent. To address this issue, we propose two important design improvements to strengthen TransUNet: 1) a novel skip connection module, which upsamples the high-level semantic features and fuse it with low-level features, producing comprehensive semantic-aware features for the decoder. 2) an improved decoder network cascades reverse attention and spatial attention to adaptively combines features from the corresponding encoder layer and the previously decoded outputs.The results of the abdominal multi-organ segmentation experiment on the Synapse multi-organ segmentation dataset indicated that Trans-UNeter improved the mean similarity coefficient(DSC) by 3.71% compared to TransUNet. Code and models are available at https://github.com/iaoqin/Trans-UNeter.
A Practical Approach to Disease Risk Prediction: Focus on High-Risk Patients via Highest-k Loss

Disease risk prediction models play an important role in preventing disease developments in modern healthcare. However, the lack of focus on high-risk patients has hindered the large-scale practical application of these models, especially considering the limitation of medical resources available for following up on patients who are deemed high-risk. In this study, we propose a novel and practical approach that focuses on minimizing the number of false positive observations among high-risk patients by introducing the Highest-k Loss. The solution is to estimate the weights of the highest k scores with a differentiable estimation of the sorting operation and apply the weights to the loss function. We extracted 253,680 survey responses from a public dataset of the U.S. health survey system to define a diabetes prediction task. This study employs nested cross-validation as well as an aggregated model applied to an independent test set to systematically evaluate the proposed method. Compared with traditional binary cross entropy loss and Focal loss, the Highest-k loss improved the precision (positive predictive value) for the highest 1% scores by 0.05 (95% CI: 0.041-0.055), the highest 5% scores by 0.03 (95% CI: 0.024-0.032), and the highest 10% scores by 0.02 (95% CI: 0.016-0.021). The introduced Highest-k loss function addresses the problem of prevailing risk prediction models and offers a practical solution that focuses on patients with the k highest predictive scores who can realistically receive an intervention as opposed to the entire patient population.
Towards Efficient Medical Dialogue Summarization with Compacting-Abstractive Model

Medical dialogue summarization (MDS) refers to automatically generating electronic health records (EHRs) from doctor-patient dialogues to relieve doctors from the recording burden. The long-term nature of medical dialogue makes it more efficient to handle MDS with a two-stage abstractive summarization model, which first extracts salient content from the source text and then generates an abstract summary based on that. However, this commonly used extractive-abstractive paradigm struggles to identify absolute salient statements in the first stage while discards all information mistakenly considered unimportant, heavily limiting the second-stage summarization performance. In this paper, we introduce a novel two-stage model for MDS with a compact-then-abstract workflow to ensure data efficiency and information integrity. After predicting EHR-related utterances in a medical dialogue, our model adaptively compresses the remaining into a soft context and generates an EHR according to both the prediction and compression results. This way, we loosen the traditional first-stage extraction to a hybrid of extraction and compression, which makes the input context compact and avoids suffering from losing information for producing an EHR. Extensive experiments on two public datasets show that the proposed model significantly outperforms the state-of-the-art counterparts w.r.t. multiple metrics.
Sentiment Analysis in Greek Clinical Conversations: A Comparative Study of BERT, VADER, and Lexicon Approaches

Sentiment analysis, a subfield of Natural Language Processing (NLP), focuses on the study of emotions expressed in text data. While extensively used in major languages such as English, its application in Greek remains underexplored. We investigate sentiment analysis in Greek clinical dialogues, focusing on hematologic malignancies. These dialogues offer valuable insights into patient experiences, healthcare provider approaches in palliative care, and underscore the critical role of sentiment analysis in accurately assessing patients' emotional status for effective care. Our study compares three methodologies—Greek Lexicon, VADER, and BERT—while addressing the limited Greek resources in healthcare-related NLP. Key findings indicate that BERT outperformed other methods by achieving well-balanced precision and recall in assessing sentiment. The Lexicon-Based approach encountered challenges in identifying both negative and positive sentiments, while VADER showcased robust results. Our study involves data collection, data annotation, model training, and performance evaluation. We aim to address the research gap and the unique challenges of sentiment analysis in Greek, especially in context of hematologic malignancies, where NLP and sentiment analysis resources are limited.
Gaussian Process-based Active Learning for Efficient Cardiovascular Disease Inference

Cardiovascular disease (CVD) poses a significant global health challenge, and accurate inference methods are vital for early detection and intervention. However, the quality of prediction relies heavily on the availability of labeled data, which are often limited in medical applications. To cope with the challenge of limited labeled data, we are the first to propose an active learning (AL) approach that leverages a weighted ensemble of Gaussian processes to effectively infer CVD by strategically selecting the few most informative data points to label. Through experiments conducted on the SMARTool dataset, we demonstrate the effectiveness of the advocated approach, achieving superior performance in CVD inference compared to baseline methods. Our findings highlight the potential impact of the proposed AL framework in CVD diagnosis and treatment clinical cases, particularly in scenarios where labeled data are scarce, due to data confidentiality concerns or high sampling costs.
Privacy-Preserving Publishing of Individual-Level Medical Data for Cloud Services

Deep learning (DL) has been extensively adopted in many applications, including disease prediction. Most DL-based applications are executed on a cloud server because the DL models are too large and complicated to be executed on the client-side. De facto cloud-hosted inferences lead to privacy concerns regarding services that operate on personal medical data. Nevertheless, given the recent development of DL-based applications for health-diagnosis services, these applications have become a dominant means of healthcare support in our daily lives. To prevent the misuse of personal medical data, several techniques have been developed to preserve sensitive information, with a trade-off between privacy and utility. A simple method that offers privacy preservation and good prediction performance involves the deployment of a diagnostic method to the client side. However, doing so makes DL models more vulnerable to adversaries. To this end, we propose a deep private generative framework that guarantees user-data privacy while maintaining the original class information and protecting the models from reverse engineering. Experimentation with practical deep neural networks on benchmark disease datasets demonstrates that the proposed method decreases the mutual information between the original data and synthetic data by nearly 80% while preserving a prediction accuracy of nearly 95% of the original prediction accuracy.