Reviewerly/exHarmony huggingface dataset:
Clinically Coherent Radiology Report Generation with Imbalanced Chest X-rays

Automatically generating radiology reports given radiographs has considerable promise for easing clinical workflows, reducing diagnostic errors, and ultimately streamlining clinical care. Therefore, this work has attracted much attention and early studies mainly employed cutting-edge techniques from computer vision and natural language generation to improve the readability of generated reports. However, those methods often fail to accurately convey critical information such as the presence status of disease, which is the first priority of medical image-to-text generation. Additionally, the class imbalance issues are frequently found in a number of radiographic datasets, resulting in missed diagnosis of some uncommon diseases. In this manuscript, we propose a clinically coherent radiology report generation method that introduces an additional memory-enhanced feature-wise affine transformation (MFAT) layer and an imbalance-aware clinically accurate reward (ICAR) to tackle the aforementioned issues. We evaluate the proposed method by comparing with state-of-the-art methods over both natural language generation (NLG) metrics and clinical efficacy results on two datasets, namely IU X-Ray and MIMIC-CXR. The results demonstrate that our method achieves significantly higher clinical efficacy accuracy for both common or uncommon disease annotations while still maintaining acceptably high NLG metrics for readability.
Milestones on the Quantum Utility Highway

We define the quantum utility performance metric and three milestones, which define quantum utility as measured in some limited context. Current and previous-generation annealing quantum systems can outperform classical solvers on a variety of input classes, on Milestones 0 and 1; our tests of Milestone 2 show positive results on a small set of inputs. Characterization of the input properties that drive these outcomes suggests that future tests will yield more widespread successes on these and more challenging milestones.
Bayesian Deep Learning for Image Reconstruction: From structured sparsity to uncertainty estimation

Conventional wisdom in model-based computational imaging incorporates physics-based imaging models, noise characteristics, and image priors into a unified Bayesian framework. Rapid advances in deep learning have inspired a new generation of data-driven computational imaging systems with performances even better than those of their model-based counterparts. However, the design of learning-based algorithms for computational imaging often lacks transparency, making it difficult to optimize the entire imaging system in a complete manner.
Deep Neural Networks for Transmission Impairment Mitigation in Long-Reach 5G Access Networks

The mitigation of transmission impairments in optical communication systems requires implementation of digital signal processing that can easily adapt to changing conditions and work on continuous streams of data. In this paper, the application of continual learning in deep neural networks for transmission impairments in long haul transmission systems is considered. This work investigates the novel use of continual learning to overcome the problem of catastrophic forgetting and remembrance in deep neural networks that can be used for dispersion and nonlinearity mitigation in long-haul transmission systems. We perform comparisons between state-of-the-art methods such as iCaRL, DGR, EWC, XdG and traditional DNNs implementations typically used in optical networks. We demonstrate improvements in training accuracy towards signal recovery, and increased flexibility for DNN equalization applications. The results show that the method can be successfully used for fiber lengths ranging from less than 1km to 250km and over with no need to retrain the network from scratch for each fibre length. The introduction of continual learning enables training DNNs on a continuous stream of data with no loss of prior knowledge.
Explainable Pulmonary Disease Diagnosis with Prompt-Based Knowledge Extraction

Recent studies show that deep learning models perform well in many medical tasks such as medical imaging and automated diagnosis. With qualified training datasets, some models can achieve or even surpass expert-level performance on some tasks. However, as a typical black-box-style approach, deep learning lacks theoretical interpretability, which is especially important for medical tasks. On the other hand, there are many sources of domain knowledge for medical diagnosis from human experts, such as clinical guidelines. How to sufficiently integrate human knowledge in the model is crucial for explainable diagnosis. In this paper, we propose a novel framework for explainable automated diagnosis that leverages explicit medical knowledge. We automate the knowledge extraction from textual clinical guidelines with prompt-based learning, train a set of weighted first-order logical rules with constructed evidence database, and finally infer the diagnosis result with integrated knowledge and multi-sourced data. We instantiate the framework for pulmonary disease diagnosis, and our experiments on a real dataset show that our method outperforms the state-of-the-art baselines in accuracy and interpretability.
Efficient Classification of Very High Resolution Histopathological Images

Over the years, deep learning approaches have shown significant improvement in various image understanding tasks. However, analysis of high resolution images still remains a major challenge. Apart from the huge computational resources required for such images, the large image sizes make it difficult to extract effective contextual information needed for important tasks, such as classification, segmentation, or clustering of such images. In this work, we address the challenge of high resolution image classification u sing a new discriminative patch selection approach. We embed our patch selection approach inside a novel classification framework, supporting potential use of different pre-trained learning models. We show results on a high resolution image dataset, namely, gigapixel whole slide tissue images for cancer tumors. We demonstrate the performance of the proposed approaches using comparative analysis with state-of-the art methods on this dataset.
ENET: Efficient Blockchain Consensus Algorithm Based on Evil Node Elimination Tree

Blockchain consensus algorithms are generally used to ensure data consistency in the blockchain. The current blockchain consensus algorithm has some problems, such as high computing resource consumption, slow block generation speed and difficulty in dealing with malicious node and failure node, which lead to the uncertainty of authenticity of new blocks and trading transaction delay. In view of these problems, we firstly turn the question into solving the problems of electing the one node to keep ledger while ensuring the correctness of ledger information and then synchronize the distributed ledger among most nodes. we propose a consensus algorithm named ENET (Evil Node Elimination Tree) using single chain blockchain structure, with integrating Boyer-Moore majority vote algorithm ensuring the correctness of all new generated blocks. A bottom-up promotion approach to election was adopted to gradually eliminate evil nodes, meanwhile the structure of elimination tree of evil nodes is being constructed. In addition, the distributed synchronization mechanism of top-down update is adopted to recursively update the distributed ledger of all nodes. Theoretical derivation and computer multithreading simulation results show that the communication complexity of this algorithm is at a linear level, and the fault tolerance ratio is 1/4 at the lowest and 3/4 at the highest. The transaction duration is significantly shortened while the double payment problem is eliminated in the system.
Deep learning-based restaining of histopathological images

Pathology entails the visual examination of digital slides created by various staining techniques based on the chemical properties of tissue histopathology. Hematoxylin and eosin stain reveals the morphological features of tissue structure. However, in modern histopathology, immunohistochemistry provides additional information on different tissue components through the antibody-mediated visualization of specific proteins. For this, the tissue sample must be stained with multiple dyes to provide the extra information that specialists need for diagnosis. A significant l imitation of this process–restaining–is that it requires a long time in the conventional laboratory flow. Deep learning-based virtual staining techniques are a considerable alternative for rapid diagnosis and treatment by minimizing time expenditure. In this study, we present a deep neural network-based restaining method, which we call StainGAN, to replace the time-consuming and expensive traditional laboratory flow. StainGAN builds upon the self-attention GAN, supplemented by contrastive learning that provides improved segmentation performance as well as a color-based loss function for extracting recessive T-cell signals in the immunohistochemistry stained image. We demonstrate the quantitative performance of StainGAN against state-of-the-art methods, such as CycleGAN and Pix2Pix as well as its qualitative performance using the multiscale structural similarity index measure (MSSIM) and universal image quality index (UQI) metrics. Although StainGAN produced much higher qualitative and quantitative results than the Pix2Pix approach, it showed only slightly higher performance than the CycleGAN approach. According to the visual evaluation by pathologists, virtual CD8-stained images produced by the proposed model show a high similarity to real CD8-stained images. Moreover, numerical similarities were quantified by comparing the presence of immune cells in virtual CD8 WSIs to real CD8 WSIs.
Feature-Guided Logical Perception Network for Health Risk Prediction

Massive EHR (electronic health records) data contains rich health-related information and offers valuable opportunities for healthcare prediction tasks. Most existing works have achieved great success in improving the accuracy of prediction by leveraging deep learning to model sequential EHR data. However, entangled patient modeling results in inability of quantifying the importance of each type of feature. In this paper, we propose a feature-guided logical perception network FeLON for health risk prediction. FeLON is built on model-level explanation and consists of two main steps. The core part of the first step is a logical perception network to extract logical rules from patient features. In the second step, FeLON builds a fully-connected layer to integrate logical rules information to calculate health risk. Extensive experiments are conducted on two real disease datasets. The experimental results demonstrate that FeLON is able to maintain high prediction accuracy while bringing transparent reasoning process.
BioNER-CFEM: Biomedical Named Entity Recognition Based on Character Feature Enhancement with Multimodal Method

Biomedical named entity recognition (Bio-NER) is an essential task for biomedical information extraction. In this paper, we regard word-level features and character-level features as two different modalities from a novel perspective and propose a biomedical named entity recognition model based on character feature enhancement with multimodal method (called BioNER-CFEM). BioNER-CFEM can not only capture interactions between modalities, but also learn interactions within modalities. In addition, our proposed cross-attention based sparse selection mechanism can effectively alleviate the noise in the interaction process of the two ‘modalities’. Experimental results show the effectiveness of BioNER-CFEM for the Bio-NER task: it achieves performance boost over SOTA models with competitive efficiency on all six Bio-NER datasets, i.e., $+0.89, +0.64, +0.40$, $+1.40, +5.57, +2.81$ on NCBI-Disease, BC5CDR-Disease, BC5CDR-Chem, BC2GM, JNLPBA, BC4CHEMD, respectively.
Joint Extraction of Biomedical Entities and Relations based on Decomposition and Recombination Strategy

Entities and relations extraction is one of the key task to build medical knowledge graph, which is of great significance to the development of medical artificial intelligence. However, overlapping triples are great challenge for biomedical entities and relations extraction. In order to improve the performance of biomedical entities and relations extraction, we propose a joint extraction of entities and relations method based on decomposition and recombination strategy to mine biomedical text. Our method decomposes entities and relations extraction task into three related sub-modules, which are entity tagging module, relation classification module and recombination matching module. Our main contributions are as follows: first, we introduce a decomposition and recombination end-to-end learning framework for joint entities and relations extraction. Second, we propose a bi-directional prediction method to deal with the overlapping triples problem. Finally, we propose the negative samples generation method to alleviate the error accumulation among these modules. The extensive experiments demonstrate that our method can improve the F1 score by 4.36%, 2.13% and 11.72% in ADE, DDI and BB biomedical corpus.
Local-Whole-Focus: Identifying Breast Masses and Calcified Clusters on Full-Size Mammograms

The detection of breast masses and calcified clusters on mammograms is critical for early diagnosis and treatment to improve the survivals of breast cancer patients. In this study, we propose a local-whole-focus pipeline to automatically identify breast masses and calcified clusters on full-size mammograms, from local breast tissues to the whole mammograms, and then focusing on the lesion areas. We first train a deep model to learn the fine features of breast masses and calcified clusteres on local breast tissues, and then transfer the well-trained deep model to identify breast masses and calcified clusteres on full-size mammograms with image-level annotations. We also highlight the areas of the breast masses and calcified clusteres in mammograms to visualize the identification results. We evaluated the proposed local-whole-focus pipeline on a public dataset CBIS-DDSM (Curated Breast Imaging Subset of Digital Database for Screening Mammography) and a private dataset MY-Mammo (Mianyang central hospital mammograms). The experiment results showed the DenseNet embedded with squeeze-and-excitation (SE) blocks achieved competitive results on the identification of breast masses and calcified clusteres on full-size mammograms. The highlight areas of the breast masses and calcified clusteres on the entire mammograms could also explain model decision making, which are important in practical medical applications.
KIT-LSTM: Knowledge-guided Time-aware LSTM for Continuous Clinical Risk Prediction

Rapid accumulation of temporal Electronic Health Record (EHR) data and recent advances in deep learning have shown high potential in precisely and timely predicting patients' risks using AI. However, most existing risk prediction approaches ignore the complex asynchronous and irregular problems in real-world EHR data. This paper proposes a novel approach called Knowledge-guIded Time-aware LSTM (KIT-LSTM) for continuous mortality predictions using EHR. KIT-LSTM extends LSTM with two time-aware gates and a knowledge-aware gate to better model EHR and interprets results. Experiments on real-world data for patients with acute kidney injury with dialysis (AKI-D) demonstrate that KIT-LSTM performs better than the state-of-the-art methods for predicting patients' risk trajectories and model interpretation. KIT-LSTM can better support timely decision-making for clinicians.
KdINet: Knowledge-driven Interpretable Network for Medical Imaging Diagnosis

Automatic diagnosis for medical images is a significant research problem of computer-aided diagnosis to reduce the workload of doctors in recent years. However, existing deep learning approaches for diagnoses are usually black-box models with implicit decision-making processes that make them inexplainable. To alleviate the issue, in this paper, we propose a Knowledge-driven Interpretable Network (KdINet) for interpretable disease classification of medical images. KdINet first exploits the pretrained CNN module and the hierarchical representation module to learn two different disease feature representations (i.e., visual disease features and hierarchical disease features). Subsequently, the joint training of two disease features by KdINet’s disease classifier learns a hierarchical classification criterion to infer the diagnosis and generate corresponding interpretable justifications (i.e., ancestral disease paths of the diagnosis). Extensive experiments on three datasets demonstrate that our KdINet achieves significantly higher effectiveness than state-of-the-art approaches on disease classification metrics.
Compound Density Networks for Risk Prediction using Electronic Health Records

Electronic Health Records (EHRs) exhibit a high amount of missing data due to variations of patient conditions and treatment needs. Imputation of missing values has been considered an effective approach to deal with this challenge. Existing work separates imputation method and prediction model as two independent parts of an EHR-based machine learning system. We propose an integrated end-to-end approach by utilizing a Compound Density Network (CDNet) that allows the imputation method and prediction model to be tuned together within a single framework. CDNet consists of a Gated recurrent unit (GRU), a Mixture Density Network (MDN), and a Regularized Attention Network (RAN). The GRU is used as a latent variable model to model EHR data. The MDN is designed to sample latent variables generated by GRU. The RAN serves as a regularizer for less reliable imputed values. The architecture of CDNet enables GRU and MDN to iteratively leverage the output of each other to impute missing values, leading to a more accurate and robust prediction. We validate CDNet on the mortality prediction task on the MIMIC-III dataset. Our model outperforms state-of-the-art models by significant margins. We also empirically show that regularizing imputed values is a key factor for superior prediction performance. Analysis of prediction uncertainty shows that our model can capture both aleatoric and epistemic uncertainties, which offers model users a better understanding of the model results.
Few-Shot Knowledge Graph Completion based on Data Enhancement

Knowledge graphs (KGs) are widely used in various natural language processing applications. In order to expand the coverage of a KG, KG completion has attracted extensive attention. The commonly used embedding methods based on a large amount of training data can play an important role in this work. However, with few of triples, the performance of these methods will be greatly reduced. The completion of this kind of few-shot task is more challenging. In this work, we propose a method of data enhancement to increase the data quantity and solve the problem of sample shortage. Specifically, we first observe that the representation vectors of the relation in a KG are approximately subordinate to Gaussian distribution. Then we construct a Gaussian distribution for the relation of each triple in few-shot task according to the distributions of its similar relations in background graph. Further, we sample from the Gaussian distribution of each triple to expand the training data. Finally, we use an adaptive attentional network model FAAN proposed by Sheng et al. as the baseline model. Experimental results on two public datasets NELL-One and Wiki-One show that the proposed method achieves better performance.
CLD-Net: Complement Local Detail For Medical Small-Object Segmentation

Deep learning has achieved outstanding performance in biomedical image segmentation. However, it still lacks attention to medical small-object segmentation scenarios, which is essential in early disease diagnosis. The low-resolution feature and single receptive field for medical small-object lead to a significant gap compared with the normal-scale object segmentation. This paper proposes a novel complement local detail-based network architecture (CLD-Net) for medical small-object segmentation, which can complement local detailed information when up-sampling global features. In details, the CLD-Net architecture is established with two sub-modules, i.e., Local Edge Feature Extraction Block (LEFE) and Local-Global Feature Fusion Block (LGFF). LEFE aims to maintain a high-resolution edge-feature sequence corresponding to each layer of the encoder through a progressive scheme. LGFF further rectifies the difference of features to represent the layer-aware detailed information, while the segmentation map can be obtained by incorporating the layer-aware local detailed features into the low-resolution fine-grained global feature. The experimental results on the polyp segmentation task demonstrate the effectiveness of the proposed method. CLD-Net outperforms state-of-the-art methods for small-object segmentation.
Multi-scale Neighborhood Attention Transformer on U-Net for Medical Image Segmentation

U-shaped network structures with skip connections played an irreplaceable role in medical image analysis, but the limitation of convolution makes it unable to learn long-distance semantic information well. The recent success of Transformer in natural language processing and image classification shows that it can benefit from global information modeling by using self-attention mechanisms. However, both local and global features are equally important for dense prediction tasks. Transformer ignores local semantic information to a certain extent. In this study, we propose a Unet-like Transformer for medical image segmentation, named MN-Unet, which can simultaneously extract local and global features. MN-Unet consists of encoder, decoder, and skip connections. Specially, we design an encoder based on the Neighborhood Attention Transformer, which fuse three neighborhood sizes of different dimensions to simultaneously extract local and global features. In the decoder, we use bilinear interpolation to restore the image to its original size. Skip connection is added to alleviate the distortion of low resolution to high resolution. MN-Unet can achieve accurate segmentation of medical images without any pre-training. Extensive experimental results on two medical image datasets (LiTS 2017 and BraTS 2020) show that we achieve relatively better performance than state-of-the-art methods. The codes and trained models will be publicly available a https://github.com/hutchinsonian/MN_Unet