Reviewerly/exHarmony huggingface dataset:
Group Distributionally Robust Knowledge Distillation

Knowledge distillation enables fast and effective transfer of features learned from a bigger model to a smaller one. However, distillation objectives are susceptible to sub-population shifts, a common scenario in medical imaging analysis which refers to groups/domains of data that are underrepresented in the training set. For instance, training models on health data acquired from multiple scanners or hospitals can yield subpar performance for minority groups. In this paper, inspired by distributionally robust optimization (DRO) techniques, we address this shortcoming by proposing a group-aware distillation loss. During optimization, a set of weights is updated based on the per-group losses at a given iteration. This way, our method can dynamically focus on groups that have low performance during training. We empirically validate our method, GroupDistil on two benchmark datasets (natural images and cardiac MRIs) and show consistent improvement in terms of worst-group accuracy.
Cross-view Contrastive Mutual Learning Across Masked Autoencoders for Mammography Diagnosis

Mammography is a widely used screening tool for breast cancer, and accurate diagnosis is critical for the effective management of breast cancer. In this study, we propose a novel cross-view mutual learning method that leverages a Cross-view Masked Autoencoder (CMAE) and a Dual-View Affinity Matrix (DAM) to extract cross-view features and facilitate malignancy classification in mammography. CMAE aims to extract the underlying features from multi-view mammography data without relying on lesion labeling information or multi-view registration. DAM helps overcome the limitations of single-view models and identifies unique patterns and features in each view, thereby improving the accuracy and robustness of breast tissue representations. We evaluate our approach on a large-scale in-house mammography dataset and demonstrate promising results compared to existing methods. Additionally, we perform an ablation analysis to investigate the influence of different loss functions on the performance of our method. The results show that all the proposed components contribute positively to the final performance. In summary, the proposed cross-view mutual learning method shows great potential for assisting malignant classification.
RelationalUNet for Image Segmentation

Medical image segmentation is one of the most classic applications of machine learning in healthcare. A variety of Deep Learning approaches, mostly based on Convolutional Neural Networks (CNNs), have been proposed to this end. In particular, U-Shaped Network (UNet) have emerged to exhibit superior performance for medical image segmentation. However, some properties of CNNs, such as the stationary kernels, may limit them from capturing more in-depth visual and spatial relations. The recent success of transformers in both language and vision has motivated dynamic feature transforms. We propose RelationalUNet (RelationalUNet) which introduces relational feature transformation to the UNet architecture. RelationalUNet models the dynamics between visual and depth dimensions of a 3D medical image by introducing Relational Self-Attention blocks in skip connections. As the architecture is mainly intended for the semantic segmentation of 3D medical images, we aim to learn their long-range depth relations. Our method was validated on the Multi-Atlas Labeling Beyond The Cranial Vault (BTCV) dataset for multi-organ segmentation. Robustness to distribution shifts is a particular challenge in safety-critical applications such as medical imaging. We further test our model performance on realistic distributional shifts on the Shifts 2.0 White Matter Multiple Sclerosis Lesion Segmentation. Experiments show that our architecture leads to competitive performance. The code is available at https://github.com/ivaxi0s/runet .
Vector Quantized Multi-modal Guidance for Alzheimer’s Disease Diagnosis Based on Feature Imputation

Magnetic Resonance Imaging (MRI) and positron emission tomography (PET) are the most used imaging modalities for Alzheimer's disease (AD) diagnosis in clinics. Although PET can better capture AD-specific pathologies than MRI, it is less used compared with MRI due to high cost and radiation exposure. Imputing PET images from MRI is one way to bypass the issue of unavailable PET, but is challenging due to severe ill-posedness. Instead, we propose to directly impute classification-oriented PET features and combine them with real MRI to improve the overall performance of AD diagnosis. In order to more effectively impute PET features, we discretize the feature space by vector quantization and employ transformer to perform feature transition between MRI and PET. Our model is composed of three stages including codebook generation, mapping construction, and classifier enhancement based on combined features. We employ paired MRI-PET data during training to enhance the performance of MRI data during inference. Experimental results on ADNI dataset including 1346 subjects show a boost in classification performance of MRI without requiring PET. Our proposed method also outperforms other state-of-the-art data imputation methods.
Finding-Aware Anatomical Tokens for Chest X-Ray Automated Reporting

The task of radiology reporting comprises describing and interpreting the medical findings in radiographic images, including description of their location and appearance. Automated approaches to radiology reporting require the image to be encoded into a suitable token representation for input to the language model. Previous methods commonly use convolutional neural networks to encode an image into a series of image-level feature map representations. However, the generated reports often exhibit realistic style but imperfect accuracy. Inspired by recent works for image captioning in the general domain in which each visual token corresponds to an object detected in an image, we investigate whether using local tokens corresponding to anatomical structures can improve the quality of the generated reports. We introduce a novel adaptation of Faster R-CNN in which finding detection is performed for the candidate bounding boxes extracted during anatomical structure localisation. We use the resulting bounding box feature representations as our set of finding-aware anatomical tokens. This encourages the extracted anatomical tokens to be informative about the findings they contain (required for the final task of radiology reporting). Evaluating on the MIMIC-CXR dataset [12, 16, 17] of chest X-Ray images, we show that task-aware anatomical tokens give state-of-the-art performance when integrated into an automated reporting pipeline, yielding generated reports with improved clinical accuracy.
A Novel Framework to Strengthen Early Warning Systems

The impact of disasters on the population and environment is an important research area. Multiple criteria need to be analyzed while making policy decisions in order to control the effect of a disaster. Researchers have used many variants of the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS), a Multi-Criteria Decision-Making (MCDM) method for prioritizing the alternatives. Additionally, the detrimental effects of disasters have compelled stakeholders to proactively prepare by strengthening crucial key elements of an Early Warning System (EWS) so that timely alerts can be produced. In this paper, a Disaster Information Provider (DIP) framework is proposed, which employs a TOPSIS variant to bolster weak elements of a people-centric EWS. Governments may utilize delivered rankings to strengthen the weak elements of the EWS in an affected area. Extensive experimentation proves the usability of the DIP framework for strengthening EWS.
A Recommendation Engine Model for Giant Social Media Platforms using a Probabilistic Approach

Existing recommender system algorithms often find it difficult to interpret and, as a result, to extract meaningful recommendations from social media. Because of this, there is a growing demand for more powerful algorithms that are able to extract information from low-dimensional spaces. One such approach would be the cutting-edge matrix factorization technique. Facebook is one of the most widely used social networking platforms. It has more than one billion monthly active users who engage with each other on the platform by sharing status updates, images, events, and other types of content. Facebook's mission includes fostering stronger connections between individuals, and to that end, the platform employs techniques from recommender systems in an effort to better comprehend the actions and patterns of its users, after which it suggests forming new connections with other users. However, relatively little study has been done in this area to investigate the low-dimensional spaces included within the black box system by employing methods such as matrix factorization. Using a probabilistic matrix factorization approach, the interactions that users have with the posts of other users, such as liking, commenting, and other similar activities, were utilized in an effort to generate a list of potential friends that the user who is the focus of this work may not yet be familiar with. The proposed model performed better in terms of suggestion accuracy in comparison to the original matrix factorization, which resulted in the creation of a recommendation list that contained more correct information.
A BiLSTM-CF and BiGRU-based Deep Sentiment Analysis Model to Explore Customer Reviews for Effective Recommendations

The advancement of technology has led to the rise of social media forums and e-commerce platforms, which have become popular means of communication, and people can express their opinions through comments and reviews. Increased accessibility to online feedback helps individuals make informed decisions about product purchases, services, and other decisions. This study used a sentiment analysis-based approach to improve the functionality of the recommendations from user reviews and consider the features (aspects and opinions) of products and services to understand the characteristics and attributes that influence the performance of classification algorithms. The proposed model consists of data preprocessing, word embedding, character representation creation, feature extraction using BiLSTM-CF, and classification using BiGRU. The proposed model was evaluated on different multidomain benchmark datasets demonstrating impressive performance. The proposed model outperformed existing models, offering more promising performance results in recommendations.
Enhanced System Usability Scale using the Software Quality Standard Approach

The objective of this paper is to improve the current System Usability Scale (SUS) and assess its applicability in the context of Learning Management Systems (LMS). The need to evaluate the usability of systems has become increasingly important in today's market, as it can have a significant impact on the user experience. In light of the COVID-19 pandemic, e-learning has become an essential tool for students, making LMS an appropriate research case study. Through a comprehensive literature review, it was discovered that SUS is the most widely used tool for evaluating system usability. However, SUS fails to satisfy some of the usability criteria outlined in ISO 9126 and ISO 9241-11. Therefore, this paper proposes an enhanced SUS model and its conceptual framework to address these limitations. The proposed model was validated using a case study approach, involving subject matter experts and software testing students, who evaluated the reliability of the enhanced SUS. Additionally, the existing and enhanced SUS models were evaluated based on an LMS case study and the results were used to calculate the enhanced SUS's reliability coefficient using Cronbach's alpha. The validation results show that the enhanced SUS has higher reliability with improved quality coverage compared to the original SUS. The proposed model has the potential to enhance the evaluation of system usability and, consequently, improve user experience.
SPPNet: A Single-Point Prompt Network for Nuclei Image Segmentation

Image segmentation plays an essential role in nuclei image analysis. Recently, the segment anything model has made a significant breakthrough in such tasks. However, the current model exists two major issues for cell segmentation: (1) the image encoder of the segment anything model involves a large number of parameters. Retraining or even fine-tuning the model still requires expensive computational resources. (2) in point prompt mode, points are sampled from the center of the ground truth and more than one set of points is expected to achieve reliable performance, which is not efficient for practical applications. In this paper, a single-point prompt network is proposed for nuclei image segmentation, called SPPNet. We replace the original image encoder with a lightweight vision transformer. Also, an effective convolutional block is added in parallel to extract the low-level semantic information from the image and compensate for the performance degradation due to the small image encoder. We propose a new point-sampling method based on the Gaussian kernel. The proposed model is evaluated on the MoNuSeg-2018 dataset. The result demonstrated that SPPNet outperforms existing U-shape architectures and shows faster convergence in training. Compared to the segment anything model, SPPNet shows roughly 20 times faster inference, with 1/70 parameters and computational cost. Particularly, only one set of points is required in both the training and inference phases, which is more reasonable for clinical applications. The code for our work and more technical details can be found at https://github.com/xq141839/SPPNet .
unORANIC: Unsupervised Orthogonalization of Anatomy and Image-Characteristic Features

We introduce unORANIC, an unsupervised approach that uses an adapted loss function to drive the orthogonalization of anatomy and image-characteristic features. The method is versatile for diverse modalities and tasks, as it does not require domain knowledge, paired data samples, or labels. During test time unORANIC is applied to potentially corrupted images, orthogonalizing their anatomy and characteristic components, to subsequently reconstruct corruption-free images, showing their domain-invariant anatomy only. This feature orthogonalization further improves generalization and robustness against corruptions. We confirm this qualitatively and quantitatively on 5 distinct datasets by assessing unORANIC's classification accuracy, corruption detection and revision capabilities. Our approach shows promise for enhancing the generalizability and robustness of practical applications in medical image analysis. The source code is available at github.com/sdoerrich97/unORANIC .
Multi-modal Adapter for Medical Vision-and-Language Learning

Recently, medical vision-and-language learning has attracted great attention from biomedical communities. Thanks to the development of large pre-trained models, the performances on these medical multi-modal learning benchmarks have been greatly improved. However, due to the rapid growth of the model size, full fine-tuning these large pre-trained models has become costly in training and storing such huge parameters for each downstream task. Thus, we propose a parameter-efficient transfer learning method named Medical Multi-Modal Adapter (M $$^3$$ AD) to mediate this problem. We select the state-of-the-art M $$^3$$ AE model as our baseline, which is pre-trained on 30k medical image-text pairs with multiple proxy tasks and has about 340M parameters. To be specific, we first insert general adapters after multi-head attention layers and feed-forward layers in all transformer blocks of M $$^3$$ AE. Then, we specifically design a modality-fusion adapter that adopts multi-head attention mechanisms and we insert them in the cross-modal encoder to enhance the multi-modal interactions. Compared to full fine-tuning, we freeze most parameters in M $$^3$$ AE and only train these inserted adapters with much smaller sizes. Extensive experimental results on three medical visual question answering datasets and one medical multi-modal classification dataset demonstrate the effectiveness of our proposed method, where $$\mathrm M^{3}AD$$ achieves competitive performances compared to full fine-tuning with much fewer training parameters and memory consumption.
A differential evolution algorithm combined with Lévy Flight for dimensional synthesis of four-bar linkage

AbstractThe paper considers dimensional synthesis of a path generator four-bar linkage by applying the modified Differential Evolution (DE) algorithm. During the search of the space, the DE algorithm may skip the right solution and lead to premature convergence. If the step is reduced, the search space is reduced as well, so that convergence becomes slower. Search efficiency may be achieved by including Lévy Flight in the DE algorithm. Namely, when the DE algorithm identifies the space of best solution, then a Levý Flight step is used for local search in that region, which significantly increases the exploitation capability and allows obtaining a very efficient solution of the optimization problem. The algorithm was tested with three examples of dimensional synthesis of a path generator four-bar linkage. The proposed algorithm can be very efficiently applied to solving very complex optimization problems.Keywords: Differential evolutionLévy flightdimensional synthesisfour-bar linkagepath generation Disclosure statementNo potential conflict of interest was reported by the author(s).Additional informationFundingThis work has been supported by the Ministry of Education, Science and Technological Development of the Republic of Serbia, through the Contracts for the scientific-research activity realization and financing in 2023: 451-03-47/2023-01/200108, 451-03-47/2023-01/200029 and 451-03-47/2023-01/200102.
RoFormer for Position Aware Multiple Instance Learning in Whole Slide Image Classification

Whole slide image (WSI) classification is a critical task in computational pathology. However, the gigapixel-size of such images remains a major challenge for the current state of deep-learning. Current methods rely on multiple-instance learning (MIL) models with frozen feature extractors. Given the the high number of instances in each image, MIL methods have long assumed independence and permutation-invariance of patches, disregarding the tissue structure and correlation between patches. Recent works started studying this correlation between instances but the computational workload of such a high number of tokens remained a limiting factor. In particular, relative position of patches remains unaddressed. We propose to apply a straightforward encoding module, namely a RoFormer layer , relying on memory-efficient exact self-attention and relative positional encoding. This module can perform full self-attention with relative position encoding on patches of large and arbitrary shaped WSIs, solving the need for correlation between instances and spatial modeling of tissues. We demonstrate that our method outperforms state-of-the-art MIL models on three commonly used public datasets (TCGA-NSCLC, BRACS and Camelyon16)) on weakly supervised classification tasks. Code is available at https://github.com/Sanofi-Public/DDS-RoFormerMIL .
Unifying and Personalizing Weakly-Supervised Federated Medical Image Segmentation via Adaptive Representation and Aggregation

Federated learning (FL) enables multiple sites to collaboratively train powerful deep models without compromising data privacy and security. The statistical heterogeneity (e.g., non-IID data and domain shifts) is a primary obstacle in FL, impairing the generalization performance of the global model. Weakly supervised segmentation, which uses sparsely-grained (i.e., point-, bounding box-, scribble-, block-wise) supervision, is increasingly being paid attention to due to its great potential of reducing annotation costs. However, there may exist label heterogeneity, i.e., different annotation forms across sites. In this paper, we propose a novel personalized FL framework for medical image segmentation, named FedICRA, which uniformly leverages heterogeneous weak supervision via adaptIve Contrastive Representation and Aggregation. Concretely, to facilitate personalized modeling and to avoid confusion, a channel selection based site contrastive representation module is employed to adaptively cluster intra-site embeddings and separate inter-site ones. To effectively integrate the common knowledge from the global model with the unique knowledge from each local model, an adaptive aggregation module is applied for updating and initializing local models at the element level. Additionally, a weakly supervised objective function that leverages a multiscale tree energy loss and a gated CRF loss is employed to generate more precise pseudo-labels and further boost the segmentation performance. Through extensive experiments on two distinct medical image segmentation tasks of different modalities, the proposed FedICRA demonstrates overwhelming performance over other state-of-the-art personalized FL methods. Its performance even approaches that of fully supervised training on centralized data. Our code and data are available at https://github.com/llmir/FedICRA .
Deformable Medical Image Registration Under Distribution Shifts with Neural Instance Optimization

Deep-learning deformable image registration methods often struggle if test-image characteristic shifts from the training domain, such as the large variations in anatomy and contrast changes with different imaging protocols. Gradient descent-based instance optimization is often introduced to refine the solution of deep-learning methods, but the performance gain is minimal due to the high degree of freedom in the solution and the absence of robust initial deformation. In this paper, we propose a new instance optimization method, Neural Instance Optimization (NIO), to correct the bias in the deformation field caused by the distribution shifts for deep-learning methods. Our method naturally leverages the inductive bias of the convolutional neural network, the prior knowledge learned from the training domain and the multi-resolution optimization strategy to fully adapt a learning-based method to individual image pairs, avoiding registration failure during the inference phase. We evaluate our method with gold standard, human cortical and subcortical segmentation, and manually identified anatomical landmarks to contrast NIO's performance with conventional and deep-learning approaches. Our method compares favourably with both approaches and significantly improves the performance of deep-learning methods under distribution shifts with 1.5% to 3.0% and 2.3% to 6.2% gains in registration accuracy and robustness, respectively.