Reviewerly/exHarmony huggingface dataset:
Structure-preserving image translation for multi-source medical image domain adaptation

Domain adaptation is an important task for medical image analysis to improve generalization on datasets collected from diverse institutes using different scanners and protocols. For images with visible domain shift, using image translation models is an intuitive and effective way to perform domain adaptation, but the structure of the generated image may often be distorted when large content discrepancies between domains exist; resulting in poor downstream task performance. To address this, we propose a novel image translation model that disentangles structure and texture to only transfer the latter by using mutual information and texture co-occurrence losses. We translate source domain images to the target domain and employ the generated results as augmented samples for domain adaptation segmentation training. We evaluate our method on three public segmentation datasets: MMWHS, Fundus, and Prostate datasets acquired from diverse institutes. Experimental results show that a segmentation model trained using the augmented images from our approach outperforms state-of-the-art domain adaptation, image translation, and domain generalization methods.
Data heterogeneous federated learning algorithm for industrial entity extraction

Entity extraction is an important part to realize digital transformation in the industrial field. Building an entity extraction model in the industrial field requires a lot of data. The parties in industry often cannot share data due to commercial competition and security and privacy issues, thus forming “Data Island”. Federated learning provides a solution to this problem. Federated learning is a distributed machine learning framework that allows each party to train locally and independently using their own private data. The model parameters or gradient information of each party will be aggregated to the central server, thus forming a model jointly trained by all parties. This approach can not only protect the security and privacy of data from all parties, but also fully utilize their data resources. Federated learning can effectively solve the problem of data island, but it still faces some problems and challenges, among which the most typical problem is data heterogeneity. To address the data islanding problem and data heterogeneity problem faced by industrial entity extraction, this paper uses a federated learning framework to solve the data islanding problem and proposes the FedDP algorithm. This algorithm assigns weights based on the data quality performance of each participant. Participants with relatively good data quality performance have higher weights in the aggregation stage, while participants with relatively poor data quality performance have lower weights in the aggregation stage, thus optimizing the performance of federated learning in heterogeneous data scenarios.
Research Hotspots and Trends of Deep Learning in Critical Care Medicine: A Bibliometric and Visualized Study

Interest in the application of deep learning (DL) in critical care medicine (CCM) is growing rapidly. However, comprehensive bibliometric research that analyze and measure the global literature is still lacking.The present study aimed to systematically evaluate the research hotspots and trends of DL in CCM worldwide based on the output of publications, cooperative relationships of research, citations, and the co-occurrence of keywords.A total of 1708 articles in all were obtained from Web of Science. Bibliometric analysis was performed by Bibliometrix package in R software (4.2.2), Microsoft Excel 2019, VOSviewer (1.6.18), and CiteSpace (5.8.R3).The annual publications increased steeply in the past five years, accounting for 95.67% (1634/1708) of all the included literature. China and USA contributed to approximately 71.66% (1244/1708) of all publications. Seven of the top ten most productive organizations rank in the top 100 universities globally. Hot spots in research on the application of DL in CCM have focused on classifying disease phenotypes, predicting early signs of clinical deterioration, and forecasting disease progression, prognosis, and death. Convolutional neural networks, long and short-term memory networks, recurrent neural networks, transformer models, and attention mechanisms were all commonly used DL technologies.Hot spots in research on the application of DL in CCM have focused on classifying disease phenotypes, predicting early signs of clinical deterioration, and forecasting disease progression, prognosis, and death. Extensive collaborative research to improve the maturity and robustness of the model remains necessary to make DL-based model applications sufficiently compelling for conventional CCM practice.
Unpaired image super-resolution using a lightweight invertible neural network

Unpaired image super-resolution (SR) has recently attracted considerable attention in the unsupervised SR community. In contrast to supervised SR, existing unpaired SR methods inevitably resort to the generative adversarial network (GAN) to explore data distribution on the given HR and unpaired LR dataset. Nevertheless, predominant strategies often strive for sophisticated network structures or training pipelines, making them intractable to apply in real-world scenarios. In this work, a lightweight invertible neural network (INN) is proposed for unpaired SR to alleviate this limitation. Specifically, we regard image degradation and SR as a pair of mutually-inverse tasks and replace the two generators in one-stage GAN with INN. Due to the information lossless nature of INN, it is impossible to generate noise in vain during image degradation. We thus design a simple noise injection network to induce realistic noise, thereby simulating real LR images. To further maintain the stability and realism of the noise, we propose to extract the noise prior from the real-world LR image. With extracted noise prior as input, our noise injection network can narrow the gap between the generated noise and the real one, thereby encouraging the degraded images to match the real-world LR domain. Extensive experiments demonstrate that our method achieves comparable performance with other SOTA methods in quantitative and qualitative evaluations while enjoying faster speed and much smaller parameters.
Heterogeneous graph attention network with motif clique

Heterogeneous information networks (HINs) have been proven to be powerful in modeling various real-world networks, such as academic networks and social media networks. By distinguishing the types of relationships and nodes, HINs obtain the capability to model multifarious data, while bringing challenges to data mining and analysis. Specifically, how to comprehensively mine the semantic information in HINs remains an open question. Although recently studied network schemas such as meta-paths or meta-graphs have achieved empirical success, they still have limitations in modeling the complex underlying relations to facilitate the graph neural networks, e.g., different schemas may overlap in semantics and lead to redundant computations. To address this issue, we leverage a recently introduced higher-order network schema known as the motif clique (abbreviated as m-clique). This schema offers greater expressiveness, enabling us to effectively construct the semantic neighborhood of nodes. We further propose a novel heterogeneous graph attention network with m-cliques, named HAMC, which employs a two-level attention mechanism (node-level and semantic-level) to learn node representations. The two-level attention measures the importance of neighbors and m-cliques for each node, respectively. Extensive experiments demonstrate that the proposed HAMC outperforms the state-of-the-art methods in many heterogeneous network analytic tasks such as node classification and clustering. Our code is available at https://github.com/wcx21/HAMC-Heterogeneous-Graph-Attention-Network-with-Motif-Clique.
Exploring stakeholder perspectives: Enhancing robot acceptance for sustainable healthcare solutions

The pandemic has highlighted the fact that healthcare systems around the world are under pressure. Demographic change is leading to an increasing shortage of care workers in most countries, and the demographic challenge is only just beginning in most societies. While robots are widely used in industry, robotic support in healthcare is still limited to very specialized robots in the operating theater. The question of what type of deployment is likely to be successful in a healthcare scenario is not only a technological or economical question but also one of technology acceptance. The answer to this question supports entrepreneurial opportunities to develop sustainable healthcare solutions. In this paper, we analyze the acceptance of robots in elderly care from the perspective of patients, patient families, and geriatric care professionals. To understand the various positions and to identify the suitability of existing acceptance models, we applied stakeholder mapping to conduct qualitative interviews with 14 people with different knowledge backgrounds and levels of involvement in care situations, based on 9 videos showing different robots and application scenarios. The results confirmed that existing technology acceptance models need to be extended by factors such as robot appearance. We found that the background knowledge of the respondents influences the results of the questions about e.g. safety concerns. In addition, we found that the contribution to patients' self-determination and independence is an important factor that is not included in existing technology acceptance models. Finally, the discovery of a significant discrepancy between the self-perception and the external perception of the different stakeholders regarding the acceptance of a service robot can be explained by the stakeholder positions involved in caring for the benefit of a specific patient. These findings encourage further research, especially with the underlying assumption that technology acceptance in healthcare is not just a patient issue, but a stakeholder issue. Stakeholder mapping is a valid tool to analyze the interdependencies for the acceptance of robots. Therefore, we suggest using a tool such as stakeholder mapping to further analyze these issues.
Modeling and hexahedral meshing of cerebral arterial networks from centerlines

Computational fluid dynamics (CFD) simulation provides valuable information on blood flow from the vascular geometry. However, it requires extracting precise models of arteries from low-resolution medical images, which remains challenging. Centerline-based representation is widely used to model large vascular networks with small vessels, as it encodes both the geometric and topological information and facilitates manual editing. In this work, we propose an automatic method to generate a structured hexahedral mesh suitable for CFD directly from centerlines. We addressed both the modeling and meshing tasks. We proposed a vessel model based on penalized splines to overcome the limitations inherent to the centerline representation, such as noise and sparsity. The bifurcations are reconstructed using a parametric model based on the anatomy that we extended to planar n-furcations. Finally, we developed a method to produce a volume mesh with structured, hexahedral, and flow-oriented cells from the proposed vascular network model. The proposed method offers better robustness to the common defects of centerlines and increases the mesh quality compared to state-of-the-art methods. As it relies on centerlines alone, it can be applied to edit the vascular model effortlessly to study the impact of vascular geometry and topology on hemodynamics. We demonstrate the efficiency of our method by entirely meshing a dataset of 60 cerebral vascular networks. 92% of the vessels and 83% of the bifurcations were meshed without defects needing manual intervention, despite the challenging aspect of the input data. The source code is released publicly.
Efficient hand segmentation for rehabilitation tasks using a convolution neural network with attention

We designed an interface to support hand rehabilitation tasks to restore hand function and relieve discomfort. The interface requires accurate hand segmentation, which is impeded by background clutter, occlusion, and variations in illumination. To overcome these challenges, we propose a novel encoder–decoder that segments the hand by encoding spatial and channel correlations using two attention blocks. This approach requires much less computation than benchmark self-attention mechanisms. Moreover, a novel loss function optimizes the model to resolve class imbalance, ensure boundary smoothness, and retain the hand’s shape. The quantitative and qualitative results show the model’s ability to segment the hands. It performed exceptionally well for images with different hand poses and orientations, the presence of a human face, background clutter, specularity, and variations in illumination. The model attained an F1-score of 97.3% for the Ouhands and 99.3% for the HGR dataset, higher than baseline models, with faster inference times. Furthermore, the model could generalize hand segmentation to multiple hands and unseen environments. Its segmentation precision enabled the development of the hand rehabilitation interface, which guided users to perform hand exercises. For five weeks, patients steadily improved hand function while using the interface.
A balanced random learning strategy for CNN based Landsat image segmentation under imbalanced and noisy labels

Landsat image segmentation is important for obtaining large-scale land cover maps. The accuracy of CNN-based Landsat image segmentation highly depends on the quantity and quality of the training samples. However, enough accurate labels for Landsat images are difficult to access. Fortunately, traditional classifier induced segmentation results can be considered as an alternative, although they are noisy and unbalanced to a certain extent. To resist noisy labels and alleviate the impact of imbalanced samples, this paper proposes a confidence interval based balanced random learning strategy. Firstly, a confidence interval-based mask is employed to control the random learning rate of the network from the entire noisy training set. Then, the multi-layer feature maps of CNN are fully utilized to compensate for the information loss in random learning, in which down-sampled labels are used to decrease the uncertainty brought by up-sampling CNN feature maps. In addition, considering the corruption of noisy labels on different classes, a balanced random learning with different confidence levels is performed on each class to further improve the learning ability of CNN. Experimental results on two widely used backbones, namely VGGNet and ResNet, demonstrate that the proposed balanced random learning strategy can effectively improve the performance of CNN under imbalanced and noisy labels, which can be improved by 3.41%.
TODLER: A Transaction Ordering Dependency anaLyzER - for Ethereum Smart Contracts

Smart contracts are programs with data (mutable state); stored on and executed by blockchain platforms. The transactions (or function invocations) dispatched to smart contracts often change their state. In the Ethereum blockchain, nodes (aka miners/validators) can schedule a set of transactions in any order in a block. Multiple transactions in a single block operating on a contract's shared state may yield different outcomes based on their execution order, thus creating a possibility for non-determinism and races between transactions. The resulting issue in Ethereum smart contracts is Transaction Ordering Dependency (TOD). Detecting a TOD requires identifying valid transactions affecting a contract's global/state variables which is equivalent to detecting read-after-write dependencies in race detection, and we expect it to be similarly nontrivial for human developers. In this paper, we identify various TODs, including a novel type previously undocumented in the literature. To detect these TODs, we propose an information flow analysis-based static analyzer, TODler. Our manual evaluation of 108 Ethereum smart contracts shows that TODler outperforms previously available approaches in terms of both run time and precision and also detects the novel TOD pattern identified in this paper.
CE-RetinaNet: A Channel Enhancement Method for Infrared Wildlife Detection in UAV Images

Thermal infrared (TIR) technology is crucial for wildlife detection in unmanned aerial vehicles (UAVs), allowing executives to explore and detect at night. However, the images captured by TIR cameras are unavoidably affected by various unexpected challenges such as image jitter, wildlife overlap, and fog, which may drastically decrease wildlife detection ability. To overcome these challenges, we propose a high-accuracy infrared object detection method called channel enhancement RetinaNet (CE-RetinaNet). Firstly, a new channel enhancement (CE) module is proposed to strengthen the feature extraction of infrared images. Then, a new batch-norm stochastic channel attention (BSCA) module is proposed to filter occlusion-caused anomalous activations and focus on the pixel in the same position across channels. Next, a path augmentation (PA) operation is added after the feature pyramid network (FPN) to improve the localization capability at the entire feature level. Finally, we modified the output strategy of the classification and regression subnets. Additionally, we built a TIR wildlife detection dataset called the Infrared Salient Object Detection (ISOD) comprising 2534 images, which is accessible by the website: https://doi.org/10.5281/zenodo.7445307. We conduct extensive experiments on both public and ISOD datasets, and the experimental results reveal that CE-RetinaNet obtains higher average precision (AP) (e.g., 11.3% more) and Recall (e.g., 11.6% more) compared to other state-of-the-art object detectors.
Attention-Based Multimodal Deep Learning on Vision-Language Data: Models, Datasets, Tasks, Evaluation Metrics and Applications

Multimodal learning has gained immense popularity due to the explosive growth in the volume of image and textual data in various domains. Vision-language heterogeneous multimodal data has been utilized to solve a variety of tasks including classification, image segmentation, image captioning, question-answering, etc. Consequently, several attention mechanism-based approaches with deep learning have been proposed on image-text multimodal data. In this paper, we highlight the current status of attention-based deep learning approaches on vision-language multimodal data by presenting a detailed description of the existing models, their performances and the variety of evaluation metrics used therein. We revisited the various attention mechanisms on image-text multimodal data since its inception in 2015 till 2022 and considered a total of 75 articles for the survey. Our comprehensive discussion also encompasses the current tasks, datasets, application areas and future directions in this domain. This is the very first attempt to discuss the vast scope of attention-based deep learning mechanisms on image-text multimodal data.
Deformable Convolutional Network Constrained by Contrastive Learning for Underwater Image Enhancement

Autonomous underwater vehicles (AUVs) based on remote sensing technology have been widely applied in various underwater tasks. However, the complex underwater environment leads to challenges such as color distortion, blurred details, and fog effects in the underwater image directly acquired by AUVs. Although numerous existing methods aim to remove the color cast and restore image details, their effectiveness is still limited. This paper proposes a new method based on a deformable convolutional network and constrained by contrastive learning for underwater image enhancement. First, we propose a deformable convolutional residual block (DCRB) to achieve a more precise restoration of texture details by adaptively adjusting the convolution kernel shape. At the same time, we utilize the long-skip connection method of the U-Net architecture to preserve information that is prone to lose in shallow networks. Second, we propose a color contrastive loss function to compare the color difference between distorted images and the ground truth, resulting in a more realistic enhanced image. Finally, experimental results demonstrate that the proposed method outperforms the state-of-the-art methods regarding image quality and visual appeal.
GLARE: A Dataset for Traffic Sign Detection in Sun Glare

Real-time machine learning object detection algorithms are often found within autonomous vehicle technology and depend on quality datasets. It is essential that these algorithms work correctly in everyday conditions as well as under strong sun glare. Reports indicate glare is one of the two most prominent environment-related reasons for crashes. However, existing datasets, such as the Laboratory for Intelligent & Safe Automobiles Traffic Sign (LISA) Dataset and the German Traffic Sign Recognition Benchmark, do not reflect the existence of sun glare at all. This paper presents the GLARE (GLARE is available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/NicholasCG/GLARE_Dataset</uri> ) traffic sign dataset: a collection of images with U.S-based traffic signs under heavy visual interference by sunlight. GLARE contains 2,157 images of traffic signs with sun glare, pulled from 33 videos of dashcam footage of roads in the United States. It provides an essential enrichment to the widely used LISA Traffic Sign dataset. Our experimental study shows that although several state-of-the-art baseline architectures have demonstrated good performance on traffic sign detection in conditions without sun glare in the past, they performed poorly when tested against GLARE (e.g., average mAP0.5:0.95 of 19.4). We also notice that current architectures have better detection when trained on images of traffic signs in sun glare performance (e.g., average mAP0.5:0.95 of 39.6), and perform best when trained on a mixture of conditions (e.g., average mAP0.5:0.95 of 42.3).
Context Adaptive Network for Image Inpainting

In a typical image inpainting task, the location and shape of the damaged or masked area is often random and irregular. The vanilla convolutions widely used in learning-based inpainting models treat all spatial features as valid and share parameters across regions, making it difficult for them to cope with those irregular damages, and models tend to produce inpainting results with color discrepancy and blurriness. In this paper, we propose a novel Context Adaptive Network (CANet) to address this issue. The main idea of the proposed CANet is able to generate different weights depending on the miscellaneous input, which may help to complement images with multiple broken forms in a flexible way. Specifically, the proposed CANet has two novel context adaptive modules, namely, the context adaptive block (CAB) and the cross-scale contextual attention (CSCA), which utilize attention mechanisms to cope with diverse content breakdowns. The proposed CAB, during the forward propagation, uses an adaptive term to determine the importance between adaptive term and convolution kernel, so as to dynamically balance features based on the degree of breakage (confidence level or soft mask), and the overall calculation is formulated as a classic convolution implementation with an additional attention term to describe local structure. Besides, the proposed CSCA, not only takes advantage of the contextual attention module, but also considers cross-scale information transfer to generate reasonable features for damaged areas, thus alleviating the inefficiency of the long-range modeling capability of convolutional neural networks. Qualitative and quantitative experiments show that our method performs better than state-of-the-arts, producing clearer, more coherent and visually plausible inpainting results. The code can be found at github.com/dengyecode/CANet_image_inpainting.