Reviewerly/exHarmony huggingface dataset:
Fine-Grained Ship Classification by Combining CNN and Swin Transformer

The mainstream algorithms used for ship classification and detection can be improved based on convolutional neural networks (CNNs). By analyzing the characteristics of ship images, we found that the difficulty in ship image classification lies in distinguishing ships with similar hull structures but different equipment and superstructures. To extract features such as ship superstructures, this paper introduces transformer architecture with self-attention into ship classification and detection, and a CNN and Swin transformer model (CNN-Swin model) is proposed for ship image classification and detection. The main contributions of this study are as follows: (1) The proposed approach pays attention to different scale features in ship image classification and detection, introduces a transformer architecture with self-attention into ship classification and detection for the first time, and uses a parallel network of a CNN and a transformer to extract features of images. (2) To exploit the CNN’s performance and avoid overfitting as much as possible, a multi-branch CNN-Block is designed and used to construct a CNN backbone with simplicity and accessibility to extract features. (3) The performance of the CNN-Swin model is validated on the open FGSC-23 dataset and a dataset containing typical military ship categories based on open-source images. The results show that the model achieved accuracies of 90.9% and 91.9% for the FGSC-23 dataset and the military ship dataset, respectively, outperforming the existing nine state-of-the-art approaches. (4) The good extraction effect on the ship features of the CNN-Swin model is validated as the backbone of the three state-of-the-art detection methods on the open datasets HRSC2016 and FAIR1M. The results show the great potential of the CNN-Swin backbone with self-attention in ship detection.
Assessment of the utility of Mixed Reality in medical education

Immersive technologies like Mixed Reality (MR), Virtual Reality (VR) and Augmented Reality (AR) are becoming increasingly popular and gain user trust across various fields, particularly in medicine. In this paper we will use the general term Mixed Reality (MR) to refer to the various virtual reality methods, namely VR and AR. These new immersive technologies require varying degrees of instruction, both in their practice use, as well as in how to adjust to interacting with 3D virtual spaces. This study assesses the pedagogical value of these immersive technologies in medical education. We surveyed a group of 211 students and 47 academic faculty at a medical college regarding potential applications of MR in the medical curriculum by using a questionnaire comprised of eight questions. Results were analyzed accounting for user age and professional position, i.e., student vs faculty. 70% of students and 60% of the academic faculty think that MR-supplemented education is advantageous over a classical instruction. Most highly valued were the 3D visualization capabilities of MR, especially in anatomy classes. There was no significant statistical difference between students and faculty responders. Moreover, screensharing between faculty and students contributed to better, longer lasting absorption of knowledge. Surprisingly, the main issue was related to availability, i.e., only 5% of students had access to MR, while 17% of faculty use MR regularly, and 36% occasionally. MR technology can be a valuable resource that supports traditional medical education, especially via 3D anatomy classes, however MR availability needs to be increased. Moreover, MR expands the capabilities and effectiveness of remote learning, which was normalized during the COVID-19 pandemic, to ensure effective student and patient education. MR-based lessons, or even select modules, provide a unique opportunity to exchange experiences inside and outside the medical community.
Dynamic adversarial domain adaptation based on multikernel maximum mean discrepancy for breast ultrasound image classification

Due to the privacy of breast ultrasound images, it is difficult to obtain a dataset, which results in the lack of a large number of labeled datasets. Additionally, the difference in the feature distribution of datasets is also considered. In this paper, a dynamic adversarial domain adaptive network based on the multi kernel maximum mean discrepancy (MK_DAAN) is proposed. An adaptive layer is added to the model to further align the feature distribution of the datasets in the source and target domains, and the multi kernel maximum mean discrepancy is adopted in the adaptive distance measurement. The dual feature alignment of the adaptive layer and adversarial learning improves the classification performance of the model in breast ultrasound images. The experimental results show that the MK_DAAN model has better classification performance than typical unsupervised domain adaptation (DDC, DAN) and typical adjustment domain adaptation (DAAN, MADA) models.
Fast trajectory planning and control of a lab-scale 3D gantry crane for a moving target in an environment with obstacles

In this work, the real-time optimal trajectory planning, together with a cascaded tracking controller, is presented for a three-dimensional (3D) gantry crane in an environment with static obstacles and a dynamically moving target considering dynamic constraints and control input limits. State-of-the-art trajectory optimization-based approaches require long computation times and cannot quickly respond to changes in the target state. The focus of this paper lies on a novel trajectory planning algorithm, which consists of two steps. First, an offline trajectory planner is implemented to compute a time-optimal, collision-free, and dynamically feasible trajectory database that connects all possible initial states of the gantry crane from a predefined starting subspace to the target states in a target subspace. Second, based on linear constrained quadratic programming, the online trajectory replanner makes use of this trajectory database to generate an optimal trajectory in real time that accounts for all changes in the target state. Additionally, a trajectory tracking controller is developed to take into account the dynamic constraints of the gantry crane and to compensate for possible model inaccuracies, disturbances, and other non-modeled effects. Both simulation and experimental results are presented to demonstrate the performance of the proposed trajectory (re)planning algorithm and the control concept.
Fixed-Wing UAV Energy Efficient 3D Path Planning in Cluttered Environments

Abstract UAV path planning in 3D cluttered and uncertain environments centers on finding an optimal / sub-optimal collision-free path, considering in parallel geometric, physical and temporal constraints, fox example, obstacles, infrastructure, physical or artificial landmarks, etc. This paper introduces a novel node-based algorithm, called Energy Efficient A* (EEA*), which is based on the A* search algorithm, but overcomes some of its key limitations. The EEA* deals with 3D environments, it is robust converging fast to the solution, it is energy efficient and it is real-time implementable and executable. In addition to the EEA*, a local path planner is also derived to cope with unknown dynamic threats within the working environment. The EEA* and the local path planner are first implemented and evaluated via simulated experiments using a fixed-wing UAV operating in mountain-like 3D environments, and in the presence of unknown dynamic obstacles. This is followed by evaluating a set up where three UAVs are commanded to follow their respective paths in a safe way. The energy efficiency of EEA* is also tested and compared with the conventional A* algorithm.
Nanoparticle suspensions from carbon-rich fluid make high-grade gold deposits

Economic gold deposits result from a 100- to 10,000-fold enrichment in gold relative to crustal background. In hydrothermal systems, this enrichment is achieved through the transport and accumulation of metals via deeply sourced fluids to a site of deposition. However, the generally low metal solubility of Au in aqueous solutions in orogenic systems requires additional processes in order to explain high-grade gold formation. Reports of Au nanoparticles in high-grade gold veins infer that their formation is linked to mineralisation. However, processes leading to nanoparticle nucleation and deposition remain poorly understood. Here we show that formation of metal nanoparticles (Au, AuAg, Cu, Ag2O) is one of the essential contributors to efficient and focused gold deposition. We report systematic and previously unrecognized metal nanoparticles preserved in amorphous silica and/or carbonic phases in five high-grade deposits. The association of metal, silica and carbonic phases helps to constrain the multiple reactive processes involved in Au, Cu and Ag metallogenesis and formation of high-grade gold mineralisation.
A hybrid framework based on extreme learning machine, discrete wavelet transform, and autoencoder with feature penalty for stock prediction

Accurate prediction of the stock market trend can assist efficient portfolio and risk management. In recent years, with the rapid development of deep learning, it can make the classifiers more robust, which can be used for solving nonlinear problems. In our previous research, we proposed a numerical model for predicting the stock market via combination of discrete wavelet transform (DWT) denoising and extreme learning machine (ELM), and promising outcomes are achieved. The current research presents a hybrid framework using DWT, ELM, and autoencoder (AE) with feature penalty. Firstly, the backpropagation of the AE with feature penalty was deduced theoretically. Then, the raw data were denoised by DWT. The denoised data were used to train the AE with feature penalty after feature preprocessing and utilization of the labeling method. Afterward, the encoder part of the well-trained AE was utilized as the feature extraction model to train ELM model, and the hybrid framework named DAELM (DWT-AE-ELM) could be successfully developed. We also carried out experiments on the corresponding dataset of 400 stocks, and the prediction accuracy of the current study was higher than that of our previous research. According to the predicted labels, we presented an investment strategy, and the yield-to-maturity of 400 stocks was significantly higher than that of the buy-and-hold (BAH) strategy. The results confirmed the superiority of the proposed hybrid framework.
Co-saliency detection with intra-group two-stage group semantics propagation and inter-group contrastive learning

Co-salient object detection (CoSOD) aims at discovering the salient objects with repeatability from multiple relevant images and suppressing non-repetitively appearing salient and non-salient ones. Recently, learning-based methods have shown the advantages in the CoSOD tasks. However, modeling the intra-group semantics and suppressing the inter-group noise objects are still two challenges. In this paper, we propose a unified Two-stage grOup semantics PropagatIon and Contrastive learning NETwork (TopicNet) for CoSOD. TopicNet consists of an intra-group two-stage group semantics propagation module (TGSP) and an inter-group contrastive learning module (CLM). On one hand, in the first stage of TGSP, we put forward an image-to-group propagation module (IGP) to capture the inter-image similarity feature interactions. In the second stage of TGSP, a group-to-pixel propagation module (GPP) is designed to construct the connection between inter-image interactions and inter-pixel correspondences to distill consensus representation. GPP redistributes the interactions with different attention weights to each pixel so as to improve the robustness of group semantics. On the other hand, in CLM, using the design of positive samples enhances the semantic consistency of group semantics. Using the design of negative samples explores the interactive suppression relationships of co-salient objects and noise objects, which suppresses non-repetitively appearing salient and non-salient objects. TGSP and CLM are jointly integrated into an end-to-end unified network predicting the co-salient objects. Experimental results on three prevailing benchmarks reveal that TopicNet outperforms other competitors under various evaluation metrics.
A New Dataset and a Baseline Model for Breast Lesion Detection in Ultrasound Videos

Breast lesion detection in ultrasound is critical for breast cancer diagnosis. Existing methods mainly rely on individual 2D ultrasound images or combine unlabeled video and labeled 2D images to train models for breast lesion detection. In this paper, we first collect and annotate an ultrasound video dataset (188 videos) for breast lesion detection. Moreover, we propose a clip-level and video-level feature aggregated network (CVA-Net) for addressing breast lesion detection in ultrasound videos by aggregating video-level lesion classification features and clip-level temporal features. The clip-level temporal features encode local temporal information of ordered video frames and global temporal information of shuffled video frames. In our CVA-Net, an inter-video fusion module is devised to fuse local features from original video frames and global features from shuffled video frames, and an intra-video fusion module is devised to learn the temporal information among adjacent video frames. Moreover, we learn video-level features to classify the breast lesions of the original video as benign or malignant lesions to further enhance the final breast lesion detection performance in ultrasound videos. Experimental results on our annotated dataset demonstrate that our CVA-Net clearly outperforms state-of-the-art methods. The corresponding code and dataset are publicly available at https://github.com/jhl-Det/CVA-Net .
Embedded Keccak implementation on FPGA

The use of Blockchain has evolved over the last decade. It has touched many fields such as health, industry 4.0, smart cities, agriculture, education etc. Being a guarantee of security, the Blockchain requires to be careful with the data used. In the medical field, patient data is confidential and in order to have access to it, it is necessary to guarantee foolproof security. To do this, the best consensus to use is the PoW. In this paper, we will optimize the code and implement the PoW as an IP in order to accelerate its execution time and consequently minimize the energy consumption. The used Blockchain in this work is the Ethereum and its consensus is the Keccak.
Real-World Blind Super-Resolution via Feature Matching with Implicit High-Resolution Priors

A key challenge of real-world image super-resolution (SR) is to recover the missing details in low-resolution (LR) images with complex unknown degradations (\eg, downsampling, noise and compression). Most previous works restore such missing details in the image space. To cope with the high diversity of natural images, they either rely on the unstable GANs that are difficult to train and prone to artifacts, or resort to explicit references from high-resolution (HR) images that are usually unavailable. In this work, we propose Feature Matching SR (FeMaSR), which restores realistic HR images in a much more compact feature space. Unlike image-space methods, our FeMaSR restores HR images by matching distorted LR image features to their distortion-free HR counterparts in our pretrained HR priors, and decoding the matched features to obtain realistic HR images. Specifically, our HR priors contain a discrete feature codebook and its associated decoder, which are pretrained on HR images with a Vector Quantized Generative Adversarial Network (VQGAN). Notably, we incorporate a novel semantic regularization in VQGAN to improve the quality of reconstructed images. For the feature matching, we first extract LR features with an LR encoder consisting of several Swin Transformer blocks and then follow a simple nearest neighbour strategy to match them with the pretrained codebook. In particular, we equip the LR encoder with residual shortcut connections to the decoder, which is critical to the optimization of feature matching loss and also helps to complement the possible feature matching errors.Experimental results show that our approach produces more realistic HR images than previous methods. Code will be made publicly available.
A robust and efficient method for skeleton‐based human action recognition and its application for cross‐dataset evaluation

Skeleton-based human action recognition has emerged recently thanks to its compactness and robustness to appearance variations. Although impressive results have been obtained in recent years, the performance of skeleton-based action recognition methods has to be improved to be deployed in real-time applications. Recently, a lightweight network structure named Double-feature Double-motion Network (DD-Net) has been proposed for the skeleton-based human action recognition. With high speed, the DD-Net achieves state-of-the-art performance on hand and body actions. The DD-Net could not distinguish actions if they have a weak connection with the global trajectories. However, the DD-Net is suitable for human action recognition where actions strongly correlate to the global trajectories. In this paper, the authors propose TD-Net, an improved version of the DD-Net in which a new branch is added. The new branch takes the normalised coordinates of joints (NCJ) to enrich the spatial information. On five datasets for skeleton-based human activity recognition that are MSR-Action3D, CMDFall, JHMDB, FPHAB, and NTU RGB + D, the TD-Net consistently obtains superior performance compared with the baseline model DD-Net. The proposed method outperforms different state-of-the-art methods, including both hand-designed and deep learning-based methods on four datasets (MSR-Action3D, CMDFall, JHMDB, and FPHAB). Furthermore, the generalisation of the proposed method is confirmed through cross-dataset evaluation. To illustrate the potential use of the model for real-time human action recognition, the authors have deployed an application on an edge device. The experimental result shows that the application can process up to 40 fps for pose estimation using MediaPipe. It takes only 0.04 ms to recognise an action from skeleton sequences.
Integrating Multi-Label Contrastive Learning With Dual Adversarial Graph Neural Networks for Cross-Modal Retrieval

With the growing amount of multimodal data, cross-modal retrieval has attracted more and more attention and become a hot research topic. To date, most of the existing techniques mainly convert multimodal data into a common representation space where similarities in semantics between samples can be easily measured across multiple modalities. However, these approaches may suffer from the following limitations: 1) They overcome the modality gap by introducing loss in the common representation space, which may not be sufficient to eliminate the heterogeneity of various modalities; 2) They treat labels as independent entities and ignore label relationships, which is not conducive to establishing semantic connections across multimodal data; 3) They ignore the non-binary values of label similarity in multi-label scenarios, which may lead to inefficient alignment of representation similarity with label similarity. To tackle these problems, in this article, we propose two models to learn discriminative and modality-invariant representations for cross-modal retrieval. First, the dual generative adversarial networks are built to project multimodal data into a common representation space. Second, to model label relation dependencies and develop inter-dependent classifiers, we employ multi-hop graph neural networks (consisting of Probabilistic GNN and Iterative GNN), where the layer aggregation mechanism is suggested for using propagation information of various hops. Third, we propose a novel soft multi-label contrastive loss for cross-modal retrieval, with the soft positive sampling probability, which can align the representation similarity and the label similarity. Additionally, to adapt to incomplete-modal learning, which can have wider applications, we propose a modal reconstruction mechanism to generate missing features. Extensive experiments on three widely used benchmark datasets, i.e., NUS-WIDE, MIRFlickr, and MS-COCO, show the superiority of our proposed method.
A 'Pointwise-Query, Listwise-Document' based Query Performance Prediction Approach

The task of Query Performance Prediction (QPP) in Information Retrieval (IR) involves predicting the relative effectiveness of a search system for a given input query. Supervised approaches for QPP, such as NeuralQPP are often trained on pairs of queries to capture their relative retrieval performance. However, pointwise approaches, such as the recently proposed BERT-QPP, are generally preferable for efficiency reasons. In this paper, we propose a novel end-to-end neural cross-encoder-based approach that is trained pointwise on individual queries, but listwise over the top ranked documents (split into chunks). In contrast to prior work, the network is then trained to predict the number of relevant documents in each chunk for a given query. Our method is thus a split-n-merge technique that instead of predicting the likely number of relevant documents in the top-k, rather predicts the number of relevant documents for each fixed chunk size p(p<k) and then aggregates them for QPP on top-k. Experiments demonstrate that our method is significantly more effective than other supervised and unsupervised QPP approaches yielding improvements of up to 30% on the TREC-DL'20 dataset and by nearly 9% for the MS MARCO Dev set.
BERT-ER

Entity-oriented search systems often learn vector representations of entities via the introductory paragraph from the Wikipedia page of the entity. As such representations are the same for every query, our hypothesis is that the representations are not ideal for IR tasks. In this work, we present BERT Entity Representations (BERT-ER) which are query-specific vector representations of entities obtained from text that describes how an entity is relevant for a query. Using BERT-ER in a downstream entity ranking system, we achieve a performance improvement of 13-42% (Mean Average Precision) over a system that uses the BERT embedding of the introductory paragraph from Wikipedia on two large-scale test collections. Our approach also outperforms entity ranking systems using entity embeddings from Wikipedia2Vec, ERNIE, and E-BERT. We show that our entity ranking system using BERT-ER can increase precision at the top of the ranking by promoting relevant entities to the top. With this work, we release our BERT models and query-specific entity embeddings fine-tuned for the entity ranking task.
Fast changeset-based bug localization with BERT

Automatically localizing software bugs to the changesets that induced them has the potential to improve software developer efficiency and to positively affect software quality. To facilitate this automation, a bug report has to be effectively matched with source code changes, even when a significant lexical gap exists between natural language used to describe the bug and identifier naming practices used by developers. To bridge this gap, we need techniques that are able to capture software engineering-specific and project-specific semantics in order to detect relatedness between the two types of documents that goes beyond exact term matching. Popular transformer-based deep learning architectures, such as BERT, excel at leveraging contextual information, hence appear to be a suitable candidate for the task. However, BERT-like models are computationally expensive, which precludes them from being used in an environment where response time is important.