Reviewerly/exHarmony huggingface dataset:
GFedKRL: Graph Federated Knowledge Re-Learning for Effective Molecular Property Prediction via Privacy Protection

Graph Neural Networks (GNNs) are one of the primary methods for molecular property prediction due to their ability to learn state-of-the-art level representations from graph-structured molecular data. In addition, the Federated Learning (FL) paradigm, which allows multiple ends to collaborate on machine learning training without sharing local data, is being considered for introduction to improve the performance of multiple ends. However, in FL, the molecular graph data among clients are not only Non-Independent Identically Distribution (Non-IID) but also skewed in quantity distribution. In this paper, we propose the GFedKRL framework to perform knowledge distillation and re-learning during the interaction between clients and servers in each cluster after clustering the graph embeddings uploaded. We also analyze the risk of privacy leakage in the GFedKRL and propose personalized local differential privacy to protect privacy while better controlling the amount of noise input and improving model performance. In addition, to resist the impact of noise data on the clients’ model, graph representation learning is enhanced by knowledge contrast learning at the local clients. Finally, our approach achieves better results in three experimental datasets compared with four public benchmark methods.
Temporal Deformable Transformer for Action Localization

Temporal action localization (TAL) is a challenging task that has received significant attention in video understanding. Recently, Transformer-based models have demonstrated their effectiveness in capturing contextual information and achieved outstanding performance on various TAL benchmarks. However, these methods still face challenges in computational efficiency and contextual modeling rigidity. In this paper, we propose a method to address those problems in Transformer-based models. Our model introduces a temporal deformable Transformer module and the corresponding time normalization, enabling flexible aggregation of temporal context information in videos, leading to enhanced video representations. To demonstrate the effectiveness of the proposed method, we construct a Transformer-based anchor-free model with a simple prediction head, which yields superior performance on widely used benchmarks. Specifically, it achieves an average mAP of 67.4% on THUMOS14 and an average mAP of 36.8% on ActivityNet-v1.3.
Comparison of Various Mutation Operators of the Bacterial Memetic Algorithm on the Traveling Salesman Problem

Bacterial memetic algorithms (BMA) are a relatively new branch of evolutionary algorithms that are good alternatives to traditional genetic algorithms. They meant a significant breakthrough regarding order-based optimizations like the traveling salesman problem. In BMA the selection and mutation steps are replaced by bacterial mutation. During bacterial mutation each specimen of the population is improved separately by cloning them, mutating each clone, and then selecting one as a replacement for the original specimen. The bacterial mutation is one of the most innovative and critical steps of the BMA. Bacterial mutations have many variants, but documentation lacks detailed statistics and behavior models. In this publication, we compare different bacterial mutation operators. The gathered data and observations will support all future research targeting BMA.
Semantic Analysis of Transit Related Tweets in London and Prague

The semantic analysis is an important tool for processing people’s opinions, but processing data from social networking sites like Twitter is still challenging. Transit related tweets in London and Prague collected during the COVID-19 pandemic were analyzed using two corpus-based approaches – Bag-of-Words and Latent Dirichlet Allocation. Punctuality was the most frequent issue in both cities, followed by COVID-19 in London and Comfort in Prague. Analysis for the busiest London station enhanced the importance of the Breakdowns topic. Specific issues were found for some stations such as Victoria Station in London. The BoW method in our cases provides more robust results, namely for large heterogeneous samples, while LDA is well-suited for topic extraction using narrow well-specified samples focused on the explored theme.
BLR: A Multi-modal Sentiment Analysis Model

In multi-modal sentiment analysis tasks, deep learning plays an important role due to its excellent performance. Compared with the traditional statistical approaches and machine learning approaches, deep learning methods have better performance and stability. However, there are still two problems in multi-modal sentiment analysis works, one is that the fused features lead to missing important information. Another is that the affiliation of each feature after fusion is not precisely defined and calculated. To deal with issues, we propose a BLR Multi-channel dual Fusion Model based on Bert,Lstm and ResNeST framework. Our model first ensures that the important features will not be lost after fusion to the maximum extent and then will be processed and optimized according to the contribution of each feature for the fusion. Finally, we conduct experiments on two datasets, the results show that the accuracy of our model gets 76.125% and 77.5%, growing 3.025% and 2.875% over the best baseline model, respectively. Thus, the proposed model BLR model, achieves better effectiveness in multi-modal sentiment analysis tasks.
Task-Aware Adversarial Feature Perturbation for Cross-Domain Few-Shot Learning

Currently, metric-based meta-learning methods have achieved great success in few-shot learning (FSL). However, most works assume a high similarity between base classes and novel classes, and their performance can be greatly reduced when comes to domain-shift problem. As a result, cross-domain few-shot learning (CD-FSL) methods are proposed to tackle the domain-shift problem, which places a higher demand on the robustness of the meta-knowledge. To this end, we propose a feature augmentation method called Task-Aware Adversarial Feature Perturbation (TAAFP) to improve the generalization of the existing FSL models. Compared to the traditional adversarial training, our adversarial perturbations are generated from the feature space and contain more sample relationship information, which is discovered by the Task Attention Module. Task Attention Module is designed based on a transformer to capture more discriminative features in a task. Therefore, our perturbations can easily attack the extraction process for discriminative features, forcing the model to extract more robust discriminative features. In addition, a regularization loss is introduced to ensure the predictions of the adversarial augmented task remain similar to the original task. We conduct extensive classification experiments in five datasets under the setting of cross-domain few-shot classification. The result shows that our method can significantly improve the classification accuracy in both seen and unseen domains.
Bacterial Evolutionary Algorithm Based Autoencoder Architecture Search for Anomaly Detection

This paper presents an architecture optimization approach, based on evolutionary algorithms, which helps in finding the optimal architecture depth, shortening the time that one has to invest in an uncertain number of test runs to find the right architecture. The provided algorithm is applied in the field of anomaly detection, which consists of searching for small amounts (ppm) of anomalies in large datasets using autoencoder models. Nowadays, as a result of the continuous digitization efforts visible in the industry, the data of semi-finished and finished products and production processes are becoming available in increasing quantities. A lot of data can be connected through databases, which offer the possibility of recognizing connections that have been hidden to this day. Quality assurance in industry also requires that deviations within limit values and specifications are investigated (outlier detection). The analyzed data is extracted from the production of a MEMS (Micro Electro Mechanical System) based inertial sensor used in the automotive industry, thus the paper reflects on real problems arising in industry, while the results of the suggested method are validated by domain experts. The presented method in this paper shows how the bacterial evolutionary algorithm combined with an abstraction, based on rational quadratic Bézier curves, can be used in the autoencoder architecture search, and thus be applied to anomaly detection challenges appearing in the industry.
Deep-Learning Based Three Channel Defocused Projection Profilometry

Speeding up the process of fringe projection profilometry to make it more suitable for dynamic three-dimensional measurement is one of the targets of structured light research. In this work, based on the fact that the frame rate of the projector can be much higher than that of the camera, we propose a three-channel binary defocused projection method to break through the speed bottleneck of the camera. Three sequential phase-shifting binary patterns are projected in a defocused way by the three color channels of the projector respectively during one camera’s exposure time. To tackle the issue of color crosstalk, a deep-learning based end-to-end fringe rectification method is first introduced. As a result, only three camera shots are required for single scene reconstruction. Through experiments in three static scenes and one dynamic scene, we demonstrate that our method substantially speeds up the reconstruction rate with an acceptable range of accuracy loss. In addition, given the limited available datasets in the fringe projection profilometry field, a new dataset is also released.
Bilateral Mammogram Mass Detection Based on Window Cross Attention

Breast cancer is the most common cancer in the world. Mammogram mass detection aids in the early detection of breast cancer and increases patient survival rates. Because the bilateral breasts of the same patient are similar and symmetrical, information fusion of bilateral mammogram images is advantageous in improving the detection rate of masses. However, existing mass detection methods use pixel-level corresponding feature fusion methods, which are sensitive to image registration errors. In this paper, we propose WCA-RCNN, a novel mass detection framework that uses window cross attention to fuse information from bilateral mammogram images. The window cross attention module eliminates pixel-level correspondence and significantly improves mass detection accuracy. In addition, we propose a mass deformation data augmentation method to address the issue of insufficient mass samples. We evaluate the proposed method on the publicly available mammography dataset DDSM, demonstrating that it outperforms state-of-the-art mass detection methods.
Kazakh-Uzbek Speech Cascade Machine Translation on Complete Set of Endings

Studies of speech-to-speech machine translation for Turkic languages are practically absent due to the difficulties of creating parallel speech corpora for training neural models. Therefore, the actual problem is creation of synthetic parallel corpora for investigation of speech machine translation of Turkic languages. In this work, a technology of formation of Turkic speech parallel corpora on the base of CSE (Complete Set of Endings) morphology model is proposed. For this technology of formation of Turkic speech parallel corpora is used the cascade scheme: speech-to-text, text-to-text and text-to-speech. A feature of this scheme is that it is used for the phase “text-to-text” a relational model of translation based on CSE morphology model. The scientific contribution of this work is the development of the technology of forming parallel speech corpora of the Kazakh-Uzbek language pairs, based on a cascade scheme for machine translation of speech on relational models. In the future, the formed synthetic parallel speech corpora will be used to train the neural machine translation of the speech of the Turkic languages.
CopiFilter: An Auxiliary Module Adapts Pre-trained Transformers for Medical Dialogue Summarization

To relieve doctors from trivial recording, medical dialogue summarization (MDS) aims at automatically generating electronic health records (EHR) from the dialogues between doctors and patients. Having seen their remarkable performance on text/document summarization tasks, we naturally expect to fine-tune pre-trained language models for MSD. However, most of the models have no special interfaces for exploiting the characteristics of dialogue (e.g., speaker’s role) and have trouble dealing with the unique features of the task. Therefore, we propose a CopiFilter module to adapt pre-trained Transformers for medical dialogue summarization. The module consists of a filter and a copier. Specifically, the filter refines the coarse features of dialogue by incorporating the query and speaker information, and the copier implements a copy mechanism to help correctly generate the medical terminologies that are not contained in the predefined vocabulary. Furthermore, we propose a local correlation detecting (LCD) task and fine-tune the augmented Transformers in a multi-task fashion. Extensive experiments on two public datasets show that the augmented Transformers significantly outperform their baselines w.r.t. ROUGE (average) and entity matching rate (up to 5.01/3.64 on IMCS-MRG and 7.29/3.26 on SUM2).
Towards Energy-Efficient Sentiment Classification with Spiking Neural Networks

Artificial Neural Networks (ANNs) have recently shown surprising results in Natural Language Processing (NLP) tasks. However, high energy consumption has become a major drawback of ANNs in NLP applications, which is contrary to the goal of sustainable and efficient computation. In this paper, we propose a novel energy-efficient sentiment classification model based on Spiking Neural Networks (SNNs), which achieves high energy efficiency by exploiting the sparsity of neural activity and using spikes to encode and transmit information. Unlike conventional neural networks that perform continuous and intensive computation, SNNs only fire spikes when they receive sufficient input stimuli, thereby reducing memory and computational overhead. We evaluate our model on the IMDB movie review dataset for sentiment classification tasks. The experimental results show that compared with the current state-of-the-art Transformer model, the energy consumption of the spike encoder model is reduced to 1.36% of the former, which is a 64.93-fold improvement in energy efficiency ratio. Furthermore, our model maintains an acceptable performance variance of 2%. Our research advances the field of “high-performance NLP models” and promotes further exploration of “low-energy NLP models”.
An Echo State Network-Based Method for Identity Recognition with Continuous Blood Pressure Data

With the development of Continuous Blood Pressure (CBP) monitoring devices, we can collect real-time blood pressure non-invasively and accurately. Since CBP data can reflect the unique dynamical characteristics of the cardiovascular system for each person, it is reasonable to develop an identity recognition method based on these data. In this study, we propose an Echo State Network-based identity recognition method with CBP data. In the proposed method, we divide each CBP series data into several CBP segments. Then we use a Bi-directional Echo State Network to transform the input segments into high-dimensional reservoir states. Finally, we compute the identity recognition results in an aggregation mode. To evaluate the proposed method, we performed person identification tasks using ten sub-datasets sampled from a large-scale CBP dataset. Our proposed method achieved higher recognition accuracy than other relevant methods in spite of its relatively low computational cost on segment-by-segment and aggregated recognition tasks, respectively.
Multi-scale Field Distillation for Multi-task Semantic Segmentation

Semantic segmentation tasks in the field of computer vision have developed rapidly in recent years, because the development of autonomous driving technology requires more accurate semantic segmentation models. However, with the increasing demand, the use of deep models to solve semantic segmentation has reached a bottleneck. This is because the utilization of image information and correlations between images are not fully mined during model operation, and segmentation accuracy can only be achieved by deepening the model depth. To this end, we propose a multi-task semantic segmentation model based on multi-scale feature fusion and distillation to fully mine complementary information between related tasks. When building the multi-task learn framework, we firstly extracted multi-scale image information from the network layer at different depths, using feature fusion and jumping connection at different scales to make up for the loss of too much spatial information in the downsampling process, using distillation module to construct intermediate auxiliary tasks to roughly process features, and high-quality features can be obtained in the middle of training, thus alleviating the calculation pressure of the decoder and improving the accuracy of its own task. The proposed method uses three task-specific decoders to train for segmentation and two estimation tasks. Experiments on NYUv2 and Cityscapes data sets show that adding multi-scale information improves the efficiency of semantic segmentation and two estimation tasks, which shows the effectiveness of the proposed method.
Image Caption with Prior Knowledge Graph and Heterogeneous Attention

Currently, most image description models are limited in their ability to generate descriptions that reflect personal experiences and subjective perspectives. This makes it difficult to produce relevant and engaging descriptions that truly capture the essence of the image. To address this issue, we propose a novel approach called Subject-awareness-driven Heterogeneous Attention (SCHA). SCHA leverages users’ knowledge and expertise to generate content-adaptive image descriptions that are more human-like and reflective of personal experiences. Our approach involves a carefully designed heterogeneous cascade annotation model that captures scene information from multiple perspectives. We also incorporate a prior knowledge graph with textual information to enhance the richness and relevance of the generated descriptions. Our method has great potential for industrial production detection and can open up new possibilities for increasing the flexibility and variety of detection steps. When compared to the results of MSCOCO and Visual Genome datasets, our approach produces richer and more adaptive descriptions than widely used baseline models.
Lightweight Human Pose Estimation Based on Densely Guided Self-Knowledge Distillation

The current human pose estimation network has difficulty to be deployed on lightweight devices due to its large number of parameters. An effective solution is knowledge distillation, but there still exists the problem of insufficient learning ability of the student network: (1) There is an error avalanche problem in multi-teacher distillation. (2) There exists noise in heatmaps generated by teachers, which causes model degradation. (3) The effect of self-knowledge distillation is ignored. (4) Pose estimation is considered to be a regression problem but people usually ignore that it is also a classification problem. To address the above problems, we propose a densely guided self-knowledge distillation framework named DSKD to solve the error avalanche problem, propose a binarization operation to reduce the noise of the teacher’s heatmaps, and add a classification loss to the total loss to guide student’s learning. Experimental results show that our method effectively improves the performance of different lightweight models.
Ga-RFR: Recurrent Feature Reasoning with Gated Convolution for Chinese Inscriptions Image Inpainting

Inscriptions were a primary means of recording historical events and literary works in ancient times, and remain an important part of Chinese ancient civilization. However, due to their large quantity and prolonged exposure to the natural environment, inscriptions have suffered significant damage. Traditional manual restoration methods are time-consuming and labor-intensive, making it necessary to explore new restoration techniques. In this study, we introduce a Ga-RFR network that uses gated convolution to replace ordinary convolution. This technique reduces feature redundancy in generated feature maps and minimizes the production of unnecessary information, thereby enhancing restoration effectiveness. We also compare our method with other advanced image restoration algorithms, and our results demonstrate that our approach outperforms other current methods.
Generalisation Approach for Banknote Authentication by Mobile Devices Trained on Incomplete Samples

Reliable Banknote Authentication is critical for economic stability. Regarding everyday use, recent studies implemented successful techniques using banknote images taken by mobile phone cameras. One challenge in mobile banknote authentication is that it is impossible to collect images by all series/brands of mobile phones. In this study, classification models are implemented that are able to generalize to the samples from a wide number of mobile phone series even though they are trained with samples from a small group of series. Existing state-of-the-art banknote authentication approaches train a separate model per sub-image of a banknote, using the extracted features of that sub-image. A new approach that trains a single global model on the concatenated features of all the sub-images is presented. Furthermore, ensemble models that combine Linear Discriminant Analysis and Deep Neural Networks are employed in order to maximize the accuracy. Implemented techniques were able to reach up to F1-score of 0.99914 on a Euro banknote data set which contain images from 16 different mobile-phone series. The results also indicate that new global model approach can improve the accuracy of the existing banknote authentication techniques in case of model training with images from restricted/incomplete phone series and brands.
Combining Edge-Guided Attention and Sparse-Connected U-Net for Detection of Image Splicing

The use of image-splicing technologies had detrimental effects on the security of multimedia information. Hence, it is necessary to develop effective methods for detecting and locating such tampering. Previous studies have mainly focused on the supervisory role of the mask on the model. The mask edges contain rich complementary signals, which help to fully understand the image and are usually ignored. In this paper, we propose a new network named EAU-Net to detect and locat the splicing regions in the image. The proposed network consists of two parts: Edge-guided SegFormer and Sparse-connected U-Net (SCU). Firstly, the feature extraction module captures local detailed cues and global environment information, which are used to deduce the initial location of the affected regions by SegFormer. Secondly, a Sobel-based edge-guided module (EGM) is proposed to guide the network to explore the complementary relationship between splicing regions and their boundaries. Thirdly, in order to achieve more precise positioning results, SCU is used as postprocessing for removing false alarm pixels outside the focusing regions. In addition, we propose an adaptive loss weight adjustment algorithm to supervise the network training, through which the weights of the mask and the mask edge can be automatically adjusted. Extensive experimental results show that the proposed method outperforms the state-of-the-art splicing detection and localization methods in terms of detection accuracy and robustness.