Reviewerly/exHarmony huggingface dataset:
STFormer: Cross-Level Feature Fusion in Object Detection

Object detection algorithms can benefit from multi-level features, which encompass both high-level semantic information and low-level location details. However, existing detection methods face numerous challenges in effectively utilizing these multi-level features. Most existing detection techniques utilize simplistic operations such as feature addition or concatenation to fuse multi-level features, thereby failing to effectively suppress redundant information. Consequently, the performance of these algorithms is significantly constrained in complex scenarios. To address these limitations, this paper presents a novel feature extraction network that incorporates joint modeling and multi-dimensional feature fusion. Specifically, the network partitions the features of each level into tiles and employs hybrid self-attention mechanisms to extract these grouped features more comprehensively. Additionally, a hybrid cross-attention-based approach is utilized to regulate the transmission proportion of each grouped feature, facilitating the seamless integration of high-level semantic features obtained from deep encoders and the low-level position details retained by the pipeline. Consequently, the network effectively suppresses noise and enhances performance. Experimental evaluation on the MS COCO dataset demonstrates the effectiveness of the proposed approach, achieving an impressive accuracy of 54.3%. Notably, the algorithm showcases exceptional performance in detecting small-scale targets, surpassing the capabilities of other state-of-the-art technologies.
Improving Handwritten Mathematical Expression Recognition via an Attention Refinement Network

Handwritten mathematical expression recognition (HMER), typically regarding as a sequence-to-sequence problem, has made great progress in recent years, where RNN based models have been widely adopted. Although Transformer based model has demonstrated success in many areas, its performance is not satisfied due to the issue of standard attention mechanism in HMER. Therefore, we propose to improve the performance via an attention refinement network in the Transformer framework for HMER. We firstly adopt a shift window attention (SWA) from Swin Transformer to capture spatial contexts of the whole image for HMER. Moreover, we propose a refined coverage attention (RCA) to overcome the issue of lack of converge in the standard attention mechanism, where we utilize a convolutional kernel with a gating function to obtain coverage features. With the proposed RCA, we refine coverage attentions to attenuate the repeating issue of focused areas in the long-sequence. In addition, we utilize a pyramid data augmentation method to generate mathematical expression images with multiple resolutions to enhance the model generalization. We evaluate the proposed attention refinement network on the HMER benchmark datasets of CROHME2014/2016/2019, and extensive experiments demonstrate its effectiveness.
Multi-scale Directed Graph Convolution Neural Network for Node Classification Task

The existence of problems and objects in the real world which can be naturally modeled by complex graph structure has motivated researchers to combine deep learning techniques with graph theory. Despite the proposal of various spectral-based graph neural networks (GNNs), they still have shortcomings in dealing with directed graph-structured data and aggregating neighborhood information of nodes at larger scales. In this paper, we first improve the Lanczos algorithm by orthogonality checking method and Modified Gram-Schmidt orthogonalization technique. Then, we build a long-scale convolution filter based on the improved Lanczos algorithm and combine it with a short-scale filter based on Chebyshev polynomial truncation to construct a multi-scale directed graph convolution neural network (MSDGCNN) which can aggregate multi-scale neighborhood information of directed graph nodes in larger scales. We validate our improved Lanczos algorithm on the atom classification task of the QM8 quantum chemistry dataset. We also apply the MSDGCNN on various real-world directed graph datasets (including WebKB, Citeseer, Telegram and Cora-ML) for node classification task. The result shows that our improved Lanczos algorithm has much better stability, and the MSDGCNN outperforms other state-of-the-art GNNs on such task of real-world datasets.
Incomplete Multi-view Subspace Clustering Using Non-uniform Hyper-graph for High-Order Information

Incomplete multi-view subspace clustering (IMSC) is intended to exploit the information of multiple incomplete views to partition data into their intrinsic subspaces. Existing methods try to exploit the high-order information of data to improve the clustering performance, many tools are used such as tensor factorization and hyper-Laplacian regularization. Compared to using complex mathematical tools to solves problems, why not considering to get considerable improvements through some simple ways? To address this issue, we propose an incomplete multi-view subspace clustering method using non-uniform hyper-graph (NUHG-IMSC) method which makes slightly change to the usual way of constructing uniform hyper-graph. We find a set of data points that have high similarity with the center point of each hyper-edge in high-dimensional space to be its neighbor samples, the cardinality of each hyper-edge is decided based on the distribution of the corresponding center point. This is a simple but effective way to utilize high-order information without bringing computational burden and extra parameters. Besides the advantage that the partial samples can be reconstructed more reasonably, our method also brings benefits to other parts in the whole framework of IMSC, such as learning the view-specific affinity matrices, the weight of each view, and the unified affinity matrix. Experimental results on three multi-view data sets indicate the effectiveness of the proposed method.
Challenging Assumptions of Normality in AES s-Box Configurations under Side-Channel Analysis

Power-based Side-Channel Analysis (SCA) began with visual-based examinations and has progressed to utilize data-driven statistical analysis. Two distinct classifications of these methods have emerged over the years; those focused on leakage exploitation and those dedicated to leakage detection. This work primarily focuses on a leakage detection-based schema that utilizes Welch’s t-test, known as Test Vector Leakage Assessment (TVLA). Both classes of methods process collected data using statistical frameworks that result in the successful exfiltration of information via SCA. Often, statistical testing used during analysis requires the assumption that collected power consumption data originates from a normal distribution. To date, this assumption has remained largely uncontested. This work seeks to demonstrate that while past studies have assumed the normality of collected power traces, this assumption should be properly evaluated. In order to evaluate this assumption, an implementation of Tiny-AES-c with nine unique substitution-box (s-box) configurations is conducted using TVLA to guide experimental design. By leveraging the complexity of the AES algorithm, a sufficiently diverse and complex dataset was developed. Under this dataset, statistical tests for normality such as the Shapiro-Wilk test and the Kolmogorov-Smirnov test provide significant evidence to reject the null hypothesis that the power consumption data is normally distributed. To address this observation, existing non-parametric equivalents such as the Wilcoxon Signed-Rank Test and the Kruskal-Wallis Test are discussed in relation to currently used parametric tests such as Welch’s t-test.
Dynamic Feature Distillation

Feature-based knowledge distillation utilizes features from superior and complex teacher networks as knowledge to help portable student networks improve their generalization capability. Recent feature distillation algorithms focus on various feature processing and transmission methods while ignoring the flexibility of feature selection, resulting in limited distillation effects for students. In this paper, we propose Dynamic Feature Distillation to increase the flexibility of feature distillation by dynamically managing feature transfer sites. Our method leverages Online Feature Estimation to monitor the learning status of the student network in the feature dimension. Adaptive Position Selection then dynamically updates valuable feature transmission locations for efficient feature transmission. Notably, our approach can be easily integrated as a strategy for feature management into other feature-based knowledge transfer methods to improve their performance. We conduct extensive experiments on the CIFAR-100 and Tiny-ImageNet datasets to validate the effectiveness of Dynamic Feature Distillation.
Uformer++: Light Uformer for Image Restoration

Based on UNet, numerous outstanding image restoration models have been developed, and Uformer is no exception. The exceptional restoration performance of Uformer is not only attributable to its novel modules but also to the network's greater depth. Increased depth does not always lead to better performance, but it does increase the number of parameters and the training difficulty. In this paper, we propose Uformer++, a reconstructed Uformer based on an efficient ensemble of UNets of varying depths that partially share an encoder and co-learn simultaneously under deep supervision. Our proposed new architecture has significantly fewer parameters than the vanilla Uformer, but still with promising results achieved. Considering that different channel-wise features contain totally different weighted information and so are pixel-wise features, a novel Nonlinear Activation Free Feature Attention (NAFFA) module combining Simplified Channel Attention (SCA) and Simplified Pixel Attention (SPA) is added to the model. The experimental results on various challenging benchmarks demonstrate that Uformer++ has the least computational cost while maintaining performance.
A Hybrid Approach Using Convolution and Transformer for Mongolian Ancient Documents Recognition

Mongolian ancient documents are an indispensable source for studying Mongolian traditional culture. To thoroughly explore and effectively utilize these ancient documents, conducting a comprehensive study on Mongolian ancient document words recognition is essential. In order to better recognize the word images in Mongolian ancient documents, this paper proposes an approach that combines convolutional neural networks with Transformer models. The approach used in this paper takes word images as the input for the model. After passing through a feature extractor composed of convolutional neural networks, the extracted features are fed into a Transformer model for prediction. Finally, the corresponding recognition results of the word images are obtained. Due to the common existence of imbalanced distribution of character classes in recognition tasks, models often tend to excessively focus on common characters while neglecting rare characters. Our proposed approach integrates focal loss to enhance the model's attention towards rare characters, thereby improving the overall recognition performance of the model for all characters. After training, the model is capable of rapidly and efficiently performing end-to-end recognition of words in Mongolian ancient documents. The experimental results indicate that our proposed approach outperforms existing methods for word recognition in Mongolian ancient documents, effectively improving the performance of Mongolian ancient document words recognition.
Multimodal Event Classification in Social Media

Currently, research on events mainly focuses on the task of event extraction, which aims to extract trigger words and arguments from text and is a fine-grained classification task. Although some researchers have improved the event extraction task by additionally constructing external image datasets, these images do not come from the original source of the text and cannot be used for detecting real-time events. To detect events in multimodal data on social media, we propose a new multimodal approach which utilizes text-image pairs for event classification. Our model uses a unified language pre-trained model CLIP to obtain visual and textual features, and builds a Transformer encoder as a fusion module to achieve interaction between modalities, thereby obtaining a good multimodal joint representation. Experimental results show that the proposed model outperforms several state-of-the-art baselines.
Co-GAN: A Text-to-Image Synthesis Model with Local and Integral Features

Text-to-Image synthesis is a promising technology that generates realistic images from textual descriptions by deep learning model. However, the state-of-the-art text-to-image synthesis models often struggle to balance the overall integrity and local diversity of objects with rich details, leading to unsatisfactory generation results of some domain-specific images, such as industrial applications. To address this issue, we propose Co-GAN, a text-to-image synthesis model that introduces two modules to enhance local diversity and maintain overall structural integrity respectively. Local Feature Enhancement (LFE) module improves the local diversity of generated images, while Integral Structural Maintenance (ISM) module ensures that the integral information is preserved. Furthermore, a cascaded central loss is proposed to address the instability during the generative training. To tackle the problem of incomplete image types in existing datasets, we create a new text-to-image synthesis dataset containing seven types of industrial components, and test the effects of various existing methods based on the dataset. The results of comparative and ablation experiments show that, compared with other current methods, the images generated by Co-GAN incorporate more details and maintain the integrity.
Deep Hashing for Multi-label Image Retrieval with Similarity Matrix Optimization of Hash Centers and Anchor Constraint of Center Pairs

Deep hashing can improve computational efficiency and save storage space, which is the most significant part of image retrieval task and has received extensive research attention. Existing deep hashing frameworks mainly fall into two categories: single-stage and two-stage. For multi-label image retrieval, most single-stage and two-stage deep hashing methods usually consider two images to be similar if one pair of the corresponding category labels is the same, and do not make full use of the multi-label information. Meanwhile, some novel two-stage deep hashing methods proposed in recent years construct hash centers firstly and then train through deep neural networks. For multi-label processing, these two-stage methods usually convert multi-label into single-label objective, which also leads to insufficient use of label information. In this paper, a novel multi-label deep hashing method is proposed by constructing the similarity matrix and designing the optimization algorithm to construct the hash centers, and the proposed method constructs the training loss function through the multi-label hash centers constraint and anchor constraint of center pairs. Experiments on several multi-label image benchmark datasets show that the proposed method can achieve the state-of-the-art results.
A Corpus of Quotation Element Annotation for Chinese Novels: Construction, Extraction and Application

Quotations or dialogues are important for literary works, like novels. In the famous Jin Yong's novels, about a half of all sentences contain quotations. Quotation elements like speaker, speech mode, speech cue and the quotation itself are very useful to the analysis of fictional characters. To build models for automatic quotation element extraction, we construct the first quotation corpus with annotation of all the four quotation elements, and the corpus size of 31,922 quotations is one of the largest to our knowledge. Based on the corpus, we compare different models for quotation element extraction and conduct extensive experiments. For the application of extracted quotation elements, we explore character recognition and gender classification, and find out that quotation and speech mode are effective for the two tasks. We will extend our work from Jin Yong's novels to other novels to analyze various characters from different angles based on quotation structures.
Multi-level Feature Enhancement Method for Medical Text Detection

In recent years, Segmentation-based text detection methods have been widely applied in the field of text detection. However, when it comes to tasks involving text-dense, extreme aspect ratios, and multi-oriented text, the current detection methods still fail to achieve satisfactory performance. In this paper, we propose an efficient and accurate text detection system that incorporates an efficient segmentation module and a learnable post-processing method. More specifically, the segmentation head consists of an Efficient Feature Enhancement Module (EFEM) and a Multi-Scale Feature Fusion Module(MSFM). The EFEM is a cascaded U-shaped module that incorporates spatial attention mechanism to introduce multi-level information for improved segmentation performance. MSFM integrates features from the EFEM at different depths and scales to generate the final features for segmentation. Moreover, a post-processing module employing a differentiable binarization approach is utilized, allowing the segmentation network to adaptively set the binarization threshold. The proposed model demonstrates excellent performance on medical text image datasets. Multiple benchmark experiments further validate the superior performance of the proposed method. Code is available at: https://github.com/csworkcode/EFDBNet .
Improve Conversational Search with Multi-document Information

Conversational Search (CS) aims to satisfy complex information needs via multi-turn user-agent interactions. During this process, multiple documents need to be retrieved based on the conversation history to respond to the user. However, existing approaches still make it difficult to distinguish irrelevant information from the user's question at the semantic level. When a conversation involves multiple documents, it sometimes affects the retrieval performance negatively. In order to enhance the model's ability to comprehend conversations and distinguish passages in irrelevant documents via multi-document information, we propose an unsupervised multi-document conversation segmentation method and a zero-shot Large Language Model (LLM)-based document summarization method to extract multi-document information from conversation history and documents respectively for amending the lack of training data for extracting multiple document information. We further present the Passage-Segment-Document (PSD) post-training method to train the reranker using the extracted multi-document information in combination with a multi-task learning method. The results on the MultiDoc2Dial dataset verifies the improvement of our method on retrieval performance. Extensive experiments show the strong performance of our method for dealing with conversation histories that contain multi-document information.
Sub-Instruction and Local Map Relationship Enhanced Model for Vision and Language Navigation

In this paper, different from most methods in vision and language navigation, which primarily relies on vision-language cross-modal attention modeling and the agent's egocentric observations. We establish connections between sub-instructions and local maps to elaborately encode environment information and learn a path responsible for the whole instructions rather than the ultimate goal. We first obtain a local semantic map by ground projecting the RGB semantic segmentation map and the depth map. The segmented sub-instruction is passed through the sub-instruction attention module and then taken as input, together with the local map, to the cross-modal attention module. Finally, a set of waypoints are predicted by the navigation module until all sub-instructions of the long instruction are executed, which is the completion of an episode. Comparison experiments and ablation studies on the VLN-CE dataset show that our method outperforms most methods and has a good whole path predicting ability.
Assessing and Enhancing LLMs: A Physics and History Dataset and One-More-Check Pipeline Method

Large language models(LLMs) demonstrate significant capabilities in traditional natural language processing(NLP) tasks and many examinations. However, there are few evaluations in regard to specific subjects in the Chinese educational context. This study, focusing on secondary physics and history, explores the potential and limitations of LLMs in Chinese education. Our contributions are as follows: a PH dataset is established, which concludes secondary school physics and history in Chinese, comprising thousands of multiple-choice questions; an evaluation on three prevalent LLMs: ChatGPT, GPT-3, ChatGLM on our PH dataset is made; a new prompt method called One-More-Check(OMC) is proposed to enhance the logical reasoning capacity of LLMs; finally, three LLMs are set to attend an actual secondary history exam. Our findings suggest that our OMC method improves the performance of LLMs on logical reasoning and LLMs underperform average level of age-appropriate students on the exam of history. All datasets, code and evaluation results are available at https://github.com/hcffffff/PH-dataset-OMC .
Dual-Domain Learning for JPEG Artifacts Removal

JPEG compression brings artifacts into the compressed image, which not only degrades visual quality but also affects the performance of other image processing tasks. Many learning-based compression artifacts removal methods have been developed to address this issue in recent years, with remarkable success. However, existing learning-based methods generally only exploit spatial information and lack exploration of frequency domain information. Exploring frequency domain information is critical because JPEG compression is actually performed in the frequency domain using the Discrete Cosine Transform (DCT). To effectively leverage information from both the spatial and frequency domains, we propose a novel Dual-Domain Learning Network for JPEG artifacts removal (D2LNet). Our approach first transforms the spatial domain image to the frequency domain by the fast Fourier transform (FFT). We then introduce two core modules, Amplitude Correction Module (ACM) and Phase Correction Module (PCM), which facilitate interactive learning of spatial and frequency domain information. Extensive experimental results performed on color and grayscale images have clearly demonstrated that our method achieves better results than the previous state-of-the-art methods. Code will be available at https://github.com/YeunkSuzy/Dual_Domain_Learning .
B-PSA: A Binary Pendulum Search Algorithm for the Feature Selection Problem

The digitization of information and technological advancements have enabled us to gather vast amounts of data from various domains, including but not limited to medicine, commerce, and mining. Machine learning techniques use this information to improve decision-making, but they have a big problem: they are very sensitive to data variation, so it is necessary to clean them to remove irrelevant and redundant information. This removal of information is known as the Feature Selection Problem. This work presents the Pendulum Search Algorithm applied to solve the Feature Selection Problem. As the Pendulum Search Algorithm is a metaheuristic designed for continuous optimization problems, a binarization process is performed using the Two-Step Technique. Preliminary results indicate that our proposal obtains competitive results when compared to other metaheuristics extracted from the literature, solving well-known benchmarks.