Reviewerly/exHarmony huggingface dataset:
Complex Network-Based Data Classification Using Minimum Spanning Tree Metric and Optimization

Generally speaking, data classification techniques divide the data or feature space into sub-spaces, where each one corresponds to a class. For this purpose, physical features, such as distance and distributions, are considered. Such an approach has difficulty to characterize complex patterns embedded in the training and testing data sets. On the other hand, complex networks are a powerful tool to capture internal relationships and class structures to realize High-Level Classification. Several complex network-based classification techniques have been proposed. The main advantage of high-level classification is its ability to classify data according to the pattern formation. In this work, we present a new network-based classification technique considering a unique measure extracted from the Minimum Spanning Tree (MST) to characterize the data pattern represented by the network constructed for each class. Promising numerical results have been obtained in comparison to classic high-level classification techniques. Still in this paper, we apply the proposed technique to chest X-ray image classification for COVID-19 diagnosis and snoring signal classification. Moreover, the proposed model presents the following distinguished features in comparison to the previous high-level classification techniques: 1) Since only one network measure is used to characterize data pattern, no wight parameters among network measures is need to be determined. Consequently, the model is largely simplified and, at the same time, can obtain better classification result; 2) an optimization approach is introduced to construct more suitable network for each class of the training data; 3) the network measure based on MST is more sensitive to the insertion of new nodes. This is an important feature because each time only one testing data sample is inserted in each class network for classification.
Auto-TabTransformer: Hierarchical Transformers for Self and Semi Supervised Learning in Tabular Data

Self and Semi-Supervised Learning have shown promising results in language and computer vision but are still underexplored in the context of tabular data. This paper focuses on exploring self and semi-supervised methods for tabular data. Towards this, we have proposed Auto-Tab Transformer, a method for training hierarchical transformers in a self and semi-supervised setup using redundancy reduction. The technique focuses on key aspects of self and semi-supervised learning: feature encoding, pre-training objective, training methodology and neural architecture. Performing extensive experiments on four publically accessible datasets, we show that Auto-Tab Transformer achieves state of the art (SOTA) results in the less labelled data domain. We conduct extensive ablation studies detailing the importance of all the components used.
Personalized Federated Learning via Gradient Modulation for Heterogeneous Text Summarization

Text summarization is essential for information aggregation and demands large amounts of training data. However, concerns about data privacy and security limit data collection and model training. To eliminate this concern, we propose a federated learning text summarization scheme, which allows users to share the global model in a cooperative learning manner without sharing raw data. Personalized federated learning (PFL) balances personalization and generalization in the process of optimizing the global model, to guide the training of local models. However, multiple local data have different distributions of semantics and context, which may cause the local model to learn deviated semantic and context information. In this paper, we propose FedSUMM, a dynamic gradient adapter to provide more appropriate local parameters for local model. Simultaneously, FedSUMM uses differential privacy to prevent parameter leakage during distributed training. Experimental evidence verifies FedSUMM can achieve faster model convergence on PFL algorithm for task-specific text summarization, and the method achieves superior performance for different optimization metrics for text summarization.
Decoupling Algorithms from Schedules for Easy Optimization of Image Processing Pipelines

Using existing programming tools, writing high-performance image processing code requires sacrificing readability, portability, and modularity. We argue that this is a consequence of conflating what computations define the algorithm, with decisions about storage and the order of computation. We refer to these latter two concerns as the schedule, including choices of tiling, fusion, recomputation vs. storage, vectorization, and parallelism
Exploiting User Preference in GNN-based Social Recommendation with Contrastive Learning

Social recommendation enhances the learning of user preferences by incorporating user social information. Recently, graph neural network models have gradually become the subject of the social recommendation. However, most graph neural network-based approaches fail to fully learn the high-order collaborative semantics of user interest and social domains, and ignore the unique self-supervised signals in user social domains. To alleviate these problems, we propose a novel lightweight GCN-based social recommendation method SGSR that jointly models the high-order collaborative relations of user/item nodes in both domains. Meanwhile, in the process of message transmission of the bipartite graph and social graph, we respectively introduce a self-attention mechanism to measure the contributions of different nodes. In particular, to take full advantage of the self-supervised signals between user node messages in the social domain, we innovatively incorporate contrastive learning into this system to enable user-side node features to self-learn and update. Extensive experiments conducted on two real datasets demonstrate the effectiveness and necessity of our proposed approach.
CSARUNet: an attention mechanism-based model for image tampering localization with ringed residual block

Nowadays, image tampering technology is becoming more and more mature, tampered images have posed a serious threat to personal privacy, social order, and national security, so techniques for detecting and locating tampered areas in an image are gaining attention. Most existing methods use convolutional blocks to build networks to accomplish this task, without considering the local receptive fields of the convolutional operation. In this paper, we propose an effective method for image tampering localization. In order to extract the features of the tampered region efficiently, we use the ringed residual block in our model, which is a modification of the classic residual block in ResNet. Also, to enable the model to pay more attention to the tampered region, we use various attention modules such as SE block, CA block, and AttentionGate to enhance the ability to focus on important information. The comprehensive experiments on benchmark datasets justify the effectiveness of the proposed method.
Multilingual Pre-training Model-Assisted Contrastive Learning Neural Machine Translation

Since pre-training and fine-tuning have been a successful paradigm in Natural Language Processing (NLP), this paper adopts the SOTA pre-training model-CeMAT as a strong assistant for low-resource ethnic language translation tasks. Aiming at the exposure bias problem in the fine-tuning process, we use the contrastive learning framework and propose a new contrastive examples generation method, which uses self- generated predictions as contrastive examples to expose the model to errors during inference. Moreover, in order to effectively utilize the limited bilingual data in low-resource tasks, this paper proposes a dynamic training strategy to fine-tune the model, and refines the model step by step by taking word embedding norm and uncertainty as the criteria of evaluate data and model respectively. Experimental results demonstrate that our method significantly improves the quality compared to the baselines, which fully verifies the effectiveness.
A Novel Cross-Fusion Method of Different Types of Features for Image Captioning

Multi-modal tasks are receiving more and more attention, including image captioning. Based on X-Linear attention, we simultaneously introduce grid features and region features extracted by Faster RCNN. We obtain a global feature vector of each type of original features through mean pooling. The two types of features are encoded by two parallel encoders. Each encoder has two inputs: a set of feature vectors (region/grid) and the corresponding global feature vector. Each encoding layer outputs an encoded global feature vector and a set of encoded feature vectors. We cross-fuse the global feature vector output by each encoding layer for region features and the set of encoded feature vectors for grid features. In the same way, we cross-fuse another pair of the global feature (grid) and the set of encoded feature vectors (region). Finally, we fuse the two global feature vectors output by the two encoders as the final global features, and the two sets of encoded feature vectors output by the two encoders as the final visual features. Experimental results on the COCO dataset show that our model achieves a new SOTA performance of BLEU-1 81.5%, BLEU-4 40.5%, METEOR 29.6%, and ROUGE 59.5% on the Karpathy test split.
AutoDes: Few-Shot Named Entity Recognition with Class Descriptions

Few-shot named entity recognition is extremely important for the domain lacking annotation data. Existing approaches ignore that class descriptions can provide additional information and rich prior knowledge for the model. Most datasets do not provide class description information, and the same entity class may have different definitions in different datasets. In this paper, we proposed a simple and effective method, i.e., AutoDes, that can extract class descriptions from annotated data automatically. AutoDes uses examples to construct class descriptions and takes the prediction words of entity-oriented prompts as candidate examples. Experiments with different few-shot settings on multiple datasets show that AutoDes is superior to the state-of-the-art methods in low-resource settings, improving F1 scores by 1.2% to 7.5% absolute points.
A Semantic and Structural Transformer for Code Summarization Generation

Currently most methods cast code summarization generation as a machine translation task. Wherein the Transformer framework is a representative among them. Thanks to the attention mechanism in the Transformer, such a framework has achieved the state-of-the-art performance. Unfortunately, the Transformer encounters a series of challenges when generalizing to code summarization generation domain. Compared with natural language, code sequence is characterized by more complex multi-modal features, and difficult to extract these features only by the original Transformer structure. To further improve the performance, we make full use of code semantic and structural information in abstract syntax tree to build a simple yet effective framework, which consists of self-attention and graph based module to integrate code semantic information and syntax tree structure information. Besides, to compensate for the insufficiency of Transformer in encoding local features, we present a well-designed local RNN module. Extensive experiments show that the proposed method performs on par with the state-of-the-art methods on two public benchmarks, including Java and Python datasets. The comprehensive ablation studies further demonstrate the effectiveness of architecture design choices. The source code is released at https://github.com/tzv314159/SSTrans.git.
Video-based Driver Action Recognition via Spatial-Temporal and Motion Deep Learning

Driver action recognition aims to identify different driving actions of drivers, which is of great significance for traffic safety monitoring. However, it is still challenging for the existing methods to identify similar actions which are common situations in driving scenarios. The fundamental reason is that they ignore the global features of videos and fail to extract the most discriminative features. To address this issue, we propose a video-based driver action recognition network via deep learning of spatial-temporal and motion features, which includes two important modules: the global spatial-temporal modeling (GSTM) module and the motion-spatial-temporal joint attention (MSTJA) module. GSTM effectively extracts the global spatial-temporal features of driving actions by expanding the equivalent receptive field of the spatial-temporal dimension through spatial-temporal separable convolution and hierarchical residual connection structures. MSTJA forces the network to focus on the most discriminative areas by jointly exciting the motion patterns of subtle variations and significant spatial-temporal features through the extracted dual path motion features and spatial-temporal features. Experiments demonstrate that the proposed network achieves state-of-the-art classification accuracy on two benchmark driver action datasets Drive&Act and SFD3.
Cortex Inspired Learning to Recover Damaged Signal Modality with ReD-SOM Model

Recent progress in the fields of AI and cognitive sciences opens up new challenges and problems that were previously inaccessible to study. One of such modern tasks is recovering lost data of one modality by using the data from another one. A similar effect (called the McGurk Effect) has been found in the functioning of the human brain. Observing this effect, one modality of information interferes with another, changing its perception. In this paper, we propose a way to reproduce such an effect and use it to reconstruct lost data modalities by combining Variational Auto-Encoders, Self-Organizing Maps, and Hebb connections in a unified ReD-SOM (Reentering Deep Self-organizing Map) model. We are inspired by human's capability to use different zones of the brain in different modalities, in case of having a lack of information in one of the modalities. This new approach not only improves the analysis of ambiguous data but also restores the intended signal. The results obtained on the multimodal dataset show an increase of quality of the signal reconstruction. The effect is remarkable both visually and quantitatively, specifically in presence of a significant degree of signal's distortion.
Scene Completion Using Millions of Photographs

What can you do with a million images? In this paper we present a new image completion algorithm powered by a huge database of photographs gathered from the Web. The algorithm patches up holes in images by finding similar image regions in the database that are not only seamless but also semantically valid. Our chief insight is that while the space of images is effectively infinite, the space of semantically differentiable scenes is actually not that large. For many image completion tasks we are able to find similar scenes which contain image fragments that will convincingly complete the image. Our algorithm is entirely data-driven, requiring no annotations or labelling by the user. Unlike existing image completion methods, our algorithm can generate a diverse set of results for each input image and we allow users to select among them. We demonstrate the superiority of our algorithm over existing image completion approaches
DDNet: Density and depth-aware network for object detection in foggy scenes

Fog causes serious degradation in image quality that in turn can degrade the performance of object detection. The main reason can be concluded that (i) the degraded images make object localization difficult, (ii) the difficulty in extracting robust features for accurate detection results in various fog densities. To address the above two problems, in this paper, we propose a simple yet efficient network named density and depth-aware network (DDNet), which consists of a density-aware attention network (DAANet) and a depth-aware non-local contextual network (DNCNet). The DNCNet captures long-range dependencies guided by depth information to improve object localization. DAANet employs an attention mechanism guided by predicted fog densities to ensure the robustness of features under different fog densities. Experiments are performed on the FoggyDriving dataset. Our approach achieves the state-of-the-art performance.
Multi-Task Collaborative Attention Network for Pedestrian Attribute Recognition

Pedestrian Attribute Recognition (PAR) is a multi-task attribute leaning problem. Research into person attributes recognition has focused on approaches to describe a person in terms of their appearance. Combination of some attributes is helpful to strengthen each other's learning such as upper clothing style and upper clothing length, while others are not, such as hair style and upper clothing length. Thus, how to effectively combine different task is the key challenges in PAR. To effectively utilizing the relationship between attributes and further improve the effects of PAR, we propose a novel Multi-Task Collaborative Attention Network (MTCAN), which consists of three modules. Specifically, we first design a Feature Division Module (FDM) to focus on reliable and flexible attribute-related regions. Based on the precise attribute-related locations, we further construct a Spatial and Channel Collaborative Attention Module (SCCAM) to facilitate the beneficial features and weaken mutually suppressed features. Thirdly, a newly weights fusion strategy named adaptive-soups is proposed to mine the optimal model which is universal for deep learning models in all fields. Experiments on two pedestrian attribute recognition datasets show that our proposed method achieves superior performance against other state-of-the-art methods.
A Trigger-Free Method Enhanced by Coreference Information for Document-Level Event Extraction

Document-level event extraction (DEE) aims to extract structured event information from a document. Previous document-level event extraction methods relied on trigger annotation, which was very expensive and time-consuming. In addition, entity representation plays an important role in the overall event extraction task, but we found that the previous work could not effectively use entity abbreviations and coreference information, which limited the representation ability of event-related entities in documents. Based on the above two aspects, we propose a new model named TFECI. In our model, we propose an efficient and effective method to make full use of abbreviations and coreference information to better exert the representational ability of event-related entities. Besides, triggers play an extremely central role in an event, but trigger annotation is often very difficult, so we propose a new strategy to select pseudo-trigger automatically. Experiments show that, compared with the previous system, our system can extract events without trigger annotation and achieve competitive results.
Benchmarking Feature Extraction Techniques for Textual Data Stream Classification

Feature extraction regards transforming unstructured or semi-structured data into structured data that can be used as input for classification and sentiment analysis algorithms, among other applications. This task becomes even more challenging and relevant when textual data becomes available over time as a continuous data stream since the lexicon and semantics can be ever-evolving. Data streams are, by definition, potentially infinite sequences of data that may have ephemeral characteristics, that is, where the data behavior changes, it leads to a phenomenon named concept drift. Textual data streams are specialized data streams, in which texts arrive over time from a continual data source, such as social media, raising challenges in which feature extractors are of great help. In this paper, we benchmark different feature extraction algorithms, i.e., Hashing Trick, Word2Vec, BERT, and Incremental Word-Vectors; in textual data stream classification, considering different stream lengths. The evaluation was performed over a binary and a multiclass classification task, considering two different datasets. Results show that pre-trained models, such as BERT, achieve interesting results, while Hashing Trick also performs competitively. We also observe that incremental methods such as Word2Vec and Incremental Word-Vectors are the most prepared for changing scenarios, yet, they are much more computationally intensive compared to the former when applied to larger streams.
A transformer-based network for perceptual contrastive underwater image enhancement

Vision-based underwater image enhancement methods have received much attention for application in the fields of marine engineering and marine science. The absorption and scattering of light in real underwater scenes leads to severe information degradation in the acquired underwater images, thus limiting further development of underwater tasks. To solve these problems, a novel transformer-based perceptual contrastive network for underwater image enhancement methods (TPC-UIE) is proposed to achieve visually friendly and high-quality images, where contrastive learning is applied to the underwater image enhancement (UIE) task for the first time. Specifically, to address the limitations of the pure convolution-based network, we embed the transformer into the UIE network to improve its ability to capture global dependencies. Then, the limits of the transformer are then taken into account as convolution is reintroduced to better capture local attention. At the same time, the dual-attention module strengthens the network’s focus on the spatial and color channels that are more severely attenuated. Finally, a perceptual contrastive regularization method is proposed, where a multi-loss function made up of reconstruction loss, perceptual loss, and contrastive loss jointly optimizes the model to simultaneously ensure texture detail, contrast, and color consistency. Experimental results on several existing datasets show that the TPC-UIE obtains excellent performance in both subjective and objective evaluations compared to other methods. In addition, the visual quality of the underwater images is significantly improved by the enhancement of the method and effectively facilitates further development of the underwater task.
A water surface garbage recognition method based on transfer learning and image enhancement

In this paper, we propose a solution to the escalating issue of water pollution caused by garbage in rivers, reservoirs, and oceans. We introduce the implementation of the You Only Look Once version 5 (YOLOv5) algorithm to address the challenges associated with identifying garbage targets, such as light reflection and varying shooting angles. Our algorithm effectively tackles the issue of identifying small and variable-sized targets, while incorporating image enhancement techniques to enhance the adaptability of the target recognition algorithm to different lighting conditions for accurate identification of water surface garbage. To prevent overfitting, we employ the k-fold cross-validation method, and further enhance the algorithm's performance through transfer learning, ensemble learning, and test-time augmentation (TTA). During the ensemble learning stage, we evaluate a half-precision model to strike a balance between prediction speed and accuracy, ultimately achieving an optimal solution. Using our self-made dataset, our study demonstrates a detection speed of 67.7 ms and a mean average precision at 50% IOU (mAP50) of 98.1%. These results exhibit practical application value, surpassing other deep learning methods in this field.