Reviewerly/exHarmony huggingface dataset:
Taper Residual Dense Network for Audio Super-Resolution

Audio Super-Resolution (Audio SR) aims to seek a mapping method from low-resolution audio to high-resolution audio, in which deep-learning-based approaches have achieved encouraging results in recent years. However, most Audio SR models do not break out of the limitations of the framework of UNet, which only have skip connections between the corresponding encoders and decoders. In this paper, we propose a taper residual dense network (TNet) for Audio SR. Specifically, 1) we introduce Residual Dense Block (RDB) and modify it to make it suitable for processing audio signals, which can fully utilize the features extracted by the network at different levels; 2) we design a taper network architecture that models and extracts features based on frequency-domain amplitude, frequency-domain phase as well as time-domain waveform in different lower branches, and then performs feature fusion in the upper branch. Experimental results show that our proposed TNet outperforms other baselines with fewer parameters.
Temporal Attention Signatures for Interpretable Time-Series Prediction

Deep neural networks have become a staple in time-series prediction due to their remarkable accuracy. However, their internal workings often remain elusive. Significant advancements have been made in the interpretability of these networks, with attention mechanisms and feature maps being notably effective for image classification by highlighting the crucial data points. While human observers can readily confirm the significance of features in image classification, the interpretability of time-series data and its modeling remains challenging. To address this, we put forth an innovative approach that unifies temporal attention and visualization as a blend of recurrent neural networks, self-attention, and general attention. This synergy results in the generation of temporal attention signatures, akin to image attention heat maps. Temporal attention not only enhances prediction accuracy beyond that of recurrent networks alone but also demonstrates that varying label classes yield distinct attention signatures. This observation indicates that neural networks focus on different sections of time-series sequences contingent on the prediction target. We conclude with a discussion on the practical implications of this novel approach, including its applicability to model interpretation, sequence length selection, and model validation. This leads to more accurate, robust, and interpretable models, instilling greater confidence in their results.
LorenTzE: Temporal Knowledge Graph Embedding Based on Lorentz Transformation

Inferring missing facts in knowledge graphs(KGs) has been proved to be a significant task due to the incompleteness problem. Temporal knowledge graphs(TKGs) in which each fact is incorporated with a timestamp to model the dynamic properties of facts, suffer from the incompleteness either. Thus temporal knowledge graph completion(TKGC) is introduced to infer missing dynamic facts. However, most of the existing TKGC models are proposed based on existing static KG embedding models and then try to learn embeddings for timestamps and incorporate them into the score functions, which, in several cases, impairs models’ expressivity and interpretabilities. In this paper, we provide a novel physical view for TKGC and propose LorenTzE, an embedding based model inspired by Lorentz transformation [1] in which each embedding is endowed with physical meanings. We also prove that LorenTzE is capable of modeling various relation patterns under temporal settings. Experimental results on three main TKGC benchmarks demonstrate the effectiveness and superiorities to several static and temporal models of our proposed method.
GanNeXt: A New Convolutional GAN for Anomaly Detection

Anomaly detection refers to the process of detecting anomalies from data that do not follow its distribution. In recent years, Transformer-based methods utilizing generative adversarial networks (GANs) have shown remarkable performance in this field. Unlike traditional convolutional architectures, Transformer structures have advantages in capturing long-range dependencies, leading to a substantial improvement in detection performance. However, transformer-based models may be limited in capturing fine-grained details as well as the inference speed. In this paper, we propose a scalable convolutional Generative Adversarial Network (GAN) called GanNeXt. Our design incorporates a new convolutional architecture that utilizes depthwise convolutional layers and pointwise convolutional layers as extension layers. In addition, we introduce skip connections to capture multi-scale local details. Experiments demonstrate that our proposed method achieves a 58% reduction in floating-point operations per second (FLOPs), while outperforming state-of-the-art Transformer-based GAN baselines on CIFAR10 and STL10 datasets. The codes will be available at https://github.com/SYLan2019/GanNeXt .
Conditional Convolution Residual Network for Efficient Super-Resolution

With the continuous development of deep learning, single-image super-resolution (SISR) based on convolutional neural networks (CNNs) has made significant progress. Although CNN-based methods have achieved great success, these methods are difficult to apply to edge devices due to the need for large amounts of computing resources. To address this problem, the latest advancements in efficient SISR techniques focus on reducing the number of parameters and multiply-add operations (MAdds). In this paper, we propose a novel Conditional Convolution Residual Network (CCRN) to tackle this challenge. The main idea is to use conditional convolution instead of ordinary convolutional layers for residual feature learning and to combine Contrast-aware Channel Attention (CCA) and Enhanced Spatial Attention (ESA) mechanisms to improve the model’s performance. The model’s performance is ensured while reducing the computational complexity. Experimental results demonstrate that CCRN has fewer MAdds than existing SISR methods while achieving state-of-the-art performance.
pFedLHNs: Personalized Federated Learning via Local Hypernetworks

As an emerging paradigm, federated learning (FL) trains a shared global model by multi-party collaboration without leaking privacy since no private data transmission between the server and clients. However, it still faces two challenges: statistical heterogeneity and communication efficiency. To tackle them simultaneously, we propose pFedLHNs, which assigns each client with both a small hypernetwork (HN) and a large target network (NN) whose parameters are generated by the hypernetwork. Each client pulls other clients’ hypernetworks from the server for local aggregation to personalize its local target model and only interacts the small hypernetwork with other clients via the central server to reduce communication costs. Besides, the server also aggregates received local hypernetworks to construct a global hypernetwork and uses it to initialize new joining out-of-distribution (OOD) clients for cold start. Extensive experiments on three datasets with Non-IID distributions demonstrate the superiority of pFedLHNs in the trade-off between model accuracy and communication efficiency. The case studies justify its tolerance to statistical heterogeneity and new OOD clients.
EGCN: A Node Classification Model Based on Transformer and Spatial Feature Attention GCN for Dynamic Graph

Node classification is an important area in graph data-related research and has attracted the attention of many researchers. Graph structure data can be divided into static graphs and dynamic graphs. Due to the temporal characteristics of dynamic graphs, dynamic graphs have higher information density and richer data features than static graph data. In order to better complete the extraction of the features for dynamic graphs, we design a comprehensive feature extraction module and then create the dynamic graph node classification model EGCN. The introduced time dimension in the dynamic graph not only brings the temporal feature but also brings the implicit spatial feature, which is mostly ignored in related research. So we propose a method for modeling implicit spatial features and combine it with GCN to introduce SAGCN for implicit spatial feature attention. We also introduce a temporal feature extraction module called TEncoder based on the transformer and combine them to design the dynamic graph node classification model EGCN. EGCN has two variants, EGCN-S and EGCN-T, which differ in their temporal and spatial characteristics of attention. Experiments show that EGCN achieves state-of-the-art performance. EGCN-S and EGCN-T achieved 1% and 5% improvement over the main baseline DS-TAGCN on dataset DBLP.
Detecting Negative Sentiment on Sarcastic Tweets for Sentiment Analysis

Sentiment Analysis (SA) is a fundamental and practical research problem in the field of natural language understanding(NLU). Meanwhile, sarcasm detection is a task to detect sarcasm in textual data. Previous works solve these two problems independently and neglect the fact that sarcasm is omnipresent and non-negligible during sentiment analysis. To explore this issue, in this paper, we formulate a general sentiment Analysis (GSA) problem where sarcastic data could be input and point out the limitations of current mainstream frameworks by systematic investigation. To address the GSA problem, we propose a sarcasm-perceivable SA (Sp-SA) training framework to train a model that is robust to sarcasm and able to achieve state-of-the-art performance. Extensive experiments and detailed analysis demonstrate our Sp-SA framework’s effectiveness and interpretability. Code and dataset will be publicly available for future research.
A Comparative Study of Sentence Embedding Models for Assessing Semantic Variation

Analyzing the pattern of semantic variation in long real-world texts such as books or transcripts is interesting from the stylistic, cognitive, and linguistic perspectives. It is also useful for applications such as text segmentation, document summarization, and detection of semantic novelty. The recent emergence of several vector-space methods for sentence embedding has made such analysis feasible. However, this raises the issue of how consistent and meaningful the semantic representations produced by various methods are in themselves. In this paper, we compare several recent sentence embedding methods via time-series of semantic similarity between successive sentences and matrices of pairwise sentence similarity for multiple books of literature. In contrast to previous work using target tasks and curated datasets to compare sentence embedding methods, our approach provides an evaluation of the methods “in the wild”. We find that most of the sentence embedding methods considered do infer highly correlated patterns of semantic similarity in a given document, but show interesting differences.
Sh-DistilBERT: New Transfer Learning Model for Arabic Sentiment Analysis and Aspect Category Detection

Arabic sentiment analysis is the process of computationally identifying and categorizing opinions expressed in a piece of text, particularly to assess whether the writer’s attitude toward a given topic, product, etc. is positive, negative, or neutral. In sentiment analysis, aspect category detection (ACD) attempts to identify the aspect categories mentioned in a sentence. Our study investigates the effects of transfer learning across several Arabic NLP tasks. We proposed a new shared DistilBERT model, which is a fine-tuned version of the basic DistilBERT. Our results demonstrate the outperforming of the proposed approach for the two tasks presented in the study, with a small variation. We also showed the limited effects of transfer learning on the performance of the proposed approach, particularly for highly dialectic comments.
LSTM-Based QoE Evaluation for Web Microservices’ Reputation Scoring

Sentiment analysis is the task of mining the authors’ opinions about specific entities. It allows organizations to monitor different services in real time and act accordingly. Reputation is what is generally said or believed about people or things. Informally, reputation combines the measure of reliability derived from feedback, reviews, and ratings gathered from users, which reflect their quality of experience (QoE) and can either increase or harm the reputation of the provided services. In this study, we propose to perform sentiment analysis on web microservices reviews to exploit the provided information to assess and score the microservices’ reputation. Our proposed approach uses the Long Short-Term Memory (LSTM) model to perform sentiment analysis and the Net Brand Reputation (NBR) algorithm to assess reputation scores for microservices. This approach is tested on a set of more than 10,000 reviews related to 15 Amazon Web microservices, and the experimental results have shown that our approach is more accurate than existing approaches, with an accuracy and precision of 93% obtained after applying an oversampling strategy and a resulting reputation score of the considered microservices community of 89%.
RA-Net: A Deep Learning Approach Based on Residual Structure and Attention Mechanism for Image Copy-Move Forgery Detection

To reduce the difficulty of image forensics on forgery images, in this paper, we present an efficient end-to-end deep learning approach using Residual Structure and Attention Mechanism (RA-Net) for image copy-move forgery detection (CMFD). The RA-Net can locate the forged areas and corresponding genuine areas, and it is composed of two modules, Residual Feature Extraction module (RFEM) and Feature Matching & Up-sampling module (FMUM). RFEM is designed to extract deep feature maps, which enriches the combination of gradient information and attention mechanism that focuses the attention of RA-Net to the forged areas. The FMUM assists RA-Net is used to detect copy-move forgery areas and return the previous output to the size of the input image for analysis and visualization of the results. Furthermore, we create a RANet-CMFD dataset for the training, the way to generate RA-Net-CMFD dataset could help solve the problem of not having enough dataset in some research areas. Otherwise, comparison results show that our model can achieve satisfied performance on CoMoFoD dataset at the pixel level, and performs superior than the compared methods.
Hierarchical Vision and Language Transformer for Efficient Visual Dialog

The visual dialog task requires a deep understanding of an image and a dialog history to answer multiple consecutive questions. Existing research focuses on enhancing cross-modal interaction and fusion but often overlooks the computational complexity and higher-level interaction between the two modalities. This paper proposes a hierarchical vision and language Transformer (HVLT) to address these issues. Specifically, HVLT employs a convolution-like design to learn the interaction and fusion of images and text at different levels. We employ a token merging module to aggregate four spatially adjacent image tokens and four temporally adjacent text tokens into one token and use the expanded [CLS] token to fuse image and text information in a new dimension. This hierarchical architecture allows the model to focus on feature maps of different sizes and dialog history at word, phrase, and sentence levels and reduces the time overhead. We tailor two training objectives for HVLT: masked language regression (MLR) and next sentence prediction (NSP), which help the model understand images and language and learn their relationships. Experimental results on the VisDial v0.9 and v1.0 datasets demonstrate the competitive performance of HVLT. Finally, we visualize the attention to gain insights into how HVLT works in practice, shedding light on its interpretability.
Enhance Representational Differentiation Step by Step: A Two-Stage Encoder-Decoder Network for Implicit Discourse Relation Classification

We tackle implicit discourse relation classification (abbr., IDRC). The current neural IDRC models still suffer from the unbalanced data distributions and one-to-many equivocal correspondence. Different classes of instances have hard-to-distinguish representations in a semantic space, which hurts the classification process. We propose a new encoder-decoder network to improve the representational differentiation in encoding and decoding, by two cold-start and hot-start learning phases. We use metric learning and self-generation for constrained differentiation. We also use a KNN-based discriminator with confidence-based sampling. It improves the distribution-sensitive relation classification. The results show that our model, in PDTB 2.0 and PDTB 3.0 benchmarks, outperforms the state-of-the-art methods in most cases. We will share the models and codes for reproducible research ( https://github.com/Destiny-Lu/HCS-KNN ).
Ternary Data, Triangle Decoding, Three Tasks, a Multitask Learning Speech Translation Model

Direct end-to-end approaches for speech translation (ST) are now competing with the traditional cascade solutions. However, end-to-end models still suffer from the challenge of ST data scarcity. How to effectively utilize the limited ST data or more text machine translation (MT) data is appealing but still an open problem. The end-to-end model requires the model to have both cross-modal and cross-language capabilities, which increases the mapping difficulty. In this paper, we propose a tightly tied multitask ST model. By adding a lightweight adapter, we make the ASR decoder also be the MT encoder, where they use one language model and share the source text semantic space. Thus, our model can utilize the MT data. Our end-to-end model can accomplish the ST, ASR and MT tasks simultaneously, and multitask learning can promote the overall performance of an ST model. Our method can make efficient and full use of the limited ternary ST data and even more intelligently utilize external data. When using the limited ternary data, our ST method can achieve state-of-the-art performance in end-to-end models. When adding the external data, our method shows a significant improvement on the strong baselines.
Knowledge Graph Transformer for Sequential Recommendation

The problem of sequential recommendation aims at predicting the most likely item that user will interact based on historical interaction sequence. However, the previous methods only consider the proximity correlations among items and neglect the internal correlations when exploiting auxiliary information, and thus are insufficient to obtain accurate item embedding. Inspired by the success of transformer in NLP, we propose a novel Knowledge Graph Transformer for Sequential Recommendation, KGT-SR for brevity. The main idea of KGT-SR is to extract the rich semantic information of items by utilizing knowledge graph and feed the fused position and item information into the transformer to well learn item representation. KGT-SR consists of embedding layer, knowledge extraction layer and prediction layer. Extensive experiments results on three real world recommendation scenarios show that KGT-SR not only outperforms state-of-the-art sequential recommendation methods but also alleviates the problem of data sparsity.
Combination of DE-GAN with CNN-LSTM for Arabic OCR on Images with Colorful Backgrounds

In this paper, a combination of the conditional generative adversarial network (DE-GAN) with the convolutional neural network (CNN) and long short time memory (LSTM) is proposed in order to improve the performance of optical character recognition (OCR) in the case of images with a colored background. For DE-GAN, it is introduced for the purpose of generating cleaned images. Then, these cleaned images are entered into the CNN model to extract their feature representations. These are next entered as sequences into the LSTM method to label them. Ultimately, Connectionist temporal component (CTC) model is applied to align the data. Experimental results showed that the combination of DE-GAN with CNN-LSTM has succeeded in achieving good results in terms of accuracy for OCR task.
Comparative Analysis of Human Action Recognition Classification for Ambient Assisted Living

Technologies of Ambient Intelligence - an environment filled with computing systems and devices that react and have a certain impact on the presence of a person in this environment. Namely, by making life easier and better for the elderly by designing and developing new functions, services and information and communication technology systems. This study is aimed at reviewing and presenting a new approach to the issue of detecting anomalies and potentially dangerous situations for human health and life - a system for calculating a person's posture by constructing a vector skeleton. The advantage of this method is rather moderate system requirements for computing software and the possibility of its rapid implementation from the planning stage to the final implementation. Extracting the main features from a sequence of images to evaluate parameters such as posture, position and activity of a person and their subsequent categorization and ordering using neural network algorithms. This procedure is necessary to determine the frames for the content of scenarios with a potential threat to an elderly person, for example, a fall. In particular, the most recent and advanced body of libraries has been created in the last five years to make skeleton-based algorithms more accessible for research and subsequent application. Since these libraries will be included for practical applications in video surveillance, medical care. This report presents a comparative classification of various libraries of the human pose calculation system by constructing a vector skeleton based on prepared images and videos. In all tests performed, the parameter for evaluating the performance of these libraries for algorithms such as PoseNet, MoveNet and BlazePose is the percentage of correctly calculated joint vectors. According to the results of the study and the tests carried out, the most effective method for identifying various positions of a person in videos is MoveNet.
PTSTEP: Prompt Tuning for Semantic Typing of Event Processes

Giving machines the ability to understand the intent of human actions is a basic goal of Natural Language Understanding. In the context of that, a task called the Multi-axis Event Processes Typing is proposed, which aims to comprehend the overall goal of an event sequence from the aspect of action and object. Existing works utilize fine-tuning to mine the semantic information of the event processes in the pre-trained language models and achieve good performance. Prompt tuning is effective in fully exploiting the capabilities of pre-trained language models. To mine more sufficient semantic information of the event process, it is crucial to utilize appropriate prompts to guide the pre-trained language models. Moreover, most existing prompt tuning methods use unified prompt encodings. Due to the complex correlations between events of event processes, it is hard to capture context-sensitive semantic information of the event processes. In this paper, we propose PTSTEP, an encoder-decoder based method with continuous prompts. Specifically, we propose a context-aware prompt encoder to obtain a more expressive continuous prompt. Parameters in the pre-trained language model are fixed. On the encoder, the continuous prompt guide the model to mine more semantic information of the event process. On the decoder, the context-aware continuous prompt guide the model to better understand the event processes. PTSTEP outperforms the state-of-the-art method by 0.82% and 3.74% respectively on action MRR and object MRR. The significant improvements prove the effectiveness of our method.