Reviewerly/exHarmony huggingface dataset:
Enhanced approach of multilabel learning for the Arabic aspect category detection of the hotel reviews

Abstract In many fields, like aspect category detection (ACD) in aspect‐based sentiment analysis, it is necessary to label each instance with more than one label at the same time. This study tackles the multilabel classification problem in the ACD task for the Arabic language. For this purpose, we used Arabic hotel reviews from the SemEval‐2016 dataset, comprising 13,113 annotated tuples provided for training (10,509) and testing (2,604). To extract valuable information, we first propose specific data preprocessing. Then, we suggest using the dynamic weighted loss function and a data augmentation method to fix the problem with this dataset's imbalance. Using two possible approaches, we develop new ways to find different categories of things in a review sentence. The first is based on classifier chains using machine learning models. The second is based on transfer learning using pretrained AraBERT fine‐tuning for contextual representation. Our findings show that both approaches outperformed the related works for ACD on the Arabic SemEval‐2016. Moreover, we observed that AraBERT fine‐tuning performed much better and achieved a promising ‐score of .
TKGR-RHETNE: A New Temporal Knowledge Graph Reasoning Model via Jointly Modeling Relevant Historical Event and Temporal Neighborhood Event Context

Temporal knowledge graph reasoning (TKGR) has been of great interest for its role in enriching the naturally incomplete temporal knowledge graph (TKG) by uncovering new events from existing ones with temporal information. At present, the majority of existing TKGR methods have attained commendable performance. Nevertheless, they still suffer from several problems, specifically their limited ability to adeptly capture intricate long-term event dependencies within the context of pertinent historical events, as well as to address the occurrence of an event with insufficient historical information or be influenced by other events. To alleviate such issues, we propose a novel TKGR method named TKGR-RHETNE, which jointly models the context of relevant historical events and temporal neighborhood events. In terms of the historical event view, we introduce an encoder based on the transformer Hawkes process and self-attention mechanism to effectively capture long-term event dependencies, thus modeling the event evolution process continuously. In terms of the neighborhood event view, we propose a neighborhood aggregator to model the potential influence between events with insufficient historical information and other events, which is implemented by integrating the random walk strategy with the TKG topological structure. Comprehensive experiments on five benchmark datasets demonstrate the superior performance of our proposed model (Code is publicly available at https://github.com/wanwano/TKGR-RHETNE ).
TASFormer: Task-Aware Image Segmentation Transformer

In image segmentation tasks for real-world applications, the number of semantic categories can be very large, and the number of objects in them can vary greatly. In this case, the multi-channel representation of the output mask for the segmentation model is inefficient. In this paper we explore approaches to overcome such a problem by using a single-channel output mask and additional input information about the desired class for segmentation. We call this information task embedding and we learn it in the process of the neural network model training. In our case, the number of tasks is equal to the number of segmentation categories. This approach allows us to build universal models that can be conveniently extended to an arbitrary number of categories without changing the architecture of the neural network. To investigate this idea we developed a transformer neural network segmentation model named TASFormer. We demonstrated that the highest quality results for task-aware segmentation are obtained using adapter technology as part of the model. To evaluate the quality of segmentation, we introduce a binary intersection over union (bIoU) metric, which is an adaptation of the standard mIoU for the models with a single-channel output. We analyze its distinguishing properties and use it to compare modern neural network methods. The experiments were carried out on the universal ADE20K dataset. The proposed TASFormer-based approach demonstrated state-of-the-art segmentation quality on it. The software implementation of the TASFormer method and the bIoU metric is publicly available at www.github.com/subake/TASFormer .
Vessel Behavior Anomaly Detection Using Graph Attention Network

Vessel behavior anomaly detection is of great significance for ensuring navigation safety, combating maritime crimes, and maritime management. Unfortunately, most current researches ignore the temporal dependencies and correlations between ship features. We propose a novel vessel behavior anomaly detection using graph attention network (i.e., VBAD-GAT) framework, which characterizes these complicated relationships and dependencies through a graph attention module that consists of a time graph attention module and a feature graph attention module. We also adopt a process of graph structure learning to obtain the correct feature graph structure. Moreover, we propose a joint detection strategy combining reconstruction and prediction modules to capture the local ship features and long-term relationships between ship features. We demonstrate the effectiveness of the graph attention module and the joint detection strategy through the ablation study. In addition, the comparative experiments with three baselines, including the quantitative analysis and visualization, show that VBAD-GAT outperforms all other baselines.
P-IoU: Accurate Motion Prediction Based Data Association for Multi-object Tracking

Multi-object tracking in complex scenarios remains a challenging task due to objects' irregular motions and indistinguishable appearances. Traditional methods often approximate the motion direction of objects solely based on their bounding box information, leading to cumulative noise and incorrect association. Furthermore, the lack of depth information in these methods can result in failed discrimination between foreground and background objects due to the perspective projection of the camera. To address these limitations, we propose a Pose Intersection over Union (P-IoU) method to predict the true motion direction of objects by incorporating body pose information, specifically the motion of the human torso. Based on P-IoU, we propose PoseTracker, a novel approach that combines bounding box IoU and P-IoU effectively during association to improve tracking performance. Exploiting the relative stability of the human torso and the confidence of keypoints, our method effectively captures the genuine motion cues, reducing identity switches caused by irregular movements. Experiments on the DanceTrack and MOT17 datasets demonstrate that the proposed PoseTracker outperforms existing methods. Our method highlights the importance of accurate motion prediction of objects for data association in MOT and provides a new perspective for addressing the challenges posed by irregular object motion.
Label Selection Algorithm Based on Ant Colony Optimization and Reinforcement Learning for Multi-label Classification

Multi-label classification handles scenarios where an instance can be annotated with multiple non-exclusive but semantically related labels simultaneously. Despite significant progress, multi-label classification is still challenging due to the emergence of multiple applications leading to high-dimensional label spaces. Researchers have generalized feature dimensionality reduction techniques to label space by using label correlation information, and obtained two techniques: label embedding and label selection. There have been many successful algorithms in label embedding, but less attention has been paid to label selection. In this paper, we propose a label selection algorithm for multi-label classification: LS-AntRL, which combines ant colony optimization (ACO) and reinforcement learning (RL). This method helps ant colony algorithms search better in the search space by using temporal difference (TD) RL algorithm to learn directly from the experience of ants. For heuristic learning, we need to model the ACO problem as a RL problem, that is, to model label selection as a Markov decision process (MDP), where the label represents the state, and each ant selecting unvisited labels represents a set of actions. The state transition rules of the ACO algorithm constitute the transition function in the MDP, and the state value function is updated by TD formula to form a heuristic function in ACO. After performing label selection, we train a binary weighted neural network to recover low-dimensional label space back to the original label space. We apply the above model to five benchmark datasets with more than 100 labels. Experimental results show that our method achieves better classification performance than other advanced methods in terms of two performance evaluation metrics (Precision@n and DCG@n).
SCME: A Self-contrastive Method for Data-Free and Query-Limited Model Extraction Attack

Previous studies have revealed that artificial intelligence (AI) systems are vulnerable to adversarial attacks. Among them, model extraction attacks fool the target model by generating adversarial examples on a substitute model. The core of such an attack is training a substitute model as similar to the target model as possible, where the simulation process can be categorized in a data-dependent and data-free manner. Compared with the data-dependent method, the data-free one has been proven to be more practical in the real world since it trains the substitute model with synthesized data. However, the distribution of these fake data lacks diversity and cannot detect the decision boundary of the target model well, resulting in the dissatisfactory simulation effect. Besides, these data-free techniques need a vast number of queries to train the substitute model, increasing the time and computing consumption and the risk of exposure. To solve the aforementioned problems, in this paper, we propose a novel data-free model extraction method named SCME (Self-Contrastive Model Extraction), which considers both the inter- and intra-class diversity in synthesizing fake data. In addition, SCME introduces the Mixup operation to augment the fake data, which can explore the target model's decision boundary effectively and improve the simulating capacity. Extensive experiments show that the proposed method can yield diversified fake data. Moreover, our method has shown superiority in many different attack settings under the query-limited scenario, especially for untargeted attacks, the SCME outperforms SOTA methods by 11.43% on average for five baseline datasets.
Direct Inter-Intra View Association for Light Field Super-Resolution

Light field (LF) cameras record both intensity and directions of light rays in a scene with a single exposure. However, due to the inevitable trade-off between spatial and angular dimensions, the spatial resolution of LF images is limited which makes LF super-resolution (LFSR) a research hotspot. The key of LFSR is the complementation across views and the extraction of high-frequency information inside each view. Due to the high-dimensinality of LF data, previous methods usually model these two processes separately, which results in insufficient inter-view information fusion. In this paper, LF Transformer is proposed for comprehensive perception of 4D LF data. Necessary inter-intra view correlations can be directly established inside each LF Transformer block. Therefore it can handle complex disparity variations of LF. Then based on LF Transformers, 4DTNet is designed which comprehensively performs inter-intra view high-frequency information extraction. Extensive experiments on public datasets demonstrate that 4DTNet outperforms the current state-of-the-art methods both numerically and visually.
PYGC: A PinYin Language Model Guided Correction Model for Chinese Spell Checking

Chinese Spell Checking (CSC) is an NLP task that detects and corrects erroneous characters in Chinese texts. Since people often use pinyin (pronunciation of Chinese characters) input methods or speech recognition to type text, most of these errors are misuse of phonetically or semantically similar characters. Previous attempts fuse pinyin information into the embedding layer of pre-trained language models. However, although they can learn from phonetic knowledge, they can not make good use of this knowledge for error correction. In this paper, we propose a PinYin language model Guided Correction model (PYGC), which regards the Chinese pinyin sequence as an independent language model. Our model builds on two parallel transformer encoders to capture pinyin and semantic features respectively, with a late fusion module to fuse these two hidden representations to generate the final prediction. Besides, we perform an additional pronunciation prediction task on pinyin embeddings to ensure the reliability of the pinyin language model. Experiments on the widely used SIGHAN benchmark and a newly released CSCD-IME dataset with mainly pinyin-related errors show that our method outperforms current state-of-the-art approaches by a remarkable margin. Furthermore, isolation tests demonstrate that our model has the best generalization ability on unseen spelling errors. (Code available in https://github.com/Imposingapple/PYGC ).
Unsupervised Joint-Semantics Autoencoder Hashing for Multimedia Retrieval

Cross-modal hashing has emerged as a prominent approach for large-scale multimedia information retrieval, offering advantages in computational speed and storage efficiency over traditional methods. However, unsupervised cross-modal hashing methods still face challenges in the lack of practical semantic labeling guidance and handling of cross-modal heterogeneity. In this paper, we propose a new unsupervised cross-modal hashing method called Unsupervised Joint-Semantics Autoencoder Hashing(UJSAH) for multimedia retrieval. First, we introduce a joint-semantics similarity matrix that effectively preserves the semantic information in multimodal data. This matrix integrates the original neighborhood structure information of the data, allowing it to better capture the associations between different modalities. This ensures that the similarity matrix can accurately mine the underlying relationships within the data. Second, we design a dual prediction network-based autoencoder, which implements the interconversion of semantic information from different modalities and ensures that the generated binary hash codes maintain the semantic information of different modalities. Experimental results on several classical datasets show a significant improvement in the performance of UJSAH in multimodal retrieval tasks relative to existing methods. The experimental code is published at https://github.com/YunfeiChenMY/UJSAH .
Reversible Data Hiding Based on Adaptive Embedding with Local Complexity

In recent years, most reversible data hiding (RDH) algorithms have considered the impact of texture information on embedding performance. The distortion caused by embedding secret data in the image's smooth region is much less than in the non-smooth region. It is because embedding secret data in the smooth region corresponds to fewer invalid shifting pixels (ISPs) in histogram shifting. However, though effective, the local complexity is not calculated precisely enough, which results in inaccurate texture division and does not considerably reduce distortion. Therefore, a new RDH scheme based on adaptive embedding with local complexity (AELC) is proposed to improve the embedding performance effectively. Specifically, the cover image is divided into two subsets by the checkerboard pattern. Then the local complexity of each pixel is computed by the correlation between adjacent pixels (CBAP). Finally, secret data are adaptive and preferentially embedded into the regions with lower local complexity in each subset. Experimental results show that the proposed algorithm performs best regarding invalid shifted pixels, maximum embedding capacity (EC), and peak signal-to-noise ratio (PSNR) compared to some state-of-the-art RDH methods.
Ignored Details in Eyes: Exposing GAN-Generated Faces by Sclera

Advances in Generative adversarial networks (GAN) have significantly improved the quality of synthetic facial images, posing threats to many vital areas. Thus, identifying whether a presented facial image is synthesized is of forensic importance. Our fundamental discovery is the lack of capillaries in the sclera of the GAN-generated faces, which is caused by the lack of physical/physiological constraints in the GAN model. Because there are more or fewer capillaries in people's eyes, one can distinguish real faces from GAN-generated ones by carefully examining the sclera area. Following this idea, we first extract the sclera area from a probe image, then feed it into a residual attention network to distinguish GAN-generated faces from real ones. The proposed method is validated on the Flickr-Faces-HQ and StyleGAN2/StyleGAN3-generated face datasets. Experiments demonstrate that the capillary in the sclera is a very effective feature for identifying GAN-generated faces. Our code is available at: https://github.com/10961020/Deepfake-detector-based-on-blood-vessels .
VFIQ: A Novel Model of ViT-FSIMc Hybrid Siamese Network for Image Quality Assessment

The Image Quality Assessment (IQA) is to measure how humans perceive the quality of images. In this paper, we propose a new model named for VFIQ – a ViT-FSIMc Hybrid Siamese Network for Full Reference IQA – that combines signal processing and leaning-based approaches, the two categories of IQA algorithms. Specifically, we design a hybrid Siamese network that leverages the Vision Transformer (ViT) and the feature similarity index measurement (FSIMc). To evaluate the performance of the proposed VFIQ model, we first pre-train the ViT module on the PIPAL dataset, and then evaluate our VFIQ model on several popular benchmark datasets including TID2008, TID2013, and LIVE. The experiment results show that our VFIQ model outperforms the state-of-the-art IQA models in the commonly used correlation metrics of PLCC, KRCC, and SRCC. We also demonstrate the usefulness of our VFIQ model in different vision tasks, such as image recovery and generative model evaluation.
Resource-Aware DNN Partitioning for Privacy-Sensitive Edge-Cloud Systems

With recent advances in deep neural networks (DNNs), there is a significant increase in IoT applications leveraging AI with edge-cloud infrastructures. Nevertheless, deploying large DNN models on resource-constrained edge devices is still challenging due to limitations in computation, power, and application-specific privacy requirements. Existing model partitioning methods, which deploy a partial DNN on an edge device while processing the remaining portion of the DNN on the cloud, mainly emphasize communication and power efficiency. However, DNN partitioning based on the privacy requirements and resource budgets of edge devices has not been sufficiently explored in the literature. In this paper, we propose awareSL, a model partitioning framework that splits DNN models based on the computational resources available on edge devices, preserving the privacy of input samples while maintaining high accuracy. In our evaluation of multiple DNN architectures, awareSL effectively identifies the split points that adapt to resource budgets of edge devices. Meanwhile, we demonstrate the privacy-preserving capability of awareSL against existing input reconstruction attacks without sacrificing inference accuracy in image classification tasks.
Multi-view Contrastive Learning for Knowledge-Aware Recommendation

Knowledge-aware recommendation has attracted increasing attention due to its wide application in alleviating data-sparse and cold-start, but the real-world knowledge graph (KG) contains many noises from irrelevant entities. Recently, contrastive learning, a self-supervised learning (SSL) method, has shown excellent anti-noise performance in recommendation task. However, the inconsistency between the use of noisy embeddings in SSL tasks and the original embeddings in recommendation tasks limits the model's ability. We propose a Multi-view Contrastive learning for Knowledge-aware Recommendation framework (MCKR) to solve the above problems. To remove inconsistencies, MCKR unifies the input of SSL and recommendation tasks and learns more representations from the contrastive learning method. To alleviate the noises from irrelevant entities, MCKR preprocesses the KG triples according to the type and randomly perturbs of graph structure with different weights. Then, a novel distance-based graph convolutional network is proposed to learn more reliable entity information in KG. Extensive experiments on three popular benchmark datasets present that our approach achieves state-of-the-art. Further analysis shows that MCKR also performs well in reducing data noise.
UATR: An Uncertainty Aware Two-Stage Refinement Model for Targeted Sentiment Analysis

Target sentiment analysis aims to predict the fine-grained sentiment polarity of a given term. Although some achievements have been made in recent years, the accuracy of targeted sentiment multi-classification technology is still insufficient-a considerable proportion of samples are incorrectly predicted as the opposite polarity. To this end, we investigate the effectiveness of utilizing model uncertainty and propose a two-stage refinement predicting model based on uncertainty called UATR. UATR can model uncertainty by inferring the distribution of model weights and is more robust to small data learning. Experiments on standard benchmark SemEval14 show that our model can not only reduce the proportion of samples incorrectly predicted as the opposite polarity, but also improves accuracy and F1 values by more than 2% and 3% compared to the current state-of-the-art models, respectively.
Empirical Analysis of Multi-label Classification on GitterCom Using BERT and ML Classifiers

To maintain development consciousness, simplify project coordination, and prevent misinterpretation, communication is essential for software development teams. Instant private messaging, group chats, and sharing code are just a few of the capabilities that chat rooms provide to assist and meet the communication demands of software development teams. All of this is capacitated to happen in real-time. Consequently, chat rooms have gained popularity among developers. Gitter is one of these platforms that has gained popularity, and the conversations it contains may be a treasure trove of data for academics researching open-source software systems. This research made use of the GitterCom dataset, The largest collection of Gitter developer messages that have been carefully labelled and curated and perform multi-label classification for the 'Purpose' category in the dataset. An extensive empirical analysis is performed on 6 feature selection techniques, 14 machine learning classifiers, and BERT transformer layer architecture with layer-by-layer comparison. Consequently, we achieve proficient results through our research pipeline involving Extra Trees Classifier and Random Forest classifiers with AUC (OvR) median performance of 0.94 and 0.92 respectively. Furthermore, The research proposed research pipeline could be utilized for generic multi-label text classification on software developer forum text data.