Reviewerly/exHarmony huggingface dataset:
Detecting Offensive Language Based on Graph Attention Networks and Fusion Features

Authors:
raw_author_name: Zhenxiong Miao
raw_affiliation_string: School of Cyber Science and Engineering, Sichuan University, Chengdu, China
raw_author_name: Xingshu Chen
raw_affiliation_string: School of Cyber Science and Engineering and the Cyber Science Research Institute, Sichuan University, Chengdu, China
raw_author_name: Haizhou Wang
raw_affiliation_string: School of Cyber Science and Engineering, Sichuan University, Chengdu, China
raw_author_name: Zhou Yang
raw_affiliation_string: School of Cyber Science and Engineering, Sichuan University, Chengdu, China
raw_author_name: Wenyi Tang
raw_affiliation_string: School of Cyber Science and Engineering, Sichuan University, Chengdu, China

Abstract: The pervasiveness of offensive language on social networks has caused adverse effects on society, such as abusive behavior online. It is urgent to detect offensive language and curb its spread. In the popular datasets, the distribution of users and tweets is imbalanced, which limits the generalization ability of the model. In addition, existing research shows that methods with community information extracted from the social graphs effectively improve the performance of offensive language detection. However, the existing models deal with social graphs independently, which seriously affects the effectiveness of detection models. In this article, we release a new dataset with users and social relationships. To encode community information, we construct the social graphs based on the user historical behavior information and social relationships. Moreover, we propose a model based on graph attention networks (GATs) and fusion features for offensive language detection (GF-OLD). Specifically, the community information is directly captured by the GAT module, and the text embeddings are taken from the last hidden layer of bidirectional encoder representation from transformer (BERT). Attention mechanisms and position encoding are used to fuse these features. Our method outperforms baselines with the F1-score of 89.94%. The results show that our model effectively learns the potential information of social graphs and text, and user historical behavior information is more suitable for user attribute in the social graphs.
An Adaptive and Dynamical Neural Network for Machine Remaining Useful Life Prediction

Authors:
raw_author_name: Ruibing Jin
raw_affiliation_string: Institute for Infocomm Research, Agency for Science, Technology and Research, Singapore
raw_author_name: Min Wu
raw_affiliation_string: Institute for Infocomm Research, Agency for Science, Technology and Research, Singapore
raw_author_name: Xiaoli Li
raw_affiliation_string: Institute for Infocomm Research, Agency for Science, Technology and Research, Singapore
raw_author_name: Zhenghua Chen
raw_affiliation_string: Institute for Infocomm Research, Agency for Science, Technology and Research, Singapore

Abstract: Recently, many neural networks have been proposed for machine remaining useful life (RUL) prediction. However, most network architectures of existing approaches are fixed. Since the sequential information depends on the input data and distributes differently, these fixed networks which cannot be dynamically adjusted according to the input data, may not be able to capture these sequential information well, resulting sub-optimal performances. To mitigate this issue, we propose an adaptive and dynamical neural network (AdaNet), which can dynamically adjust its architecture according to the input data. Neural network is generally determined by kernel size, depth and channel size. In this paper, we aim to enable our proposed AdaNet to adjust its kernel size and channel size dynamically. Firstly, we explore to adapt the deformable convolution to time series data, which allows the convolutional kernel to change according to the feature map. With this deformable convolution, the convolutional kernels in AdaNet become adjustable, which is beneficial to fully exploit the sequential information in time series data, leading to accurate RUL prediction. Additionally, a channel selection module is devised, which can selectively activate the feature channel according to the input, further improving the performance of our AdaNet. Extensive experiments have been carried out on the C-MAPSS dataset, demonstrating that our proposed AdaNet achieves state-of-the-art performances.
DYnet++: A Deep Learning Based Single-Shot Phase-Measuring Deflectometry for the 3-D Measurement of Complex Free-Form Surfaces

Authors:
raw_author_name: Manh The Nguyen
raw_affiliation_string: Advanced Instrumentation Institute, Korea Research Institute of Standards and Science, Daejeon, South Korea
raw_author_name: Young-Sik Ghim
raw_affiliation_string: Advanced Instrumentation Institute, Korea Research Institute of Standards and Science, Daejeon, South Korea
raw_author_name: Hyug-Gyo Rhee
raw_affiliation_string: Advanced Instrumentation Institute, Korea Research Institute of Standards and Science, Daejeon, South Korea

Abstract: Deflectometry is a three-dimensional optical technique based on structured light projection for measuring specular free-form surfaces. Much research on single-shot deflectometry has been conducted to develop reliable measurement techniques, even for harsh industrial environments. But the measurement of complex surfaces with low reflectivity is still a big challenge in single-shot deflectometry since it is very tricky to obtain the phase from a poor quality single complex pattern. Inspired by recent applications of deep learning in the optical metrology field, we propose a novel single-shot deflectometry that utilizes deep learning to measure complex surfaces with low quality in just a single shot. A deformable mirror with 9 actuators was developed to generate big data for various surface shapes to train the deep learning model. We developed a deep learning network model, DYnet++, to retrieve the phase from single composite patterns even where both closed- and opened- loops are presented. We verified the feasibility of the proposed deep learning based single-shot deflectometry by comparing measurement results with those from 16 step phase-shifting method.
Towards Efficient Cross-Modal Anomaly Detection Using Triple-Adaptive Network and Bi-Quintuple Contrastive Learning

Authors:
raw_author_name: Shu-Juan Peng
raw_affiliation_string: Department of Artificial Intelligence, Huaqiao University, Xiamen, China
raw_author_name: Ye Fan
raw_affiliation_string: Xiamen Key Laboratory of Computer Vision, and Pattern Recognition, Xiamen, China
raw_author_name: Yiu-ming Cheung
raw_affiliation_string: Department of Computer Science, Hong Kong Baptist University, Hong Kong SAR, China
raw_author_name: Xin Liu
raw_affiliation_string: Department of Computer Science, Huaqiao University, Xiamen, China
raw_author_name: Zhen Cui
raw_affiliation_string: Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, Nanjing University of Science and Technology, Nanjing, China
raw_author_name: Taihao Li
raw_affiliation_string: Artificial Intelligence Research Institute, Zhejiang Lab, Hangzhou, China

Abstract: Cross-modal anomaly detection is a relatively new and challenging research topic in machine learning field, which aims at identifying the anomalies whose patterns are disparate across different modalities. As far as we know, this topic has yet to be well studied, and existing works often suffer from the incomplete anomalous data detection and low data utilization problems. To alleviate these limitations, this paper proposes an efficient deep cross-modal anomaly detection approach via Triple-adaptive Network and Bi-quintuple Contrastive Learning (TN-BCL), which lies among the earliest attempt to detect various cross-modal anomalies within the heterogeneous multi-modal data. To be specific, a triple-adaptive network is explicitly designed to identify various anomalies, whose patterns are disparate in both single-modal scenario and cross-modal scenario. On the one hand, the top branch network is utilized to adaptively detect the attribute anomalies and part of mixed anomalies in multi-modal data samples. On the other hand, the bottom two-branch network, with shared residual blocks, is leveraged to learn the discriminative cross-modal embeddings. At the same time, an efficient bi-quintuple contrastive learning method is designed to enhance the feature correlation between the same attribute data, while maximally enlarging the feature difference between different attribute data. Besides that, the bidirectional learning scheme is employed to significantly improve the data utilization. Through the joint exploitation of the above, different kinds of anomalous samples can be well detected across different modalities. Extensive experiments show that the proposed framework outperforms the state-of-the-art competing methods, with a large improvement margin.
Hierarchical Patch Selection: An Improved Patch Sampling for No Reference Image Quality Assessment

Authors:
raw_author_name: C. Nandhini
raw_affiliation_string: Department of Computer Science and Engineering, National Institute of Technology, Tiruchirappalli, India
raw_author_name: M. Brindha
raw_affiliation_string: Department of Computer Science and Engineering, National Institute of Technology, Tiruchirappalli, India

Abstract: Quality degradation due to compression and transmission of images is a significant threat for multimedia applications. Blind image quality assessment (BIQA) is a principal technique to measure the distortion and dynamically set the optimal parameters for developing image compression standards, image restoration algorithms, etc. Insufficient training data with quality scores is a challenge for image quality assessment (IQA) tasks. The existing solutions to deep convolution neural network (DCNN) based BIQA which rely on patch-wise training, struggle to find an ideal set of patches consistent with the human visual system. To address these issues, HIerarchical paTch Selection (HITS) is proposed. HITS keeps the patches with considerable details in each quadrant according to their intensity variance scatter ratio (IVSR). IVSR identifies the best non-homogeneous patches by computing patch intensity variance (PIV). Extensive trials are conducted, and the performance reveals that the proposed approach achieves excellent performance on synthetic and authentic distortion datasets with less memory and processing power. Moreover, the proposed approach outperforms other BIQA methods that adapt patch-wise training strategies.
FDGNN: Feature-Aware Disentangled Graph Neural Network for Recommendation

Authors:
raw_author_name: Xiao Liu
raw_affiliation_string: Department of Cyber Science and Engineering, Nanjing University of Science and Technology, Nanjing, China
raw_author_name: Shunmei Meng
raw_affiliation_string: Department of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China
raw_author_name: Qianmu Li
raw_affiliation_string: Department of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China
raw_author_name: Qiyan Liu
raw_affiliation_string: Department of Computer Engineering, University of Toronto, Toronto, Canada
raw_author_name: Qiang He
raw_affiliation_string: School of Software and Electrical Engineering, Swinburne University of Technology, Melbourne, VIC, Australia
raw_author_name: Dharavath Ramesh
raw_affiliation_string: Department of Computer Science and Engineering, Indian Institute of Technology (Indian School of Mines), Dhanbad, India
raw_author_name: Lianyong Qi
raw_affiliation_string: College of Computer Science and Technology, China University of Petroleum (East China), Dongying, China

Abstract: Collaborative filtering (CF) is dedicated to learning the representations of users and items based on interactive data. Regrettably, the lack of fine-grained modeling of interactive motivation makes the model less interpretable. A feasible solution is to combine the disentangling idea with the graph neural network (GNN) and capture different types of interaction relationships by using a message propagation mechanism on the graph of user–item interaction. However, this process typically relies on the disentangling of users’ hidden intents, ignoring the significance of item features to user engagement. This fact leads to the inadequate interpretability of existing models. To make up for the deficiency, this article proposes a new feature-aware disentangled GNN (FDGNN) for the recommendation. By learning the relationship between user behavior and important features of items, the model aims to achieve better recommendation performance and model interpretability. In the end, we first realize the feature partition based on mutual information and then design an attention-based graph disentangling model to realize the fine-grained disentangling of user intents. In addition, to further ensure the independence of the disentangled intents, we augment the model with disagreement regularization. Through multilayer embedding propagation, FDGNN can display a capture CF effect in feature semantics. The interpretability and efficiency of our proposed approach are demonstrated by numerous pertinent experiments.
Deep Image Feature Learning With Fuzzy Rules

Authors:
raw_author_name: Liangzhe Chen
raw_affiliation_string: School of Artificial Intelligence and Computer Science, Jiangsu Key Laboratory of Digital Design and Software Technology, Jiangnan University, Wuxi, China
raw_author_name: Zhaohong Deng
raw_affiliation_string: School of Artificial Intelligence and Computer Science, Jiangsu Key Laboratory of Digital Design and Software Technology, Jiangnan University, Wuxi, China
raw_author_name: Peng Xu
raw_affiliation_string: School of Artificial Intelligence and Computer Science, Jiangsu Key Laboratory of Digital Design and Software Technology, Jiangnan University, Wuxi, China
raw_author_name: Kup-Sze Choi
raw_affiliation_string: The Centre for Smart Health, The Hong Kong Polytechnic University, Hong Kong, China
raw_author_name: Shitong Wang
raw_affiliation_string: School of Artificial Intelligence and Computer Science, Jiangsu Key Laboratory of Digital Design and Software Technology, Jiangnan University, Wuxi, China

Abstract: Feature extraction methods are key to many image processing tasks. At present, the most popular method is to use a deep neural network to automatically extract features through end-to-end training instead of the traditional hand-crafted feature extraction. However, the training of deep neural network relies heavily on data quality and quantity, and the network is a black-box model that has poor interpretability. Human intelligence can be leveraged here to improve the deep neural network model, where the human decision process can be integrated in feature learning and object classification to enhance robustness and interpretability. In this paper, the method Deep Image Feature Learning with Fuzzy Rules (DIFL-FR) is proposed, where human decision process is embedded in feature extraction by combining fuzzy logic rule-based modeling with deep-stacked learning strategy. The proposed method has the following distinctive characteristics. First, since the method is based on fuzzy sets and fuzzy inference, it can extract more robust features from noisy data scenes. Second, the method progressively learns the image features through a layer-by-layer approach based on fuzzy rules, so that the feature learning process can be better explained by the rules generated. Third, the learning process of the method has a high efficiency since it is only based on forward propagation without back propagation and iterative learning. Finally, while the method is based on unsupervised learning, it can be easily extended to supervised and semi-supervised learning cases. The results of extensive experiments conducted on image datasets of different scales clearly show the effectiveness of the proposed method.
Self-Supervised Deep Clustering Method for Detecting Abnormal Data of Wastewater Treatment Process

Authors:
raw_author_name: Honggui Han
raw_affiliation_string: Faculty of Information Technology, the Beijing Key Laboratory of Computational Intelligence and Intelligent System, and the Engineering Research Center of Digital Community, Ministry of Education, Beijing University of Technology, Beijing, China
raw_author_name: Fangyu Li
raw_affiliation_string: Faculty of Information Technology, the Beijing Key Laboratory of Computational Intelligence and Intelligent System, and the Engineering Research Center of Digital Community, Ministry of Education, Beijing University of Technology, Beijing, China
raw_author_name: Chen Wang
raw_affiliation_string: Beijing Aerospace Smart Manufacturing Technology Development Company Ltd, Beijing, China

Abstract: In wastewater treatment process (WWTP), abnormal data seriously reduce data quality rendering the application techniques impractical. The implementation of abnormal data detection is challenging due to the nonlinear nature of WWTP. Typically, constructing an accurate anomaly detector requires large amounts of labeled data, which is difficult in practice. Thus, a self-supervised memory enhanced deep clustering method (SMEL) is proposed to detect abnormal data without using any labels. First, a self-supervised deep clustering network, combining stacked autoencoders and the clustering algorithm, leverages unlabeled data to extract nonlinear features and capture the normal pattern. Second, an adaptive weight objective function, jointly optimizing the reconstruct error and clustering error, is designed to obtain a robust clustering structure. Third, double memory enhanced modules, consisting of a centroid memory and a score memory, are presented to enhance training stability and detection accuracy. Finally, experiments on three WWTP datasets show that SMEL achieves the highest detection accuracy.
Naive Feature Selection: A Nearly Tight Convex Relaxation for Sparse Naive Bayes

Authors:
raw_author_name: Armin Askari
raw_affiliation_string: Electrical Engineering and Computer Science, University of California, Berkeley, California 94720;
raw_author_name: Alexandre d’Aspremont
raw_affiliation_string: Département d’Informatique, Centre National de la Recherche Scientifique, and École Normale Supérieure, 75005 Paris, France
raw_author_name: Laurent El Ghaoui
raw_affiliation_string: Electrical Engineering and Computer Science, University of California, Berkeley, California 94720;

Abstract: Because of its linear complexity, naive Bayes classification remains an attractive supervised learning method, especially in very large-scale settings. We propose a sparse version of naive Bayes, which can be used for feature selection. This leads to a combinatorial maximum-likelihood problem, for which we provide an exact solution in the case of binary data, or a bound in the multinomial case. We prove that our convex relaxation bounds become tight as the marginal contribution of additional features decreases using a priori duality gap bounds derived from the Shapley–Folkman theorem. We show how to produce primal solutions satisfying these bounds. Both binary and multinomial sparse models are solvable in time almost linear in problem size, representing a very small extra relative cost compared with the classical naive Bayes. Numerical experiments on text data show that the naive Bayes feature selection method is as statistically effective as state-of-the-art feature selection methods such as recursive feature elimination, l 1 -penalized logistic regression, and LASSO while being orders of magnitude faster (a python implementation can be found at https://github.com/aspremon/NaiveFeatureSelection ). Funding: A. d’Aspremont acknowledges support from the Fonds AXA Pour la Recherche and Kamet Ventures [Machine Learning and Optimisation Joint Research Initiative] and a Google-focused award as well as funding from Agence Nationale de la Recherche [Grant ANR-19-P3IA-0001]. L. El Ghaoui acknowledges support from Berkeley Artificial Intelligence Research and the Tsinghua–Berkeley–Shenzhen Institute.
Masked Siamese Prompt Tuning for Few-Shot Natural Language Understanding

Authors:
raw_author_name: Shiwen Ni
raw_affiliation_string: Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
raw_author_name: Hung-Yu Kao
raw_affiliation_string: Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan

Abstract: Recently, prompt-based learning has shown excellent performance on few-shot scenarios. Using frozen language models to tune trainable continuous prompt embeddings has become a popular and powerful methodology. For few-shot natural language understanding, even if we freeze the parameters of the pre-trained language model, the learned pseudo-prompts might still be overfitted. In this paper, we propose a novel masked siamese prompt tuning (MSP-tuning) to improve few-shot natural language understanding. Concretely, MSP-tuning masks randomly out part of the prompt tokens to get a pair of masked siamese prompts for each sample. Each training sample is then fed to the model twice with the masked siamese prompts. Finally, MSP-tuning minimizes the JS-divergence between the two output probability distributions of the pre-trained language model to regularize the model further. Experiment results on the few-shot GLUE benchmark and SuperGLUE benchmark show that MSP-tuning outperforms previous approaches. Numerically, our MSP-tuning achieves an average improvement of 1.79% (BERT-base) and 1.39% (BERT-large) of the GLUE benchmark and 1.90% (RoBERTa-base) and 1.71% (RoBERTa-large) of the SuperGLUE benchmark compared to state-of-the-art method P-tuning. Our method facilitates applying large pre-trained language models in natural language understanding.
Integrating Blockchain and Deep Learning Into Extremely Resource-Constrained IoT: An Energy-Saving Zero-Knowledge PoL Approach

Authors:
raw_author_name: Jun Wu
raw_affiliation_string: Graduate School of Information, Production and Systems, Waseda University, Tokyo, Japan
raw_author_name: Xi Lin
raw_affiliation_string: School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai, China
raw_author_name: Ali Kashif Bashir
raw_affiliation_string: Department of Computing and Mathematics, Manchester Metropolitan University, Manchester, U.K.
raw_author_name: Yasser D. Al-Otaibi
raw_affiliation_string: Department of Information Systems, Faculty of Computing and Information Technology in Rabigh, King Abdulaziz University, Jeddah, Saudi Arabia

Abstract: The convergence of blockchain and deep learning (DL) drives the intelligence of the Internet of Things (IoT) with security guarantees. However, the soaring resource consumption resulting from blockchain mining and DL model training has overwhelmed the extremely resource-constrained IoT. In this paper, we first build a blockchain and DL-empowered cloud-edge orchestrated framework for an extremely resource-constrained IoT environment. To solve the resource bottleneck of this framework, we then propose a Zero-Knowledge Proof of Learning (ZPoL) consensus approach to channel the meaningless Proof of Work (PoW) mining energy waste to valuable DL model training, while protecting the DL model privacy. Besides, to encourage resource-constrained IoT devices to perform meaningful DL model mining in our ZPoL consensus, we design a model quality-aware incentive mechanism based on a two-stage Stackelberg game. Moreover, we conduct extensive simulations and experiments to evaluate our proposed ZPoL-based framework. The numerical simulation illustrates our proposed incentive mechanism could motivate IoT devices to actively join in DL model mining. Compared with the existing blockchain and DL-enabled IoT system, experimental results demonstrate our proposed ZPoL-based framework could significantly reduce the communication, computation, and storage cost, which is more applicable to a resource-constrained IoT environment.