Reviewerly/exHarmony huggingface dataset:
Learning Framework of Entity and Entity Type Composition Representation for Knowledge Graph Completion

This paper proposes a simple Knowledge Graph Embedding (KGE) framework that considers the entity types. The KGE finds appropriate representations of entities and relations by learning structured information in triples and uses these representations to predict the missing links in a knowledge graph (KG). The entity types are included in many KGs, but most of the existing methods ignored its potential for the link prediction task. The proposed framework, which is called EETCRL (Entity and Entity Type Composition Representation Learning), learns representations for entities and entity types in a simple way. Those combined representations are used for link prediction. The proposed framework is evaluated on three datasets and the results show that while the overall performance was comparable to baseline methods, its performance depends on the entity type. This paper also discusses how the information about entity types affects the link prediction through analysis of the fine-grained results.
Abusive Language Detection with Graph based Multi-task Learning

To counter the online abusive language in social media, it is desirable to develop automated detection methods. Previous research has primarily formulated this problem as a sentence-level classification task, ignoring the crucial role of abusive lexicons that can strengthen the model explainability and enable more faithful predictions. Although a few methods have introduced the abusive lexicons for detection, the lexicons they use are either externally provided or labeled by human annotators, suffering from two limitations: (1) lack adaptability to diverse and evolving offensive scenarios; (2) require large human efforts to annotate the words.This paper overcomes the limitations of prior work with a multi-task abusive language detection framework. It combines sentence-level and word-level classification tasks, based on dependency tree based graph attention networks (GAT). With the two tasks, it is encouraged to capture both global and local data properties to produce better sentence representations. It is also advantageous in automatic lexicon construction during the learning process, without human annotations. Extensive experiments on two public datasets exhibit that our proposal can outperform the state-of-the-art baselines. Case studies show that the model explainability can be strengthened with the abusive parts identified by our framework. Our code is released to public. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
An Open Dataset for Deep Learning-based Earthquake Detection using MEMS Sensors

Due to the high population density and economic value of contemporary cities, earthquakes inflict greater damage on these cities. Consequently, the importance of quick earthquake early warning (EEW) is expanding, yet it is challenging to create a dense seismic monitoring network due to high installation and management costs. In order to overcome such limitations, MEMS sensors to monitor earthquakes and artificial intelligence (AI) technologies to analyze massive earthquake monitoring data are widely used today. In AI-based earthquake detection, the key to accurate detection is the use of sufficient data that accurately represents the various earthquake patterns. Unfortunately, how-ever, there is no publicly accessible database containing IoT-based seismic data. This is the result of relatively short research efforts. During the last two years of operation of CrowdQuake, a MEMS-based earthquake detection system, we collected earthquake and non-earthquake events, as well as normal noise data, which was greatly useful to improve the accuracy of AI models. As a result, we present an open dataset that is publicly available for MEMS-based earthquake detection research.
Continuous and Silent User Authentication Through Mouse Dynamics and Explainable Deep Learning: A Proposal

Over the years, the number of compromised accounts dramatically increased. To avoid this type of attack various types of authentication methods were introduced. In particular, currently, researchers are focusing on biometric-based techniques such as physical-biometric and behavior-biometric. The idea at the bottom of the last technique is that each person exhibits a unique behavior. Starting from the touch dynamics, and keyboard dynamics nowadays, one of the most promising investigation areas is currently represented by mouse dynamics. Because of the simpler technology necessary to gather biometric data without employing user sensor data, the latter has recently been a popular study area. In this paper, we propose an approach for continuous and silent user authentication based on mouse dynamics and explainable deep learning. We build a set of images using an existing dataset of mouse dynamics in CSV format. The images obtained were then used to train a deep-learning model to discriminate between legitimate and malicious users. We also adopted the Gradient-weighted Class Activation Mapping, to allow highlighting the areas of the images which are responsible for a specific legitimate/attack prediction, thus providing explainability behind the model classification. The preliminary experimental analysis based on ten different users shows that the proposed method can be promising in silent and continuous user authentication.
Adaptive Attacks and Targeted Fingerprinting of Relational Data

Fingerprinting is a method of embedding a traceable mark into digital data to (i) verify the owner and (ii) identify the recipient of a released copy of a data set. This is crucial when releasing data to third parties, especially if it involves a fee, or if the data is of sensitive nature and further sharing and leaks should be discouraged and deterred from. A fingerprint is required to (i) be robust against modifications t o t he d ata to achieve successful ownership protection, while (ii) affecting the quality and utility of the data as little as possible.So far, literature mostly assumes attackers with rather limited capabilities who perform random modification t o t he dataset. With a certain task in mind to perform on the data, the attacker can however perform an adaptive and targeted attack that maximises its chances of removing or invalidating the fingerprint, while reducing the data utility the least. In the same line, the data owner can optimise the robustness of the scheme by anticipating a specific f ocus o f t he a ttacker a nd f ocusing t he fingerprint embedding on the most valuable parts of the data. In this paper, we, therefore, provide an in-depth discussion on threat models, targeted attacks and adaptive defences. We further demonstrate the impact of targeted attacks on classical and, in comparison, adaptive fingerprinting i n a n e mpirical manner.
Incremental Fuzzy Utility Mining with Tree Structure

High-utility-itemset mining, extended from frequent-itemset mining, considers external utilities of items in quantitative databases to obtain the itemsets with high utility. To easily understand those patterns with high utility values, fuzzy utility mining adopts fuzzy sets to increase the readability of the derived utility itemsets. However, real-world databases are usually dynamic. New transactions may be intermittently added, and the corresponding mined patterns must be updated to keep correct knowledge. In this paper, we propose an incremental method based on our previously proposed batch-processing fuzzy utility tree-based mining algorithm. The proposed approach adopts the fast-update (FUP) strategy. It considers newly coming data to readjust the head table and the tree structure, which generate desired itemsets in two phases. The experimental results reveal that the proposed algorithm outperforms the tree-based batch mining method.
Fine-Grained Early Frequency Attention for Deep Speaker Representation Learning

Deep learning techniques have considerably improved speech processing in recent years. Speaker representations extracted by deep learning models are being used in a wide range of tasks, such as speaker recognition and speech emotion recognition. Attention mechanisms have started to play an important role in improving deep learning models in the field of speech processing. Nonetheless, despite the fact that important speaker-related information can be embedded in individual frequency-bins of the input spectral representations, current attention models are unable to attend to fine-grained information items in spectral representations. In this article, we propose Fine-grained Early Frequency Attention (FEFA) for speaker representation learning. Our model is a simple and lightweight model that can be integrated into various convolutional neural networks (CNN) pipelines and is capable of focusing on information items as small as frequency-bins. We evaluate the proposed model on three tasks of speaker recognition, speech emotion recognition, and spoken digit recognition. We use three widely used public datasets, namely VoxCeleb, IEMOCAP, and free spoken digit dataset for our experiments. We attach FEFA to several prominent deep learning models and evaluate its impact on the final performance. We also compare our work with other related works in the area. Our experiments show that by adding FEFA to different CNN architectures, performance is consistently improved by substantial margins, and the models equipped with FEFA outperform all the other attentive models. We also test our model against different levels of the added noise showing improvements in robustness and less sensitivity compared to the backbone networks.
How Did COVID-19 Change Faculty Members’ Use of Technology?

COVID-19 has caused great changes in education. Routines, practices, and especially the technologies used in teaching have differentiated at all levels of education. The purpose of this study is to determine the technologies used by faculty members for instructional purposes before and during the COVID-19 pandemic and their perspectives on the use of these technologies after the end of pandemic. A close-ended questionnaire was used for gathering data in this survey research study. Participants are150 faculty members from different ages and fields. Findings indicate that, although faculty members experienced various issues (e.g., infrastructure problems, access to technology, and lack of experience in technology) while teaching through technological facilities during the pandemic, they are in favor of using tools such as WhatsApp, Google classroom, Zoom Meeting, Facebook, and e-mail after the end of pandemic due to the different facilities they provide.
AIOps Essential to Unified Resiliency Management in Data Lakehouses

AIOps can provide essential value for data lake-houses as lakehouses pose complex operational challenges for Site Reliability Engineers (SRE). This paper proposes that the unified approach of data lakehouses creates a unique opportunity for unified data resiliency management. We focus on AIOps applied to disaster recovery and backup/restore. In particular, we focus on managing data lakehouse hardware resources to ensure that lakehouse data Recovery Point Objectives (RPO) are met with a high degree of accuracy. The goal is to warn an SRE about an impending RPO violation and to suggest adding given amounts of hardware resources before a given time to avoid violation of the lakehouse data’s RPO. We claim AIOps can achieve this goal with an ensemble of machine learning and time series analysis.
Emotion-Cause Relationship Between Clauses Prediction: a Novel Method Based on BERT for Emotion-Cause Pair Extraction

Emotion-cause pair extraction (ECPE) aims to obtain all emotion-cause pairs consisting of the emotion clause and the corresponding cause clause in a document. Many existing works for ECPE utilize BERT to obtain representation on each clause in the document, and then perform the classification of Cartesian product among all clause representations or the clause-level sequence tagging. In this paper, we propose to redefine ECPE as the emotion-cause relationship between clauses prediction (ECRP). ECRP fits well with the form of the next sentence prediction task in BERT, which effectively unifies the BERT’s pre-training and the ECPE-specific fi ne-tuning process. According to the task form of ECRP, we reconstruct the original ECPE dataset from the document format to the clause-pair format. The scale of data is effectively expanded, and the imbalance of data is alleviated to a certain extent because some redundant data is filtered out based o n t he r elative distance between clauses. Experiments demonstrate that our ECRP-BERT model outperforms many competitive baselines. Especially in the case of low resources, the ECRP-BERT model still achieves a good performance.
Data protections for minors with named entity recognition

The difference between minors and adults is an important legal distinction codified within multiple legal frameworks. Moreover, the distinction is important information to extract from unstructured text when applying natural language processing tasks such as anonymization, and abuse detection. Despite the value, identifying minors remains a manually performed task of which little research has been dedicated and no benchmarks are established. This paper seeks to address this need with two goals. First, the creation of labelled and publicly available dataset. Second, to demonstrate proof-of-concept with a BERT named entity recognition model, fine-tuned to make the subject distinction based on surrounding context. Toward this end, we created a custom dataset consisting of 2,534 sequences with 8,770 labelled instances of minors. This data was used to fine-tune a BERT named entity recognition model which demonstrated context aware tagging of minors versus adults. The model achieved an 89% f1 score in detecting minors and a 61% f1 score in detecting adults. While the performance of this model may not be suited for a production environment, we have established a starting point for future research in specific privacy protections for minors within a data processing stack.
Perceptual-IQ: Visual Commonsense Reasoning about Perceptual Imagination

In this paper, we present a new dataset Perceptual Imagination: Question-Answering (Perceptual-IQ) to evaluate the visual systems’ commonsense reasoning ability when confronted with perceptual changes. In our dataset, the machines are given a question that includes a perceptual change over an image and they have to predict human response to the change. Perceptual-IQ consists of 3.7K manually annotated QA pairs from 1.6K curated images and covers various types of perceptual changes. Through the evaluation of vision-language models with Perceptual-IQ, we identify the performance gap (~25%) with human performance.
User-Entity Differential Privacy in Learning Natural Language Models

In this paper, we introduce a novel concept of user-entity differential privacy (UeDP) to provide formal privacy protection simultaneously to both sensitive entities in textual data and data owners in learning natural language models (NLMs). To preserve UeDP, we developed a novel algorithm, called UeDP-Alg, optimizing the trade-off between privacy loss and model utility with a tight sensitivity bound derived from seamlessly combining user and sensitive entity sampling processes. An extensive theoretical analysis and evaluation show that our UeDP-Alg outperforms baseline approaches in model utility under the same privacy budget consumption on several NLM tasks, using benchmark datasets.
Road Scene Segmentation Based on Multi-scale Attention Mechanism

Road scene segmentation is mainly used in the field of autonomous driving today. However, the complexity of road scenes brings great challenges and difficulties to the perception and understanding of the vehicle environment, so the semantic segmentation of complex traffic scenes is a challenging research topic in the field of computer vision. When facing complex background and variable scale real scenes, the accuracy of segmentation method based on full convolution neural network needs to be further improved. In response to this problem, this research proposed a model based on a multi-scale attention to improve the performance of semantic segmentation algorithms from multiple perspectives. First, we design a multi-scale attention module, which can combine multi-scale information and attention to obtain semantic correlation between spatial dimensions and channel dimensions at different scales in encoder. Secondly, the number of fusions of low-level features and high-level features is increased to alleviate the problem of low-level information loss caused by multiple downsampling in the decoder. Finally, a better feature fusion is achieved by adding an adaptive feature fusion module after concatenating the decoder feature maps. The experimental results show that on the Cityscapes, compared with the baseline DeepLabV3+, the MIoU of the model is improved by 1.3% on the validation set and 1.2% on the test set.
Dynamic Graph Node Classification via Time Augmentation

Node classification for graph-structured data aims to classify nodes whose labels are unknown. While studies on static graphs are prevalent, few studies have focused on dynamic graph node classification. Node classification on dynamic graphs is challenging for two reasons. First, the model needs to capture both structural and temporal information, particularly on dynamic graphs with a long history and require large receptive fields. Second, model scalability becomes a significant concern as the size of the dynamic graph increases. To address these problems, we propose the Time Augmented Dynamic Graph Neural Network (TADGNN) framework. TADGNN consists of two modules: 1) a time augmentation module that captures the temporal evolution of nodes across time structurally, creating a time-augmented spatio-temporal graph, and 2) an information propagation module that learns the dynamic representations for each node across time using the constructed time-augmented graph. We perform node classification experiments on four dynamic graph benchmarks. Experimental results demonstrate that TADGNN framework outperforms several static and dynamic state-of-the-art (SOTA) GNN models while demonstrating superior scalability. We also conduct theoretical and empirical analyses to validate the efficiency of the proposed method.
Research on Mural Restoration Method Based on Generative Multi-column Transformer

Ancient mural are splendid artistic treasures of ancient civilizations, but their blurring and destruction are becoming more and more serious due to the effects of time and environment. In this paper, a Generative Multi-column Transformer method is proposed to recover the missing areas of mural. The method synthesizes different image components in a parallel manner by transformer and enhances local details by employing implicit diversified Markov random fields regularization. Generative Multi-column Transformer is used in combination with reconstruction and Markov random field loss methods to propagate local and global information from the background to the target defaced region to reconstruct the loss in that part. The experimental results show that the method based on Generative Multi-column Transformer adopted in this paper is able to restore mural images with complex textures and missing areas very well, while the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) of the method are improved, this paper provides theoretical guidance for the digital restoration of murals.
Simple and Efficient Identification of Personally Identifiable Information on a Public Website

Personally Identifiable Information (PII) is a key concept in privacy regulation. This form of information can provide revealing information about individuals, which may be collected and used for malicious purposes, such as social engineering and identity theft [1]. Consequently, privacy preserving legislations, such as GDPR place the responsibility of appropriately handling PII onto organisations who may process large amounts of personal data as part of their day to day operations. Therefore, it is necessary to develop processes that can provide privacy assurances but also to do so with increased automation and reliability [2]. This work will focus on assessing the ability of the Natural Language Processing tool, sentiment analysis and text classification algorithms to detect PII automatically, reliably and without too much complexity. To achieve this a dataset containing web pages from Newcastle University’s website, with a focus on staff profiles was created and manually labelled to indicate which sentences contained PII. The dataset was then used to train three text classification algorithms: Multinominal Naïve Bayes, Random Forest Classifier and LSTM model in order to predict the labels of an unseen portion of the dataset. The algorithms all performed well at detecting PII, with Random Forest achieving the highest accuracy at 96% and 96% F1-Score. Nevertheless, the models all mislabelled more sentences containing PII as not containing PII, than those which did not contain PII but were labelled as doing so.
Zoned Mapping Network from SDR Video to HDR Video

HDR video is popular for its brilliant colors, incredible brightness and rich details. Considering the strict production conditions of HDR video, most videos are currently saved in SDR format. Converting massive SDR videos into HDR format through technical methods can quickly fill the vacancy of HDR videos in the market and improve the utilization efficiency of film and television resources. In this paper, we propose a zoned mapping network to convert the video format. According to SDR OETF, the video frames are firstly segmented into three regions, highlight, medium and darkness. Then a tone mapping model is designed and respectively trained for the above three regions. Moreover, a detail enhancement model is proposed to form a composite inverse tone mapping network. Experiments show that our method can not only achieve good subjective visual quality, but also accomplish excellent results on various objective metrics, such as PSNR, SSIM, VIF, etc.
A Convergence Theory for Federated Average: Beyond Smoothness

Federated learning enables a large amount of edge computing devices to learn a model without data sharing jointly. As a leading algorithm in this setting, Federated Average (FedAvg), which runs Stochastic Gradient Descent (SGD) in parallel on local devices and averages the sequences only once in a while, have been widely used due to their simplicity and low communication cost. However, despite recent research efforts, it lacks theoretical analysis under assumptions beyond smoothness. In this paper, we analyze the convergence of FedAvg. Different from the existing work, we relax the assumption of strong smoothness. More specifically, we assume the semi-smoothness and semi-Lipschitz properties for the loss function, which have an additional first-order term in assumption definitions. In addition, we also assume bound on the gradient, which is weaker than the commonly used bounded gradient assumption in the convergence analysis scheme. As a solution, this paper provides a theoretical convergence study on Federated Learning.