Reviewerly/exHarmony huggingface dataset:
K-means Clustering Algorithm and Its Improvement Research

Abstract Clustering is a typical unsupervised learning method, and it is also very important in natural language processing. K-means is one of the classical algorithms in clustering. In k-means algorithm, the processing mode of abnormal data and the similarity calculation method will affect the clustering division. Aiming at the defect of K-means, this paper proposes a new similarity calculation method, that is, a similarity calculation method based on weighted and Euclidean distance. Experiments show that the new algorithm is superior to k-means algorithm in efficiency, correctness and stability.
Named entity recognition method in health preserving field based on BERT

With the aging of the population development, people pay more attention to health preserving. In order to build a knowledge graph of health-preserving field, named entity recognition is required first. In the health-preserving field, due to there is no publicly available data sets, building corpus by getting data from websites and defining seven types of entities. Considering the complexity and ambiguity of data, a named entity recognition method based on BERT in the health-preserving field is proposed. Because BERT can generate different vectors for the same word and make full use of context semantic. CNN can extract local features and BILSTM can capture long distance characteristics. Finally selecting the best sentence through the conditional random field. By comparing with other models in the same data set, it is verified that the new model is better in precision, recall and F1 score, reaching the optimal value of 87.84%, which can meet the task requirements in the health-preserving field.
Tweet-Based Bot Detection Using Big Data Analytics

Twitter is one of the most popular micro-blogging social media platforms that has millions of users. Due to its popularity, Twitter has been targeted by different attacks such as spreading rumors, phishing links, and malware. Tweet-based botnets represent a serious threat to users as they can launch large-scale attacks and manipulation campaigns. To deal with these threats, big data analytics techniques, particularly shallow and deep learning techniques have been leveraged in order to accurately distinguish between human accounts and tweet-based bot accounts. In this paper, we discuss existing techniques, and provide a taxonomy that classifies the state-of-the-art of tweet-based bot detection techniques. We also describe the shallow and deep learning techniques for tweet-based bot detection, along with their performance results. Finally, we present and discuss the challenges and open issues in the area of tweet-based bot detection.
Heterogeneous Graph Representation Learning with Relation Awareness

Representation learning on heterogeneous graphs aims to obtain meaningful node representations to facilitate various downstream tasks, such as node classification and link prediction. Existing heterogeneous graph learning methods are primarily developed by following the propagation mechanism of node representations. There are few efforts on studying the role of relations for improving the learning of more fine-grained node representations. Indeed, it is important to collaboratively learn the semantic representations of relations and discern node representations with respect to different relation types. To this end, in this paper, we propose a novel Relation-aware Heterogeneous Graph Neural Network, namely R-HGNN, to learn node representations on heterogeneous graphs at a fine-grained level by considering relation-aware characteristics. Specifically, a dedicated graph convolution component is first designed to learn unique node representations from each relation-specific graph separately. Then, a cross-relation message passing module is developed to improve the interactions of node representations across different relations. Also, the relation representations are learned in a layer-wise manner to capture relation semantics, which are used to guide the node representation learning process. Moreover, a semantic fusing module is presented to aggregate relation-aware node representations into a compact representation with the learned relation representations. Finally, we conduct extensive experiments on a variety of graph learning tasks, and experimental results demonstrate that our approach consistently outperforms existing methods among all the tasks.
Efficient Non-Sampling Knowledge Graph Embedding

Knowledge Graph (KG) is a flexible structure that is able to describe the complex relationship between data entities. Currently, most KG embedding models are trained based on negative sampling, i.e., the model aims to maximize some similarity of the connected entities in the KG, while minimizing the similarity of the sampled disconnected entities. Negative sampling helps to reduce the time complexity of model learning by only considering a subset of negative instances, which may fail to deliver stable model performance due to the uncertainty in the sampling procedure. To avoid such deficiency, we propose a new framework for KG embedding—Efficient Non-Sampling Knowledge Graph Embedding (NS-KGE). The basic idea is to consider all of the negative instances in the KG for model learning, and thus to avoid negative sampling. The framework can be applied to square-loss based knowledge graph embedding models or models whose loss can be converted to a square loss. A natural side-effect of this non-sampling strategy is the increased computational complexity of model learning. To solve the problem, we leverage mathematical derivations to reduce the complexity of non-sampling loss function, which eventually provides us both better efficiency and better accuracy in KG embedding compared with existing models. Experiments on benchmark datasets show that our NS-KGE framework can achieve a better performance on efficiency and accuracy over traditional negative sampling based models, and that the framework is applicable to a large class of knowledge graph embedding models.
Implicitly Abusive Language – What does it actually look like and why are we not getting there?

Abusive language detection is an emerging field in natural language processing which has received a large amount of attention recently.Still the success of automatic detection is limited.Particularly, the detection of implicitly abusive language, i.e. abusive language that is not conveyed by abusive words (e.g.dumbass or scum), is not working well.In this position paper, we explain why existing datasets make learning implicit abuse difficult and what needs to be changed in the design of such datasets.Arguing for a divide-and-conquer strategy, we present a list of subtypes of implicitly abusive language and formulate research tasks and questions for future research.
G-TADOC: Enabling Efficient GPU-Based Text Analytics without Decompression

Text analytics directly on compression (TADOC) has proven to be a promising technology for big data analytics. GPUs are extremely popular accelerators for data analytics systems. Unfortunately, no work so far shows how to utilize GPUs to accelerate TADOC. We describe G-TADOC, the first framework that provides GPU-based text analytics directly on compression, effectively enabling efficient text analytics on GPUs without decompressing the input data. G-TADOC solves three major challenges. First, TADOC involves a large amount of dependencies, which makes it difficult to exploit massive parallelism on a GPU. We develop a novel fine-grained thread-level workload scheduling strategy for GPU threads, which partitions heavily-dependent loads adaptively in a fine-grained manner. Second, in developing G-TADOC, thousands of GPU threads writing to the same result buffer leads to inconsistency while directly using locks and atomic operations lead to large synchronization overheads. We develop a memory pool with thread-safe data structures on GPUs to handle such difficulties. Third, maintaining the sequence information among words is essential for lossless compression. We design a sequence-support strategy, which maintains high GPU parallelism while ensuring sequence information. Our experimental evaluations show that G-TADOC provides 31.1× average speedup compared to state-of-the-art TADOC.
A Novel Access Control Method Via Smart Contracts for Internet-Based Service Provisioning

The dramatic rise in internet-based service provisioning has highlighted the importance of deploying scalable access control methods, facilitating service authorization for eligible users. Existing centralized methods suffer from single-point-of-failure, low scalability, and high computational overhead. In addition, in these methods, users pay for the service provider as well as the network provider independently for a specific service, imposing extra cost for the user. New business models are needed to resolve such shortcomings. The realization of these models calls for sophisticated access control methods which consider the requirements of all parties who want to: 1) access a service; 2) provide that service; and 3) provide the network connection. Blockchain is an enabling technology that provides unprecedented opportunities to novel distributed access control methods for new business models. We propose an Attribute-based access control solution by leveraging Blockchain to share network providers' and service providers' resources. Our solution offers access flexibility based on the requirements of the parties while fulfilling reliability, accountability, and immutability. Besides, it decreases the overall service cost which is beneficial for each party. Our solution makes it possible for service providers to outsource their access control procedures without requiring a trusted third party. The experiments confirm that our solution can provide a fast, comprehensive, and scalable access control mechanism.
Facemask Detection Algorithm on COVID Community Spread Control using EfficientNet Algorithm

Facemask has become mandatory in all COVID-infected communities present across the world. However, in real-life situations, checking the facemask code on each individual has become a difficult task. On the other hand, Automation systems are playing a widespread role in human community to automate different applications. As a result, it necessitates the need to develop a dependable automated method to monitor the facemask code to benefit humans. Recently, deep learning algorithms are emerging as a fast growing application, which has been developed for performing huge number of analysis and detection process. Henceforth, this paper proposes a deep learning based facemask detection process for automating the human effort involved in monitoring process. This work utilizes an openly available facemask detection dataset with 7553 images for the training and verification process, which is based on CNN driven EfficientNet architecture with an accuracy of about 97.12%.
Edge detection algorithm of medical image based on Canny operator

Abstract Edge detection is an important part of image segmentation, in this paper, the edge detection algorithm based on traditional Canny operator for medical images is studied. The Canny operator is improved using Otsu algorithm and double-gate limit detection method, and the ability of Canny operator edge detection is strengthened. The simulation of the algorithm is realized on the computer platform by MATLAB, and the experimental results are analyzed from two image objective evaluation indexes of information entropy and mean square error. The experimental results show that compared with the traditional Canny algorithm, the improved adaptive double threshold Canny algorithm has better edge detection effect, richer image details, better noise suppression and less false edges.
Deep Learning-Based Robust Automatic Modulation Classification for Cognitive Radio Networks

In this paper, a novel deep learning-based robust automatic modulation classification (AMC) method is proposed for cognitive radio networks. Generally, as network input of AMC convolutional neural networks (CNNs) images or complex signals are utilized in time domain or frequency domain. In terms of the image that contains RGB(Red, Green, Blue) levels the input size may be larger than the complex signal, which represents the increase of computational complexity. In terms of the complex signal it is normally used as <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2 \times N$ </tex-math></inline-formula> size for the input, which is divided into in-phase and quadrature-phase (IQ) components. In this paper, the input size is extended as <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$4 \times N$ </tex-math></inline-formula> size by copying IQ components and concatenating in reverse order to improve the classification accuracy. Since the increase in the amount of computation complexity due to the extended input size, the proposed CNN archiecture is designed to reduce the size from <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$4 \times N$ </tex-math></inline-formula> to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2 \times N$ </tex-math></inline-formula> by an average pooling layer, which can enhence the classification accuracy as well. The simulation results show that the classification accuracy of the proposed model is higher than the conventional models in the almost signal-to-noise ratio (SNR) range.
Aligning Artificial Neural Networks and Ontologies towards Explainable AI

Neural networks have been the key to solve a variety of different problems. However, neural network models are still regarded as black boxes, since they do not provide any human-interpretable evidence as to why they output a certain result. We address this issue by leveraging on ontologies and building small classifiers that map a neural network model's internal state to concepts from an ontology, enabling the generation of symbolic justifications for the output of neural network models. Using an image classification problem as testing ground, we discuss how to map the internal state of a neural network to the concepts of an ontology, examine whether the results obtained by the established mappings match our understanding of the mapped concepts, and analyze the justifications obtained through this method.
CoSQA: 20,000+ Web Queries for Code Search and Question Answering

Finding codes given natural language query is beneficial to the productivity of software developers.Future progress towards better semantic matching between query and code requires richer supervised training resources.To remedy this, we introduce the CoSQA dataset.It includes 20,604 labels for pairs of natural language queries and codes, each annotated by at least 3 human annotators.We further introduce a contrastive learning method dubbed Co-CLR to enhance query-code matching, which works as a data augmenter to bring more artificially generated training instances.We show that evaluated on CodeXGLUE with the same CodeBERT model, training on CoSQA improves the accuracy of code question answering by 5.1%, and incorporating CoCLR brings a further improvement of 10.5%. 1 .
Dynamic Vulnerability Detection on Smart Contracts Using Machine Learning

In this work we propose Dynamit, a monitoring framework to detect reentrancy vulnerabilities in Ethereum smart contracts. The novelty of our framework is that it relies only on transaction metadata and balance data from the blockchain system; our approach requires no domain knowledge, code instrumentation, or special execution environment. Dynamit extracts features from transaction data and uses a machine learning model to classify transactions as benign or harmful. Therefore, not only can we find the contracts that are vulnerable to reentrancy attacks, but we also get an execution trace that reproduces the attack. Using a random forest classifier, our model achieved more than 90 percent accuracy on 105 transactions, showing the potential of our technique.
Implementation of efficient quantum search algorithms on NISQ computers

Despite the advent of Grover’s algorithm for the unstructured search, its successful implementation on near-term quantum devices is still limited. We apply three strategies to reduce the errors associated with implementing quantum search algorithms. Our improved search algorithms have been implemented on the IBM quantum processors. Using them, we demonstrate three- and four-qubit search algorithm with higher average success probabilities compared to previous works. We present the successful execution of the five-qubit search on the IBM quantum processor for the first time. The results have been benchmarked using degraded ratio, which is the ratio between the experimental and the theoretical success probabilities. The fast decay of the degraded ratio supports our divide-and-conquer strategy. Our proposed strategies are also useful for implementation of quantum search algorithms in the post-NISQ era.
Transformers-sklearn: a toolkit for medical language understanding with transformer-based models

Abstract Background Transformer is an attention-based architecture proven the state-of-the-art model in natural language processing (NLP). To reduce the difficulty of beginning to use transformer-based models in medical language understanding and expand the capability of the scikit-learn toolkit in deep learning, we proposed an easy to learn Python toolkit named transformers-sklearn. By wrapping the interfaces of transformers in only three functions (i.e., fit, score, and predict), transformers-sklearn combines the advantages of the transformers and scikit-learn toolkits. Methods In transformers-sklearn , three Python classes were implemented, namely, BERTologyClassifier for the classification task, BERTologyNERClassifier for the named entity recognition (NER) task, and BERTologyRegressor for the regression task. Each class contains three methods, i.e., fit for fine-tuning transformer-based models with the training dataset, score for evaluating the performance of the fine-tuned model, and predict for predicting the labels of the test dataset. transformers-sklearn is a user-friendly toolkit that (1) Is customizable via a few parameters (e.g., model_name_or_path and model_type ), (2) Supports multilingual NLP tasks, and (3) Requires less coding. The input data format is automatically generated by transformers-sklearn with the annotated corpus. Newcomers only need to prepare the dataset. The model framework and training methods are predefined in transformers-sklearn . Results We collected four open-source medical language datasets, including TrialClassification for Chinese medical trial text multi label classification, BC5CDR for English biomedical text name entity recognition, DiabetesNER for Chinese diabetes entity recognition and BIOSSES for English biomedical sentence similarity estimation. In the four medical NLP tasks, the average code size of our script is 45 lines/task, which is one-sixth the size of transformers ’ script. The experimental results show that transformers-sklearn based on pretrained BERT models achieved macro F1 scores of 0.8225, 0.8703 and 0.6908, respectively, on the TrialClassification , BC5CDR and DiabetesNER tasks and a Pearson correlation of 0.8260 on the BIOSSES task, which is consistent with the results of transformers . Conclusions The proposed toolkit could help newcomers address medical language understanding tasks using the scikit-learn coding style easily. The code and tutorials of transformers-sklearn are available at https://doi.org/10.5281/zenodo.4453803 . In future, more medical language understanding tasks will be supported to improve the applications of transformers_sklearn .
Embedding-based real-time change point detection with application to activity segmentation in smart home time series data

A C THuman activity recognition systems are essential to enable many assistive applications.Those systems can be sensor-based or vision-based.When sensor-based systems are deployed in real environments, they must segment sensor data streams on the fly in order to extract features and recognize the ongoing activities.This segmentation can be done with different approaches.One effective approach is to employ change point detection (CPD) algorithms to detect activity transitions (i.e.determine when activities start and end).In this paper, we present a novel real-time CPD method to perform activity segmentation, where neural embeddings (vectors of continuous numbers) are used to represent sensor events.Through empirical evaluation with 3 publicly available benchmark datasets, we conclude that our method is useful for segmenting sensor data, offering significant better performance than state of the art algorithms in two of them.Besides, we propose the use of retrofitting, a graph-based technique, to adjust the embeddings and introduce expert knowledge in the activity segmentation task, showing empirically that it can improve the performance of our method using three graphs generated from two sources of information.Finally, we discuss the advantages of our approach regarding computational cost, manual effort reduction (no need of hand-crafted features) and cross-environment possibilities (transfer learning) in comparison to others.
An SMT Solver for Regular Expressions and Linear Arithmetic over String Length

Abstract We present a novel length-aware solving algorithm for the quantifier-free first-order theory over regex membership predicate and linear arithmetic over string length. We implement and evaluate this algorithm and related heuristics in the Z3 theorem prover. A crucial insight that underpins our algorithm is that real-world regex and string formulas contain a wealth of information about upper and lower bounds on lengths of strings, and such information can be used very effectively to simplify operations on automata representing regular expressions. Additionally, we present a number of novel general heuristics, such as the prefix/suffix method, that can be used to make a variety of regex solving algorithms more efficient in practice. We showcase the power of our algorithm and heuristics via an extensive empirical evaluation over a large and diverse benchmark of 57256 regex-heavy instances, almost 75% of which are derived from industrial applications or contributed by other solver developers. Our solver outperforms five other state-of-the-art string solvers, namely, CVC4, OSTRICH, Z3seq, Z3str3, and Z3-Trau, over this benchmark, in particular achieving a speedup of 2.4 $$\times $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mo>×</mml:mo></mml:math> over CVC4, 4.4 $$\times $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mo>×</mml:mo></mml:math> over Z3seq, 6.4 $$\times $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mo>×</mml:mo></mml:math> over Z3-Trau, 9.1 $$\times $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mo>×</mml:mo></mml:math> over Z3str3, and 13 $$\times $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mo>×</mml:mo></mml:math> over OSTRICH.