Reviewerly/exHarmony huggingface dataset:
A BiLSTM-CRF Entity Type Tagger for Question Answering System

Question answering system over linked data (QALD) has been a very important research field in natural language processing (NLP). And the process of detecting useful words and assigning them with right entity types is crucial to the performance of QALD systems. Although entity-type taggers achieved good results using probability graph models such as MEMM and CRF, the design and selection of features may pose limitations. Due to the popularity of deep learning architectures, many studies employed Recurrent Neural Network (RNN) framework and achieved state-of-art performances in NLP. Therefore, we choose to use BiLSTM-CRF in the design of entity-type tagger. It can be seen from the experimental results that the proposed BiLSTM-CRF model outperformed other probability graph models, which also lead to the best performance of overall Question Answering system than other competitor systems.
Improving Convolutional Neural Network-Based Webshell Detection Through Reinforcement Learning

Webshell detection is highly important for network security protection. Conventional methods are based on keywords matching, which heavily relies on experiences of domain experts when facing emerging malicious webshells of various kinds. Recently, machine learning, especially supervised learning, is introduced for webshell detection and has proved to be a great success. As one of state-of-the-art work, neural network (NN) is designed to input a large number of features and enable deep learning. Thus, how to properly combine the advantages of automatic feature selection and the advantages of expert knowledge-based way has become a key issue. Considering that special features to indicate unexpected webshell behaviors for a target business system are usually simple but effective, in this work, we propose a novel approach for improving webshell detection based on convolutional neural network (CNN) through reinforcement learning. We utilize the reinforcement learning of asynchronous advantage actor-critic (A3C) for automatic feature selection, aiming to maximize the expected accuracy of the CNN classifier on a validation dataset by sequentially interacting with the feature space. Moreover, considering the sparseness of feature values, we build the CNN classifier with two convolutional layers and a global pooling. Extensive experiments and analysis have been conducted to demonstrate the effectiveness of our proposed method.
Efficient Text Classification with Echo State Networks

We consider echo state networks (ESNs) for text classification. More specifically, we investigate the learning capabilities of ESNs with pre-trained word embedding as input features, trained on the IMDb and TREC sentiment and question classification datasets, respectively. First, we introduce a customized training paradigm for the processing of multiple input time series (the inputs texts) associated with categorical targets (their corresponding classes). For sentiment tasks, we use an additional frozen attention mechanism which is based on an external lexicon, and hence requires only negligible computational cost. Within this paradigm, ESNs can be trained in tens of seconds on a GPU. We show that ESNs significantly outperform their Ridge regression baselines provided with the same embedded features. ESNs also compete with classical Bi-LSTM networks while keeping a training time of up to 23 times faster. These results show that ESNs can be considered as robust, efficient and fast candidates for text classification tasks. Overall, this study falls within the context of light and fast-to-train models for NLP.
DIRECT: A Differential Dynamic Programming Based Framework for Trajectory Generation

This letter introduces a differential dynamic programming (DDP) based framework for polynomial trajectory generation for differentially flat systems. In particular, instead of using a linear equation with increasing size to represent multiple polynomial segments as in literature, we take a new perspective from state-space representation such that the linear equation reduces to a finite horizon control system with a fixed state dimension and the required continuity conditions for consecutive polynomials are automatically satisfied. Consequently, the constrained trajectory generation problem (both with and without time optimization) can be converted to a discrete-time finite-horizon optimal control problem with inequality constraints, which can be approached by a recently developed interior-point DDP (IPDDP) algorithm. Furthermore, for unconstrained trajectory generation with preallocated time, we show that this problem is indeed a linear-quadratic tracking (LQT) problem (DDP algorithm with exact one iteration). All these algorithms enjoy linear complexity with respect to the number of segments. Both numerical comparisons with state-of-the-art methods and physical experiments are presented to verify and validate the effectiveness of our theoretical findings. The implementation code will be open-sourced. <xref ref-type="fn" rid="fn1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><sup>1</sup></xref> <fn id="fn1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><label><sup>1</sup></label> [Online] Available: <uri>https://github.com/ntu-caokun/DIRECT</uri> </fn>
Measuring chaos in the Lorenz and Rössler models: Fidelity tests for reservoir computing

This study focuses on the qualitative and quantitative characterization of chaotic systems with the use of a symbolic description. We consider two famous systems, Lorenz and Rössler models with their iconic attractors, and demonstrate that with adequately chosen symbolic partition, three measures of complexity, such as the Shannon source entropy, the Lempel–Ziv complexity, and the Markov transition matrix, work remarkably well for characterizing the degree of chaoticity and precise detecting stability windows in the parameter space. The second message of this study is to showcase the utility of symbolic dynamics with the introduction of a fidelity test for reservoir computing for simulating the properties of the chaos in both models’ replicas. The results of these measures are validated by the comparison approach based on one-dimensional return maps and the complexity measures.
Semantically Rich Framework to Automate Cyber Insurance Services

With the rapid enhancements in technology and the adoption of web services, there has been a significant increase in cyber threats faced by organizations in cyberspace. It has become essential to get financial cover to mitigate the expenses due to a security incident. Organizations want to purchase adequate cyber insurance to safeguard against the third-party services they use. However, cyber insurance policies describe their coverages and exclusions using legal jargon that can be difficult to comprehend. Parsing these policy documents and extracting the rules embedded in them is currently a very manual time-consuming process. We have developed a novel framework that automatically extracts the coverage and exclusion key terms and rules embedded in a cyber policy. We have built our framework using Information Retrieval and Artificial Intelligence techniques, specifically Semantic Web and Modal Logic. We have also developed a web interface where users can find the best matching cyber insurance policy based on particular coverage criteria. To validate our approach, we used industry standards proposed by the Federal Trade Commission document (FTC) and have applied it against publicly available policies of seven insurance providers. Our system will allow cyber insurance seekers to explore various policy documents and compare the paradigms mentioned in those documents while selecting the best relevant policy documents.
A Linguistic System for Predicting Sentiment in Arabic Tweets

The term sentiment analysis is considered very important in our current era, especially with widespread of social media, as it helps understand people's feelings, behavior and opinions about a specific behavior or entity, individuals, organizations and any related topic. Recently, with the development of machine learning, there have been many studies concerned with analyzing feelings. Still, most of these researches are concerned with the English language more than other languages. This paper proposes a model for working with Standard Arabic and some other Arabic dialects such as Levantine, Egyptian, and Gulf. Working with the Arabic language poses several challenges due to the complex structure of the language, the large number of dialects used and the lack of associated resources. The data collected was divided into positive, negative, and neutral. Several algorithms were used to predict sentiment in Arabic texts such as Naive Bayes classifiers (NB), Support Vector Machine (SVM), Random Forest Classifier, and BERT model (Bidirectional Encoder Representations from Transformers). The results obtained are very encouraging, especially with the Bert model (Bidirectional Encoder Representations from Transformers) that gave very accurate results during the test, reaching more than 83%.
Exploiting Multi-aspect Interactions for God Class Detection with Dataset Fine-tuning

God class refers to a class that undertakes too many responsibilities for tasks that should more appropriately be handled by multiple classes. The existence of god classes seriously affects the maintainability and understandability of software. To eliminate god class, we first need to identify them. Researchers have proposed traditional methods using code metrics and deep learning methods using code metrics and text information to detect god classes. However, the relationship existing in metrics and text information is often ignored; moreover, deep learning methods require a large number of reliable datasets, while authentic god class datasets are scarce. To solve the above problems, we propose a novel god class detection method based on multi-aspect interactions and dataset fine-tuning. First, we use proposed model to extract multi-aspect interaction information, including three parts: (i) the interaction information existing in code metrics; (ii) the interaction information existing in texts; (iii) the interaction information existing in texts and code metrics. In this way, we can not only make use of code metrics and text information, but also fully exploit the multi-aspect interaction information. Second, we train with large-scale synthetic datasets to obtain a pre-trained model, then fine-tune the pre-trained model parameters with high-quality authentic datasets. Using the training method of pre-training and fine-tuning, we can solve the problem of low-reliability synthetic datasets and scarce authentic datasets. Finally, evaluation results on open-source applications suggest that the proposed approach improves on the state-of-the-art.
Advances in Morphological Theory: Construction Morphology and Relational Morphology

In recent years, construction-based approaches to morphology have gained ground in the research community. This framework is characterized by the assumption that the mental lexicon is extensive and richly structured, containing not only a large number of stored words but also a wide variety of generalizations in the form of schemas. This review explores two construction-based theories, Construction Morphology and Relational Morphology. After outlining the basic theoretical architecture, the article presents an array of recent applications of a construction-based approach to morphological phenomena in various languages. In addition, it offers reflections on challenges and opportunities for further research. The review highlights those aspects of the theory that have proved particularly helpful in accommodating both the regularities and the quirks that are typical of the grammar of words.
Empirical Analysis of On-chain Voting with Smart Contracts

Blockchains and smart contracts promise transparency, verifiability, and self-enforcing agreements. Against this background, novel use cases such as decentralized governance platforms that implement voting to collectively manage funds have emerged. While a number of arguments against blockchain-based voting exist, we still see a relevance. In this paper, we therefore present a quantitative analysis of the Ethereum blockchain with respect to voting. To this end, we develop a blockchain analysis toolchain that we use to analyze 3 173 smart contracts on the Ethereum Mainnet with voting functionality. We extract insights on the complexity of deployed voting methods and reveal a trend towards a centralization of funds, i.e., five smart contracts manage 98% of funds comprising more than four million USD. We additionally analyze the feasibility of on-chain voting for Ethereum as well as other well-established blockchains that are used for voting, i.e., Bitcoin and Dash.
Multiple Run Ensemble Learning with Low-Dimensional Knowledge Graph Embeddings

Knowledge graphs (KGs) represent world facts in a structured form. Although knowledge graphs are quantitatively huge and consist of millions of triples, the coverage is still only a small fraction of world's knowledge. Among the top approaches of recent years, link prediction using knowledge graph embedding (KGE) models has gained significant attention for knowledge graph completion. Various embedding models have been proposed so far, among which, some recent KGE models obtain state-of-the-art performance on link prediction tasks by using embeddings with a high dimension (e.g. 1000) which accelerate the costs of training and evaluation considering the large scale of KGs. In this paper, we propose a simple but effective performance boosting strategy for KGE models by using multiple low dimensions in different repetition rounds of the same model. For example, instead of training a model one time with a large embedding size of 1200, we repeat the training of the model 6 times in parallel with an embedding size of 200 and then combine the 6 separate models for testing while the overall numbers of adjustable parameters are same (6*200=1200) and the total memory footprint remains the same. We show that our approach enables different models to better cope with their expressiveness issues on modeling various graph patterns such as symmetric, 1-n, n-1 and n-n. In order to justify our findings, we conduct experiments on various KGE models. Experimental results on standard benchmark datasets, namely FB15K, FB15K-237 and WN18RR, show that multiple low-dimensional models of the same kind outperform the corresponding single high-dimensional models on link prediction in a certain range and have advantages in training efficiency by using parallel training while the overall numbers of adjustable parameters are same.
Real-time deep learning based traffic analytics

The increased interest towards Deep Learning (DL) technologies has led to the development of a new generation of specialized hardware accelerator [8] such as Graphic Processing Unit (GPU) and Tensor Processing Unit (TPU) [1, 2]. Although attractive for implementing real-time analytics based traffic engineering fostering the development of self-driving networks [5], the integration of such components in network routers is not trivial. Indeed, routers typically aim to minimize the overhead of per-packet processing (e.g., Ethernet switching, IP forwarding, telemetry) and design choices (e.g., power, memory consumption) to integrate a new accelerator need to factor in these key requirements. Previous works on DL hardware accelerators have overlooked specific router constraints (e.g., strict latency) and focused instead on cloud deployment [4] and image processing. Likewise, there is limited literature regarding DL application on traffic processing at line-rate [6, 9].
Privacy Preserving Collaborative Filtering by Distributed Mediation

Recommender systems have become very influential in our everyday decision making, e.g., helping us choose a movie from a content platform, or offering us suitable products on e-commerce websites. While most vendors who utilize recommender systems rely exclusively on training data consisting of past transactions that took place through them, the accuracy of recommendations can be improved if several vendors conjoin their datasets. Alas, such data sharing poses grave privacy concerns for both the vendors and the users. In this study we present secure multi-party protocols that enable several vendors to share their data, in a privacy-preserving manner, in order to allow more accurate Collaborative Filtering (CF). Shmueli and Tassa (RecSys 2017) introduced privacy-preserving CF protocols that rely on a mediator; namely, a third party that assists in performing the computations. They demonstrated the significant advantages of mediation in that context. We take here the mediation approach into the next level by using several independent mediators. Such distributed mediation maintains all of the advantages that were identified by Shmueli and Tassa, and offers additional ones, in comparison with the single-mediator protocols: stronger security and dramatically shorter runtimes. In addition, while all prior art assumed limited and unrealistic settings, in which each user can purchase any given item through only one vendor, we consider here a general and more realistic setting, which encompasses all previously considered settings, where users can choose between different competing vendors. We demonstrate the appealing performance of our protocols through extensive experimentation.
VeriDL: Integrity Verification of Outsourced Deep Learning Services

Deep neural networks (DNNs) are prominent due to their superior performance in many fields. The deep-learning-as-a-service (DLaaS) paradigm enables individuals and organizations (clients) to outsource their DNN learning tasks to the cloud-based platforms. However, the DLaaS server may return incorrect DNN models due to various reasons (e.g., Byzantine failures). This raises the serious concern of how to verify if the DNN models trained by potentially untrusted DLaaS servers are indeed correct. To address this concern, in this paper, we design VeriDL, a framework that supports efficient correctness verification of DNN models in the DLaaS paradigm. The key idea of VeriDL is the design of a small-size cryptographic proof of the training process of the DNN model, which is associated with the model and returned to the client. Through the proof, VeriDL can verify the correctness of the DNN model returned by the DLaaS server with a deterministic guarantee and cheap overhead. Our experiments on four real-world datasets demonstrate the efficiency and effectiveness of VeriDL.
Intelligent Search of Values for a Controller Using the Artificial Bee Colony Algorithm to Control the Velocity of Displacement of a Robot

The optimization is essential in the engineering area and, in conjunction with use of meta-heuristics, has had a great impact in recent years; this is because of its great precision in search of optimal parameters for the solution of problems. In this work, the use of the Artificial Bee Colony Algorithm (ABC) is presented to optimize the values for the variables of a proportional integral controller (PI) to observe the behavior of the controller with the optimized Ti and Kp values. It is proposed using a robot built using the MINDSTORMS version EV3 kit. The objective of this work is to demonstrate the improvement and efficiency of the controllers in conjunction with optimization meta-heuristics. In the results section, we observe that the results improve considerably compared to traditional methods. In this work, the main contribution is the implementation of an optimization algorithm (ABC) applied to a controller (PI), and the results are tested to control the movement of a robot. There are many papers where the kit is used in various domains such as education as well as research for science and technology tasks and some real-world problems by engineering scholars, showing the acceptable result.
Differentiating Interpreting Types: Connecting Complex Networks to Cognitive Complexity

Prominent interpreting models have illustrated different processing mechanisms of simultaneous interpreting and consecutive interpreting. Although great efforts have been made, a macroscopic examination into interpreting outputs is sparse. Since complex network is a powerful and feasible tool to capture the holistic features of language, the present study adopts this novel approach to investigate different properties of syntactic dependency networks based on simultaneous interpreting and consecutive interpreting outputs. Our results show that consecutive interpreting networks demonstrate higher degrees, higher clustering coefficients, and a more important role of function words among the central vertices than simultaneous interpreting networks. These findings suggest a better connectivity, better transitivity, and a lower degree of vocabulary richness in consecutive interpreting outputs. Our research provides an integrative framework for the understanding of underlying mechanisms in diverse interpreting types.
A Wasserstein Index of Dependence for Random Measures

Optimal transport and Wasserstein distances are flourishing in many scientific fields as a means for comparing and connecting random structures. Here we pioneer the use of an optimal transport distance between Lévy measures to solve a statistical problem. Dependent Bayesian nonparametric models provide flexible inference on distinct, yet related, groups of observations. Each component of a vector of random measures models a group of exchangeable observations, while their dependence regulates the borrowing of information across groups. We derive the first statistical index of dependence in [0,1] for (completely) random measures that accounts for their whole infinite-dimensional distribution, which is assumed to be equal across different groups. This is accomplished by using the geometric properties of the Wasserstein distance to solve a max–min problem at the level of the underlying Lévy measures. The Wasserstein index of dependence sheds light on the models’ deep structure and has desirable properties: (i) it is 0 if and only if the random measures are independent; (ii) it is 1 if and only if the random measures are completely dependent; (iii) it simultaneously quantifies the dependence of d≥2 random measures, avoiding the need for pairwise comparisons; (iv) it can be evaluated numerically. Moreover, the index allows for informed prior specifications and fair model comparisons for Bayesian nonparametric models. Supplementary materials for this article are available online.