Reviewerly/exHarmony huggingface dataset:
Supervised Extreme Learning Machine-Based Auto-Encoder for Discriminative Feature Learning

In this paper, a Supervised Extreme Learning Machine-based Auto-Encoder (SELM-AE) is proposed for discriminative Feature Learning. Different from traditional ELM-AE (designed based on data information X only), SELM-AE is designed based on both data information X and label information T. In detail, SELM-AE not only minimizes the reconstruction error of input data but also minimizes the intra-class distance and maximizes the inter-class distance in the new feature space. Under this way, the new data representation extracted by proposed SELM-AE is more discriminative than traditional ELM-AE for further classification. Then multiple SELM-AEs are stacked layer by layer to develop a new multi-layer perceptron (MLP) network called ML-SAE-ELM. Benefit from SELM-AE, the proposed ML-SAE-ELM is highly effective on classification than ELM-AE based MLP. Moreover, different from ELM-AE based MLP that requires large number of hidden nodes to achieve satisfactory accuracy, ML-SAE-ELM usually takes very small number of hidden nodes on both feature learning and classification stages to achieve better accuracy, which highly lightens the network memory requirement. The proposed method has been evaluated over 13 benchmark binary and multi-class datasets and one complicated image dataset. As shown in the experimental results, through the visualization of data representation, the proposed SELM-AE extracts more discriminative data representation than ELM-AE. Moreover, the shallow ML-SAE-ELM with smaller hidden nodes achieves higher classification accuracy than hierarchical ELM (a commonly used effective ELM-AE based MLP) on most evaluated datasets.
Graph-Hist: Graph Classification from Latent Feature Histograms with Application to Bot Detection

Neural networks are increasingly used for graph classification in a variety of contexts. Social media is a critical application area in this space, however the characteristics of social media graphs differ from those seen in most popular benchmark datasets. Social networks tend to be large and sparse, while benchmarks are small and dense. Classically, large and sparse networks are analyzed by studying the distribution of local properties. Inspired by this, we introduce Graph-Hist: an end-to-end architecture that extracts a graph's latent local features, bins nodes together along 1-D cross sections of the feature space, and classifies the graph based on this multi-channel histogram. We show that Graph-Hist improves state of the art performance on true social media benchmark datasets, while still performing well on other benchmarks. Finally, we demonstrate Graph-Hist's performance by conducting bot detection in social media. While sophisticated bot and cyborg accounts increasingly evade traditional detection methods, they leave artificial artifacts in their conversational graph that are detected through graph classification. We apply Graph-Hist to classify these conversational graphs. In the process, we confirm that social media graphs are different than most baselines and that Graph-Hist outperforms existing bot-detection models.
Pairwise Matching for 3D Fragment Reassembly Based on Boundary Curves and Concave-Convex Patches

We present a simple pairwise matching method for 3D fragment reassembly that uses boundary curves and concave-convex patches to accelerate and optimize the matching.Given the boundary curves of two fracture surfaces that completely or partially coincide, we can quickly exclude the fracture surface pairs with low boundary curve similarities, which can substantially reduce the computational cost of the subsequent patch matching, where we extract and delineate concave-convex patches of the selected fracture surfaces.A modified iterative closest point algorithm is applied on these concave-convex patches to refine the alignment.Finally, we determine the matched 3D fragments according to the overlap ratio of their fracture surfaces.The results of experiments on real-world examples demonstrate that our proposed algorithm is both accurate and efficient.
Ambient Light Based Hand Gesture Recognition Enabled by Recurrent Neural Network

As an essential requirement of pervasive smart devices, free hand gestural input considered as necessary for user interactions has attracted lots of research attention for nearly decades. Nevertheless, existing proposals heavily rely on either expensive pre-deployed equipment or user on-body sensors, thus confine their application scenarios. In this paper, we propose a novel hand gesture recognition system which purely relies on ubiquitous ambient light and low-cost photodiodes. The proposed system does not need any modification to existing lighting infrastructure. While without complex signal pre-processing for modulated light, very low-cost photodiodes and processors can capture and process the light variations caused by hand gesture. To produce accurate hand gesture recognition, we design efficient algorithms based on recurrent neural network to process sensing data collected by a photodiode array. We implement a prototype consisting of an array of 8 photodiodes and extensive experiments demonstrate that the proposed solution can achieve a very high overall recognition accuracy of 99.31%.
Hypergraph Label Propagation Network

In recent years, with the explosion of information on the Internet, there has been a large amount of data produced, and analyzing these data is useful and has been widely employed in real world applications. Since data labeling is costly, lots of research has focused on how to efficiently label data through semi-supervised learning. Among the methods, graph and hypergraph based label propagation algorithms have been a widely used method. However, traditional hypergraph learning methods may suffer from their high computational cost. In this paper, we propose a Hypergraph Label Propagation Network (HLPN) which combines hypergraph-based label propagation and deep neural networks in order to optimize the feature embedding for optimal hypergraph learning through an end-to-end architecture. The proposed method is more effective and also efficient for data labeling compared with traditional hypergraph learning methods. We verify the effectiveness of our proposed HLPN method on a real-world microblog dataset gathered from Sina Weibo. Experiments demonstrate that the proposed method can significantly outperform the state-of-the-art methods and alternative approaches.
Characterizing Membership Privacy in Stochastic Gradient Langevin Dynamics

Bayesian deep learning is recently regarded as an intrinsic way to characterize the weight uncertainty of deep neural networks (DNNs). Stochastic Gradient Langevin Dynamics (SGLD) is an effective method to enable Bayesian deep learning on large-scale datasets. Previous theoretical studies have shown various appealing properties of SGLD, ranging from the convergence properties to the generalization bounds. In this paper, we study the properties of SGLD from a novel perspective of membership privacy protection (i.e., preventing the membership attack). The membership attack, which aims to determine whether a specific sample is used for training a given DNN model, has emerged as a common threat against deep learning algorithms. To this end, we build a theoretical framework to analyze the information leakage (w.r.t. the training dataset) of a model trained using SGLD. Based on this framework, we demonstrate that SGLD can prevent the information leakage of the training dataset to a certain extent. Moreover, our theoretical analysis can be naturally extended to other types of Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) methods. Empirical results on different datasets and models verify our theoretical findings and suggest that the SGLD algorithm can not only reduce the information leakage but also improve the generalization ability of the DNN models in real-world applications.
Bare-Hand Depth Perception Used in Augmented Reality Assembly Supporting

Augmented Reality (AR) had been used in mechanical assembly supporting. The current reported researches of AR assembly supporting system (ARASS) less concerned about the interaction between the operators' hands and virtual objects, especially on the occlusion between hands and virtual objects. The correct occlusion results would make ARASS more immersive and then the operators' interaction experiences were increased. To address this issue, this paper presented a bared-hand depth perception method that was designed for improving interaction experience in ARASS. The method was based on an interaction scene that was designed for AR mechanical assembly supporting. The method consisted of two components: hand segmentation and hand mesh generation. Hand segmentation method segmented operator's hand areas from depth image of scene and divided hand area into several sub areas that provided hand information. Hand surface mesh generation method generated hand surface meshes in 3D space based on the results of hand segmentation. The meshes were used to solve the occlusion problem of hand area in AR scene. The results verified that the bared-hand depth perception method could handle the occlusion between operator's hands and virtual objects correctly in real-time and recognize hand information in a limited space. The method could increase operators' depth perception of bared hand and make ARASS more immersive.
Cyberbullying identification in twitter using support vector machine and information gain based feature selection

&lt;span&gt;Cyberbullying is one of the actions that violate the ITE Law where the crime is committed on social media applications such as Twitter. This action is difficult to detect if no one is reporting the tweet. Cyberbullying tweet identification aims to classify tweets that contain bullying. Classification is done using Support Vector Machine method where this method aims to find the dividing hyperplane between negative and positive class. This study is a text classification where more data is used, the more features are produced, therefore this research also uses Information Gain as feature selection to select features that are not relevant to the classification. The process of the system starts from text preprocessing with tokenizing, filtering, stemming and term weighting. Then perform the information gain feature selection by calculating the entropy value of each term. After that perform the classification process based on the terms that have been selected, and the output of the system is identification whether the tweet is bullying or not. The result of using SVM method is accuracy 75%, precision 70.27%, recall 86.66% and f-measure 77.61% on experiment maximum iteration = 20, λ = 0.5, γ = 0.001, ε = 0.000001, and C = 1. The best threshold of information gain is 90%, with accuracy 76.66%, precision 72.22%, recall 86.66% and f-measure 78.78%.&lt;/span&gt;
ReCO: A Large Scale Chinese Reading Comprehension Dataset on Opinion

This paper presents the ReCO, a human-curated Chinese Reading Comprehension dataset on Opinion. The questions in ReCO are opinion based queries issued to commercial search engine. The passages are provided by the crowdworkers who extract the support snippet from the retrieved documents. Finally, an abstractive yes/no/uncertain answer was given by the crowdworkers. The release of ReCO consists of 300k questions that to our knowledge is the largest in Chinese reading comprehension. A prominent characteristic of ReCO is that in addition to the original context paragraph, we also provided the support evidence that could be directly used to answer the question. Quality analysis demonstrates the challenge of ReCO that it requires various types of reasoning skills such as causal inference, logical reasoning, etc. Current QA models that perform very well on many question answering problems, such as BERT (Devlin et al. 2018), only achieves 77% accuracy on this dataset, a large margin behind humans nearly 92% performance, indicating ReCO present a good challenge for machine reading comprehension. The codes, dataset and leaderboard will be freely available at https://github.com/benywon/ReCO.
Human Activity Recognition Using Accelerometer, Gyroscope and Magnetometer Sensors: Deep Neural Network Approaches

Nowadays, smartphone with various extraordinary and notable sensors makes new invigorating open entryways for Data Mining and Machine Learning; other than makes another exploration field for Human Activity Recognition a.k.a. HAR. Such development advanced the manufacture of an assortment of useful datasets, which encourages the investigation of certainties of different research area. The MHEALTH, a multivariate time series dataset is such an optional dataset that was arranged in order to encourage the investigation with respect to HAR. The dataset includes data with respect to twelve human activities oppressing ten volunteers. Sensors were utilized for the information amassing process to be specific Accelerometer, Gyroscope, Magnetometer and Electrocardiogram signal. This paper plays out a relative report on HAR process as far as work of four unique information preprocessing strategies joined by six popular classifiers entitled as Random Forest, Support Vector Machine, Naive Bayes, along with three deep learning approaches, Multilayer Perceptron, Deep Convolutional Neural Network and Long-Short Term Memory. Information preprocessing strategies that were utilized on the referenced dataset are disposal of invalid name occurrences, consistency of unequal classes, low-pass filtering and Principle Component Analysis. The objective of this examination is to break down execution of various classifiers as far as the referenced information preprocessing techniques and furthermore distinguishing the procedure for which the classifiers display prevalent precision.
Probabilistic Reasoning Across the Causal Hierarchy

We propose a formalization of the three-tier causal hierarchy of association, intervention, and counterfactuals as a series of probabilistic logical languages. Our languages are of strictly increasing expressivity, the first capable of expressing quantitative probabilistic reasoning—including conditional independence and Bayesian inference—the second encoding do-calculus reasoning for causal effects, and the third capturing a fully expressive do-calculus for arbitrary counterfactual queries. We give a corresponding series of finitary axiomatizations complete over both structural causal models and probabilistic programs, and show that satisfiability and validity for each language are decidable in polynomial space.
Learning to Reason: Leveraging Neural Networks for Approximate DNF Counting

Weighted model counting (WMC) has emerged as a prevalent approach for probabilistic inference. In its most general form, WMC is #P-hard. Weighted DNF counting (weighted #DNF) is a special case, where approximations with probabilistic guarantees are obtained in O(nm), where n denotes the number of variables, and m the number of clauses of the input DNF, but this is not scalable in practice. In this paper, we propose a neural model counting approach for weighted #DNF that combines approximate model counting with deep learning, and accurately approximates model counts in linear time when width is bounded. We conduct experiments to validate our method, and show that our model learns and generalizes very well to large-scale #DNF instances.
Dynamic Reward-Based Dueling Deep Dyna-Q: Robust Policy Learning in Noisy Environments

Task-oriented dialogue systems provide a convenient interface to help users complete tasks. An important consideration for task-oriented dialogue systems is the ability to against the noise commonly existed in the real-world conversation. Both rule-based strategies and statistical modeling techniques can solve noise problems, but they are costly. In this paper, we propose a new approach, called Dynamic Reward-based Dueling Deep Dyna-Q (DR-D3Q). The DR-D3Q can learn policies in noise robustly, and it is easy to implement by combining dynamic reward and the Dueling Deep Q-Network (Dueling DQN) into Deep Dyna-Q (DDQ) framework. The Dueling DQN can mitigate the negative impact of noise on learning policies, but it is inapplicable to dialogue domain due to different reward mechanisms. Unlike typical dialogue reward function, we integrate dynamic reward that provides reward in real-time for agent to make Dueling DQN adapt to dialogue domain. For the purpose of supplementing the limited amount of real user experiences, we take the DDQ framework as the basic framework. Experiments using simulation and human evaluation show that the DR-D3Q significantly improve the performance of policy learning tasks in noisy environments.1
MultiSumm: Towards a Unified Model for Multi-Lingual Abstractive Summarization

Automatic text summarization aims at producing a shorter version of the input text that conveys the most important information. However, multi-lingual text summarization, where the goal is to process texts in multiple languages and output summaries in the corresponding languages with a single model, has been rarely studied. In this paper, we present MultiSumm, a novel multi-lingual model for abstractive summarization. The MultiSumm model uses the following training regime: (I) multi-lingual learning that contains language model training, auto-encoder training, translation and back-translation training, and (II) joint summary generation training. We conduct experiments on summarization datasets for five rich-resource languages: English, Chinese, French, Spanish, and German, as well as two low-resource languages: Bosnian and Croatian. Experimental results show that our proposed model significantly outperforms a multi-lingual baseline model. Specifically, our model achieves comparable or even better performance than models trained separately on each language. As an additional contribution, we construct the first summarization dataset for Bosnian and Croatian, containing 177,406 and 204,748 samples, respectively.
Learning from Weak-Label Data: A Deep Forest Expedition

Weak-label learning deals with the problem where each training example is associated with multiple ground-truth labels simultaneously but only partially provided. This circumstance is frequently encountered when the number of classes is very large or when there exists a large ambiguity between class labels, and significantly influences the performance of multi-label learning. In this paper, we propose LCForest, which is the first tree ensemble based deep learning method for weak-label learning. Rather than formulating the problem as a regularized framework, we employ the recently proposed cascade forest structure, which processes information layer-by-layer, and endow it with the ability of exploiting from weak-label data by a concise and highly efficient label complement structure. Specifically, in each layer, the label vector of each instance from testing-fold is modified with the predictions of random forests trained with the corresponding training-fold. Since the ground-truth label matrix is inaccessible, we can not estimate the performance via cross-validation directly. In order to control the growth of cascade forest, we adopt label frequency estimation and the complement flag mechanism. Experiments show that the proposed LCForest method compares favorably against the existing state-of-the-art multi-label and weak-label learning methods.
Hierarchical Coloured Petri-Net Based Multi-Agent System for Flood Monitoring, Prediction, and Rescue (FMPR)

In Punjab Pakistan, a river flood event is the most important natural disasters that every year causes high human casualties as well as heavy economic losses. A safety-critical system should be correct, reliable, complete, consistent, and unambiguous for accurate and precise flood monitoring, prediction, as well as emergency and rescue services during or after a flood event. System correctness must be ensured in every phase of the development of a safety-critical Flood Monitoring, Prediction, and Rescue (FMPR) system. Correctness is a functional property related to the behavior of the system having sub-properties of safety and liveness. These sub-properties must also be correct, complete, consistent, and unambiguous. Rigorous mathematical based software engineering methods like formal methods are suitable for the specification, design, modeling, verification, and validation of an FMPR safety-critical system. The proposed safety-critical system for FMPR is distributed as it is based on multi-agents. This system is specified, analyzed and designed by using Gaia multi-agent methodology, which is based on organizational abstractions. Gaia role model-based agent roles are specified, regular expression based liveness properties are specified, and first-order predicate calculus based safety properties are specified. Thus, safety and liveness properties are formally specified. After the detailed design phase, the system is formally modeled and verified by Hierarchical Coloured-Petri Nets (CP-Nets). As a result, a novel formal system for the specification, analysis, design, modeling, and verification of the FMPR system is proposed. This safety-critical system is distributed based on multi-agents.
Data Poison Detection Schemes for Distributed Machine Learning

Distributed machine learning (DML) can realize massive dataset training when no single node can work out the accurate results within an acceptable time. However, this will inevitably expose more potential targets to attackers compared with the non-distributed environment. In this paper, we classify DML into basic-DML and semi-DML. In basic-DML, the center server dispatches learning tasks to distributed machines and aggregates their learning results. While in semi-DML, the center server further devotes resources into dataset learning in addition to its duty in basic-DML. We firstly put forward a novel data poison detection scheme for basic-DML, which utilizes a cross-learning mechanism to find out the poisoned data. We prove that the proposed cross-learning mechanism would generate training loops, based on which a mathematical model is established to find the optimal number of training loops. Then, for semi-DML, we present an improved data poison detection scheme to provide better learning protection with the aid of the central resource. To efficiently utilize the system resources, an optimal resource allocation approach is developed. Simulation results show that the proposed scheme can significantly improve the accuracy of the final model by up to 20% for support vector machine and 60% for logistic regression in the basic-DML scenario. Moreover, in the semi-DML scenario, the improved data poison detection scheme with optimal resource allocation can decrease the wasted resources for 20-100%.
Residual Continual Learning

We propose a novel continual learning method called Residual Continual Learning (ResCL). Our method can prevent the catastrophic forgetting phenomenon in sequential learning of multiple tasks, without any source task information except the original network. ResCL reparameterizes network parameters by linearly combining each layer of the original network and a fine-tuned network; therefore, the size of the network does not increase at all. To apply the proposed method to general convolutional neural networks, the effects of batch normalization layers are also considered. By utilizing residual-learning-like reparameterization and a special weight decay loss, the trade-off between source and target performance is effectively controlled. The proposed method exhibits state-of-the-art performance in various continual learning scenarios.