Reviewerly/exHarmony huggingface dataset:
LSTM based prediction algorithm and abnormal change detection for temperature in aerospace gyroscope shell

Purpose Abnormal changes in temperature directly affect the stability and reliability of a gyroscope. Predicting the temperature and detecting the abnormal change is great value for timely understanding of the working state of the gyroscope. Considering that the actual collected gyroscope shell temperature data have strong non-linearity and are accompanied by random noise pollution, the prediction accuracy and convergence speed of the traditional method need to be improved. The purpose of this paper is to use a predictive model with strong nonlinear mapping ability to predict the temperature of the gyroscope to improve the prediction accuracy and detect the abnormal change. Design/methodology/approach In this paper, an double hidden layer long-short term memory (LSTM) is presented to predict temperature data for the gyroscope (including single point and period prediction), and the evaluation index of the prediction effect is also proposed, and the prediction effects of shell temperature data are compared by BP network, support vector machine (SVM) and LSTM network. Using the estimated value detects the abnormal change of the gyroscope. Findings By combined simulation calculation with the gyroscope measured data, the effect of different network hyperparameters on shell temperature prediction of the gyroscope is analyzed, and the LSTM network can be used to predict the temperature (time series data). By comparing the performance indicators of different prediction methods, the accuracy of the shell temperature estimation by LSTM is better, which can meet the requirements of abnormal change detection. Quick and accurate diagnosis of different types of gyroscope faults (steps and drifts) can be achieved by setting reasonable data window lengths and thresholds. Practical implications The LSTM model is a deep neural network model with multiple non-linear mapping levels, and can abstract the input signal layer by layer and extract features to discover deeper underlying laws. The improved method has been used to solve the problem of strong non-linearity and random noise pollution in time series, and the estimated value can detect the abnormal change of the gyroscope. Originality/value In this paper, based on the LSTM network, an double hidden layer LSTM is presented to predict temperature data for the gyroscope (including single point and period prediction), and validate the effectiveness and feasibility of the algorithm by using shell temperature measurement data. The prediction effects of shell temperature data are compared by BP network, SVM and LSTM network. The LSTM network has the best prediction effect, and is used to predict the temperature of the gyroscope to improve the prediction accuracy and detect the abnormal change.
Alignment over Heterogeneous Embeddings for Question Answering

Vikas Yadav, Steven Bethard, Mihai Surdeanu. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019.
A Comparison of Machine Learning Attributes for Detecting Malicious Websites

The number of Malicious Websites has increased manifold in the past few years. As on start of year 2018, 1 in every 13 URL was malicious, amounting to 7.8% URLs identified as malicious [1]. These figures have increased by 2.8%, thereby showing an increasing trend of attack vectors through Malicious Websites. These statistics clearly highlight the need to detect Malicious Websites on the Internet. Many research works have suggested Machine Learning techniques to detect Malicious Websites. Research has also been done to compare Machine Learning algorithms for their detection. However, the aspect of attribute selection for detecting Malicious Websites using Machine Learning has not been delved in detail. In Machine Learning techniques, attribute selection outweighs the importance of any other aspect in the process. Thus, there is a need to compare and analyze the various attributes that can help find Malicious Websites faster and better. This paper is focused to address this research gap, so that, fewer and optimal attributes can do a better job.
Multi-level Dense Capsule Networks

Past few years have witnessed an exponential growth of interest in deep learning methodologies with rapidly improving accuracy and reduced computational complexity. In particular, architectures using Convolutional Neural Networks (CNNs) have produced state-of-the-art performances for image classification and object recognition tasks. Recently, Capsule Networks (CapsNets) achieved a significant increase in performance by addressing an inherent limitation of CNNs in encoding pose and deformation. Inspired by such an advancement, we propose Multi-level Dense Capsule Networks (multi-level DCNets). The proposed framework customizes CapsNet by adding multi-level capsules and replacing the standard convolutional layers with densely connected convolutions. A single-level DCNet essentially adds a deeper convolution network, which leads to learning of discriminative feature maps learned by different layers to form the primary capsules. Additionally, multi-level capsule networks uses a hierarchical architecture to learn new capsules from former capsules that represent spatial information in a fine-to-coarser manner, which makes it more efficient for learning complex data. Experiments on image classification task using benchmark datasets demonstrate the efficacy of the proposed architectures. DCNet achieves state-of-the-art performance (99.75%) on the MNIST dataset with approximately twenty-fold decrease in total training iterations, over the conventional CapsNet. Furthermore, multi-level DCNet performs better than CapsNet on SVHN dataset (96.90%), and outperforms the ensemble of seven CapsNet models on CIFAR-10 by $$+$$ 0.31% with seven-fold decrease in the number of parameters. Source codes, models and figures are available at https://github.com/ssrp/Multi-level-DCNet .
Semantic-filtered Soft-Split-Aware video captioning with audio-augmented feature

Automatic video description, or video captioning, is a challenging yet much attractive task. It aims to combine video with text. Multiple methods have been proposed based on neural networks, utilizing Convolutional Neural Networks (CNN) to extract features, and Recurrent Neural Networks (RNN) to encode and decode videos to generate descriptions. Previously, a number of methods used in video captioning task are motivated by image captioning approaches. However, videos carry much more information than images. This increases the difficulty of video captioning task. Current methods commonly lack the ability to utilize the additional information provided by videos, especially the semantic and structural information of the videos. To address the above shortcoming, we propose a Semantic-Filtered Soft-Split-Aware-Gated LSTM (SF-SSAG-LSTM) model, that would improve video captioning quality by combining semantic concepts with audio-augmented feature extracted from input videos, while understanding the underlying structure of input videos. In the experiments, we quantitatively evaluate the performance of our model which matches other prominent methods on three benchmark datasets. We also qualitatively examine the result of our model, and show that our generated descriptions are more detailed and logical.
Hybrid bag of approaches to characterize selection criteria for cohort identification

Abstract Objective The 2018 National NLP Clinical Challenge (2018 n2c2) focused on the task of cohort selection for clinical trials, where participating systems were tasked with analyzing longitudinal patient records to determine if the patients met or did not meet any of the 13 selection criteria. This article describes our participation in this shared task. Materials and Methods We followed a hybrid approach combining pattern-based, knowledge-intensive, and feature weighting techniques. After preprocessing the notes using publicly available natural language processing tools, we developed individual criterion-specific components that relied on collecting knowledge resources relevant for these criteria and pattern-based and weighting approaches to identify “met” and “not met” cases. Results As part of the 2018 n2c2 challenge, 3 runs were submitted. The overall micro-averaged F1 on the training set was 0.9444. On the test set, the micro-averaged F1 for the 3 submitted runs were 0.9075, 0.9065, and 0.9056. The best run was placed second in the overall challenge and all 3 runs were statistically similar to the top-ranked system. A reimplemented system achieved the best overall F1 of 0.9111 on the test set. Discussion We highlight the need for a focused resource-intensive effort to address the class imbalance in the cohort selection identification task. Conclusion Our hybrid approach was able to identify all selection criteria with high F1 performance on both training and test sets. Based on our participation in the 2018 n2c2 task, we conclude that there is merit in continuing a focused criterion-specific analysis and developing appropriate knowledge resources to build a quality cohort selection system.
Transfer Learning Based Free-Form Speech Command Classification for Low-Resource Languages

Current state-of-the-art speech-based user interfaces use data intense methodologies to recognize free-form speech commands.However, this is not viable for low-resource languages, which lack speech data.This restricts the usability of such interfaces to a limited number of languages.In this paper, we propose a methodology to develop a robust domainspecific speech command classification system for low-resource languages using speech data of a high-resource language.In this transfer learning-based approach, we used a Convolution Neural Network (CNN) to identify a fixed set of intents using an ASR-based character probability map.We were able to achieve significant results for Sinhala and Tamil datasets using an English based ASR, which attests the robustness of the proposed approach..
Model-based clustering of censored data via mixtures of factor analyzers

Mixtures of factor analyzers (MFA) provide a promising tool for modeling and clustering high-dimensional data that contain an overwhelmingly large number of attributes measured on individuals arisen from a heterogeneous population. Due to the restriction of experimental apparatus, measurements can be limited to some lower and/or upper detection bounds and thus the data are possibly censored. In this paper, we extend the MFA to accommodate censored data, and the new model is called the MFA with censoring (MFAC). A computationally feasible alternating expectation conditional maximization (AECM) algorithm is developed to carry out maximum likelihood estimation of the MFAC model. Practical issues related to model-based clustering and recovery of censored data are also discussed. Simulation studies are conducted to examine the effect of censoring in classification, estimation and cluster validation. We also present an application of the proposed approach to two real data examples in which a certain number of left-censored observations are present.
Results Analysis of Russian Students’ Participation in the Online International Educational Project X-Culture

The purpose of the paper is to assess the experience of Russian students in the online project class. The authors studied the impact of the international project “X-Culture” on the level of development of socio-cultural competencies of students and their knowledge of the English language. An additional aspect of the analysis was the question of the connection of students’ perception of the goal of participation in the international project “X-Culture” with their self-motivation and self-assessment of the success of the results achieved. The study used both quantitative and qualitative methods—student testing, database analysis provided by the project “X-culture”, and focus group records. The result of the participation has been inconsistent. As expected, such sociocultural competence as the level of proficiency in English increased among the majority of students, as confirmed by the values of the student’s criterion for the results of language testing, conducted before and after the project. At the same time, average indicators of sociocultural competences such as “interpersonal skills”, “creativity”, “leadership”, and “friendliness” have deteriorated during the project that is confirmed by the trends lines of time series. The focus group revealed differences in terms of participation in the X-Culture project, their connection with self-motivation, and student satisfaction with the results achieved. Students who have achieved high levels of sociocultural competences have set themselves the goals of participation in the project related to the improvement of professional competencies and intercultural communications. At the same time, students were most satisfied with their activities in the project and the results achieved, aimed at obtaining new professional knowledge and skills, and students whose goal was simply to obtain an international certificate of the project participant.
Self-Guiding Multimodal LSTM—When We Do Not Have a Perfect Training Dataset for Image Captioning

In this paper, a self-guiding multimodal LSTM (sgLSTM) image captioning model is proposed to handle an uncontrolled imbalanced real-world image-sentence dataset. We collect a FlickrNYC dataset from Flickr as our testbed with 306,165 images and the original text descriptions uploaded by the users are utilized as the ground truth for training. Descriptions in the FlickrNYC dataset vary dramatically ranging from short term-descriptions to long paragraph-descriptions and can describe any visual aspects, or even refer to objects that are not depicted. To deal with the imbalanced and noisy situation and to fully explore the dataset itself, we propose a novel guiding textual feature extracted utilizing a multimodal LSTM (mLSTM) model. Training of mLSTM is based on the portion of data in which the image content and the corresponding descriptions are strongly bonded. Afterward, during the training of sgLSTM on the rest training data, this guiding information serves as additional input to the network along with the image representations and the ground-truth descriptions. By integrating these input components into a multimodal block, we aim to form a training scheme with the textual information tightly coupled with the image content. The experimental results demonstrate that the proposed sgLSTM model outperforms the traditional state-of-the-art multimodal RNN captioning framework in successfully describing the key components of the input images.
Few-shot learning of neural networks from scratch by pseudo example optimization

© 2018. The copyright of this document resides with its authors. In this paper, we propose a simple but effective method for training neural networks with a limited amount of training data. Our approach inherits the idea of knowledge distillation that transfers knowledge from a deep or wide reference model to a shallow or narrow target model. The proposed method employs this idea to mimic predictions of reference estimators that are more robust against overfitting than the network we want to train. Different from almost all the previous work for knowledge distillation that requires a large amount of labeled training data, the proposed method requires only a small amount of training data. Instead, we introduce pseudo training examples that are optimized as a part of model parameters. Experimental results for several benchmark datasets demonstrate that the proposed method outperformed all the other baselines, such as naive training of the target model and standard knowledge distillation.
FLightNNs

To improve the throughput and energy efficiency of Deep Neural Networks (DNNs) on customized hardware, lightweight neural networks constrain the weights of DNNs to be a limited combination (denoted as k ϵ {1, 2}) of powers of 2. In such networks, the multiply-accumulate operation can be replaced with a single shift operation, or two shifts and an add operation. To provide even more design flexibility, the k for each convolutional filter can be optimally chosen instead of being fixed for every filter. In this paper, we formulate the selection of k to be differentiable, and describe model training for determining k-based weights on a per-filter basis. Over 46 FPGA-design experiments involving eight configurations and four data sets reveal that lightweight neural networks with a flexible k value (dubbed FLightNNs) fully utilize the hardware resources on Field Programmable Gate Arrays (FPGAs), our experimental results show that FLightNNs can achieve 2× speedup when compared to lightweight NNs with k = 2, with only 0.1% accuracy degradation. Compared to a 4-bit fixed-point quantization, FLightNNs achieve higher accuracy and up to 2× inference speedup, due to their lightweight shift operations. In addition, our experiments also demonstrate that FLightNNs can achieve higher computational energy efficiency for ASIC implementation.
Efficient &amp; secure cipher scheme with dynamic key-dependent mode of operation

Security attacks are constantly on the rise leading to drastic consequences. Several security services are required more than ever to prevent both passive and active attacks such as Data Confidentiality (DC). A DC security service is typically based on a strong symmetric cipher algorithm. However, some of today’s applications, such as real-time applications and those running on constrained devices, require efficient lightweight cipher schemes that can achieve a good balance between the security level and system performance. Recently, a set of lightweight cryptographic algorithms has been proposed to that end, which is based on a dynamic key approach. The dynamic structure enables the reduction of the number of rounds to the minimum possible value of just one or two rounds, which minimizes the computational overhead without degrading the security level. This paper follows the dynamic key-dependent cipher logic and proposes a new flexible lightweight technique with or without the reliance on the chaining mode of operation. Furthermore, the dynamic key changes for each input message, which leads to different cipher primitives such as substitution and permutation tables, in addition to round keys. Also, the proposed mode of operation is based on the dynamic key approach whereby blocks are selected and mixed according to a dynamic permutation table. Accordingly, different plaintext messages are encrypted differently while preserving the avalanche effect. Finally, we conduct security and performance analysis to validate the efficiency and robustness of the proposed cipher scheme as compared to traditional ciphers and to the recently proposed dynamic key-dependent ciphers.
Intrinsically Efficient, Stable, and Bounded Off-Policy Evaluation for Reinforcement Learning

Off-policy evaluation (OPE) in both contextual bandits and reinforcement learning allows one to evaluate novel decision policies without needing to conduct exploration, which is often costly or otherwise infeasible. The problem's importance has attracted many proposed solutions, including importance sampling (IS), self-normalized IS (SNIS), and doubly robust (DR) estimates. DR and its variants ensure semiparametric local efficiency if Q-functions are well-specified, but if they are not they can be worse than both IS and SNIS. It also does not enjoy SNIS's inherent stability and boundedness. We propose new estimators for OPE based on empirical likelihood that are always more efficient than IS, SNIS, and DR and satisfy the same stability and boundedness properties as SNIS. On the way, we categorize various properties and classify existing estimators by them. Besides the theoretical guarantees, empirical studies suggest the new estimators provide advantages.
Variance Reduction for Matrix Games

We present a randomized primal-dual algorithm that solves the problem min_x max_y y^T A x to additive error epsilon in time nnz(A) + sqrt{nnz(A) n} / epsilon, for matrix A with larger dimension n and nnz(A) nonzero entries. This improves the best known exact gradient methods by a factor of sqrt{nnz(A) / n} and is faster than fully stochastic gradient methods in the accurate and/or sparse regime epsilon < sqrt{n / nnz(A)$. Our results hold for x,y in the simplex (matrix games, linear programming) and for x in an \ell_2 ball and y in the simplex (perceptron / SVM, minimum enclosing ball). Our algorithm combines the Nemirovski's conceptual prox-method and a novel reduced-variance gradient estimator based on sampling from the difference between the current iterate and a reference point.
Learning to Bootstrap for Entity Set Expansion

Lingyong Yan, Xianpei Han, Le Sun, Ben He. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
The Case for Evaluating Causal Models Using Interventional Measures and Empirical Data

Causal inference is central to many areas of artificial intelligence, including complex reasoning, planning, knowledge-base construction, robotics, explanation, and fairness. An active community of researchers develops and enhances algorithms that learn causal models from data, and this work has produced a series of impressive technical advances. However, evaluation techniques for causal modeling algorithms have remained somewhat primitive, limiting what we can learn from experimental studies of algorithm performance, constraining the types of algorithms and model representations that researchers consider, and creating a gap between theory and practice. We argue for more frequent use of evaluation techniques that examine interventional measures rather than structural or observational measures, and that evaluate those measures on empirical data rather than synthetic data. We survey the current practice in evaluation and show that the techniques we recommend are rarely used in practice. We show that such techniques are feasible and that data sets are available to conduct such evaluations. We also show that these techniques produce substantially different results than using structural measures and synthetic data.
The continuous Bernoulli: fixing a pervasive error in variational autoencoders

Variational autoencoders (VAE) have quickly become a central tool in machine learning, applicable to a broad range of data types and latent variable models. By far the most common first step, taken by seminal papers and by core software libraries alike, is to model MNIST data using a deep network parameterizing a Bernoulli likelihood. This practice contains what appears to be and what is often set aside as a minor inconvenience: the pixel data is [0,1] valued, not {0,1} as supported by the Bernoulli likelihood. Here we show that, far from being a triviality or nuisance that is convenient to ignore, this error has profound importance to VAE, both qualitative and quantitative. We introduce and fully characterize a new [0,1]-supported, single parameter distribution: the continuous Bernoulli, which patches this pervasive bug in VAE. This distribution is not nitpicking; it produces meaningful performance improvements across a range of metrics and datasets, including sharper image samples, and suggests a broader class of performant VAE.