OpenRaiser/Intern-Atlas huggingface dataset:
Assessment of the Relative Importance of different hyper-parameters of LSTM for an IDS
Venue: arXiv cs.CR 2020
Year: 2020

Recurrent deep learning language models like the LSTM are often used to provide advanced cyber-defense for high-value assets. The underlying assumption for using LSTM networks for malware-detection is that the op-code sequence of malware could be treated as a (spoken) language representation. There are differences between any spoken-language (sequence of words/sentences) and the machine-language (sequence of op-codes). In this paper, we demonstrate that due to these inherent differences, an LSTM model with its default configuration as tuned for a spoken-language, may not work well to detect malware (using its op-code sequence) unless the network's essential hyper-parameters are tuned appropriately. In the process, we also determine the relative importance of all the different hyper-parameters of an LSTM network as applied to malware detection using their op-code sequence representations. We experimented with different configurations of LSTM networks, and altered hyper-parameters like the embedding-size, number of hidden layers, number of LSTM-units in a hidden layer, pruning/padding-length of the input-vector, activation-function, and batch-size. We discovered that owing to the enhanced complexity of the malware/machine-language, the performance of an LSTM network configured for an Intrusion Detection System, is very sensitive towards the number-of-hidden-layers, input sequence-length, and the choice of the activation-function. Also, for (spoken) language-modeling, the recurrent architectures by-far outperform their non-recurrent counterparts. Therefore, we also assess how sequential DL architectures like the LSTM compare against their non-sequential counterparts like the MLP-DNN for the purpose of malware-detection.
Enhancing Handwritten Text Recognition with N-gram sequence decomposition and Multitask Learning
Venue: arXiv cs.CV 2020
Year: 2020

Current state-of-the-art approaches in the field of Handwritten Text Recognition are predominately single task with unigram, character level target units. In our work, we utilize a Multi-task Learning scheme, training the model to perform decompositions of the target sequence with target units of different granularity, from fine to coarse. We consider this method as a way to utilize n-gram information, implicitly, in the training process, while the final recognition is performed using only the unigram output. % in order to highlight the difference of the internal Unigram decoding of such a multi-task approach highlights the capability of the learned internal representations, imposed by the different n-grams at the training step. We select n-grams as our target units and we experiment from unigrams to fourgrams, namely subword level granularities. These multiple decompositions are learned from the network with task-specific CTC losses. Concerning network architectures, we propose two alternatives, namely the Hierarchical and the Block Multi-task. Overall, our proposed model, even though evaluated only on the unigram task, outperforms its counterpart single-task by absolute 2.52\% WER and 1.02\% CER, in the greedy decoding, without any computational overhead during inference, hinting towards successfully imposing an implicit language model.
On the computation of the slow dynamics of nonlinear modes of mechanical systems
Venue: arXiv cs.CE 2020
Year: 2020

A novel method for the numerical prediction of the slowly varying dynamics of nonlinear mechanical systems has been developed. The method is restricted to the regime of an isolated nonlinear mode and consists of a two-step procedure: In the first step, a multiharmonic analysis of the autonomous system is performed to directly compute the amplitude-dependent characteristics of the considered nonlinear mode. In the second step, these modal properties are used to construct a two-dimensional reduced order model (ROM) that facilitates the efficient computation of steady-state and unsteady dynamics provided that nonlinear modal interactions are absent. The proposed methodology is applied to several nonlinear mechanical systems ranging form single degree-of-freedom to Finite Element models. Unsteady vibration phenomena such as approaching behavior towards an equilibrium point or limit cycles, and resonance passages are studied regarding the effect of various nonlinearities such as cubic springs, unilateral contact and friction. It is found that the proposed ROM facilitates very fast and accurate analysis of the slow dynamics of nonlinear systems. Moreover, the ROM concept offers a huge parameter space including additional linear damping, stiffness and near-resonant forcing.
A Survey on Vulnerabilities of Ethereum Smart Contracts
Venue: arXiv cs.CR 2020
Year: 2020

Smart contract (SC) is an extension of BlockChain technology. Ethereum BlockChain was the first to incorporate SC and thus started a new era of crypto-currencies and electronic transactions. Solidity helps to program the SCs. Still, soon after Solidity's emergence in 2014, Solidity-based SCs suffered many attacks that deprived the SC account holders of their precious funds. The main reason for these attacks was the presence of vulnerabilities in SC. This paper discusses SC vulnerabilities and classifies them according to the domain knowledge of the faulty operations. This classification is a source of reminding developers and software engineers that for SC's safety, each SC requires proper testing with effective tools to catch those classes' vulnerabilities.
Sparse PCA via $l_{2,p}$-Norm Regularization for Unsupervised Feature Selection
Venue: arXiv cs.LG 2020
Year: 2020

In the field of data mining, how to deal with high-dimensional data is an inevitable problem. Unsupervised feature selection has attracted more and more attention because it does not rely on labels. The performance of spectral-based unsupervised methods depends on the quality of constructed similarity matrix, which is used to depict the intrinsic structure of data. However, real-world data contain a large number of noise samples and features, making the similarity matrix constructed by original data cannot be completely reliable. Worse still, the size of similarity matrix expands rapidly as the number of samples increases, making the computational cost increase significantly. Inspired by principal component analysis, we propose a simple and efficient unsupervised feature selection method, by combining reconstruction error with $l_{2,p}$-norm regularization. The projection matrix, which is used for feature selection, is learned by minimizing the reconstruction error under the sparse constraint. Then, we present an efficient optimization algorithm to solve the proposed unsupervised model, and analyse the convergence and computational complexity of the algorithm theoretically. Finally, extensive experiments on real-world data sets demonstrate the effectiveness of our proposed method.
Multiple Structural Priors Guided Self Attention Network for Language Understanding
Venue: arXiv cs.CL 2020
Year: 2020

Self attention networks (SANs) have been widely utilized in recent NLP studies. Unlike CNNs or RNNs, standard SANs are usually position-independent, and thus are incapable of capturing the structural priors between sequences of words. Existing studies commonly apply one single mask strategy on SANs for incorporating structural priors while failing at modeling more abundant structural information of texts. In this paper, we aim at introducing multiple types of structural priors into SAN models, proposing the Multiple Structural Priors Guided Self Attention Network (MS-SAN) that transforms different structural priors into different attention heads by using a novel multi-mask based multi-head attention mechanism. In particular, we integrate two categories of structural priors, including the sequential order and the relative position of words. For the purpose of capturing the latent hierarchical structure of the texts, we extract these information not only from the word contexts but also from the dependency syntax trees. Experimental results on two tasks show that MS-SAN achieves significant improvements against other strong baselines.
Parzen Window Approximation on Riemannian Manifold
Venue: arXiv cs.LG 2020
Year: 2020

In graph motivated learning, label propagation largely depends on data affinity represented as edges between connected data points. The affinity assignment implicitly assumes even distribution of data on the manifold. This assumption may not hold and may lead to inaccurate metric assignment due to drift towards high-density regions. The drift affected heat kernel based affinity with a globally fixed Parzen window either discards genuine neighbors or forces distant data points to become a member of the neighborhood. This yields a biased affinity matrix. In this paper, the bias due to uneven data sampling on the Riemannian manifold is catered to by a variable Parzen window determined as a function of neighborhood size, ambient dimension, flatness range, etc. Additionally, affinity adjustment is used which offsets the effect of uneven sampling responsible for the bias. An affinity metric which takes into consideration the irregular sampling effect to yield accurate label propagation is proposed. Extensive experiments on synthetic and real-world data sets confirm that the proposed method increases the classification accuracy significantly and outperforms existing Parzen window estimators in graph Laplacian manifold regularization methods.
RADDLE: An Evaluation Benchmark and Analysis Platform for Robust Task-oriented Dialog Systems
Venue: arXiv cs.CL 2020
Year: 2020

For task-oriented dialog systems to be maximally useful, it must be able to process conversations in a way that is (1) generalizable with a small number of training examples for new task domains, and (2) robust to user input in various styles, modalities or domains. In pursuit of these goals, we introduce the RADDLE benchmark, a collection of corpora and tools for evaluating the performance of models across a diverse set of domains. By including tasks with limited training data, RADDLE is designed to favor and encourage models with a strong generalization ability. RADDLE also includes a diagnostic checklist that facilitates detailed robustness analysis in aspects such as language variations, speech errors, unseen entities, and out-of-domain utterances. We evaluate recent state-of-the-art systems based on pre-training and fine-tuning, and find that grounded pre-training on heterogeneous dialog corpora performs better than training a separate model per domain. Overall, existing models are less than satisfactory in robustness evaluation, which suggests opportunities for future improvement.
Fast Incremental Expectation Maximization for finite-sum optimization: nonasymptotic convergence
Venue: arXiv cs.LG 2020
Year: 2020

Fast Incremental Expectation Maximization (FIEM) is a version of the EM framework for large datasets. In this paper, we first recast FIEM and other incremental EM type algorithms in the {\em Stochastic Approximation within EM} framework. Then, we provide nonasymptotic bounds for the convergence in expectation as a function of the number of examples $n$ and of the maximal number of iterations $\kmax$. We propose two strategies for achieving an $ε$-approximate stationary point, respectively with $\kmax = O(n^{2/3}/ε)$ and $\kmax = O(\sqrt{n}/ε^{3/2})$, both strategies relying on a random termination rule before $\kmax$ and on a constant step size in the Stochastic Approximation step. Our bounds provide some improvements on the literature. First, they allow $\kmax$ to scale as $\sqrt{n}$ which is better than $n^{2/3}$ which was the best rate obtained so far; it is at the cost of a larger dependence upon the tolerance $ε$, thus making this control relevant for small to medium accuracy with respect to the number of examples $n$. Second, for the $n^{2/3}$-rate, the numerical illustrations show that thanks to an optimized choice of the step size and of the bounds in terms of quantities characterizing the optimization problem at hand, our results desig a less conservative choice of the step size and provide a better control of the convergence in expectation.
Deep Hashing for Secure Multimodal Biometrics
Venue: arXiv cs.CV 2020
Year: 2020

When compared to unimodal systems, multimodal biometric systems have several advantages, including lower error rate, higher accuracy, and larger population coverage. However, multimodal systems have an increased demand for integrity and privacy because they must store multiple biometric traits associated with each user. In this paper, we present a deep learning framework for feature-level fusion that generates a secure multimodal template from each user's face and iris biometrics. We integrate a deep hashing (binarization) technique into the fusion architecture to generate a robust binary multimodal shared latent representation. Further, we employ a hybrid secure architecture by combining cancelable biometrics with secure sketch techniques and integrate it with a deep hashing framework, which makes it computationally prohibitive to forge a combination of multiple biometrics that pass the authentication. The efficacy of the proposed approach is shown using a multimodal database of face and iris and it is observed that the matching performance is improved due to the fusion of multiple biometrics. Furthermore, the proposed approach also provides cancelability and unlinkability of the templates along with improved privacy of the biometric data. Additionally, we also test the proposed hashing function for an image retrieval application using a benchmark dataset. The main goal of this paper is to develop a method for integrating multimodal fusion, deep hashing, and biometric security, with an emphasis on structural data from modalities like face and iris. The proposed approach is in no way a general biometric security framework that can be applied to all biometric modalities, as further research is needed to extend the proposed framework to other unconstrained biometric modalities.
Generating Adversarial Examples in Chinese Texts Using Sentence-Pieces
Venue: arXiv cs.CL 2020
Year: 2020

Adversarial attacks in texts are mostly substitution-based methods that replace words or characters in the original texts to achieve success attacks. Recent methods use pre-trained language models as the substitutes generator. While in Chinese, such methods are not applicable since words in Chinese require segmentations first. In this paper, we propose a pre-train language model as the substitutes generator using sentence-pieces to craft adversarial examples in Chinese. The substitutions in the generated adversarial examples are not characters or words but \textit{'pieces'}, which are more natural to Chinese readers. Experiments results show that the generated adversarial samples can mislead strong target models and remain fluent and semantically preserved.
Hybrid Interest Modeling for Long-tailed Users
Venue: arXiv cs.IR 2020
Year: 2020

User behavior modeling is a key technique for recommender systems. However, most methods focus on head users with large-scale interactions and hence suffer from data sparsity issues. Several solutions integrate side information such as demographic features and product reviews, another is to transfer knowledge from other rich data sources. We argue that current methods are limited by the strict privacy policy and have low scalability in real-world applications and few works consider the behavioral characteristics behind long-tailed users. In this work, we propose the Hybrid Interest Modeling (HIM) network to hybrid both personalized interest and semi-personalized interest in learning long-tailed users' preferences in the recommendation. To achieve this, we first design the User Behavior Pyramid (UBP) module to capture the fine-grained personalized interest of high confidence from sparse even noisy positive feedbacks. Moreover, the individual interaction is too sparse and not enough for modeling user interest adequately, we design the User Behavior Clustering (UBC) module to learn latent user interest groups with self-supervised learning mechanism novelly, which capture coarse-grained semi-personalized interest from group-item interaction data. Extensive experiments on both public and industrial datasets verify the superiority of HIM compared with the state-of-the-art baselines.
ALVIO: Adaptive Line and Point Feature-based Visual Inertial Odometry for Robust Localization in Indoor Environments
Venue: arXiv cs.RO 2020
Year: 2020

The amount of texture can be rich or deficient depending on the objects and the structures of the building. The conventional mono visual-initial navigation system (VINS)-based localization techniques perform well in environments where stable features are guaranteed. However, their performance is not assured in a changing indoor environment. As a solution to this, we propose Adaptive Line and point feature-based Visual Inertial Odometry (ALVIO) in this paper. ALVIO actively exploits the geometrical information of lines that exist in abundance in an indoor space. By using a strong line tracker and adaptive selection of feature-based tightly coupled optimization, it is possible to perform robust localization in a variable texture environment. The structural characteristics of ALVIO are as follows: First, the proposed optical flow-based line tracker performs robust line feature tracking and management. By using epipolar geometry and trigonometry, accurate 3D lines are recovered. These 3D lines are used to calculate the line re-projection error. Finally, with the sensitivity-analysis-based adaptive feature selection in the optimization process, we can estimate the pose robustly in various indoor environments. We validate the performance of our system on public datasets and compare it against other state-of the-art algorithms (S-MSKCF, VINS-Mono). In the proposed algorithm based on point and line feature selection, translation RMSE increased by 16.06% compared to VINS-Mono, while total optimization time decreased by up to 49.31%. Through this, we proved that it is a useful algorithm as a real-time pose estimation algorithm.
Language Identification of Devanagari Poems
Venue: arXiv cs.CL 2020
Year: 2020

Language Identification is a very important part of several text processing pipelines. Extensive research has been done in this field. This paper proposes a procedure for automatic language identification of poems for poem analysis task, consisting of 10 Devanagari based languages of India i.e. Angika, Awadhi, Braj, Bhojpuri, Chhattisgarhi, Garhwali, Haryanvi, Hindi, Magahi, and Maithili. We collated corpora of poems of varying length and studied the similarity of poems among the 10 languages at the lexical level. Finally, various language identification systems based on supervised machine learning and deep learning techniques are applied and evaluated.
Improved Approximation Algorithms for Weighted Edge Coloring of Graphs
Venue: arXiv cs.DS 2020
Year: 2020

We study weighted edge coloring of graphs, where we are given an undirected edge-weighted general multi-graph $G := (V, E)$ with weights $w : E \rightarrow [0, 1]$. The goal is to find a proper weighted coloring of the edges with as few colors as possible. An edge coloring is called a proper weighted coloring if the sum of the weights of the edges incident to a vertex of any color is at most one. In the online setting, the edges are revealed one by one and have to be colored irrevocably as soon as they are revealed. We show that $3.39m+o(m)$ colors are enough when the maximum number of neighbors of a vertex over all the vertices is $o(m)$ and where $m$ is the maximum over all vertices of the minimum number of unit-sized bins needed to pack the weights of the incident edges to that vertex. We also prove the tightness of our analysis. This improves upon the previous best upper bound of $5m$ by Correa and Goemans [STOC 2004]. For the offline case, we show that for a simple graph with edge disjoint cycles, $m+1$ colors are sufficient and for a multi-graph tree, we show that $1.693m+12$ colors are sufficient.
Enhancing Pre-trained Language Model with Lexical Simplification
Venue: arXiv cs.CL 2020
Year: 2020

For both human readers and pre-trained language models (PrLMs), lexical diversity may lead to confusion and inaccuracy when understanding the underlying semantic meanings of given sentences. By substituting complex words with simple alternatives, lexical simplification (LS) is a recognized method to reduce such lexical diversity, and therefore to improve the understandability of sentences. In this paper, we leverage LS and propose a novel approach which can effectively improve the performance of PrLMs in text classification. A rule-based simplification process is applied to a given sentence. PrLMs are encouraged to predict the real label of the given sentence with auxiliary inputs from the simplified version. Using strong PrLMs (BERT and ELECTRA) as baselines, our approach can still further improve the performance in various text classification tasks.
RTS3D: Real-time Stereo 3D Detection from 4D Feature-Consistency Embedding Space for Autonomous Driving
Venue: arXiv cs.CV 2020
Year: 2020

Although the recent image-based 3D object detection methods using Pseudo-LiDAR representation have shown great capabilities, a notable gap in efficiency and accuracy still exist compared with LiDAR-based methods. Besides, over-reliance on the stand-alone depth estimator, requiring a large number of pixel-wise annotations in the training stage and more computation in the inferencing stage, limits the scaling application in the real world. In this paper, we propose an efficient and accurate 3D object detection method from stereo images, named RTS3D. Different from the 3D occupancy space in the Pseudo-LiDAR similar methods, we design a novel 4D feature-consistent embedding (FCE) space as the intermediate representation of the 3D scene without depth supervision. The FCE space encodes the object's structural and semantic information by exploring the multi-scale feature consistency warped from stereo pair. Furthermore, a semantic-guided RBF (Radial Basis Function) and a structure-aware attention module are devised to reduce the influence of FCE space noise without instance mask supervision. Experiments on the KITTI benchmark show that RTS3D is the first true real-time system (FPS$>$24) for stereo image 3D detection meanwhile achieves $10\%$ improvement in average precision comparing with the previous state-of-the-art method. The code will be available at https://github.com/Banconxuan/RTS3D
New Constructions of Optimal Locally Repairable Codes with Super-Linear Length
Venue: arXiv cs.IT 2020
Year: 2020

As an important coding scheme in modern distributed storage systems, locally repairable codes (LRCs) have attracted a lot of attentions from perspectives of both practical applications and theoretical research. As a major topic in the research of LRCs, bounds and constructions of the corresponding optimal codes are of particular concerns. In this work, codes with $(r,δ)$-locality which have optimal minimal distance w.r.t. the bound given by Prakash et al. \cite{Prakash2012Optimal} are considered. Through parity check matrix approach, constructions of both optimal $(r,δ)$-LRCs with all symbol locality ($(r,δ)_a$-LRCs) and optimal $(r,δ)$-LRCs with information locality ($(r,δ)_i$-LRCs) are provided. As a generalization of a work of Xing and Yuan \cite{XY19}, these constructions are built on a connection between sparse hypergraphs and optimal $(r,δ)$-LRCs. With the help of constructions of large sparse hypergraphs, the length of codes constructed can be super-linear in the alphabet size. This improves upon previous constructions when the minimal distance of the code is at least $3δ+1$. As two applications, optimal H-LRCs with super-linear length and GSD codes with unbounded length are also constructed.