OpenRaiser/Intern-Atlas huggingface dataset:
Characterization of Android malware based on subgraph isomorphism
Venue: arXiv cs.CR 2021
Year: 2021

The Android operating system is the most spread mobile platform in the world. Therefor attackers are producing an incredible number of malware applications for Android. Our aim is to detect Android's malware in order to protect the user. To do so really good results are obtained by dynamic analysis of software, but it requires complex environments. In order to achieve the same level of precision we analyze the machine code and investigate the frequencies of ngrams of opcodes in order to detect singular code blocks. This allow us to construct a database of infected code blocks. Then, because attacker may modify and organized differently the infected injected code in their new malware, we perform not only a semantic comparison of the tested software with the database of infected code blocks but also a structured comparison. To do such comparison we compute subgraph isomorphism. It allows us to characterize precisely if the tested software is a malware and if so in witch family it belongs. Our method is tested both on a laboratory database and a set of real data. It achieves an almost perfect detection rate.
msolve: A Library for Solving Polynomial Systems
Venue: arXiv cs.SC 2021
Year: 2021

We present a new open source C library \texttt{msolve} dedicated to solving multivariate polynomial systems of dimension zero through computer algebra methods. The core algorithmic framework of \texttt{msolve} relies on Gr\''obner bases and linear algebra based algorithms for polynomial system solving. It relies on Gr\''obner basis computation w.r.t.\ the degree reverse lexicographical order, Gr\''obner conversion to a lexicographical Gr\''obner basis and real solving of univariate polynomials. We explain in detail how these three main steps of the solving process are implemented, how we exploit \texttt{AVX2} instruction processors and the more general implementation ideas we put into practice to better exploit the computational capabilities of this algorithmic framework. We compare the practical performances of \texttt{msolve} with leading computer algebra systems such as \textsc{Magma}, \textsc{Maple}, \textsc{Singular} on a wide range of systems with finitely many complex solutions, showing that \texttt{msolve} can tackle systems which were out of reach by the computer algebra software state-of-the-art.
WNARS: WFST based Non-autoregressive Streaming End-to-End Speech Recognition
Venue: arXiv cs.SD 2021
Year: 2021

Recently, attention-based encoder-decoder (AED) end-to-end (E2E) models have drawn more and more attention in the field of automatic speech recognition (ASR). AED models, however, still have drawbacks when deploying in commercial applications. Autoregressive beam search decoding makes it inefficient for high-concurrency applications. It is also inconvenient to integrate external word-level language models. The most important thing is that AED models are difficult for streaming recognition due to global attention mechanism. In this paper, we propose a novel framework, namely WNARS, using hybrid CTC-attention AED models and weighted finite-state transducers (WFST) to solve these problems together. We switch from autoregressive beam search to CTC branch decoding, which performs first-pass decoding with WFST in chunk-wise streaming way. The decoder branch then performs second-pass rescoring on the generated hypotheses non-autoregressively. On the AISHELL-1 task, our WNARS achieves a character error rate of 5.22% with 640ms latency, to the best of our knowledge, which is the state-of-the-art performance for online ASR. Further experiments on our 10,000-hour Mandarin task show the proposed method achieves more than 20% improvements with 50% latency compared to a strong TDNN-BLSTM lattice-free MMI baseline.
The educational process organization in the distance learning model: problems and features
Venue: arXiv cs.SI 2021
Year: 2021

This article discusses the current issues of providing the educational process in a remote format on the example of the Tomsk State University of Control Systems and Radioelectronics (TUSUR). On the basis of the conducted re-search, the problems and features of distance learning are identified and analyzed. It is concluded that it is neces-sary to modernize education on the basis of constant monitoring of the effectiveness of providing distance learn-ing, taking into account the opinions of all participants in the educational process.
Uncertainty-aware Remaining Useful Life predictor
Venue: arXiv cs.LG 2021
Year: 2021

Remaining Useful Life (RUL) estimation is the problem of inferring how long a certain industrial asset can be expected to operate within its defined specifications. Deploying successful RUL prediction methods in real-life applications is a prerequisite for the design of intelligent maintenance strategies with the potential of drastically reducing maintenance costs and machine downtimes. In light of their superior performance in a wide range of engineering fields, Machine Learning (ML) algorithms are natural candidates to tackle the challenges involved in the design of intelligent maintenance systems. In particular, given the potentially catastrophic consequences or substantial costs associated with maintenance decisions that are either too late or too early, it is desirable that ML algorithms provide uncertainty estimates alongside their predictions. However, standard data-driven methods used for uncertainty estimation in RUL problems do not scale well to large datasets or are not sufficiently expressive to model the high-dimensional mapping from raw sensor data to RUL estimates. In this work, we consider Deep Gaussian Processes (DGPs) as possible solutions to the aforementioned limitations. We perform a thorough evaluation and comparison of several variants of DGPs applied to RUL predictions. The performance of the algorithms is evaluated on the N-CMAPSS (New Commercial Modular Aero-Propulsion System Simulation) dataset from NASA for aircraft engines. The results show that the proposed methods are able to provide very accurate RUL predictions along with sensible uncertainty estimates, providing more reliable solutions for (safety-critical) real-life industrial applications.
Extending Hierarchical Delta Debugging with Hoisting
Venue: arXiv cs.SE 2021
Year: 2021

Minimizing failing test cases is an important pre-processing step on the path of debugging. If much of a test case that triggered a bug does not contribute to the actual failure, then the time required to fix the bug can increase considerably. However, test case reduction itself can be a time consuming task, especially if done manually. Therefore, automated minimization techniques have been proposed, the minimizing Delta Debugging (DDMIN) and the Hierarchical Delta Debugging (HDD) algorithms being the most well known. DDMIN does not need any information about the structure of the test case, thus it works for any kind of input. If the structure is known, however, it can be utilized to create smaller test cases faster. This is exemplified by HDD, which works on tree-structured inputs, pruning subtrees at each level of the tree with the help of DDMIN. In this paper, we propose to extend HDD with a reduction method that does not prune subtrees, but replaces them with compatible subtrees further down the hierarchy, called hoisting. We have evaluated various combinations of pruning and hoisting on multiple test suites and found that hoisting can help to further reduce the size of test cases by as much as 80% compared to the baseline HDD. We have also compared our results to other state-of-the-art test case reduction algorithms and found that HDD extended with hoisting can produce smaller output in most of the cases.
Media Cloud: Massive Open Source Collection of Global News on the Open Web
Venue: arXiv cs.SI 2021
Year: 2021

We present the first full description of Media Cloud, an open source platform based on crawling hyperlink structure in operation for over 10 years, that for many uses will be the best way to collect data for studying the media ecosystem on the open web. We document the key choices behind what data Media Cloud collects and stores, how it processes and organizes these data, and its open API access as well as user-facing tools. We also highlight the strengths and limitations of the Media Cloud collection strategy compared to relevant alternatives. We give an overview two sample datasets generated using Media Cloud and discuss how researchers can use the platform to create their own datasets.
A Decentralized and Autonomous Model to Administer University Examinations
Venue: arXiv cs.CY 2021
Year: 2021

Administering standardized examinations is a challenging task, especially for those universities for which colleges affiliated to it are geographically distributed over a wide area. Some of the challenges include maintaining integrity and confidentiality of examination records, preventing mal-practices, issuing unique identification numbers to a large student population and managing assets required for the smooth conduct of examinations. These challenges aggravate when colleges affiliated to universities demand academic and administrative autonomy by demonstrating best practices consistently over a long period. In this chapter, we describe a model for decentralized and autonomous examination system to provide the necessary administrative support. The model is based on two emerging technologies of Blockchain Technology and Internet of Things (IoT). We adopt a software architecture approach to describe the model. The prescriptive architecture consists of {\em architectural mappings} which map functional and non-functional requirements to architectural elements of blockchain technology and IoT. In architectural mappings, first, we identify common use-cases in administering standardized examinations. Then we map these use-cases to the core elements of blockchain, i.e. distributed ledgers, cryptography, consensus protocols and smart-contracts and IoT. Such kind of prescriptive architecture guide downstream software engineering processes of implementation and testing
Social Big Data: An Overview and Applications
Venue: arXiv cs.SI 2021
Year: 2021

The emergence of online social media services has made a qualitative leap and brought profound changes to various aspects of human, cultural, intellectual, and social life. These significant Big data tributaries have further transformed the businesses processes by establishing convergent and transparent dialogues between businesses and their customers. Therefore, analysing the flow of social data content is necessary in order to enhance business practices, to augment brand awareness, to develop insights on target markets, to detect and identify positive and negative customer sentiments, etc., thereby achieving the hoped-for added value. This chapter presents an overview of Social Big Data term and definition. This chapter also lays the foundation for several applications and analytics that are broadly discussed in this book.
BR-NS: an Archive-less Approach to Novelty Search
Venue: arXiv cs.AI 2021
Year: 2021

As open-ended learning based on divergent search algorithms such as Novelty Search (NS) draws more and more attention from the research community, it is natural to expect that its application to increasingly complex real-world problems will require the exploration to operate in higher dimensional Behavior Spaces which will not necessarily be Euclidean. Novelty Search traditionally relies on k-nearest neighbours search and an archive of previously visited behavior descriptors which are assumed to live in a Euclidean space. This is problematic because of a number of issues. On one hand, Euclidean distance and Nearest-neighbour search are known to behave differently and become less meaningful in high dimensional spaces. On the other hand, the archive has to be bounded since, memory considerations aside, the computational complexity of finding nearest neighbours in that archive grows linearithmically with its size. A sub-optimal bound can result in "cycling" in the behavior space, which inhibits the progress of the exploration. Furthermore, the performance of NS depends on a number of algorithmic choices and hyperparameters, such as the strategies to add or remove elements to the archive and the number of neighbours to use in k-nn search. In this paper, we discuss an alternative approach to novelty estimation, dubbed Behavior Recognition based Novelty Search (BR-NS), which does not require an archive, makes no assumption on the metrics that can be defined in the behavior space and does not rely on nearest neighbours search. We conduct experiments to gain insight into its feasibility and dynamics as well as potential advantages over archive-based NS in terms of time complexity.
Handwriting Transformers
Venue: arXiv cs.CV 2021
Year: 2021

We propose a novel transformer-based styled handwritten text image generation approach, HWT, that strives to learn both style-content entanglement as well as global and local writing style patterns. The proposed HWT captures the long and short range relationships within the style examples through a self-attention mechanism, thereby encoding both global and local style patterns. Further, the proposed transformer-based HWT comprises an encoder-decoder attention that enables style-content entanglement by gathering the style representation of each query character. To the best of our knowledge, we are the first to introduce a transformer-based generative network for styled handwritten text generation. Our proposed HWT generates realistic styled handwritten text images and significantly outperforms the state-of-the-art demonstrated through extensive qualitative, quantitative and human-based evaluations. The proposed HWT can handle arbitrary length of text and any desired writing style in a few-shot setting. Further, our HWT generalizes well to the challenging scenario where both words and writing style are unseen during training, generating realistic styled handwritten text images.
Quantum-Assisted Feature Selection for Vehicle Price Prediction Modeling
Venue: arXiv cs.LG 2021
Year: 2021

Within machine learning model evaluation regimes, feature selection is a technique to reduce model complexity and improve model performance in regards to generalization, model fit, and accuracy of prediction. However, the search over the space of features to find the subset of $k$ optimal features is a known NP-Hard problem. In this work, we study metrics for encoding the combinatorial search as a binary quadratic model, such as Generalized Mean Information Coefficient and Pearson Correlation Coefficient in application to the underlying regression problem of price prediction. We investigate trade-offs in the form of run-times and model performance, of leveraging quantum-assisted vs. classical subroutines for the combinatorial search, using minimum redundancy maximal relevancy as the heuristic for our approach. We achieve accuracy scores of 0.9 (in the range of [0,1]) for finding optimal subsets on synthetic data using a new metric that we define. We test and cross-validate predictive models on a real-world problem of price prediction, and show a performance improvement of mean absolute error scores for our quantum-assisted method $(1471.02 \pm{135.6})$, vs. similar methodologies such as recursive feature elimination $(1678.3 \pm{143.7})$. Our findings show that by leveraging quantum-assisted routines we find solutions that increase the quality of predictive model output while reducing the input dimensionality to the learning algorithm on synthetic and real-world data.
Manchester Encoding for Non-coherent Detection of Ambient Backscatter in Time-Selective Fading
Venue: arXiv cs.IT 2021
Year: 2021

Simplifying the detection procedure and improving the bit error rate (BER) performance of a non-coherent receiver in ambient backscatter is vital for enhancing its ability to function without the channel state information (CSI). In this work, we analyze the BER performance of Manchester encoding which is implemented at the transmitter for data transmission, and demonstrate that the optimal decision rule is independent of the system parameters. Further, through extensive numerical results, it is shown that the ambient backscatter system can achieve a signal-to-noise ratio (SNR) gain with Manchester encoding compared to the commonly used uncoded direct on-off keying (OOK) modulation, when used in conjunction with a multi-antenna receiver employing the direct-link cancellation.
Increasing the Speed and Accuracy of Data LabelingThrough an AI Assisted Interface
Venue: arXiv cs.HC 2021
Year: 2021

Labeling data is an important step in the supervised machine learning lifecycle. It is a laborious human activity comprised of repeated decision making: the human labeler decides which of several potential labels to apply to each example. Prior work has shown that providing AI assistance can improve the accuracy of binary decision tasks. However, the role of AI assistance in more complex data-labeling scenarios with a larger set of labels has not yet been explored. We designed an AI labeling assistant that uses a semi-supervised learning algorithm to predict the most probable labels for each example. We leverage these predictions to provide assistance in two ways: (i) providing a label recommendation and (ii) reducing the labeler's decision space by focusing their attention on only the most probable labels. We conducted a user study (n=54) to evaluate an AI-assisted interface for data labeling in this context. Our results highlight that the AI assistance improves both labeler accuracy and speed, especially when the labeler finds the correct label in the reduced label space. We discuss findings related to the presentation of AI assistance and design implications for intelligent labeling interfaces.
FL-AGCNS: Federated Learning Framework for Automatic Graph Convolutional Network Search
Venue: arXiv cs.LG 2021
Year: 2021

Recently, some Neural Architecture Search (NAS) techniques are proposed for the automatic design of Graph Convolutional Network (GCN) architectures. They bring great convenience to the use of GCN, but could hardly apply to the Federated Learning (FL) scenarios with distributed and private datasets, which limit their applications. Moreover, they need to train many candidate GCN models from scratch, which is inefficient for FL. To address these challenges, we propose FL-AGCNS, an efficient GCN NAS algorithm suitable for FL scenarios. FL-AGCNS designs a federated evolutionary optimization strategy to enable distributed agents to cooperatively design powerful GCN models while keeping personal information on local devices. Besides, it applies the GCN SuperNet and a weight sharing strategy to speed up the evaluation of GCN models. Experimental results show that FL-AGCNS can find better GCN models in short time under the FL framework, surpassing the state-of-the-arts NAS methods and GCN models.
Shape, Size, and Fabrication Effects in 3D Printed Granular Jamming Grippers
Venue: arXiv cs.RO 2021
Year: 2021

Granular jamming is a popular soft actuation mechanism that provides high stiffness variability with minimum volume variation. Jamming is particularly interesting from a design perspective, as a myriad of design parameters can potentially be exploited to induce a diverse variety of useful behaviours. To date, grain shape has been largely ignored. Here, we focus on the use of 3D printing to expose design variables related to grain shape and size. Grains are represented by parameterised superquadrics (superellipsoids); four diverse shapes are investigated along with three size variations. Grains are 3D printed at high resolution and performance is assessed in experimental pull-off testing on a variety of benchmark test objects. We show that grain shape and size are key determinants in granular gripping performance. Moreover, there is no universally-optimal grain shape for gripping. Optical imaging assesses the accuracy of printed shapes compared to their ideal models. Results suggest that optimisation of grain shape is a key enabler for high-performance, bespoke, actuation behaviour and can be exploited to expand the range and performance of granular grippers across a range of diverse usage scenarios.
Piracy-Resistant DNN Watermarking by Block-Wise Image Transformation with Secret Key
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we propose a novel DNN watermarking method that utilizes a learnable image transformation method with a secret key. The proposed method embeds a watermark pattern in a model by using learnable transformed images and allows us to remotely verify the ownership of the model. As a result, it is piracy-resistant, so the original watermark cannot be overwritten by a pirated watermark, and adding a new watermark decreases the model accuracy unlike most of the existing DNN watermarking methods. In addition, it does not require a special pre-defined training set or trigger set. We empirically evaluated the proposed method on the CIFAR-10 dataset. The results show that it was resilient against fine-tuning and pruning attacks while maintaining a high watermark-detection accuracy.
Creating Robust Deep Neural Networks With Coded Distributed Computing for IoT Systems
Venue: arXiv cs.DC 2021
Year: 2021

The increasing interest in serverless computation and ubiquitous wireless networks has led to numerous connected devices in our surroundings. Among such devices, IoT devices have access to an abundance of raw data, but their inadequate resources in computing limit their capabilities. Specifically, with the emergence of deep neural networks (DNNs), not only is the demand for the computing power of IoT devices increasing but also privacy concerns are pushing computations towards the edge. To overcome inadequate resources, several studies have proposed the distribution of work among IoT devices. These promising methods harvest the aggregated computing power of the idle IoT devices in an environment. However, since such a distributed system strongly relies on each device, unstable latencies, and intermittent failures, the common characteristics of IoT devices and wireless networks, cause high recovery overheads. To reduce this overhead, we propose a novel robustness method with a close-to-zero recovery latency for DNN computations. Our solution never loses a request or spends time recovering from a failure. To do so, first, we analyze the underlying matrix-matrix computations affected by distribution. Then, we introduce a new coded distributed computing (CDC) method that has a constant cost with the increasing number of devices, unlike the linear cost of modular redundancies. Moreover, our method is applied in the library level, without requiring extensive changes to the program, while still ensuring a balanced work assignment during distribution. To illustrate our method, we perform experiments with distributed systems comprising up to 12 Raspberry Pis.
Unsupervised Class-Incremental Learning Through Confusion
Venue: arXiv cs.LG 2021
Year: 2021

While many works on Continual Learning have shown promising results for mitigating catastrophic forgetting, they have relied on supervised training. To successfully learn in a label-agnostic incremental setting, a model must distinguish between learned and novel classes to properly include samples for training. We introduce a novelty detection method that leverages network confusion caused by training incoming data as a new class. We found that incorporating a class-imbalance during this detection method substantially enhances performance. The effectiveness of our approach is demonstrated across a set of image classification benchmarks: MNIST, SVHN, CIFAR-10, CIFAR-100, and CRIB.