OpenRaiser/Intern-Atlas huggingface dataset:
EC P-256: Successful Simple Power Analysis
Venue: arXiv cs.CR 2021
Year: 2021

In this work we discuss the resistance of atomic pattern algorithms for elliptic curve point multiplication against simple side channel analysis attacks using our own implementation as an example. The idea of the atomicity principle is to make kP implementations resistant against simple side channel analysis attacks. One of the assumptions, on which the atomicity principle is based, is the indistinguishability of register operations, i.e. two write-to-register operations cannot be distinguished if their old and new data values are the same. But before the data can be stored to a register/block, this register/block has to be addressed for storing the data. Different registers/blocks have different addresses. In praxis, this different and key dependent addressing can be used to reveal the key, even by running simple SCA attacks. The key dependent addressing of registers/blocks allows to reveal the key and is an inherent feature of the binary kP algorithms. This means that the assumption, that addressing of different registers/blocks is an indistinguishable operation, may no longer be applied when realizing kP implementations, at least not for hardware implementations.
First Step Towards EXPLAINable DGA Multiclass Classification
Venue: arXiv cs.CR 2021
Year: 2021

Numerous malware families rely on domain generation algorithms (DGAs) to establish a connection to their command and control (C2) server. Counteracting DGAs, several machine learning classifiers have been proposed enabling the identification of the DGA that generated a specific domain name and thus triggering targeted remediation measures. However, the proposed state-of-the-art classifiers are based on deep learning models. The black box nature of these makes it difficult to evaluate their reasoning. The resulting lack of confidence makes the utilization of such models impracticable. In this paper, we propose EXPLAIN, a feature-based and contextless DGA multiclass classifier. We comparatively evaluate several combinations of feature sets and hyperparameters for our approach against several state-of-the-art classifiers in a unified setting on the same real-world data. Our classifier achieves competitive results, is real-time capable, and its predictions are easier to trace back to features than the predictions made by the DGA multiclass classifiers proposed in related work.
Gradient-Based Interpretability Methods and Binarized Neural Networks
Venue: arXiv cs.CV 2021
Year: 2021

Binarized Neural Networks (BNNs) have the potential to revolutionize the way that deep learning is carried out in edge computing platforms. However, the effectiveness of interpretability methods on these networks has not been assessed. In this paper, we compare the performance of several widely used saliency map-based interpretabilty techniques (Gradient, SmoothGrad and GradCAM), when applied to Binarized or Full Precision Neural Networks (FPNNs). We found that the basic Gradient method produces very similar-looking maps for both types of network. However, SmoothGrad produces significantly noisier maps for BNNs. GradCAM also produces saliency maps which differ between network types, with some of the BNNs having seemingly nonsensical explanations. We comment on possible reasons for these differences in explanations and present it as an example of why interpretability techniques should be tested on a wider range of network types.
Dealing with training and test segmentation mismatch: FBK@IWSLT2021
Venue: arXiv cs.CL 2021
Year: 2021

This paper describes FBK's system submission to the IWSLT 2021 Offline Speech Translation task. We participated with a direct model, which is a Transformer-based architecture trained to translate English speech audio data into German texts. The training pipeline is characterized by knowledge distillation and a two-step fine-tuning procedure. Both knowledge distillation and the first fine-tuning step are carried out on manually segmented real and synthetic data, the latter being generated with an MT system trained on the available corpora. Differently, the second fine-tuning step is carried out on a random segmentation of the MuST-C v2 En-De dataset. Its main goal is to reduce the performance drops occurring when a speech translation model trained on manually segmented data (i.e. an ideal, sentence-like segmentation) is evaluated on automatically segmented audio (i.e. actual, more realistic testing conditions). For the same purpose, a custom hybrid segmentation procedure that accounts for both audio content (pauses) and for the length of the produced segments is applied to the test data before passing them to the system. At inference time, we compared this procedure with a baseline segmentation method based on Voice Activity Detection (VAD). Our results indicate the effectiveness of the proposed hybrid approach, shown by a reduction of the gap with manual segmentation from 8.3 to 1.4 BLEU points.
Handwritten Digit Recognition using Machine and Deep Learning Algorithms
Venue: arXiv cs.CV 2021
Year: 2021

The reliance of humans over machines has never been so high such that from object classification in photographs to adding sound to silent movies everything can be performed with the help of deep learning and machine learning algorithms. Likewise, Handwritten text recognition is one of the significant areas of research and development with a streaming number of possibilities that could be attained. Handwriting recognition (HWR), also known as Handwritten Text Recognition (HTR), is the ability of a computer to receive and interpret intelligible handwritten input from sources such as paper documents, photographs, touch-screens and other devices [1]. Apparently, in this paper, we have performed handwritten digit recognition with the help of MNIST datasets using Support Vector Machines (SVM), Multi-Layer Perceptron (MLP) and Convolution Neural Network (CNN) models. Our main objective is to compare the accuracy of the models stated above along with their execution time to get the best possible model for digit recognition.
Leveraging semantically similar queries for ranking via combining representations
Venue: arXiv cs.LG 2021
Year: 2021

In modern ranking problems, different and disparate representations of the items to be ranked are often available. It is sensible, then, to try to combine these representations to improve ranking. Indeed, learning to rank via combining representations is both principled and practical for learning a ranking function for a particular query. In extremely data-scarce settings, however, the amount of labeled data available for a particular query can lead to a highly variable and ineffective ranking function. One way to mitigate the effect of the small amount of data is to leverage information from semantically similar queries. Indeed, as we demonstrate in simulation settings and real data examples, when semantically similar queries are available it is possible to gainfully use them when ranking with respect to a particular query. We describe and explore this phenomenon in the context of the bias-variance trade off and apply it to the data-scarce settings of a Bing navigational graph and the Drosophila larva connectome.
The Stereotyping Problem in Collaboratively Filtered Recommender Systems
Venue: arXiv cs.IR 2021
Year: 2021

Recommender systems play a crucial role in mediating our access to online information. We show that such algorithms induce a particular kind of stereotyping: if preferences for a set of items are anti-correlated in the general user population, then those items may not be recommended together to a user, regardless of that user's preferences and rating history. First, we introduce a notion of joint accessibility, which measures the extent to which a set of items can jointly be accessed by users. We then study joint accessibility under the standard factorization-based collaborative filtering framework, and provide theoretical necessary and sufficient conditions when joint accessibility is violated. Moreover, we show that these conditions can easily be violated when the users are represented by a single feature vector. To improve joint accessibility, we further propose an alternative modelling fix, which is designed to capture the diverse multiple interests of each user using a multi-vector representation. We conduct extensive experiments on real and simulated datasets, demonstrating the stereotyping problem with standard single-vector matrix factorization models.
Gender differences in scientific careers: A large-scale bibliometric analysis
Venue: arXiv cs.DL 2021
Year: 2021

We present a large-scale bibliometric analysis of gender differences in scientific careers, covering all scientific disciplines and a large number of countries worldwide. We take a longitudinal perspective in which we trace the publication careers of almost six million male and female researchers in the period 1996-2018. Our analysis reveals an increasing trend in the percentage of women starting a career as publishing researcher, from 33% in 2000 to about 40% in recent years. Looking at cohorts of male and female researchers that started their publication career in the same year, we find that women seem to be somewhat less likely to continue their career as publishing researcher than men, but the difference is small. We also observe that men produce on average between 15% and 20% more publications than women. Moreover, in biomedical disciplines, men are about 25% more likely than women to be last author of a publication, suggesting that men tend to have more senior roles than women. Compared with cross-sectional studies, our longitudinal analysis has the advantage of providing a more in-depth understanding of gender imbalances among authors of scientific publications.
On the Differential Cryptanalysis of SEPAR Cipher
Venue: arXiv cs.CR 2021
Year: 2021

SEPAR is a lightweight cryptographic algorithm, designed to implement on resource-constrained devices especially those employed in IoT environments. Meanwhile, the mixed structure design of cipher leads to speed improvement while guaranteeing its resistance against common cryptographic attacks, especially differential and linear attacks. In order to confirm the resistance of the cipher against differential attack, an extensive investigation was presented in our previous work. In his study, we conduct new research continuing the previously presented research. We prove that there are enough active S-boxes so as to resist cipher against differential cryptanalysis. Moreover, this can provide a tight bound of resisting cipher against this attack.
A Pure HTTP/3 Alternative to MQTT-over-QUIC in Resource-Constrained IoT
Venue: arXiv cs.NI 2021
Year: 2021

In this paper, we address the issue of scalable, interoperable, and timely dissemination of information in resource-constrained IoT. Scalability is addressed by adopting a publish-subscribe architecture. To address interoperable and timely dissemination, we propose an HTTP/3 (H3) solution that exploits the wide-ranging improvements made over H2. We evaluated our solution by comparing it to a state-of-the-art work: MQTT-over-QUIC. Because QUIC and H3 have undergone standardization in tandem, we hypothesized that H3 would take better advantage of QUIC transport than an MQTT mapping would. Performance, network overhead, and device overhead were investigated for both protocols. Our H3-based solution satisfied our timely dissemination requirement by offering a key performance savings of 1 RoundTrip Time (RTT) for publish messages to arrive at the broker. In IoT networks, with typically high RTT, this savings is significant. On the other hand, we found that MQTT-over-QUIC put marginally less strain over the network.
Comparative Error Analysis in Neural and Finite-state Models for Unsupervised Character-level Transduction
Venue: arXiv cs.CL 2021
Year: 2021

Traditionally, character-level transduction problems have been solved with finite-state models designed to encode structural and linguistic knowledge of the underlying process, whereas recent approaches rely on the power and flexibility of sequence-to-sequence models with attention. Focusing on the less explored unsupervised learning scenario, we compare the two model classes side by side and find that they tend to make different types of errors even when achieving comparable performance. We analyze the distributions of different error classes using two unsupervised tasks as testbeds: converting informally romanized text into the native script of its language (for Russian, Arabic, and Kannada) and translating between a pair of closely related languages (Serbian and Bosnian). Finally, we investigate how combining finite-state and sequence-to-sequence models at decoding time affects the output quantitatively and qualitatively.
Distilling the Knowledge from Conditional Normalizing Flows
Venue: arXiv cs.LG 2021
Year: 2021

Normalizing flows are a powerful class of generative models demonstrating strong performance in several speech and vision problems. In contrast to other generative models, normalizing flows are latent variable models with tractable likelihoods and allow for stable training. However, they have to be carefully designed to represent invertible functions with efficient Jacobian determinant calculation. In practice, these requirements lead to overparameterized and sophisticated architectures that are inferior to alternative feed-forward models in terms of inference time and memory consumption. In this work, we investigate whether one can distill flow-based models into more efficient alternatives. We provide a positive answer to this question by proposing a simple distillation approach and demonstrating its effectiveness on state-of-the-art conditional flow-based models for image super-resolution and speech synthesis.
A review of systematic selection of clustering algorithms and their evaluation
Venue: arXiv cs.LG 2021
Year: 2021

Data analysis plays an indispensable role for value creation in industry. Cluster analysis in this context is able to explore given datasets with little or no prior knowledge and to identify unknown patterns. As (big) data complexity increases in the dimensions volume, variety, and velocity, this becomes even more important. Many tools for cluster analysis have been developed from early on and the variety of different clustering algorithms is huge. As the selection of the right clustering procedure is crucial to the results of the data analysis, users are in need for support on their journey of extracting knowledge from raw data. Thus, the objective of this paper lies in the identification of a systematic selection logic for clustering algorithms and corresponding validation concepts. The goal is to enable potential users to choose an algorithm that fits best to their needs and the properties of their underlying data clustering problem. Moreover, users are supported in selecting the right validation concepts to make sense of the clustering results. Based on a comprehensive literature review, this paper provides assessment criteria for clustering method evaluation and validation concept selection. The criteria are applied to several common algorithms and the selection process of an algorithm is supported by the introduction of pseudocode-based routines that consider the underlying data structure.
The maximum discrete surface-to-volume ratio of space-filling curve partitions
Venue: arXiv cs.CE 2021
Year: 2021

Space-filling curves (SFCs) are used in high performance computing to distribute a computational domain or its mesh, respectively, amongst different compute units, i.e.~cores or nodes or accelerators. The part of the domain allocated to each compute unit is called a partition. Besides the balancing of the work, the communication cost to exchange data between units determines the quality of a chosen partition. This cost can be approximated by the surface-to-volume ratio of partitions: the volume represents the amount of local work, while the surface represents the amount of data to be transmitted. Empirical evidence suggests that space-filling curves yield advantageous surface-to-volume ratios. Formal proofs are available only for regular grids. We investigate the surface-to-volume ratio of space-filling curve partitions for adaptive grids and derive the maximum surface-to-volume ratio as a function of the number of cells in the partition. In order to prove our main theorem, we construct a new framework for the study of adaptive grids, notably introducing the concepts of a shape and of classified partitions. The new methodological framework yields insight about the SFC-induced partition character even if the grids refine rather aggressively in localised areas: it quantifies the obtained surface-to-volume ratio. This framework thus has the potential to guide the design of better load balancing algorithms on the long term.
DCoM: A Deep Column Mapper for Semantic Data Type Detection
Venue: arXiv cs.LG 2021
Year: 2021

Detection of semantic data types is a very crucial task in data science for automated data cleaning, schema matching, data discovery, semantic data type normalization and sensitive data identification. Existing methods include regular expression-based or dictionary lookup-based methods that are not robust to dirty as well unseen data and are limited to a very less number of semantic data types to predict. Existing Machine Learning methods extract large number of engineered features from data and build logistic regression, random forest or feedforward neural network for this purpose. In this paper, we introduce DCoM, a collection of multi-input NLP-based deep neural networks to detect semantic data types where instead of extracting large number of features from the data, we feed the raw values of columns (or instances) to the model as texts. We train DCoM on 686,765 data columns extracted from VizNet corpus with 78 different semantic data types. DCoM outperforms other contemporary results with a quite significant margin on the same dataset.
Towards Measuring and Quantifying the Comprehensibility of Process Models -- The Process Model Comprehension Framework
Venue: arXiv cs.SE 2021
Year: 2021

Process models constitute crucial artifacts in modern information systems and, hence, the proper comprehension of these models is of utmost importance in the utilization of such systems. Generally, process models are considered from two different perspectives: process modelers and readers. Both perspectives share similarities and differences in the comprehension of process models (e.g., diverse experiences when working with process models). The literature proposed many rules and guidelines to ensure a proper comprehension of process models for both perspectives. As a novel contribution in this context, this paper introduces the Process Model Comprehension Framework (PMCF) as a first step towards the measurement and quantification of the perspectives of process modelers and readers as well as the interaction of both regarding the comprehension of process models. Therefore, the PMCF describes an Evaluation Theory Tree based on the Communication Theory as well as the Conceptual Modeling Quality Framework and considers a total of 96 quality metrics in order to quantify process model comprehension. Furthermore, the PMCF was evaluated in a survey with 131 participants and has been implemented as well as applied successfully in a practical case study including 33 participants. To conclude, the PMCF allows for the identification of pitfalls and provides related information about how to assist process modelers as well as readers in order to foster and enable a proper comprehension of process models.
A Novel Compact Tri-Band Antenna Design for WiMAX, WLAN and Bluetooth Applications
Venue: arXiv cs.NI 2021
Year: 2021

A novel and compact tri-band planar antenna for 2.4/5.2/5.8-GHz wireless local area network (WLAN), 2.3/3.5/5.5GHz Worldwide Interoperability for Microwave Access (WiMAX) and Bluetooth applications is proposed and studied in this paper. The antenna comprises of a L-shaped element which is coupled with a ground shorted parasitic resonator to generate three resonant modes for tri-band operation. The L-shaped element which is placed on top of the substrate is fed by a 50$Ω$ microstrip feed line and is responsible for the generation of a wide band at 5.5 GHz. The parasitic resonator is placed on the other side of the substrate and is directly connected to the ground plane. The presence of the parasitic resonator gives rise to two additional resonant bands at 2.3 GHz and 3.5 GHz. Thus, together the two elements generate three resonant bands to cover WLAN, WiMAX and Bluetooth bands of operation. A thorough parametric study has been performed on the antenna and it has been found that the three bands can be tuned by varying certain dimensions of the antenna. Hence, the same design can be used for frequencies in adjacent bands as well with minor changes in its dimensions. Important antenna parameters such as return loss, radiation pattern and peak gains in the operating bands have been studied in detail to prove that the proposed design is a promising candidate for the aforementioned wireless technologies.
Long-term Cross Adversarial Training: A Robust Meta-learning Method for Few-shot Classification Tasks
Venue: arXiv cs.LG 2021
Year: 2021

Meta-learning model can quickly adapt to new tasks using few-shot labeled data. However, despite achieving good generalization on few-shot classification tasks, it is still challenging to improve the adversarial robustness of the meta-learning model in few-shot learning. Although adversarial training (AT) methods such as Adversarial Query (AQ) can improve the adversarially robust performance of meta-learning models, AT is still computationally expensive training. On the other hand, meta-learning models trained with AT will drop significant accuracy on the original clean images. This paper proposed a meta-learning method on the adversarially robust neural network called Long-term Cross Adversarial Training (LCAT). LCAT will update meta-learning model parameters cross along the natural and adversarial sample distribution direction with long-term to improve both adversarial and clean few-shot classification accuracy. Due to cross-adversarial training, LCAT only needs half of the adversarial training epoch than AQ, resulting in a low adversarial training computation. Experiment results show that LCAT achieves superior performance both on the clean and adversarial few-shot classification accuracy than SOTA adversarial training methods for meta-learning models.
Making Tezos smart contracts more reliable with Coq
Venue: arXiv cs.PL 2021
Year: 2021

Tezos is a smart-contract blockchain. Tezos smart contracts are written in a low-level stack-based language called Michelson. This article gives an overview of efforts using the Coq proof assistant to have stronger guarantees on Michelson smart contracts: the Mi-Cho-Coq framework, a Coq library defining formal semantics of Michelson, as well as an interpreter, a simple optimiser and a weakest-precondition calculus to reason about Michelson smart contracts; Albert, an intermediate language that abstracts Michelson stacks with a compiler written in Coq that targets Mi-Cho-Coq.