Miracle-dz/UniFAR-FTUs huggingface dataset:
MalPhase: Fine-Grained Malware Detection Using Network Flow Data

Economic incentives encourage malware authors to constantly develop new, increasingly complex malware to steal sensitive data or blackmail individuals and companies into paying large ransoms. In 2017, the worldwide economic impact of cyberattacks is estimated to be between 445 and 600 billion USD, or 0.8% of global GDP. Traditionally, one of the approaches used to defend against malware is network traffic analysis, which relies on network data to detect the presence of potentially malicious software. However, to keep up with increasing network speeds and amount of traffic, network analysis is generally limited to work on aggregated network data, which is traditionally challenging and yields mixed results. In this paper we present MalPhase, a system that was designed to cope with the limitations of aggregated flows. MalPhase features a multi-phase pipeline for malware detection, type and family classification. The use of an extended set of network flow features and a simultaneous multi-tier architecture facilitates a performance improvement for deep learning models, making them able to detect malicious flows (>98% F1) and categorize them to a respective malware type (>93% F1) and family (>91% F1). Furthermore, the use of robust features and denoising autoencoders allows MalPhase to perform well on samples with varying amounts of benign traffic mixed in. Finally, MalPhase detects unseen malware samples with performance comparable to that of known samples, even when interlaced with benign flows to reflect realistic network environments.
Spin Hall and inverse spin galvanic effects in graphene with strong interfacial spin-orbit coupling: A quasi-classical Green's function approach

van der Waals heterostructures assembled from atomically thin crystals are ideal model systems to study spin-orbital coupled transport because they exhibit a strong interplay between spin, lattice and valley degrees of freedom that can be manipulated by strain, electric bias and proximity effects. The recently predicted spin-helical regime in graphene on transition metal dichalcogenides, in which spin and pseudospin degrees of freedom are locked together [M. Offidani et al . Phys. Rev. Lett. 119, 196801 (2017)], suggests their potential application in spintronics. Here, by deriving an Eilenberger equation for the quasiclassical Green’s function of two-dimensional Dirac fermions in the presence of spin-orbit coupling (SOC) and scalar disorder, we obtain analytical expressions for the dc spin galvanic susceptibility and spin Hall conductivity in the spin-helical regime. Our results disclose a sign change in the spin Hall angle (SHA) when the Fermi energy relative to the Dirac point matches the Bychkov-Rashba energy scale, irrespective of the magnitude of the spin-valley interaction imprinted on the graphene layer. The behavior of the SHA is connected to a reversal of the total internal angular momentum of Bloch electrons that reflects the spin-pseudospin entanglement induced by SOC. We also show that the charge-spin conversion reaches a maximum when the Fermi level lies at the edge of the spin-minority band in agreement with previous findings. Both features are fingerprints of spin-helical Dirac fermions and suggest a direct way to estimate the strength of proximity-induced SOC from transport data. The relevance of these findings for interpreting recent spin-charge conversion measurements in nonlocal spin-valve geometry is also discussed.
Families of hybridizable interior penalty discontinuous Galerkin methods for degenerate advection-diffusion-reaction problems

We analyze families of primal high-order hybridizable discontinuous Galerkin (HDG) methods for solving degenerate (second-order) elliptic problems. One major trouble regarding this class of PDEs concerns its mathematical nature, which may be nonuniform over the domain. Due to the local degeneracy of the diffusion term, it can be purely hyperbolic in a subregion and elliptic in the rest. This problem is thus quite delicate to solve since the exact solution is discontinuous at interfaces separating both elliptic and hyperbolic parts. The proposed HDG method is developed in a unified and compact fashion. It can efficiently handle pure diffusive or advective regimes and intermediate regimes that combine the above mechanisms for a wide range of P\'eclet numbers, including the delicate situation of local evanescent diffusion. To this end, an adaptive stabilization strategy based on the addition of jump-penalty terms is then considered. A $\theta$-upwind-based scheme is favored for the hyperbolic region, and an inspired Scharfetter--Gummel-based technique is preferred for the elliptic region. The well-posedness of the HDG method is also discussed by analyzing the consistency and discrete coercivity properties. Extensive numerical experiments are finally considered to verify the model's robustness for all the abovementioned regimes.
Low-Resource Spoken Language Identification Using Self-Attentive Pooling and Deep 1D Time-Channel Separable Convolutions

This memo describes NTR/TSU winning submission for Low Resource ASR challenge at Dialog2021 conference, language identification track. Spoken Language Identification (LID) is an important step in a multilingual Automated Speech Recognition (ASR) system pipeline. Traditionally, the ASR task requires large volumes of labeled data that are unattainable for most of the world's languages, including most of the languages of Russia. In this memo, we show that a convolutional neural network with a Self-Attentive Pooling layer shows promising results in low-resource setting for the language identification task and set up a SOTA for the Low Resource ASR challenge dataset. Additionally, we compare the structure of confusion matrices for this and significantly more diverse VoxForge dataset and state and substantiate the hypothesis that whenever the dataset is diverse enough so that the other classification factors, like gender, age etc. are well-averaged, the confusion matrix for LID system bears the language similarity measure.
The Care Label Concept: A Certification Suite for Trustworthy and Resource-Aware Machine Learning

Machine learning applications have become ubiquitous. This has led to an increased effort of making machine learning trustworthy. Explainable and fair AI have already matured. They address knowledgeable users and application engineers. For those who do not want to invest time into understanding the method or the learned model, we offer care labels: easy to understand at a glance, allowing for method or model comparisons, and, at the same time, scientifically well-based. On one hand, this transforms descriptions as given by, e.g., Fact Sheets or Model Cards, into a form that is well-suited for end-users. On the other hand, care labels are the result of a certification suite that tests whether stated guarantees hold. In this paper, we present two experiments with our certification suite. One shows the care labels for configurations of Markov random fields (MRFs). Based on the underlying theory of MRFs, each choice leads to its specific rating of static properties like, e.g., expressivity and reliability. In addition, the implementation is tested and resource consumption is measured yielding dynamic properties. This two-level procedure is followed by another experiment certifying deep neural network (DNN) models. There, we draw the static properties from the literature on a particular model and data set. At the second level, experiments are generated that deliver measurements of robustness against certain attacks. We illustrate this by ResNet-18 and MobileNetV3 applied to ImageNet.
Pattern Discovery in Time Series with Byte Pair Encoding

The growing popularity of wearable sensors has generated large quantities of temporal physiological and activity data. Ability to analyze this data offers new opportunities for real-time health monitoring and forecasting. However, temporal physiological data presents many analytic challenges: the data is noisy, contains many missing values, and each series has a different length. Most methods proposed for time series analysis and classification do not handle datasets with these characteristics nor do they offer interpretability and explainability, a critical requirement in the health domain. We propose an unsupervised method for learning representations of time series based on common patterns identified within them. The patterns are, interpretable, variable in length, and extracted using Byte Pair Encoding compression technique. In this way the method can capture both long-term and short-term dependencies present in the data. We show that this method applies to both univariate and multivariate time series and beats state-of-the-art approaches on a real world dataset collected from wearable sensors.
Spectral reconstruction in Fourier modal methods: on exclusive robustness of Gegenbauer method

Fourier spectral methods constitute a rigorous memory-minimizing technology for the analysis of photonic structures. However, the discrete geometry of these artifacts can lead to the Gibbs phenomenon, and consequently, a dramatic deterioration of the exponential rate convergence. In this paper, we study the applicability of spectral reconstructions for the resolution of the Gibbs phenomenon in Fourier modal methods. We show in practice, the emergence and propagation of a computational disorder potentially incapacitate most of the popular post-processing techniques from retrieving a Fourier spectral solution in the vicinity of discontinuities. Nonetheless, we numerically substantiate and rigorously prove that the classical Gegenbauer method remains exceptionally robust against this phenomenon. Specifically, we show the Gegenbauer reconstruction error at worst diminishes proportional to O(N−1) for the Fourier order N , while the convergence can be even exponentially fast up to a constant summand.
PUDLE: Implicit Acceleration of Dictionary Learning by Backpropagation

The dictionary learning problem, representing data as a combination of few atoms, has long stood as a popular method for learning representations in statistics and signal processing. The most popular dictionary learning algorithm alternates between sparse coding and dictionary update steps, and a rich literature has studied its theoretical convergence. The growing popularity of neurally plausible unfolded sparse coding networks has led to the empirical ﬁnding that backpropagation through such networks performs dictionary learning. This paper offers the ﬁrst theoretical proof for these empirical results through PUDLE, a Provable Unfolded Dictionary LEarning method. We highlight the impact of loss, unfolding, and backpropagation on convergence. We discover an implicit acceleration: as a function of unfolding, the backpropagated gradient converges faster and is more accurate than the gradient from alternating minimization. We complement our ﬁndings through synthetic and image denoising experiments. The ﬁndings support the use of accelerated deep learning optimizers and unfolded networks for dictionary learning.
LenAtten: An Effective Length Controlling Unit For Text Summarization

Fixed length summarization aims at generating summaries with a preset number of words or characters. Most recent researches incorporate length information with word embeddings as the input to the recurrent decoding unit, causing a compromise between length controllability and summary quality. In this work, we present an effective length controlling unit Length Attention (LenAtten) to break this trade-off. Experimental results show that LenAtten not only brings improvements in length controllability and ROGUE scores but also has great generalization ability. In the task of generating a summary with the target length, our model is 732 times better than the best-performing length controllable summarizer in length controllability on the CNN/Daily Mail dataset.
Potential applications of modular representation theory to quantum mechanics

There is a unique finite group that lies inside the 2-dimensional unitary group but not in the special unitary group, and maps by the symmetric square to an irreducible subgroup of the 3-dimensional real special orthogonal group. In an earlier paper I showed how the representation theory of this group over the real numbers gives rise to much of the structure of the standard model of particle physics, but with a number of added twists. In this theory the group is quantised, but the representations are not. In this paper I consider how a quantisation of the representations might lead to a more fundamental theory.
On the Stability of Mixed Finite-Element Formulations for High-Temperature Superconductors

In this article, we present and analyze the numerical stability of two coupled finite element formulations. The first one is the <inline-formula><tex-math notation="LaTeX">$h$</tex-math></inline-formula>-<inline-formula><tex-math notation="LaTeX">$a$</tex-math></inline-formula>-formulation and is well suited for modeling systems with superconductors and ferromagnetic materials. The second one, the so-called <inline-formula><tex-math notation="LaTeX">$t$</tex-math></inline-formula>-<inline-formula><tex-math notation="LaTeX">$a$</tex-math></inline-formula>-formulation with thin-shell approximation, applies for systems with thin superconducting domains. Both formulations involve two coupled unknown fields and are mixed on the coupling interfaces. Function spaces in mixed formulations must satisfy compatibility conditions to ensure stability of the problem and reliability of the numerical solution. We propose stable choices of function spaces using hierarchical basis functions and demonstrate the effectiveness of the approach on simple 2D examples.
Unsupervised detection of mouse behavioural anomalies using two-stream convolutional autoencoders

This paper explores the application of unsupervised learning to detecting anomalies in mouse video data. The two models presented in this paper are a dual-stream, 3D convolutional autoencoder (with residual connections) and a dual-stream, 2D convolutional autoencoder. The publicly available dataset used here contains twelve videos of single home-caged mice alongside frame-level annotations. Under the pretext that the autoencoder only sees normal events, the video data was handcrafted to treat each behaviour as a pseudo-anomaly thereby eliminating them from the others during training. The results are presented for one conspicuous behaviour (hang) and one inconspicuous behaviour (groom). The performance of these models is compared to a single stream autoencoder and a supervised learning model, which are both based on the custom CAE. Both models are also tested on the CUHK Avenue dataset were found to perform as well as some state-of-the-art architectures.
Effect of large-scale pre-training on full and few-shot transfer learning for natural and medical images

Transfer learning aims to exploit pre-trained models for more efﬁcient follow-up training on wide range of downstream tasks and datasets, enabling successful training also on small data. Recent line of work posits strong beneﬁts for model generalization and transfer when model size, data size, and compute budget are increased for the pre-training. It remains however still largely unclear whether the observed transfer improvement due to increase in scale also holds when source and target data distributions are far apart from each other. In this work we conduct large-scale pre-training on large source datasets of either natural (ImageNet-21k/1k) or medical chest X-Ray images and compare full and few-shot transfer using different target datasets from both natural and medical imaging domains 1 . Our observations provide evidence that while pre-training and transfer on closely related datasets do show clear beneﬁt of increasing model and data size during pre-training, such beneﬁts are not clearly visible when source and target datasets are further apart. These observations hold across both full and few-shot transfer and indicate that scaling laws hinting improvement of generalization and transfer with increasing model and data size are incomplete and should also take into account the degree of how distinct the source and target data distributions are, to correctly predict effect of model size and data size variation during pre-training on transfer.
Multi-Scale Attention Neural Network for Acoustic Echo Cancellation

Acoustic Echo Cancellation (AEC) plays a key role in speech interaction by suppressing the echo received at microphone introduced by acoustic reverberations from loudspeakers. Since the performance of linear adaptive filter (AF) would degrade severely due to nonlinear distortions, background noises, and microphone clipping in real scenarios, deep learning has been employed for AEC for its good nonlinear modelling ability. In this paper, we constructed an end-to-end multi-scale attention neural network for AEC. Temporal convolution is first used to transform waveform into spectrogram. The spectrograms of the far-end reference and the near-end mixture are concatenated, and fed to a temporal convolution network (TCN) with stacked dilated convolution layers. Attention mechanism is performed among these representations from different layers to adaptively extract relevant features by referring to the previous hidden state in the encoder long short-term memory (LSTM) unit. The representations are weighted averaged and fed to the encoder LSTM for the near-end speech estimation. Experiments show the superiority of our method in terms of the echo return loss enhancement (ERLE) for single-talk periods and the perceptual evaluation of speech quality (PESQ) score for double-talk periods in background noise and nonlinear distortion scenarios.
More than just Frequency? Demasking Unsupervised Hypernymy Prediction Methods

This paper presents a comparison of unsupervised methods of hypernymy prediction (i.e., to predict which word in a pair of words such as fish-cod is the hypernym and which the hyponym). Most importantly, we demonstrate across datasets for English and for German that the predictions of three methods (WeedsPrec, invCL, SLQS Row) strongly overlap and are highly correlated with frequency-based predictions. In contrast, the second-order method SLQS shows an overall lower accuracy but makes correct predictions where the others go wrong. Our study once more confirms the general need to check the frequency bias of a computational method in order to identify frequency-(un)related effects.
3D map creation using crowdsourced GNSS data

3D maps are increasingly useful for many applications such as drone navigation, emergency services, and urban planning. However, creating 3D maps and keeping them up-to-date using existing technologies, such as laser scanners, is expensive. This paper proposes and implements a novel approach to generate 2.5D (otherwise known as 3D level-of-detail (LOD) 1) maps for free using Global Navigation Satellite Systems (GNSS) signals, which are globally available and are blocked only by obstacles between the satellites and the receivers. This enables us to find the patterns of GNSS signal availability and create 3D maps. The paper applies algorithms to GNSS signal strength patterns based on a boot-strapped technique that iteratively trains the signal classifiers while generating the map. Results of the proposed technique demonstrate the ability to create 3D maps using automatically processed GNSS data. The results show that the third dimension, i.e. height of the buildings, can be estimated with below 5 metre accuracy, which is the benchmark recommended by the CityGML standard.
Decoupling Shape and Density for Liver Lesion Synthesis Using Conditional Generative Adversarial Networks

Lesion synthesis received much attention with the rise of efficient generative models for augmenting training data, drawing lesion evolution scenarios, or aiding expert training. The quality and diversity of synthesized data are highly dependent on the annotated data used to train the models, which not rarely struggle to derive very different yet realistic samples from the training ones. That adds an inherent bias to lesion segmentation algorithms and limits synthesizing lesion evolution scenarios efficiently. This paper presents a method for decoupling shape and density for liver lesion synthesis, creating a framework that allows straight-forwardly driving the synthesis. We offer qualitative results that show the synthesis control by modifying shape and density individually, and quantitative results that demonstrate that embedding the density information in the generator model helps to increase lesion segmentation performance compared to using the shape solely.
A Blueprint for the Study of the Brain's Spatiotemporal Patterns

The functioning of an organ such as the brain emerges from interactions between its constituent parts. Further, this interaction is not immutable in time but rather unfolds in a succession of patterns, thereby allowing the brain to adapt to constantly changing exterior and interior milieus. This calls for a framework able to study patterned spatiotemporal interactions between components of the brain. A theoretical and methodological framework is developed to study the brain’s coordination dynamics. Here we present a toolset designed to decipher the continuous dynamics of electrophysiological data and its relation to (dys-) function. Understanding the spatiotemporal organization of brain patterns and their association with behavioral, cognitive and clinically-relevant variables is an important challenge for the fields of neuroscience and biologically-inspired engineering. It is hoped that such a comprehensive framework will shed light not only on human behavior and the human mind but also help in understanding the growing number of pathologies that are linked to disorders of brain connectivity. Keywords—4D, dynamics, coordination, spatial, temporal, visualization
KGPool: Dynamic Knowledge Graph Context Selection for Relation Extraction

We present a novel method for relation extraction (RE) from a single sentence, mapping the sentence and two given entities to a canonical fact in a knowledge graph (KG). Especially in this presumed sentential RE setting, the context of a single sentence is often sparse. This paper introduces the KGPool method to address this sparsity, dynamically expanding the context with additional facts from the KG. It learns the representation of these facts (entity alias, entity descriptions, etc.) using neural methods, supplementing the sentential context. Unlike existing methods that statically use all expanded facts, KGPool conditions this expansion on the sentence. We study the efficacy of KGPool by evaluating it with different neural models and KGs (Wikidata and NYT Freebase). Our experimental evaluation on standard datasets shows that by feeding the KGPool representation into a Graph Neural Network, the overall method is significantly more accurate than state-of-the-art methods.
UniStore: A fault-tolerant marriage of causal and strong consistency (extended version)

Modern online services rely on data stores that replicate their data across geographically distributed data centers. Providing strong consistency in such data stores results in high latencies and makes the system vulnerable to network partitions. The alternative of relaxing consistency violates crucial correctness properties. A compromise is to allow multiple consistency levels to coexist in the data store. In this paper we present UniStore, the first fault-tolerant and scalable data store that combines causal and strong consistency. The key challenge we address in UniStore is to maintain liveness despite data center failures: this could be compromised if a strong transaction takes a dependency on a causal transaction that is later lost because of a failure. UniStore ensures that such situations do not arise while paying the cost of durability for causal transactions only when necessary. We evaluate UniStore on Amazon EC2 using both microbenchmarks and a sample application. Our results show that UniStore effectively and scalably combines causal and strong consistency.