OpenRaiser/Intern-Atlas huggingface dataset:
Generative Extraction of Audio Classifiers for Speaker Identification
Venue: arXiv cs.CR 2022
Year: 2022

It is perhaps no longer surprising that machine learning models, especially deep neural networks, are particularly vulnerable to attacks. One such vulnerability that has been well studied is model extraction: a phenomenon in which the attacker attempts to steal a victim's model by training a surrogate model to mimic the decision boundaries of the victim model. Previous works have demonstrated the effectiveness of such an attack and its devastating consequences, but much of this work has been done primarily for image and text processing tasks. Our work is the first attempt to perform model extraction on {\em audio classification models}. We are motivated by an attacker whose goal is to mimic the behavior of the victim's model trained to identify a speaker. This is particularly problematic in security-sensitive domains such as biometric authentication. We find that prior model extraction techniques, where the attacker \textit{naively} uses a proxy dataset to attack a potential victim's model, fail. We therefore propose the use of a generative model to create a sufficiently large and diverse pool of synthetic attack queries. We find that our approach is able to extract a victim's model trained on \texttt{LibriSpeech} using queries synthesized with a proxy dataset based off of \texttt{VoxCeleb}; we achieve a test accuracy of 84.41\% with a budget of 3 million queries.
Data-aided Active User Detection with False Alarm Correction in Grant-Free Transmission
Venue: arXiv cs.IT 2022
Year: 2022

In most existing grant-free (GF) studies, the two key tasks, namely active user detection (AUD) and payload data decoding, are handled separately. In this paper, a two-step dataaided AUD scheme is proposed, namely the initial AUD step and the false alarm correction step respectively. To implement the initial AUD step, an embedded low-density-signature (LDS) based preamble pool is constructed. In addition, two message passing algorithm (MPA) based initial estimators are developed. In the false alarm correction step, a redundant factor graph is constructed based on the initial active user set, on which MPA is employed for data decoding. The remaining false detected inactive users will be further recognized by the false alarm corrector with the aid of decoded data symbols. Simulation results reveal that both the data decoding performance and the AUD performance are significantly enhanced by more than 1:5 dB at the target accuracy of 10^3 compared with the traditional compressed sensing (CS) based counterparts
KinePose: A temporally optimized inverse kinematics technique for 6DOF human pose estimation with biomechanical constraints
Venue: arXiv cs.CV 2022
Year: 2022

Computer vision/deep learning-based 3D human pose estimation methods aim to localize human joints from images and videos. Pose representation is normally limited to 3D joint positional/translational degrees of freedom (3DOFs), however, a further three rotational DOFs (6DOFs) are required for many potential biomechanical applications. Positional DOFs are insufficient to analytically solve for joint rotational DOFs in a 3D human skeletal model. Therefore, we propose a temporal inverse kinematics (IK) optimization technique to infer joint orientations throughout a biomechanically informed, and subject-specific kinematic chain. For this, we prescribe link directions from a position-based 3D pose estimate. Sequential least squares quadratic programming is used to solve a minimization problem that involves both frame-based pose terms, and a temporal term. The solution space is constrained using joint DOFs, and ranges of motion (ROMs). We generate 3D pose motion sequences to assess the IK approach both for general accuracy, and accuracy in boundary cases. Our temporal algorithm achieves 6DOF pose estimates with low Mean Per Joint Angular Separation (MPJAS) errors (3.7°/joint overall, & 1.6°/joint for lower limbs). With frame-by-frame IK we obtain low errors in the case of bent elbows and knees, however, motion sequences with phases of extended/straight limbs results in ambiguity in twist angle. With temporal IK, we reduce ambiguity for these poses, resulting in lower average errors.
Extreme compression of sentence-transformer ranker models: faster inference, longer battery life, and less storage on edge devices
Venue: arXiv cs.LG 2022
Year: 2022

Modern search systems use several large ranker models with transformer architectures. These models require large computational resources and are not suitable for usage on devices with limited computational resources. Knowledge distillation is a popular compression technique that can reduce the resource needs of such models, where a large teacher model transfers knowledge to a small student model. To drastically reduce memory requirements and energy consumption, we propose two extensions for a popular sentence-transformer distillation procedure: generation of an optimal size vocabulary and dimensionality reduction of the embedding dimension of teachers prior to distillation. We evaluate these extensions on two different types of ranker models. This results in extremely compressed student models whose analysis on a test dataset shows the significance and utility of our proposed extensions.
Variational multiscale reinforcement learning for discovering reduced order closure models of nonlinear spatiotemporal transport systems
Venue: arXiv cs.LG 2022
Year: 2022

A central challenge in the computational modeling and simulation of a multitude of science applications is to achieve robust and accurate closures for their coarse-grained representations due to underlying highly nonlinear multiscale interactions. These closure models are common in many nonlinear spatiotemporal systems to account for losses due to reduced order representations, including many transport phenomena in fluids. Previous data-driven closure modeling efforts have mostly focused on supervised learning approaches using high fidelity simulation data. On the other hand, reinforcement learning (RL) is a powerful yet relatively uncharted method in spatiotemporally extended systems. In this study, we put forth a modular dynamic closure modeling and discovery framework to stabilize the Galerkin projection based reduced order models that may arise in many nonlinear spatiotemporal dynamical systems with quadratic nonlinearity. However, a key element in creating a robust RL agent is to introduce a feasible reward function, which can be constituted of any difference metrics between the RL model and high fidelity simulation data. First, we introduce a multi-modal RL (MMRL) to discover mode-dependant closure policies that utilize the high fidelity data in rewarding our RL agent. We then formulate a variational multiscale RL (VMRL) approach to discover closure models without requiring access to the high fidelity data in designing the reward function. Specifically, our chief innovation is to leverage variational multiscale formalism to quantify the difference between modal interactions in Galerkin systems. Our results in simulating the viscous Burgers equation indicate that the proposed VMRL method leads to robust and accurate closure parameterizations, and it may potentially be used to discover scale-aware closure models for complex dynamical systems.
Perun: Performance Version System
Venue: arXiv cs.PF 2022
Year: 2022

In this paper, we present Perun: an open-source tool suite for profiling-based performance analysis. At its core, Perun maintains links between project versions and the corresponding stored performance profiles, which are then leveraged for automated detection of performance changes in new project versions. The Perun tool suite further includes multiple profilers (and is designed such that further profilers can be easily added), a performance fuzz-tester for workload generation, methods for deriving performance models, and numerous visualization methods. We demonstrate how Perun can help developers to analyze their program performance on two examples: detection and localization of a performance degradation and generation of inputs forcing performance issues to show up.
Review of Advanced Monitoring Mechanisms in Peer-to-Peer (P2P) Botnets
Venue: arXiv cs.CR 2022
Year: 2022

Internet security is getting less secure because of the existing of botnet threats. An attack plan can only be planned out to take down the botnet after the monitoring activities to understand the behaviour of a botnet. Nowadays, the architecture of the botnet is developed using Peer-to-Peer (P2P) connection causing it to be harder to be monitored and track down. This paper is mainly about existing botnet monitoring tools. The purpose of this paper is to study the ways to monitor a botnet and how monitoring mechanism works. The monitoring tools are categorized into active and passive mechanism. A crawler is an active mechanism while sensor and Honeypot are the passive mechanisms. Previous work about each mechanism is present in this paper as well.
On the Security of IO-Link Wireless Communication in the Safety Domain
Venue: arXiv cs.CR 2022
Year: 2022

Security is an essential requirement of Industrial Control System (ICS) environments and its underlying communication infrastructure. Especially the lowest communication level within Supervisory Control and Data Acquisition (SCADA) systems - the field level - commonly lacks security measures. Since emerging wireless technologies within field level expose the lowest communication infrastructure towards potential attackers, additional security measures above the prevalent concept of air-gapped communication must be considered. Therefore, this work analyzes security aspects for the wireless communication protocol IO-LinkWireless (IOLW), which is commonly used for sensor and actuator field level communication. A possible architecture for an IOLW safety layer has already been presented recently [1]. In this paper, the overall attack surface of IOLW within its typical environment is analyzed and attack preconditions are investigated to assess the effectiveness of different security measures. Additionally, enhanced security measures are evaluated for the communication systems and the results are summarized. Also, interference of security measures and functional safety principles within the communication are investigated, which do not necessarily complement one another but may also have contradictory requirements. This work is intended to discuss and propose enhancements of the IOLW standard with additional security considerations in future implementations.
Learning structures of the French clinical language:development and validation of word embedding models using 21 million clinical reports from electronic health records
Venue: arXiv cs.CL 2022
Year: 2022

Background Clinical studies using real-world data may benefit from exploiting clinical reports, a particularly rich albeit unstructured medium. To that end, natural language processing can extract relevant information. Methods based on transfer learning using pre-trained language models have achieved state-of-the-art results in most NLP applications; however, publicly available models lack exposure to speciality-languages, especially in the medical field. Objective We aimed to evaluate the impact of adapting a language model to French clinical reports on downstream medical NLP tasks. Methods We leveraged a corpus of 21M clinical reports collected from August 2017 to July 2021 at the Greater Paris University Hospitals (APHP) to produce two CamemBERT architectures on speciality language: one retrained from scratch and the other using CamemBERT as its initialisation. We used two French annotated medical datasets to compare our language models to the original CamemBERT network, evaluating the statistical significance of improvement with the Wilcoxon test. Results Our models pretrained on clinical reports increased the average F1-score on APMed (an APHP-specific task) by 3 percentage points to 91%, a statistically significant improvement. They also achieved performance comparable to the original CamemBERT on QUAERO. These results hold true for the fine-tuned and from-scratch versions alike, starting from very few pre-training samples. Conclusions We confirm previous literature showing that adapting generalist pre-train language models such as CamenBERT on speciality corpora improves their performance for downstream clinical NLP tasks. Our results suggest that retraining from scratch does not induce a statistically significant performance gain compared to fine-tuning.
AMF: Adaptable Weighting Fusion with Multiple Fine-tuning for Image Classification
Venue: arXiv cs.CV 2022
Year: 2022

Fine-tuning is widely applied in image classification tasks as a transfer learning approach. It re-uses the knowledge from a source task to learn and obtain a high performance in target tasks. Fine-tuning is able to alleviate the challenge of insufficient training data and expensive labelling of new data. However, standard fine-tuning has limited performance in complex data distributions. To address this issue, we propose the Adaptable Multi-tuning method, which adaptively determines each data sample's fine-tuning strategy. In this framework, multiple fine-tuning settings and one policy network are defined. The policy network in Adaptable Multi-tuning can dynamically adjust to an optimal weighting to feed different samples into models that are trained using different fine-tuning strategies. Our method outperforms the standard fine-tuning approach by 1.69%, 2.79% on the datasets FGVC-Aircraft, and Describable Texture, yielding comparable performance on the datasets Stanford Cars, CIFAR-10, and Fashion-MNIST.
Revisited Containment for Graph Patterns
Venue: arXiv cs.DB 2022
Year: 2022

We consider the class of conditional graph patterns (\emph{CGPs}) that allow user to query data graphs with complex patterns that contain negation and predicates. To overcome the prohibitive cost of subgraph isomorphism, we consider matching of \emph{CGPs} under simulation semantics which can be conducted in quadratic time. In emerging applications, one would like to reduce more this matching time, and the static analysis of patterns may allow ensuring part of this reduction. We study the containment problem of \emph{CGPs} that aims to check whether the matches of some pattern $P_1$, over any data graph, are contained in those of another pattern $P_2$ (written $P_1\sqsubseteq P_2$). The optimization process consists to extract matches of $P_1$ only from those of $P_2$ without querying the (possibly large) data graph. We show that the traditional semantics of containment is decidable in quadratic time, but it fails to meet the optimization goal in the presence of negation and predicates. To overcome this limit, we propose a new semantics of containment, called \emph{strong containment}, that is more suitable for \emph{CGPs} and allows to reduce their matching time. We show that \emph{strong containment} can be decided in cubic time by providing such an algorithm. We are planing to use results of this paper to answer \emph{CGPs} using views. This paper is part of an ongoing project that aims to design a caching system for complex graph patterns.
Jacobian Norm with Selective Input Gradient Regularization for Improved and Interpretable Adversarial Defense
Venue: arXiv cs.LG 2022
Year: 2022

Deep neural networks (DNNs) are known to be vulnerable to adversarial examples that are crafted with imperceptible perturbations, i.e., a small change in an input image can induce a mis-classification, and thus threatens the reliability of deep learning based deployment systems. Adversarial training (AT) is often adopted to improve robustness through training a mixture of corrupted and clean data. However, most of AT based methods are ineffective in dealing with transferred adversarial examples which are generated to fool a wide spectrum of defense models, and thus cannot satisfy the generalization requirement raised in real-world scenarios. Moreover, adversarially training a defense model in general cannot produce interpretable predictions towards the inputs with perturbations, whilst a highly interpretable robust model is required by different domain experts to understand the behaviour of a DNN. In this work, we propose a novel approach based on Jacobian norm and Selective Input Gradient Regularization (J-SIGR), which suggests the linearized robustness through Jacobian normalization and also regularizes the perturbation-based saliency maps to imitate the model's interpretable predictions. As such, we achieve both the improved defense and high interpretability of DNNs. Finally, we evaluate our method across different architectures against powerful adversarial attacks. Experiments demonstrate that the proposed J-SIGR confers improved robustness against transferred adversarial attacks, and we also show that the predictions from the neural network are easy to interpret.
Efficient High-Resolution Deep Learning: A Survey
Venue: arXiv cs.CV 2022
Year: 2022

Cameras in modern devices such as smartphones, satellites and medical equipment are capable of capturing very high resolution images and videos. Such high-resolution data often need to be processed by deep learning models for cancer detection, automated road navigation, weather prediction, surveillance, optimizing agricultural processes and many other applications. Using high-resolution images and videos as direct inputs for deep learning models creates many challenges due to their high number of parameters, computation cost, inference latency and GPU memory consumption. Simple approaches such as resizing the images to a lower resolution are common in the literature, however, they typically significantly decrease accuracy. Several works in the literature propose better alternatives in order to deal with the challenges of high-resolution data and improve accuracy and speed while complying with hardware limitations and time restrictions. This survey describes such efficient high-resolution deep learning methods, summarizes real-world applications of high-resolution deep learning, and provides comprehensive information about available high-resolution datasets.
Task Agnostic and Post-hoc Unseen Distribution Detection
Venue: arXiv cs.LG 2022
Year: 2022

Despite the recent advances in out-of-distribution(OOD) detection, anomaly detection, and uncertainty estimation tasks, there do not exist a task-agnostic and post-hoc approach. To address this limitation, we design a novel clustering-based ensembling method, called Task Agnostic and Post-hoc Unseen Distribution Detection (TAPUDD) that utilizes the features extracted from the model trained on a specific task. Explicitly, it comprises of TAP-Mahalanobis, which clusters the training datasets' features and determines the minimum Mahalanobis distance of the test sample from all clusters. Further, we propose the Ensembling module that aggregates the computation of iterative TAP-Mahalanobis for a different number of clusters to provide reliable and efficient cluster computation. Through extensive experiments on synthetic and real-world datasets, we observe that our approach can detect unseen samples effectively across diverse tasks and performs better or on-par with the existing baselines. To this end, we eliminate the necessity of determining the optimal value of the number of clusters and demonstrate that our method is more viable for large-scale classification tasks.
SAR-to-EO Image Translation with Multi-Conditional Adversarial Networks
Venue: arXiv cs.CV 2022
Year: 2022

This paper explores the use of multi-conditional adversarial networks for SAR-to-EO image translation. Previous methods condition adversarial networks only on the input SAR. We show that incorporating multiple complementary modalities such as Google maps and IR can further improve SAR-to-EO image translation especially on preserving sharp edges of manmade objects. We demonstrate effectiveness of our approach on a diverse set of datasets including SEN12MS, DFC2020, and SpaceNet6. Our experimental results suggest that additional information provided by complementary modalities improves the performance of SAR-to-EO image translation compared to the models trained on paired SAR and EO data only. To best of our knowledge, our approach is the first to leverage multiple modalities for improving SAR-to-EO image translation performance.
Information We Can Extract About a User From 'One Minute Mobile Application Usage'
Venue: arXiv cs.LG 2022
Year: 2022

Understanding human behavior is an important task and has applications in many domains such as targeted advertisement, health analytics, security, and entertainment, etc. For this purpose, designing a system for activity recognition (AR) is important. However, since every human can have different behaviors, understanding and analyzing common patterns become a challenging task. Since smartphones are easily available to every human being in the modern world, using them to track the human activities becomes possible. In this paper, we extracted different human activities using accelerometer, magnetometer, and gyroscope sensors of android smartphones by building an android mobile applications. Using different social media applications, such as Facebook, Instagram, Whatsapp, and Twitter, we extracted the raw sensor values along with the attributes of $29$ subjects along with their attributes (class labels) such as age, gender, and left/right/both hands application usage. We extract features from the raw signals and use them to perform classification using different machine learning (ML) algorithms. Using statistical analysis, we show the importance of different features towards the prediction of class labels. In the end, we use the trained ML model on our data to extract unknown features from a well known activity recognition data from UCI repository, which highlights the potential of privacy breach using ML models. This security analysis could help researchers in future to take appropriate steps to preserve the privacy of human subjects.
XADLiME: eXplainable Alzheimer's Disease Likelihood Map Estimation via Clinically-guided Prototype Learning
Venue: arXiv cs.LG 2022
Year: 2022

Diagnosing Alzheimer's disease (AD) involves a deliberate diagnostic process owing to its innate traits of irreversibility with subtle and gradual progression. These characteristics make AD biomarker identification from structural brain imaging (e.g., structural MRI) scans quite challenging. Furthermore, there is a high possibility of getting entangled with normal aging. We propose a novel deep-learning approach through eXplainable AD Likelihood Map Estimation (XADLiME) for AD progression modeling over 3D sMRIs using clinically-guided prototype learning. Specifically, we establish a set of topologically-aware prototypes onto the clusters of latent clinical features, uncovering an AD spectrum manifold. We then measure the similarities between latent clinical features and well-established prototypes, estimating a "pseudo" likelihood map. By considering this pseudo map as an enriched reference, we employ an estimating network to estimate the AD likelihood map over a 3D sMRI scan. Additionally, we promote the explainability of such a likelihood map by revealing a comprehensible overview from two perspectives: clinical and morphological. During the inference, this estimated likelihood map served as a substitute over unseen sMRI scans for effectively conducting the downstream task while providing thorough explainable states.