OpenRaiser/Intern-Atlas huggingface dataset:
On the sensitivity of pose estimation neural networks: rotation parameterizations, Lipschitz constants, and provable bounds
Venue: arXiv cs.CV 2022
Year: 2022

In this paper, we approach the task of determining sensitivity bounds for pose estimation neural networks. This task is particularly challenging as it requires characterizing the sensitivity of 3D rotations. We develop a sensitivity measure that describes the maximum rotational change in a network's output with respect to a Euclidean change in its input. We show that this measure is a type of Lipschitz constant, and that it is bounded by the product of a network's Euclidean Lipschitz constant and an intrinsic property of a rotation parameterization which we call the "distance ratio constant". We derive the distance ratio constant for several rotation parameterizations, and then discuss why the structure of most of these parameterizations makes it difficult to construct a pose estimation network with provable sensitivity bounds. However, we show that sensitivity bounds can be computed for networks which parameterize rotation using unconstrained exponential coordinates. We then construct and train such a network and compute sensitivity bounds for it.
A Comparison of Static, Dynamic, and Hybrid Analysis for Malware Detection
Venue: arXiv cs.CR 2022
Year: 2022

In this research, we compare malware detection techniques based on static, dynamic, and hybrid analysis. Specifically, we train Hidden Markov Models (HMMs ) on both static and dynamic feature sets and compare the resulting detection rates over a substantial number of malware families. We also consider hybrid cases, where dynamic analysis is used in the training phase, with static techniques used in the detection phase, and vice versa. In our experiments, a fully dynamic approach generally yields the best detection rates. We discuss the implications of this research for malware detection based on hybrid techniques.
Neural Enhanced Belief Propagation for Data Association in Multiobject Tracking
Venue: arXiv cs.CV 2022
Year: 2022

Situation-aware technologies enabled by multiobject tracking (MOT) methods will create new services and applications in fields such as autonomous navigation and applied ocean sciences. Belief propagation (BP) is a state-of-the-art method for Bayesian MOT but fully relies on a statistical model and preprocessed sensor measurements. In this paper, we establish a hybrid method for model-based and data-driven MOT. The proposed neural enhanced belief propagation (NEBP) approach complements BP by information learned from raw sensor data with the goal to improve data association and to reject false alarm measurements. We evaluate the performance of our NEBP approach for MOT on the nuScenes autonomous driving dataset and demonstrate that it can outperform state-of-the-art reference methods.
Improving Semantic Consistency of Variable Names with Use-Flow Graph Analysis
Venue: arXiv cs.SE 2022
Year: 2022

Consistency is one of the keys to maintainable source code and hence a successful software project. We propose a novel method of extracting the intent of programmers from source code of a large project (~300kLOC) and checking the semantic consistency of its variable names. Our system learns a project-specific naming convention for variables based on its role solely from source code, and suggest alternatives when it violates its internal consistency. The system can also show the reasoning why a certain variable should be named in a specific way. The system does not rely on any external knowledge. We applied our method to 12 open-source projects and evaluated its results with human reviewers. Our system proposed alternative variable names for 416 out of 1080 (39%) instances that are considered better than ones originally used by the developers. Based on the results, we created patches to correct the inconsistent names and sent them to its developers. Three open-source projects adopted it.
BIOS: An Algorithmically Generated Biomedical Knowledge Graph
Venue: arXiv cs.CL 2022
Year: 2022

Biomedical knowledge graphs (BioMedKGs) are essential infrastructures for biomedical and healthcare big data and artificial intelligence (AI), facilitating natural language processing, model development, and data exchange. For decades, these knowledge graphs have been developed via expert curation; however, this method can no longer keep up with today's AI development, and a transition to algorithmically generated BioMedKGs is necessary. In this work, we introduce the Biomedical Informatics Ontology System (BIOS), the first large-scale publicly available BioMedKG generated completely by machine learning algorithms. BIOS currently contains 4.1 million concepts, 7.4 million terms in two languages, and 7.3 million relation triplets. We present the methodology for developing BIOS, including the curation of raw biomedical terms, computational identification of synonymous terms and aggregation of these terms to create concept nodes, semantic type classification of the concepts, relation identification, and biomedical machine translation. We provide statistics on the current BIOS content and perform preliminary assessments of term quality, synonym grouping, and relation extraction. The results suggest that machine learning-based BioMedKG development is a viable alternative to traditional expert curation.
Analyzing EEG Data with Machine and Deep Learning: A Benchmark
Venue: arXiv cs.LG 2022
Year: 2022

Nowadays, machine and deep learning techniques are widely used in different areas, ranging from economics to biology. In general, these techniques can be used in two ways: trying to adapt well-known models and architectures to the available data, or designing custom architectures. In both cases, to speed up the research process, it is useful to know which type of models work best for a specific problem and/or data type. By focusing on EEG signal analysis, and for the first time in literature, in this paper a benchmark of machine and deep learning for EEG signal classification is proposed. For our experiments we used the four most widespread models, i.e., multilayer perceptron, convolutional neural network, long short-term memory, and gated recurrent unit, highlighting which one can be a good starting point for developing EEG classification models.
Challenges and Strategies in Cross-Cultural NLP
Venue: arXiv cs.CL 2022
Year: 2022

Various efforts in the Natural Language Processing (NLP) community have been made to accommodate linguistic diversity and serve speakers of many different languages. However, it is important to acknowledge that speakers and the content they produce and require, vary not just by language, but also by culture. Although language and culture are tightly linked, there are important differences. Analogous to cross-lingual and multilingual NLP, cross-cultural and multicultural NLP considers these differences in order to better serve users of NLP systems. We propose a principled framework to frame these efforts, and survey existing and potential strategies.
On the Generalization Mystery in Deep Learning
Venue: arXiv cs.LG 2022
Year: 2022

The generalization mystery in deep learning is the following: Why do over-parameterized neural networks trained with gradient descent (GD) generalize well on real datasets even though they are capable of fitting random datasets of comparable size? Furthermore, from among all solutions that fit the training data, how does GD find one that generalizes well (when such a well-generalizing solution exists)? We argue that the answer to both questions lies in the interaction of the gradients of different examples during training. Intuitively, if the per-example gradients are well-aligned, that is, if they are coherent, then one may expect GD to be (algorithmically) stable, and hence generalize well. We formalize this argument with an easy to compute and interpretable metric for coherence, and show that the metric takes on very different values on real and random datasets for several common vision networks. The theory also explains a number of other phenomena in deep learning, such as why some examples are reliably learned earlier than others, why early stopping works, and why it is possible to learn from noisy labels. Moreover, since the theory provides a causal explanation of how GD finds a well-generalizing solution when one exists, it motivates a class of simple modifications to GD that attenuate memorization and improve generalization. Generalization in deep learning is an extremely broad phenomenon, and therefore, it requires an equally general explanation. We conclude with a survey of alternative lines of attack on this problem, and argue that the proposed approach is the most viable one on this basis.
RoSS: Utilizing Robotic Rotation for Audio Source Separation
Venue: arXiv cs.SD 2022
Year: 2022

This paper considers the problem of audio source separation where the goal is to isolate a target audio signal (say Alice's speech) from a mixture of multiple interfering signals (e.g., when many people are talking). This problem has gained renewed interest mainly due to the significant growth in voice controlled devices, including robots in homes, offices, and other public facilities. Although a rich body of work exists on the core topic of source separation, we find that robotic motion of the microphone -- say the robot's head -- is a complementary opportunity to past approaches. Briefly, we show that rotating the microphone array to the correct orientation can produce desired aliasing between two interferers, causing the two interferers to pose as one. In other words, a mixture of K signals becomes a mixture of (K-1), a mathematically concrete gain. We show that the gain translates well to practice provided two mobility-related challenges can be mitigated. This paper is focused on mitigating these challenges and demonstrating the end-to-end performance on a fully functional prototype. We believe that our Rotational Source Separation module RoSS could be plugged into actual robot heads, or into other devices (like Amazon Show) that are also capable of rotation.
Learning Compressed Embeddings for On-Device Inference
Venue: arXiv cs.LG 2022
Year: 2022

In deep learning, embeddings are widely used to represent categorical entities such as words, apps, and movies. An embedding layer maps each entity to a unique vector, causing the layer's memory requirement to be proportional to the number of entities. In the recommendation domain, a given category can have hundreds of thousands of entities, and its embedding layer can take gigabytes of memory. The scale of these networks makes them difficult to deploy in resource constrained environments. In this paper, we propose a novel approach for reducing the size of an embedding table while still mapping each entity to its own unique embedding. Rather than maintaining the full embedding table, we construct each entity's embedding "on the fly" using two separate embedding tables. The first table employs hashing to force multiple entities to share an embedding. The second table contains one trainable weight per entity, allowing the model to distinguish between entities sharing the same embedding. Since these two tables are trained jointly, the network is able to learn a unique embedding per entity, helping it maintain a discriminative capability similar to a model with an uncompressed embedding table. We call this approach MEmCom (Multi-Embedding Compression). We compare with state-of-the-art model compression techniques for multiple problem classes including classification and ranking. On four popular recommender system datasets, MEmCom had a 4% relative loss in nDCG while compressing the input embedding sizes of our recommendation models by 16x, 4x, 12x, and 40x. MEmCom outperforms the state-of-the-art techniques, which achieved 16%, 6%, 10%, and 8% relative loss in nDCG at the respective compression ratios. Additionally, MEmCom is able to compress the RankNet ranking model by 32x on a dataset with millions of users' interactions with games while incurring only a 1% relative loss in nDCG.
Automated Materials Spectroscopy Analysis using Genetic Algorithms
Venue: arXiv cs.NE 2022
Year: 2022

We introduce a Genetic Algorithm (GA) based, open-source project to solve multi-objective optimization problems of materials characterization data analysis including EXAFS, XPS and nanoindentation. The modular design and multiple crossover and mutation options make the software extensible for additional materials characterization applications too. This automation of the analysis is crucial in the era when instrumentation acquires data orders of magnitude more rapidly than it can be analyzed by hand. Our results demonstrated good fitness scores with minimal human intervention.
Negative Inner-Loop Learning Rates Learn Universal Features
Venue: arXiv cs.LG 2022
Year: 2022

Model Agnostic Meta-Learning (MAML) consists of two optimization loops: the outer loop learns a meta-initialization of model parameters that is shared across tasks, and the inner loop task-specific adaptation step. A variant of MAML, Meta-SGD, uses the same two loop structure, but also learns the learning-rate for the adaptation step. Little attention has been paid to how the learned learning-rate of Meta-SGD affects feature reuse. In this paper, we study the effect that a learned learning-rate has on the per-task feature representations in Meta-SGD. The learned learning-rate of Meta-SGD often contains negative values. During the adaptation phase, these negative learning rates push features away from task-specific features and towards task-agnostic features. We performed several experiments on the Mini-Imagenet dataset. Two neural networks were trained, one with MAML, and one with Meta-SGD. The feature quality for both models was tested as follows: strip away the linear classification layer, pass labeled and unlabeled samples through this encoder, classify the unlabeled samples according to their nearest neighbor. This process was performed: 1) after training and using the meta-initialization parameters; 2) after adaptation, and validated on that task; and 3) after adaptation, and validated on a different task. The MAML trained model improved on the task it was adapted to, but had worse performance on other tasks. The Meta-SGD trained model was the opposite; it had worse performance on the task it was adapted to, but improved on other tasks. This confirms the hypothesis that Meta-SGD's negative learning rates cause the model to learn task-agnostic features rather than simply adapt to task specific features.
Analysis and Adaptation of YOLOv4 for Object Detection in Aerial Images
Venue: arXiv cs.CV 2022
Year: 2022

The recent and rapid growth in Unmanned Aerial Vehicles (UAVs) deployment for various computer vision tasks has paved the path for numerous opportunities to make them more effective and valuable. Object detection in aerial images is challenging due to variations in appearance, pose, and scale. Autonomous aerial flight systems with their inherited limited memory and computational power demand accurate and computationally efficient detection algorithms for real-time applications. Our work shows the adaptation of the popular YOLOv4 framework for predicting the objects and their locations in aerial images with high accuracy and inference speed. We utilized transfer learning for faster convergence of the model on the VisDrone DET aerial object detection dataset. The trained model resulted in a mean average precision (mAP) of 45.64% with an inference speed reaching 8.7 FPS on the Tesla K80 GPU and was highly accurate in detecting truncated and occluded objects. We experimentally evaluated the impact of varying network resolution sizes and training epochs on the performance. A comparative study with several contemporary aerial object detectors proved that YOLOv4 performed better, implying a more suitable detection algorithm to incorporate on aerial platforms.
UAV Trajectory and Beamforming Optimization for Integrated Periodic Sensing and Communication
Venue: arXiv cs.IT 2022
Year: 2022

Unmanned aerial vehicle (UAV) is expected to bring transformative improvement to the integrated sensing and communication (ISAC) system. However, due to shared spectrum resources, it is challenging to achieve a critical trade-off between these two integrated functionalities. To address this issue, we propose in this paper a new integrated \emph{periodic} sensing and communication mechanism for the UAV-enable ISAC system. Specifically, the user achievable rate is maximized via jointly optimizing UAV trajectory, transmit precoder, and sensing start instant, subject to the sensing frequency and beam pattern gain constraints. Despite that this problem is highly non-convex and involves an infinite number of variables, we obtain the optimal transmit precoder and derive the optimal achievable rate in closed-form for any given UAV location to facilitate the UAV trajectory design. Furthermore, we first prove the structural symmetry between optimal solutions in different ISAC frames without location constraints and then propose a high-quality UAV trajectory and sensing optimization algorithm for the general location-constrained case. Simulation results corroborate the effectiveness of the proposed design and also unveil a more flexible trade-off in ISAC systems over benchmark schemes.
ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning
Venue: arXiv cs.CL 2022
Year: 2022

Charts are very popular for analyzing data. When exploring charts, people often ask a variety of complex reasoning questions that involve several logical and arithmetic operations. They also commonly refer to visual features of a chart in their questions. However, most existing datasets do not focus on such complex reasoning questions as their questions are template-based and answers come from a fixed-vocabulary. In this work, we present a large-scale benchmark covering 9.6K human-written questions as well as 23.1K questions generated from human-written chart summaries. To address the unique challenges in our benchmark involving visual and logical reasoning over charts, we present two transformer-based models that combine visual features and the data table of the chart in a unified way to answer questions. While our models achieve the state-of-the-art results on the previous datasets as well as on our benchmark, the evaluation also reveals several challenges in answering complex reasoning questions.
Assessing Gender Bias in Predictive Algorithms using eXplainable AI
Venue: arXiv cs.HC 2022
Year: 2022

Predictive algorithms have a powerful potential to offer benefits in areas as varied as medicine or education. However, these algorithms and the data they use are built by humans, consequently, they can inherit the bias and prejudices present in humans. The outcomes can systematically repeat errors that create unfair results, which can even lead to situations of discrimination (e.g. gender, social or racial). In order to illustrate how important is to count with a diverse training dataset to avoid bias, we manipulate a well-known facial expression recognition dataset to explore gender bias and discuss its implications.
From meaning to perception -- exploring the space between word and odor perception embeddings
Venue: arXiv cs.CL 2022
Year: 2022

In this paper we propose the use of the Word2vec algorithm in order to obtain odor perception embeddings (or smell embeddings), only using publicly available perfume descriptions. Besides showing meaningful similarity relationships among each other, these embeddings also demonstrate to possess some shared information with their respective word embeddings. The meaningfulness of these embeddings suggests that aesthetics might provide enough constraints for using algorithms motivated by distributional semantics on non-randomly combined data. Furthermore, they provide possibilities for new ways of classifying odors and analyzing perfumes. We have also employed the embeddings in an attempt to understand the aesthetic nature of perfumes, based on the difference between real and randomly generated perfumes. In an additional tentative experiment we explore the possibility of a mapping between the word embedding space and the odor perception embedding space by fitting a regressor on the shared vocabulary and then predict the odor perception embeddings of words without an a priori associated smell, such as night or sky.
Modelling nonlinear dependencies in the latent space of inverse scattering
Venue: arXiv cs.CV 2022
Year: 2022

The problem of inverse scattering proposed by Angles and Mallat in 2018, concerns training a deep neural network to invert the scattering transform applied to an image. After such a network is trained, it can be used as a generative model given that we can sample from the distribution of principal components of scattering coefficients. For this purpose, Angles and Mallat simply use samples from independent Gaussians. However, as shown in this paper, the distribution of interest can actually be very far from normal and non-negligible dependencies might exist between different coefficients. This motivates using models for this distribution that allow for non-linear dependencies between variables. Within this paper, two such models are explored, namely a Variational AutoEncoder and a Generative Adversarial Network. We demonstrate the results obtained can be extremely realistic on some datasets and look better than those produced by Angles and Mallat. The conducted meta-analysis also shows a clear practical advantage of such constructed generative models in terms of the efficiency of their training process compared to existing generative models for images.
No Shifted Augmentations (NSA): compact distributions for robust self-supervised Anomaly Detection
Venue: arXiv cs.CV 2022
Year: 2022

Unsupervised Anomaly detection (AD) requires building a notion of normalcy, distinguishing in-distribution (ID) and out-of-distribution (OOD) data, using only available ID samples. Recently, large gains were made on this task for the domain of natural images using self-supervised contrastive feature learning as a first step followed by kNN or traditional one-class classifiers for feature scoring. Learned representations that are non-uniformly distributed on the unit hypersphere have been shown to be beneficial for this task. We go a step further and investigate how the \emph {geometrical compactness} of the ID feature distribution makes isolating and detecting outliers easier, especially in the realistic situation when ID training data is polluted (i.e. ID data contains some OOD data that is used for learning the feature extractor parameters). We propose novel architectural modifications to the self-supervised feature learning step, that enable such compact distributions for ID data to be learned. We show that the proposed modifications can be effectively applied to most existing self-supervised objectives, with large gains in performance. Furthermore, this improved OOD performance is obtained without resorting to tricks such as using strongly augmented ID images (e.g. by 90 degree rotations) as proxies for the unseen OOD data, as these impose overly prescriptive assumptions about ID data and its invariances. We perform extensive studies on benchmark datasets for one-class OOD detection and show state-of-the-art performance in the presence of pollution in the ID data, and comparable performance otherwise. We also propose and extensively evaluate a novel feature scoring technique based on the angular Mahalanobis distance, and propose a simple and novel technique for feature ensembling during evaluation that enables a big boost in performance at nearly zero run-time cost.