OpenRaiser/Intern-Atlas huggingface dataset:
Multi-Modal Music Information Retrieval: Augmenting Audio-Analysis with Visual Computing for Improved Music Video Analysis
Venue: arXiv cs.MM 2020
Year: 2020

This thesis combines audio-analysis with computer vision to approach Music Information Retrieval (MIR) tasks from a multi-modal perspective. This thesis focuses on the information provided by the visual layer of music videos and how it can be harnessed to augment and improve tasks of the MIR research domain. The main hypothesis of this work is based on the observation that certain expressive categories such as genre or theme can be recognized on the basis of the visual content alone, without the sound being heard. This leads to the hypothesis that there exists a visual language that is used to express mood or genre. In a further consequence it can be concluded that this visual information is music related and thus should be beneficial for the corresponding MIR tasks such as music genre classification or mood recognition. A series of comprehensive experiments and evaluations are conducted which are focused on the extraction of visual information and its application in different MIR tasks. A custom dataset is created, suitable to develop and test visual features which are able to represent music related information. Evaluations range from low-level visual features to high-level concepts retrieved by means of Deep Convolutional Neural Networks. Additionally, new visual features are introduced capturing rhythmic visual patterns. In all of these experiments the audio-based results serve as benchmark for the visual and audio-visual approaches. The experiments are conducted for three MIR tasks Artist Identification, Music Genre Classification and Cross-Genre Classification. Experiments show that an audio-visual approach harnessing high-level semantic information gained from visual concept detection, outperforms audio-only genre-classification accuracy by 16.43%.
Alias Resolution Based on ICMP Rate Limiting
Venue: arXiv cs.NI 2020
Year: 2020

Alias resolution techniques (e.g., Midar) associate, mostly through active measurement, a set of IP addresses as belonging to a common router. These techniques rely on distinct router features that can serve as a signature. Their applicability is affected by router support of the features and the robustness of the signature. This paper presents a new alias resolution tool called Limited Ltd. that exploits ICMP rate limiting, a feature that is increasingly supported by modern routers that has not previously been used for alias resolution. It sends ICMP probes toward target interfaces in order to trigger rate limiting, extracting features from the probe reply loss traces. It uses a machine learning classifier to designate pairs of interfaces as aliases. We describe the details of the algorithm used by Limited Ltd. and illustrate its feasibility and accuracy. Limited Ltd. not only is the first tool that can perform alias resolution on IPv6 routers that do not generate monotonically increasing fragmentation IDs (e.g., Juniper routers) but it also complements the state-of-the-art techniques for IPv4 alias resolution. All of our code and the collected dataset are publicly available.
Few-Shot Scene Adaptive Crowd Counting Using Meta-Learning
Venue: arXiv cs.CV 2020
Year: 2020

We consider the problem of few-shot scene adaptive crowd counting. Given a target camera scene, our goal is to adapt a model to this specific scene with only a few labeled images of that scene. The solution to this problem has potential applications in numerous real-world scenarios, where we ideally like to deploy a crowd counting model specially adapted to a target camera. We accomplish this challenge by taking inspiration from the recently introduced learning-to-learn paradigm in the context of few-shot regime. In training, our method learns the model parameters in a way that facilitates the fast adaptation to the target scene. At test time, given a target scene with a small number of labeled data, our method quickly adapts to that scene with a few gradient updates to the learned parameters. Our extensive experimental results show that the proposed approach outperforms other alternatives in few-shot scene adaptive crowd counting. Code is available at https://github.com/maheshkkumar/fscc.
Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension
Venue: arXiv cs.CL 2020
Year: 2020

Innovations in annotation methodology have been a catalyst for Reading Comprehension (RC) datasets and models. One recent trend to challenge current RC models is to involve a model in the annotation process: humans create questions adversarially, such that the model fails to answer them correctly. In this work we investigate this annotation methodology and apply it in three different settings, collecting a total of 36,000 samples with progressively stronger models in the annotation loop. This allows us to explore questions such as the reproducibility of the adversarial effect, transfer from data collected with varying model-in-the-loop strengths, and generalisation to data collected without a model. We find that training on adversarially collected samples leads to strong generalisation to non-adversarially collected datasets, yet with progressive performance deterioration with increasingly stronger models-in-the-loop. Furthermore, we find that stronger models can still learn from datasets collected with substantially weaker models-in-the-loop. When trained on data collected with a BiDAF model in the loop, RoBERTa achieves 39.9F1 on questions that it cannot answer when trained on SQuAD - only marginally lower than when trained on data collected using RoBERTa itself (41.0F1).
A Closer Look at Small-loss Bounds for Bandits with Graph Feedback
Venue: arXiv cs.LG 2020
Year: 2020

We study small-loss bounds for adversarial multi-armed bandits with graph feedback, that is, adaptive regret bounds that depend on the loss of the best arm or related quantities, instead of the total number of rounds. We derive the first small-loss bound for general strongly observable graphs, resolving an open problem of Lykouris et al. (2018). Specifically, we develop an algorithm with regret $\mathcal{\tilde{O}}(\sqrt{κL_*})$ where $κ$ is the clique partition number and $L_*$ is the loss of the best arm, and for the special case of self-aware graphs where every arm has a self-loop, we improve the regret to $\mathcal{\tilde{O}}(\min\{\sqrt{αT}, \sqrt{κL_*}\})$ where $α\leq κ$ is the independence number. Our results significantly improve and extend those by Lykouris et al. (2018) who only consider self-aware undirected graphs. Furthermore, we also take the first attempt at deriving small-loss bounds for weakly observable graphs. We first prove that no typical small-loss bounds are achievable in this case, and then propose algorithms with alternative small-loss bounds in terms of the loss of some specific subset of arms. A surprising side result is that $\mathcal{\tilde{O}}(\sqrt{T})$ regret is achievable even for weakly observable graphs as long as the best arm has a self-loop. Our algorithms are based on the Online Mirror Descent framework but require a suite of novel techniques that might be of independent interest. Moreover, all our algorithms can be made parameter-free without the knowledge of the environment.
Reflection Resource Management for Intelligent Reflecting Surface Aided Wireless Networks
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, the adoption of an intelligent reflecting surface (IRS) for multiple single-antenna source terminal (ST)-DT pairs in two-hop networks is investigated. Different from the previous studies on IRS that merely focused on tuning the reflection coefficient of all the reflection elements at IRS, in this paper, we consider the true reflection resource management. Specifically, the true reflection resource management can be realized via trigger module selection based on our proposed IRS architecture that all the reflection elements are partially controlled by multiple parallel switches of controller. As the number of reflection elements increases, the true reflection resource management will become urgently needed in this context, which is due to the non-ignorable energy consumption. Moreover, the proposed modular architecture of IRS is designed to make the reflection elements part independent and controllable. As such, our goal is to maximize the minimum signal-to-interference-plus-noise ratio (SINR) at DTs via a joint trigger module subset selection, transmit power allocation of STs, and the corresponding passive beamforming of the trigger modules, subject to per ST power budgets and module size constraint. Whereas this problem is NP-hard due to the module size constraint, to deal with it, we transform the hard module size constraint into the group sparse constraint by introducing the mixed row block norm, which yields a suitable semidefinite relaxation. Additionally, the parallel alternating direction method of multipliers (PADMM) is proposed to identify the trigger module subset, and then subsequently the transmit power allocation and passive beamforming can be obtained by solving the original minimum SINR maximization problem without the group sparse constraint via partial linearization for generalized fractional programs.
Developments in Modern GNSS and Its Impact on Autonomous Vehicle Architectures
Venue: arXiv cs.RO 2020
Year: 2020

This paper surveys a number of recent developments in modern Global Navigation Satellite Systems (GNSS) and investigates the possible impact on autonomous driving architectures. Modern GNSS now consist of four independent global satellite constellations delivering modernized signals at multiple civil frequencies. New ground monitoring infrastructure, mathematical models, and internet services correct for errors in the GNSS signals at continent scale. Mass-market automotive-grade receiver chipsets are available at low Cost, Size, Weight, and Power (CSWaP). The result is that GNSS in 2020 delivers better than lane-level accurate localization with 99.99999% integrity guarantees at over 95% availability. In autonomous driving, SAE Level 2 partially autonomous vehicles are now available to consumers, capable of autonomously following lanes and performing basic maneuvers under human supervision. Furthermore, the first pilot programs of SAE Level 4 driverless vehicles are being demonstrated on public roads. However, autonomous driving is not a solved problem. GNSS can help. Specifically, incorporating high-integrity GNSS lane determination into vision-based architectures can unlock lane-level maneuvers and provide oversight to guarantee safety. Incorporating precision GNSS into LiDAR-based systems can unlock robustness and additional fallbacks for safety and utility. Lastly, GNSS provides interoperability through consistent timing and reference frames for future V2X scenarios.
Adversarial Generation of Continuous Implicit Shape Representations
Venue: arXiv cs.CV 2020
Year: 2020

This work presents a generative adversarial architecture for generating three-dimensional shapes based on signed distance representations. While the deep generation of shapes has been mostly tackled by voxel and surface point cloud approaches, our generator learns to approximate the signed distance for any point in space given prior latent information. Although structurally similar to generative point cloud approaches, this formulation can be evaluated with arbitrary point density during inference, leading to fine-grained details in generated outputs. Furthermore, we study the effects of using either progressively growing voxel- or point-processing networks as discriminators, and propose a refinement scheme to strengthen the generator's capabilities in modeling the zero iso-surface decision boundary of shapes. We train our approach on the ShapeNet benchmark dataset and validate, both quantitatively and qualitatively, its performance in generating realistic 3D shapes.
Interpreting video features: a comparison of 3D convolutional networks and convolutional LSTM networks
Venue: arXiv cs.CV 2020
Year: 2020

A number of techniques for interpretability have been presented for deep learning in computer vision, typically with the goal of understanding what the networks have based their classification on. However, interpretability for deep video architectures is still in its infancy and we do not yet have a clear concept of how to decode spatiotemporal features. In this paper, we present a study comparing how 3D convolutional networks and convolutional LSTM networks learn features across temporally dependent frames. This is the first comparison of two video models that both convolve to learn spatial features but have principally different methods of modeling time. Additionally, we extend the concept of meaningful perturbation introduced by \cite{MeaningFulPert} to the temporal dimension, to identify the temporal part of a sequence most meaningful to the network for a classification decision. Our findings indicate that the 3D convolutional model concentrates on shorter events in the input sequence, and places its spatial focus on fewer, contiguous areas.
Sound Event Detection with Depthwise Separable and Dilated Convolutions
Venue: arXiv cs.SD 2020
Year: 2020

State-of-the-art sound event detection (SED) methods usually employ a series of convolutional neural networks (CNNs) to extract useful features from the input audio signal, and then recurrent neural networks (RNNs) to model longer temporal context in the extracted features. The number of the channels of the CNNs and size of the weight matrices of the RNNs have a direct effect on the total amount of parameters of the SED method, which is to a couple of millions. Additionally, the usually long sequences that are used as an input to an SED method along with the employment of an RNN, introduce implications like increased training time, difficulty at gradient flow, and impeding the parallelization of the SED method. To tackle all these problems, we propose the replacement of the CNNs with depthwise separable convolutions and the replacement of the RNNs with dilated convolutions. We compare the proposed method to a baseline convolutional neural network on a SED task, and achieve a reduction of the amount of parameters by 85% and average training time per epoch by 78%, and an increase the average frame-wise F1 score and reduction of the average error rate by 4.6% and 3.8%, respectively.
Shapeshifter: A Multi-Agent, Multi-Modal Robotic Platform for Exploration of Titan
Venue: arXiv cs.RO 2020
Year: 2020

In this paper we present a mission architecture and a robotic platform, the Shapeshifter, that allow multi-domain and redundant mobility on Saturn's moon Titan, and potentially other bodies with atmospheres. The Shapeshifter is a collection of simple and affordable robotic units, called Cobots, comparable to personal palm-size quadcopters. By attaching and detaching with each other, multiple Cobots can shape-shift into novel structures, capable of (a) rolling on the surface, to increase the traverse range, (b) flying in a flight array formation, and (c) swimming on or under liquid. A ground station complements the robotic platform, hosting science instrumentation and providing power to recharge the batteries of the Cobots. In the first part of this paper we experimentally show the flying, docking and rolling capabilities of a Shapeshifter constituted by two Cobots, presenting ad-hoc control algorithms. We additionally evaluate the energy-efficiency of the rolling-based mobility strategy by deriving an analytic model of the power consumption and by integrating it in a high-fidelity simulation environment. In the second part we tailor our mission architecture to the exploration of Titan. We show that the properties of the Shapeshifter allow the exploration of the possible cryovolcano Sotra Patera, Titan's Mare and canyons.
Mixing Patterns in Interdisciplinary Collaboration Networks: Assessing Interdisciplinarity Through Multiple Lenses
Venue: arXiv cs.SI 2020
Year: 2020

There are inherent challenges to interdisciplinary research collaboration, such as bridging cognitive gaps and balancing transaction costs with collaborative benefits. This raises the question: Does interdisciplinary research necessarily result in interdisciplinary collaborations? This study aims to explore this question and assess collaboration preferences in interdisciplinary research at the individual, dyadic, and team level by examining mixing patterns in a collaboration network. Using a network of over 2,000 researchers from the field of artificial intelligence in education, we find that "interdisciplinarity" is demonstrated by diverse research experiences of individual researchers rather than diversity among researchers within collaborations. We also examine intergroup mixing by applying a novel approach to classify the active and non-active researchers in the collaboration network based on participation in multiple teams. We find a significant difference in indicators of academic performance and experience between the clusters of active and non-active researchers, suggesting intergroup mixing as a key factor in academic success. Our results shed light on the nature of team formation in interdisciplinary research, as well as highlight the importance of interdisciplinary programs.
DiverseDepth: Affine-invariant Depth Prediction Using Diverse Data
Venue: arXiv cs.CV 2020
Year: 2020

We present a method for depth estimation with monocular images, which can predict high-quality depth on diverse scenes up to an affine transformation, thus preserving accurate shapes of a scene. Previous methods that predict metric depth often work well only for a specific scene. In contrast, learning relative depth (information of being closer or further) can enjoy better generalization, with the price of failing to recover the accurate geometric shape of the scene. In this work, we propose a dataset and methods to tackle this dilemma, aiming to predict accurate depth up to an affine transformation with good generalization to diverse scenes. First we construct a large-scale and diverse dataset, termed Diverse Scene Depth dataset (DiverseDepth), which has a broad range of scenes and foreground contents. Compared with previous learning objectives, i.e., learning metric depth or relative depth, we propose to learn the affine-invariant depth using our diverse dataset to ensure both generalization and high-quality geometric shapes of scenes. Furthermore, in order to train the model on the complex dataset effectively, we propose a multi-curriculum learning method. Experiments show that our method outperforms previous methods on 8 datasets by a large margin with the zero-shot test setting, demonstrating the excellent generalization capacity of the learned model to diverse scenes. The reconstructed point clouds with the predicted depth show that our method can recover high-quality 3D shapes. Code and dataset are available at: https://tinyurl.com/DiverseDepth
CoTK: An Open-Source Toolkit for Fast Development and Fair Evaluation of Text Generation
Venue: arXiv cs.CL 2020
Year: 2020

In text generation evaluation, many practical issues, such as inconsistent experimental settings and metric implementations, are often ignored but lead to unfair evaluation and untenable conclusions. We present CoTK, an open-source toolkit aiming to support fast development and fair evaluation of text generation. In model development, CoTK helps handle the cumbersome issues, such as data processing, metric implementation, and reproduction. It standardizes the development steps and reduces human errors which may lead to inconsistent experimental settings. In model evaluation, CoTK provides implementation for many commonly used metrics and benchmark models across different experimental settings. As a unique feature, CoTK can signify when and which metric cannot be fairly compared. We demonstrate that it is convenient to use CoTK for model development and evaluation, particularly across different experimental settings.
Validating Mathematical Structures
Venue: arXiv cs.PL 2020
Year: 2020

Sharing of notations and theories across an inheritance hierarchy of mathematical structures, e.g., groups and rings, is important for productivity when formalizing mathematics in proof assistants. The packed classes methodology is a generic design pattern to define and combine mathematical structures in a dependent type theory with records. When combined with mechanisms for implicit coercions and unification hints, packed classes enable automated structure inference and subtyping in hierarchies, e.g., that a ring can be used in place of a group. However, large hierarchies based on packed classes are challenging to implement and maintain. We identify two hierarchy invariants that ensure modularity of reasoning and predictability of inference with packed classes, and propose algorithms to check these invariants. We implement our algorithms as tools for the Coq proof assistant, and show that they significantly improve the development process of Mathematical Components, a library for formalized mathematics.
Dynamic Parameter Allocation in Parameter Servers
Venue: arXiv cs.LG 2020
Year: 2020

To keep up with increasing dataset sizes and model complexity, distributed training has become a necessity for large machine learning tasks. Parameter servers ease the implementation of distributed parameter management---a key concern in distributed training---, but can induce severe communication overhead. To reduce communication overhead, distributed machine learning algorithms use techniques to increase parameter access locality (PAL), achieving up to linear speed-ups. We found that existing parameter servers provide only limited support for PAL techniques, however, and therefore prevent efficient training. In this paper, we explore whether and to what extent PAL techniques can be supported, and whether such support is beneficial. We propose to integrate dynamic parameter allocation into parameter servers, describe an efficient implementation of such a parameter server called Lapse, and experimentally compare its performance to existing parameter servers across a number of machine learning tasks. We found that Lapse provides near-linear scaling and can be orders of magnitude faster than existing parameter servers.
FAE: A Fairness-Aware Ensemble Framework
Venue: arXiv cs.AI 2020
Year: 2020

Automated decision making based on big data and machine learning (ML) algorithms can result in discriminatory decisions against certain protected groups defined upon personal data like gender, race, sexual orientation etc. Such algorithms designed to discover patterns in big data might not only pick up any encoded societal biases in the training data, but even worse, they might reinforce such biases resulting in more severe discrimination. The majority of thus far proposed fairness-aware machine learning approaches focus solely on the pre-, in- or post-processing steps of the machine learning process, that is, input data, learning algorithms or derived models, respectively. However, the fairness problem cannot be isolated to a single step of the ML process. Rather, discrimination is often a result of complex interactions between big data and algorithms, and therefore, a more holistic approach is required. The proposed FAE (Fairness-Aware Ensemble) framework combines fairness-related interventions at both pre- and postprocessing steps of the data analysis process. In the preprocessing step, we tackle the problems of under-representation of the protected group (group imbalance) and of class-imbalance by generating balanced training samples. In the post-processing step, we tackle the problem of class overlapping by shifting the decision boundary in the direction of fairness.