OpenRaiser/Intern-Atlas huggingface dataset:
Automatic event detection in football using tracking data
Venue: arXiv cs.LG 2022
Year: 2022

One of the main shortcomings of event data in football, which has been extensively used for analytics in the recent years, is that it still requires manual collection, thus limiting its availability to a reduced number of tournaments. In this work, we propose a deterministic decision tree-based algorithm to automatically extract football events using tracking data, which consists of two steps: (1) a possession step that evaluates which player was in possession of the ball at each frame in the tracking data, as well as the distinct player configurations during the time intervals where the ball is not in play to inform set piece detection; (2) an event detection step that combines the changes in ball possession computed in the first step with the laws of football to determine in-game events and set pieces. The automatically generated events are benchmarked against manually annotated events and we show that in most event categories the proposed methodology achieves $+90\%$ detection rate across different tournaments and tracking data providers. Finally, we demonstrate how the contextual information offered by tracking data can be leveraged to increase the granularity of auto-detected events, and exhibit how the proposed framework may be used to conduct a myriad of data analyses in football.
Some Reflections on Drawing Causal Inference using Textual Data: Parallels Between Human Subjects and Organized Texts
Venue: arXiv cs.CL 2022
Year: 2022

We examine the role of textual data as study units when conducting causal inference by drawing parallels between human subjects and organized texts. %in human population research. We elaborate on key causal concepts and principles, and expose some ambiguity and sometimes fallacies. To facilitate better framing a causal query, we discuss two strategies: (i) shifting from immutable traits to perceptions of them, and (ii) shifting from some abstract concept/property to its constituent parts, i.e., adopting a constructivist perspective of an abstract concept. We hope this article would raise the awareness of the importance of articulating and clarifying fundamental concepts before delving into developing methodologies when drawing causal inference using textual data.
Image Forgery Detection with Interpretability
Venue: arXiv cs.CV 2022
Year: 2022

In this work, we present a learning based method focusing on the convolutional neural network (CNN) architecture to detect these forgeries. We consider the detection of both copy-move forgeries and inpainting based forgeries. For these, we synthesize our own large dataset. In addition to classification, the focus is also on interpretability of the forgery detection. As the CNN classification yields the image-level label, it is important to understand if forged region has indeed contributed to the classification. For this purpose, we demonstrate using the Grad-CAM heatmap, that in various correctly classified examples, that the forged region is indeed the region contributing to the classification. Interestingly, this is also applicable for small forged regions, as is depicted in our results. Such an analysis can also help in establishing the reliability of the classification.
Flipping the switch on local exploration: Genetic Algorithms with Reversals
Venue: arXiv cs.NE 2022
Year: 2022

One important feature of complex systems are problem domains that have many local minima and substructure. Biological systems manage these local minima by switching between different subsystems depending on their environmental or developmental context. Genetic Algorithms (GA) can mimic this switching property as well as provide a means to overcome problem domain complexity. However, standard GA requires additional operators that will allow for large-scale exploration in a stochastic manner. Gradient-free heuristic search techniques are suitable for providing an optimal solution in the discrete domain to such single objective optimization tasks, particularly compared to gradient-based methods which are noticeably slower. To do this, the authors turn to an optimization problem from the flight scheduling domain. The authors compare the performance of such common gradient-free heuristic search algorithms and propose variants of GAs. The Iterated Chaining (IC) method is also introduced, building upon traditional chaining techniques by triggering multiple local searches instead of the singular action of a mutation operator. The authors will show that the use of multiple local searches can improve performance on local stochastic searches, providing ample opportunity for application to a host of other problem domains. It is observed that the proposed GA variants have the least average cost across all benchmarks including the problem proposed and IC algorithm performs better than its constituents.
Dynamic Time Slot Allocation Algorithm for Quadcopter Swarms
Venue: arXiv cs.RO 2022
Year: 2022

A swarm of quadcopters can perform cooperative tasks, such as monitoring of a large area, more efficiently than a single one. However, to be able to successfully work together, the quadcopters must be aware of the position of the other swarm members, especially to avoid collisions. A quadcopter can share its own position by transmitting it via radio waves and in order to allow multiple quadcopters to communicate effectively, a decentralized channel access protocol is essential. We propose a new dynamic channel access protocol, called Dynamic time slot allocation (DTSA), where the quadcopters share the total channel access time in a non-periodic and decentralized manner. Quadcopters with higher communication demands occupy more time slots than less active ones. Our dynamic approach allows the agents to adapt to changing swarm situations and therefore to act efficiently, as compared to the state-of-the-art periodic channel access protocol, time division multiple access (TDMA). Along with simulations, we also do experiments using real Crazyflie quadcopters to show the improved performance of DTSA as compared to TDMA.
GANSlider: How Users Control Generative Models for Images using Multiple Sliders with and without Feedforward Information
Venue: arXiv cs.HC 2022
Year: 2022

We investigate how multiple sliders with and without feedforward visualizations influence users' control of generative models. In an online study (N=138), we collected a dataset of people interacting with a generative adversarial network (StyleGAN2) in an image reconstruction task. We found that more control dimensions (sliders) significantly increase task difficulty and user actions. Visual feedforward partly mitigates this by enabling more goal-directed interaction. However, we found no evidence of faster or more accurate task performance. This indicates a tradeoff between feedforward detail and implied cognitive costs, such as attention. Moreover, we found that visualizations alone are not always sufficient for users to understand individual control dimensions. Our study quantifies fundamental UI design factors and resulting interaction behavior in this context, revealing opportunities for improvement in the UI design for interactive applications of generative models. We close by discussing design directions and further aspects.
Performance Analysis and Optimization for Jammer-Aided Multi-Antenna UAV Covert Communication
Venue: arXiv cs.IT 2022
Year: 2022

Unmanned aerial vehicles (UAVs) have attracted a lot of research attention because of their high mobility and low cost in serving as temporary aerial base stations (BSs) and providing high data rates for next-generation communication networks. To protect user privacy while avoiding detection by a warden, we investigate a jammer-aided UAV covert communication system, which aims to maximize the user's covert rate with optimized transmit and jamming power. The UAV is equipped with multi-antennas to serve multi-users simultaneously and enhance the Quality of Service. By considering the general composite fading and shadowing channel models, we derive the exact probability density (PDF) and cumulative distribution functions (CDF) of the signal-to-interference-plusnoise ratio (SINR). The obtained PDF and CDF are used to derive the closed-form expressions for detection error probability and covert rate. Furthermore, the covert rate maximization problem is formulated as a Nash bargaining game, and the Nash bargaining solution (NBS) is introduced to investigate the negotiation among users. To solve the NBS, we propose two algorithms, i.e., particle swarm optimization-based and joint twostage power allocation algorithms, to achieve covertness and high data rates under the warden's optimal detection threshold. All formulated problems are proven to be convex, and the complexity is analyzed. The numerical results are presented to verify the theoretical performance analysis and show the effectiveness and success of achieving the covert communication of our algorithms.
Precoding and Decoding Schemes for Downlink MIMO-RSMA with Simultaneous Diagonalization and User Exclusion
Venue: arXiv cs.IT 2022
Year: 2022

In this paper, we consider the precoder design for downlink multiple-input multiple-output (MIMO) rate-splitting multiple access (RSMA) systems. The proposed scheme with simultaneous diagonalization (SD) decomposes the MIMO channel matrices of the users into scalar channels via higher-order generalized singular value decomposition for the common message (CM) and block diagonalization (BD) for the private messages, thereby enabling low-complexity element-by-element successive interference cancellation (SIC) and decoding at the receivers. Furthermore, the proposed SD MIMO-RSMA overcomes a critical limitation in RSMA systems, whereby the achievable rate of the CM is restricted by the users with weak effective MIMO channel for the CM, by excluding a subset of users from decoding the CM. We formulate a non-convex weighted sum rate (WSR) optimization problem for SD MIMO-RSMA and solve it via successive convex approximation to obtain a locally optimal solution. Our simulation results reveal that, for both perfect and imperfect CSI, the proposed SD MIMO-RSMA with user exclusion outperforms baseline MIMO-RSMA schemes and linear BD precoding.
Grammars for Free: Toward Grammar Inference for Ad Hoc Parsers
Venue: arXiv cs.SE 2022
Year: 2022

Ad hoc parsers are everywhere: they appear any time a string is split, looped over, interpreted, transformed, or otherwise processed. Every ad hoc parser gives rise to a language: the possibly infinite set of input strings that the program accepts without going wrong. Any language can be described by a formal grammar: a finite set of rules that can generate all strings of that language. But programmers do not write grammars for ad hoc parsers -- even though they would be eminently useful. Grammars can serve as documentation, aid program comprehension, generate test inputs, and allow reasoning about language-theoretic security. We propose an automatic grammar inference system for ad hoc parsers that would enable all of these use cases, in addition to opening up new possibilities in mining software repositories and bi-directional parser synthesis.
Knowledge Engineering in the Long Game of Artificial Intelligence: The Case of Speech Acts
Venue: arXiv cs.AI 2022
Year: 2022

This paper describes principles and practices of knowledge engineering that enable the development of holistic language-endowed intelligent agents that can function across domains and applications, as well as expand their ontological and lexical knowledge through lifelong learning. For illustration, we focus on dialog act modeling, a task that has been widely pursued in linguistics, cognitive modeling, and statistical natural language processing. We describe an integrative approach grounded in the OntoAgent knowledge-centric cognitive architecture and highlight the limitations of past approaches that isolate dialog from other agent functionalities.
Melody Extraction from Polyphonic Music by Deep Learning Approaches: A Review
Venue: arXiv cs.SD 2022
Year: 2022

Melody extraction is a vital music information retrieval task among music researchers for its potential applications in education pedagogy and the music industry. Melody extraction is a notoriously challenging task due to the presence of background instruments. Also, often melodic source exhibits similar characteristics to that of the other instruments. The interfering background accompaniment with the vocals makes extracting the melody from the mixture signal much more challenging. Until recently, classical signal processing-based melody extraction methods were quite popular among melody extraction researchers. The ability of the deep learning models to model large-scale data and the ability of the models to learn automatic features by exploiting spatial and temporal dependencies inspired many researchers to adopt deep learning models for melody extraction. In this paper, an attempt has been made to review the up-to-date data-driven deep learning approaches for melody extraction from polyphonic music. The available deep models have been categorized based on the type of neural network used and the output representation they use for predicting melody. Further, the architectures of the 25 melody extraction models are briefly presented. The loss functions used to optimize the model parameters of the melody extraction models are broadly categorized into four categories and briefly describe the loss functions used by various melody extraction models. Also, the various input representations adopted by the melody extraction models and the parameter settings are deeply described. A section describing the explainability of the block-box melody extraction deep neural networks is included. The performance of 25 melody extraction methods is compared. The possible future directions to explore/improve the melody extraction methods are also presented in the paper.
Exact Privacy Analysis of the Gaussian Sparse Histogram Mechanism
Venue: arXiv cs.CR 2022
Year: 2022

Sparse histogram methods can be useful for returning differentially private counts of items in large or infinite histograms, large group-by queries, and more generally, releasing a set of statistics with sufficient item counts. We consider the Gaussian version of the sparse histogram mechanism and study the exact $ε,δ$ differential privacy guarantees satisfied by this mechanism. We compare these exact $ε,δ$ parameters to the simpler overestimates used in prior work to quantify the impact of their looser privacy bounds.
The slurk Interaction Server Framework: Better Data for Better Dialog Models
Venue: arXiv cs.CL 2022
Year: 2022

This paper presents the slurk software, a lightweight interaction server for setting up dialog data collections and running experiments. Slurk enables a multitude of settings including text-based, speech and video interaction between two or more humans or humans and bots, and a multimodal display area for presenting shared or private interactive context. The software is implemented in Python with an HTML and JS frontend that can easily be adapted to individual needs. It also provides a setup for pairing participants on common crowdworking platforms such as Amazon Mechanical Turk and some example bot scripts for common interaction scenarios.
Error Correction in ASR using Sequence-to-Sequence Models
Venue: arXiv cs.CL 2022
Year: 2022

Post-editing in Automatic Speech Recognition (ASR) entails automatically correcting common and systematic errors produced by the ASR system. The outputs of an ASR system are largely prone to phonetic and spelling errors. In this paper, we propose to use a powerful pre-trained sequence-to-sequence model, BART, further adaptively trained to serve as a denoising model, to correct errors of such types. The adaptive training is performed on an augmented dataset obtained by synthetically inducing errors as well as by incorporating actual errors from an existing ASR system. We also propose a simple approach to rescore the outputs using word level alignments. Experimental results on accented speech data demonstrate that our strategy effectively rectifies a significant number of ASR errors and produces improved WER results when compared against a competitive baseline. We also highlight a negative result obtained on the related grammatical error correction task in Hindi language showing the limitation in capturing wider context by our proposed model.
AntidoteRT: Run-time Detection and Correction of Poison Attacks on Neural Networks
Venue: arXiv cs.CR 2022
Year: 2022

We study backdoor poisoning attacks against image classification networks, whereby an attacker inserts a trigger into a subset of the training data, in such a way that at test time, this trigger causes the classifier to predict some target class. %There are several techniques proposed in the literature that aim to detect the attack but only a few also propose to defend against it, and they typically involve retraining the network which is not always possible in practice. We propose lightweight automated detection and correction techniques against poisoning attacks, which are based on neuron patterns mined from the network using a small set of clean and poisoned test samples with known labels. The patterns built based on the mis-classified samples are used for run-time detection of new poisoned inputs. For correction, we propose an input correction technique that uses a differential analysis to identify the trigger in the detected poisoned images, which is then reset to a neutral color. Our detection and correction are performed at run-time and input level, which is in contrast to most existing work that is focused on offline model-level defenses. We demonstrate that our technique outperforms existing defenses such as NeuralCleanse and STRIP on popular benchmarks such as MNIST, CIFAR-10, and GTSRB against the popular BadNets attack and the more complex DFST attack.
NoisyMix: Boosting Model Robustness to Common Corruptions
Venue: arXiv cs.LG 2022
Year: 2022

For many real-world applications, obtaining stable and robust statistical performance is more important than simply achieving state-of-the-art predictive test accuracy, and thus robustness of neural networks is an increasingly important topic. Relatedly, data augmentation schemes have been shown to improve robustness with respect to input perturbations and domain shifts. Motivated by this, we introduce NoisyMix, a novel training scheme that promotes stability as well as leverages noisy augmentations in input and feature space to improve both model robustness and in-domain accuracy. NoisyMix produces models that are consistently more robust and that provide well-calibrated estimates of class membership probabilities. We demonstrate the benefits of NoisyMix on a range of benchmark datasets, including ImageNet-C, ImageNet-R, and ImageNet-P. Moreover, we provide theory to understand implicit regularization and robustness of NoisyMix.
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts
Venue: arXiv cs.LG 2022
Year: 2022

PromptSource is a system for creating, sharing, and using natural language prompts. Prompts are functions that map an example from a dataset to a natural language input and target output. Using prompts to train and query language models is an emerging area in NLP that requires new tools that let users develop and refine these prompts collaboratively. PromptSource addresses the emergent challenges in this new setting with (1) a templating language for defining data-linked prompts, (2) an interface that lets users quickly iterate on prompt development by observing outputs of their prompts on many examples, and (3) a community-driven set of guidelines for contributing new prompts to a common pool. Over 2,000 prompts for roughly 170 datasets are already available in PromptSource. PromptSource is available at https://github.com/bigscience-workshop/promptsource.
Harmony: Overcoming the Hurdles of GPU Memory Capacity to Train Massive DNN Models on Commodity Servers
Venue: arXiv cs.DC 2022
Year: 2022

Deep neural networks (DNNs) have grown exponentially in size over the past decade, leaving only those who have massive datacenter-based resources with the ability to develop and train such models. One of the main challenges for the long tail of researchers who might have only limited resources (e.g., a single multi-GPU server) is limited GPU memory capacity compared to model size. The problem is so acute that the memory requirement of training massive DNN models can often exceed the aggregate capacity of all available GPUs on a single server; this problem only gets worse with the trend of ever-growing model sizes. Current solutions that rely on virtualizing GPU memory (by swapping to/from CPU memory) incur excessive swapping overhead. In this paper, we present a new training framework, Harmony, and advocate rethinking how DNN frameworks schedule computation and move data to push the boundaries of training massive models efficiently on a single commodity server. Across various massive DNN models, Harmony is able to reduce swap load by up to two orders of magnitude and obtain a training throughput speedup of up to 7.6x over highly optimized baselines with virtualized memory.
PanoDepth: A Two-Stage Approach for Monocular Omnidirectional Depth Estimation
Venue: arXiv cs.CV 2022
Year: 2022

Omnidirectional 3D information is essential for a wide range of applications such as Virtual Reality, Autonomous Driving, Robotics, etc. In this paper, we propose a novel, model-agnostic, two-stage pipeline for omnidirectional monocular depth estimation. Our proposed framework PanoDepth takes one 360 image as input, produces one or more synthesized views in the first stage, and feeds the original image and the synthesized images into the subsequent stereo matching stage. In the second stage, we propose a differentiable Spherical Warping Layer to handle omnidirectional stereo geometry efficiently and effectively. By utilizing the explicit stereo-based geometric constraints in the stereo matching stage, PanoDepth can generate dense high-quality depth. We conducted extensive experiments and ablation studies to evaluate PanoDepth with both the full pipeline as well as the individual modules in each stage. Our results show that PanoDepth outperforms the state-of-the-art approaches by a large margin for 360 monocular depth estimation.
Adaptive Discrete Communication Bottlenecks with Dynamic Vector Quantization
Venue: arXiv cs.LG 2022
Year: 2022

Vector Quantization (VQ) is a method for discretizing latent representations and has become a major part of the deep learning toolkit. It has been theoretically and empirically shown that discretization of representations leads to improved generalization, including in reinforcement learning where discretization can be used to bottleneck multi-agent communication to promote agent specialization and robustness. The discretization tightness of most VQ-based methods is defined by the number of discrete codes in the representation vector and the codebook size, which are fixed as hyperparameters. In this work, we propose learning to dynamically select discretization tightness conditioned on inputs, based on the hypothesis that data naturally contains variations in complexity that call for different levels of representational coarseness. We show that dynamically varying tightness in communication bottlenecks can improve model performance on visual reasoning and reinforcement learning tasks.