OpenRaiser/Intern-Atlas huggingface dataset:
On Negative Sampling for Audio-Visual Contrastive Learning from Movies
Venue: arXiv cs.CV 2022
Year: 2022

The abundance and ease of utilizing sound, along with the fact that auditory clues reveal a plethora of information about what happens in a scene, make the audio-visual space an intuitive choice for representation learning. In this paper, we explore the efficacy of audio-visual self-supervised learning from uncurated long-form content i.e movies. Studying its differences with conventional short-form content, we identify a non-i.i.d distribution of data, driven by the nature of movies. Specifically, we find long-form content to naturally contain a diverse set of semantic concepts (semantic diversity), where a large portion of them, such as main characters and environments often reappear frequently throughout the movie (reoccurring semantic concepts). In addition, movies often contain content-exclusive artistic artifacts, such as color palettes or thematic music, which are strong signals for uniquely distinguishing a movie (non-semantic consistency). Capitalizing on these observations, we comprehensively study the effect of emphasizing within-movie negative sampling in a contrastive learning setup. Our view is different from those of prior works who consider within-video positive sampling, inspired by the notion of semantic persistency over time, and operate in a short-video regime. Our empirical findings suggest that, with certain modifications, training on uncurated long-form videos yields representations which transfer competitively with the state-of-the-art to a variety of action recognition and audio classification tasks.
"Short on time and big on ideas": Perspectives from Lab Members on DIYBio Work in Community Biolabs
Venue: arXiv cs.HC 2022
Year: 2022

DIYbio challenges the status quo by positioning laboratory biology work outside of traditional institutions. HCI has increasingly explored the DIYbio movement, but we lack insight into sites of practice such as community biolabs. Therefore, we gathered data on eleven community biolabs by interviewing sixteen lab managers and members. These labs represent half of identified organizations in scope worldwide. Participants detailed their practices and motivations, outlining the constraints and opportunities of their community biolabs. We found that lab members conducted technically challenging project work with access to high-end equipment and professional expertise. We found that the unique nature of biowork exacerbated challenges for cooperative work, partially due to the particular time sensitivities of work with living organisms. Building on our findings, we discuss how community biolab members are creating new approaches to laboratory biology and how this has design implications for systems that support non-traditional settings for scientific practice.
Vision-based system for a real-time detection and following of UAV
Venue: arXiv cs.RO 2022
Year: 2022

In this paper a vision-based system for detection, motion tracking and following of Unmanned Aerial Vehicle (UAV) with other UAV (follower) is presented. For detection of an airborne UAV we apply a convolutional neural network YOLO trained on a collected and processed dataset of 10,000 images. The trained network is capable of detecting various multirotor UAVs in indoor, outdoor and simulation environments. Furthermore, detection results are improved with Kalman filter which ensures steady and reliable information about position and velocity of a target UAV. Preserving the target UAV in the field of view (FOV) and at required distance is accomplished by a simple nonlinear controller based on visual servoing strategy. The proposed system achieves a real-time performance on Neural Compute Stick 2 with a speed of 20 frames per second (FPS) for the detection of an UAV. Justification and efficiency of the developed vision-based system are confirmed in Gazebo simulation experiment where the target UAV is executing a 3D trajectory in a shape of number eight.
Enumerating Connected Dominating Sets
Venue: arXiv cs.CC 2022
Year: 2022

The question to enumerate all inclusion-minimal connected dominating sets in a graph of order $n$ in time significantly less than $2^n$ is an open question that was asked in many places. We answer this question affirmatively, by providing an enumeration algorithm that runs in time $\mathcal{O}(1.9896^n)$, using polynomial space only. The key to this result is the consideration of this enumeration problem on 2-degenerate graphs, which is proven to be possible in time $\mathcal{O}(1.9767^n)$. We also show new lower bound results by constructing a family of graphs of order $n$ with $Ω(1.4890^n)$ minimal connected dominating sets, while previous examples achieved $Ω(1.4422^n)$. Our construction results in lower bounds for a few special graph classes. We also address essential questions concerning output-sensitive enumeration. Namely, we give reasons why our algorithm cannot be turned into an enumeration algorithm that guarantees polynomial delay without much efforts. More precisely, we prove that it is NP-complete to decide, given a graph $G$ and a vertex set $U$, if there exists a minimal connected dominating set $D$ with $U\subseteq D$, even if $G$ is known to be 2-degenerate. Our reduction also shows that even any subexponential delay is not easy to achieve for enumerating minimal connected dominating sets. Another reduction shows that no FPT-algorithms can be expected for this extension problem concerning minimal connected dominating sets, parameterized by $|U|$. We also relate our enumeration problem to the famous open Hitting Set Transversal problem, which can be phrased in our context as the question to enumerate all minimal dominating sets of a graph with polynomial delay by showing that a polynomial-delay enumeration algorithm for minimal connected dominating sets implies an affirmative algorithmic solution to the Hitting Set Transversal problem.
Perceptual Visibility Model for Temporal Contrast Changes in Periphery
Venue: arXiv cs.GR 2022
Year: 2022

Modeling perception is critical for many applications and developments in computer graphics to optimize and evaluate content generation techniques. Most of the work to date has focused on central (foveal) vision. However, this is insufficient for novel wide-field-of-view display devices, such as virtual and augmented reality headsets. Furthermore, the perceptual models proposed for the fovea do not readily extend to the off-center, peripheral visual field, where human perception is drastically different. In this paper, we focus on modeling the temporal aspect of visual perception in the periphery. We present new psychophysical experiments that measure the sensitivity of human observers to different spatio-temporal stimuli across a wide field of view. We use the collected data to build a perceptual model for the visibility of temporal changes at different eccentricities in complex video content. Finally, we discuss, demonstrate, and evaluate several problems that can be addressed using our technique. First, we show how our model enables injecting new content into the periphery without distracting the viewer, and we discuss the link between the model and human attention. Second, we demonstrate how foveated rendering methods can be evaluated and optimized to limit the visibility of temporal aliasing.
To Know by the Company Words Keep and What Else Lies in the Vicinity
Venue: arXiv cs.CL 2022
Year: 2022

The development of state-of-the-art (SOTA) Natural Language Processing (NLP) systems has steadily been establishing new techniques to absorb the statistics of linguistic data. These techniques often trace well-known constructs from traditional theories, and we study these connections to close gaps around key NLP methods as a means to orient future work. For this, we introduce an analytic model of the statistics learned by seminal algorithms (including GloVe and Word2Vec), and derive insights for systems that use these algorithms and the statistics of co-occurrence, in general. In this work, we derive -- to the best of our knowledge -- the first known solution to Word2Vec's softmax-optimized, skip-gram algorithm. This result presents exciting potential for future development as a direct solution to a deep learning (DL) language model's (LM's) matrix factorization. However, we use the solution to demonstrate a seemingly-universal existence of a property that word vectors exhibit and which allows for the prophylactic discernment of biases in data -- prior to their absorption by DL models. To qualify our work, we conduct an analysis of independence, i.e., on the density of statistical dependencies in co-occurrence models, which in turn renders insights on the distributional hypothesis' partial fulfillment by co-occurrence statistics.
AnimalTrack: A Benchmark for Multi-Animal Tracking in the Wild
Venue: arXiv cs.CV 2022
Year: 2022

Multi-animal tracking (MAT), a multi-object tracking (MOT) problem, is crucial for animal motion and behavior analysis and has many crucial applications such as biology, ecology and animal conservation. Despite its importance, MAT is largely under-explored compared to other MOT problems such as multi-human tracking due to the scarcity of dedicated benchmarks. To address this problem, we introduce AnimalTrack, a dedicated benchmark for multi-animal tracking in the wild. Specifically, AnimalTrack consists of 58 sequences from a diverse selection of 10 common animal categories. On average, each sequence comprises of 33 target objects for tracking. In order to ensure high quality, every frame in AnimalTrack is manually labeled with careful inspection and refinement. To our best knowledge, AnimalTrack is the first benchmark dedicated to multi-animal tracking. In addition, to understand how existing MOT algorithms perform on AnimalTrack and provide baselines for future comparison, we extensively evaluate 14 state-of-the-art representative trackers. The evaluation results demonstrate that, not surprisingly, most of these trackers become degenerated due to the differences between pedestrians and animals in various aspects (e.g., pose, motion, and appearance), and more efforts are desired to improve multi-animal tracking. We hope that AnimalTrack together with evaluation and analysis will foster further progress on multi-animal tracking. The dataset and evaluation as well as our analysis will be made available at https://hengfan2010.github.io/projects/AnimalTrack/.
FEDIC: Federated Learning on Non-IID and Long-Tailed Data via Calibrated Distillation
Venue: arXiv cs.LG 2022
Year: 2022

Federated learning provides a privacy guarantee for generating good deep learning models on distributed clients with different kinds of data. Nevertheless, dealing with non-IID data is one of the most challenging problems for federated learning. Researchers have proposed a variety of methods to eliminate the negative influence of non-IIDness. However, they only focus on the non-IID data provided that the universal class distribution is balanced. In many real-world applications, the universal class distribution is long-tailed, which causes the model seriously biased. Therefore, this paper studies the joint problem of non-IID and long-tailed data in federated learning and proposes a corresponding solution called Federated Ensemble Distillation with Imbalance Calibration (FEDIC). To deal with non-IID data, FEDIC uses model ensemble to take advantage of the diversity of models trained on non-IID data. Then, a new distillation method with logit adjustment and calibration gating network is proposed to solve the long-tail problem effectively. We evaluate FEDIC on CIFAR-10-LT, CIFAR-100-LT, and ImageNet-LT with a highly non-IID experimental setting, in comparison with the state-of-the-art methods of federated learning and long-tail learning. Our code is available at https://github.com/shangxinyi/FEDIC.
Gender neutrality in robots: An open living review framework
Venue: arXiv cs.HC 2022
Year: 2022

Gender is a primary characteristic by which people organize themselves. Previous research has shown that people tend to unknowingly ascribe gender to robots based on features of their embodiment. Yet, robots are not necessarily ascribed the same, or any, gender by different people. Indeed, robots may be ascribed non-human genders or used as "genderless" alternatives. This underlies the notion of gender neutrality in robots: neither masculine nor feminine but somewhere in between or even beyond gender. Responding to calls for gender as a locus of study within robotics, we offer a framework for conducting an open living review to be updated periodically as work emerges. Significantly, we provide an open, formalized submission process and open access dataset of research on gender neutrality in robots. This novel and timely approach to consensus-building is expected to pave the way for similar endeavours on other key topics within human-robot interaction research.
Reliable Label Correction is a Good Booster When Learning with Extremely Noisy Labels
Venue: arXiv cs.CV 2022
Year: 2022

Learning with noisy labels has aroused much research interest since data annotations, especially for large-scale datasets, may be inevitably imperfect. Recent approaches resort to a semi-supervised learning problem by dividing training samples into clean and noisy sets. This paradigm, however, is prone to significant degeneration under heavy label noise, as the number of clean samples is too small for conventional methods to behave well. In this paper, we introduce a novel framework, termed as LC-Booster, to explicitly tackle learning under extreme noise. The core idea of LC-Booster is to incorporate label correction into the sample selection, so that more purified samples, through the reliable label correction, can be utilized for training, thereby alleviating the confirmation bias. Experiments show that LC-Booster advances state-of-the-art results on several noisy-label benchmarks, including CIFAR-10, CIFAR-100, Clothing1M and WebVision. Remarkably, under the extreme 90\% noise ratio, LC-Booster achieves 92.9\% and 48.4\% accuracy on CIFAR-10 and CIFAR-100, surpassing state-of-the-art methods by a large margin.
Aggregation of Stack Trace Similarities for Crash Report Deduplication
Venue: arXiv cs.SE 2022
Year: 2022

The automatic collection of stack traces in bug tracking systems is an integral part of many software projects and their maintenance. However, such reports often contain a lot of duplicates, and the problem of de-duplicating them into groups arises. In this paper, we propose a new approach to solve the deduplication task and report on its use on the real-world data from JetBrains, a leading developer of IDEs and other software. Unlike most of the existing methods, which assign the incoming stack trace to a particular group in which a single most similar stack trace is located, we use the information about all the calculated similarities to the group, as well as the information about the timestamp of the stack traces. This approach to aggregating all available information shows significantly better results compared to existing solutions. The aggregation improved the results over the state-of-the-art solutions by 15 percentage points in the Recall Rate Top-1 metric on the existing NetBeans dataset and by 8 percentage points on the JetBrains data. Additionally, we evaluated a simpler k-Nearest Neighbors approach to aggregation and showed that it cannot reach the same levels of improvement. Finally, we studied what features from the aggregation contributed the most towards better quality to understand which of them to develop further. We publish the implementation of the suggested approach, and will release the newly collected industrial dataset upon acceptance to facilitate further research in the area.
An attention model for the formation of collectives in real-world domains
Venue: arXiv cs.AI 2022
Year: 2022

We consider the problem of forming collectives of agents for real-world applications aligned with Sustainable Development Goals (e.g., shared mobility, cooperative learning). We propose a general approach for the formation of collectives based on a novel combination of an attention model and an integer linear program (ILP). In more detail, we propose an attention encoder-decoder model that transforms a collective formation instance to a weighted set packing problem, which is then solved by an ILP. Results on two real-world domains (i.e., ridesharing and team formation for cooperative learning) show that our approach provides solutions that are comparable (in terms of quality) to the ones produced by state-of-the-art approaches specific to each domain. Moreover, our solution outperforms the most recent general approach for forming collectives based on Monte Carlo tree search.
A New Evaluation Method: Evaluation Data and Metrics for Chinese Grammar Error Correction
Venue: arXiv cs.CL 2022
Year: 2022

As a fundamental task in natural language processing, Chinese Grammatical Error Correction (CGEC) has gradually received widespread attention and become a research hotspot. However, one obvious deficiency for the existing CGEC evaluation system is that the evaluation values are significantly influenced by the Chinese word segmentation results or different language models. The evaluation values of the same error correction model can vary considerably under different word segmentation systems or different language models. However, it is expected that these metrics should be independent of the word segmentation results and language models, as they may lead to a lack of uniqueness and comparability in the evaluation of different methods. To this end, we propose three novel evaluation metrics for CGEC in two dimensions: reference-based and reference-less. In terms of the reference-based metric, we introduce sentence-level accuracy and char-level BLEU to evaluate the corrected sentences. Besides, in terms of the reference-less metric, we adopt char-level meaning preservation to measure the semantic preservation degree of the corrected sentences. We deeply evaluate and analyze the reasonableness and validity of the three proposed metrics, and we expect them to become a new standard for CGEC.
Approximating Permutations with Neural Network Components for Travelling Photographer Problem
Venue: arXiv cs.LG 2022
Year: 2022

Most of the current inference techniques rely upon Bayesian inference on Probabilistic Graphical Models of observations and do predictions and classification on observations. However, there is very little literature on the mining of relationships between observations and building models of the relationship between sets of observations or the generating context of the observations. Moreover, event understanding of machines with observation inputs needs to understand the relationship between observations. Thus there is a crucial need to build models and develop effective data structures to accumulate and organize relationships between observations. Given a PGM model, this paper attempts to fit a permutation of states to a sequence of observation tokens (The Travelling Photographer Problem). We have devised a machine learning inspired architecture for randomized approximation of state permutation, facilitating parallelization of heuristic search of permutations. As a result, our algorithm can solve The Travelling Photographer Problem with minimal error. Furthermore, we demonstrate that by mimicking machine learning components such as normalization, dropout, and lambda layer with a randomized algorithm, we can devise an architecture that solves TPP, a permutation NP-Hard problem. Other than TPP, we can also provide a 2-Local improvement heuristic for the Travelling Salesman Problem (TSP) with similar ideas.
Probing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders
Venue: arXiv cs.CL 2022
Year: 2022

Pretrained multilingual language models (LMs) can be successfully transformed into multilingual sentence encoders (SEs; e.g., LaBSE, xMPNet) via additional fine-tuning or model distillation with parallel data. However, it remains unclear how to best leverage them to represent sub-sentence lexical items (i.e., words and phrases) in cross-lingual lexical tasks. In this work, we probe SEs for the amount of cross-lingual lexical knowledge stored in their parameters, and compare them against the original multilingual LMs. We also devise a simple yet efficient method for exposing the cross-lingual lexical knowledge by means of additional fine-tuning through inexpensive contrastive learning that requires only a small amount of word translation pairs. Using bilingual lexical induction (BLI), cross-lingual lexical semantic similarity, and cross-lingual entity linking as lexical probing tasks, we report substantial gains on standard benchmarks (e.g., +10 Precision@1 points in BLI). The results indicate that the SEs such as LaBSE can be 'rewired' into effective cross-lingual lexical encoders via the contrastive learning procedure, and that they contain more cross-lingual lexical knowledge than what 'meets the eye' when they are used as off-the-shelf SEs. This way, we also provide an effective tool for harnessing 'covert' multilingual lexical knowledge hidden in multilingual sentence encoders.
Flexible skyline: overview and applicability
Venue: arXiv cs.DB 2022
Year: 2022

Ranking (or top-k) and skyline queries are the most popular approaches used to extract interesting data from large datasets. The first one is based on a scoring function to evaluate and rank tuples. Its computation is fast, but it is sensitive to the choice of the evaluating function. Skyline queries are based on the idea of dominance and the result is the set of all non-dominated tuples. This is a very interesting approach, but it can't allow to control the cardinality of the output. Recent researches discovered more techniques to compensate for these drawbacks. In particular, this paper will focus on the flexible skyline approach.
Blockchain Applicability for the Internet of Things: Performance and Scalability Challenges and Solutions
Venue: arXiv cs.CR 2022
Year: 2022

Blockchain has recently been able to draw wider attention throughout the research community. Since its emergence, the world has seen the mind-blowing expansion of this new technology, which was initially developed as a pawn of digital currency more than a decade back. A self-administering ledger that ensures extensive data immutability over the peer-to-peer network has made it attractive for cybersecurity applications such as a sensor-enabled system called the Internet of things (IoT). Brand new challenges and questions now demand solutions as huge IoT devices are now online in a distributed fashion to ease our everyday lives. After being motivated by those challenges, the work here has figured out the issues and perspectives an IoT infrastructure can suffer because of the wrong choice of blockchain technology. Though it may look like a typical review, however, unlike that, this paper targets sorting out the specific security challenges of the blockchain-IoT eco-system through critical findings and applicable use-cases. Therefore, the contribution includes directing Blockchain architects, designers, and researchers in the broad domain to select the unblemished combinations of Blockchain-powered IoT applications. In addition, the paper promises to bring a deep insight into the state-of-the-art Blockchain platforms, namely Ethereum, Hyperledger, and IOTA, to exhibit the respective challenges, constraints, and prospects in terms of performance and scalability.