OpenRaiser/Intern-Atlas huggingface dataset:
Fusion-GCN: Multimodal Action Recognition using Graph Convolutional Networks
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we present Fusion-GCN, an approach for multimodal action recognition using Graph Convolutional Networks (GCNs). Action recognition methods based around GCNs recently yielded state-of-the-art performance for skeleton-based action recognition. With Fusion-GCN, we propose to integrate various sensor data modalities into a graph that is trained using a GCN model for multi-modal action recognition. Additional sensor measurements are incorporated into the graph representation, either on a channel dimension (introducing additional node attributes) or spatial dimension (introducing new nodes). Fusion-GCN was evaluated on two public available datasets, the UTD-MHAD- and MMACT datasets, and demonstrates flexible fusion of RGB sequences, inertial measurements and skeleton sequences. Our approach gets comparable results on the UTD-MHAD dataset and improves the baseline on the large-scale MMACT dataset by a significant margin of up to 12.37% (F1-Measure) with the fusion of skeleton estimates and accelerometer measurements.
Degradation and Failure Mechanisms of Complex Systems: Principles
Venue: arXiv cs.SE 2021
Year: 2021

A cyber physical human complex system failure prevents the accomplishment of the systems intended function. The failure of a complex system could be a breakdown of any system hardware, human related factors, application software, or the interaction between these components. Having knowledge about all these three components would allow us to better understand the behavior, interactions, and the associated failure mechanisms of the cyber physical human systems as a whole. In this study, degradation mechanisms in these three components are classified and discussed. The main categories are hardware related degradation mechanisms including mechanical, thermal, chemical, electronic and radiation effects degradation mechanisms. In addition to hardware related degradation mechanisms, human failure modes, software errors, and the failures due to cyber physical human interactions are presented and discussed. This paper covers the main types of failure mechanisms in complex systems and is beneficial for developing conceptual risk and reliability models for complex systems.
Synerise at RecSys 2021: Twitter user engagement prediction with a fast neural model
Venue: arXiv cs.IR 2021
Year: 2021

In this paper we present our 2nd place solution to ACM RecSys 2021 Challenge organized by Twitter. The challenge aims to predict user engagement for a set of tweets, offering an exceptionally large data set of 1 billion data points sampled from over four weeks of real Twitter interactions. Each data point contains multiple sources of information, such as tweet text along with engagement features, user features, and tweet features. The challenge brings the problem close to a real production environment by introducing strict latency constraints in the model evaluation phase: the average inference time for single tweet engagement prediction is limited to 6ms on a single CPU core with 64GB memory. Our proposed model relies on extensive feature engineering performed with methods such as the Efficient Manifold Density Estimator (EMDE) - our previously introduced algorithm based on Locality Sensitive Hashing method, and novel Fourier Feature Encoding, among others. In total, we create numerous features describing a user's Twitter account status and the content of a tweet. In order to adhere to the strict latency constraints, the underlying model is a simple residual feed-forward neural network. The system is a variation of our previous methods which proved successful in KDD Cup 2021, WSDM Challenge 2021, and SIGIR eCom Challenge 2020. We release the source code at: https://github.com/Synerise/recsys-challenge-2021
Overview of the CLEF--2021 CheckThat! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake News
Venue: arXiv cs.CL 2021
Year: 2021

We describe the fourth edition of the CheckThat! Lab, part of the 2021 Conference and Labs of the Evaluation Forum (CLEF). The lab evaluates technology supporting tasks related to factuality, and covers Arabic, Bulgarian, English, Spanish, and Turkish. Task 1 asks to predict which posts in a Twitter stream are worth fact-checking, focusing on COVID-19 and politics (in all five languages). Task 2 asks to determine whether a claim in a tweet can be verified using a set of previously fact-checked claims (in Arabic and English). Task 3 asks to predict the veracity of a news article and its topical domain (in English). The evaluation is based on mean average precision or precision at rank k for the ranking tasks, and macro-F1 for the classification tasks. This was the most popular CLEF-2021 lab in terms of team registrations: 132 teams. Nearly one-third of them participated: 15, 5, and 25 teams submitted official runs for tasks 1, 2, and 3, respectively.
Experience feedback using Representation Learning for Few-Shot Object Detection on Aerial Images
Venue: arXiv cs.CV 2021
Year: 2021

This paper proposes a few-shot method based on Faster R-CNN and representation learning for object detection in aerial images. The two classification branches of Faster R-CNN are replaced by prototypical networks for online adaptation to new classes. These networks produce embeddings vectors for each generated box, which are then compared with class prototypes. The distance between an embedding and a prototype determines the corresponding classification score. The resulting networks are trained in an episodic manner. A new detection task is randomly sampled at each epoch, consisting in detecting only a subset of the classes annotated in the dataset. This training strategy encourages the network to adapt to new classes as it would at test time. In addition, several ideas are explored to improve the proposed method such as a hard negative examples mining strategy and self-supervised clustering for background objects. The performance of our method is assessed on DOTA, a large-scale remote sensing images dataset. The experiments conducted provide a broader understanding of the capabilities of representation learning. It highlights in particular some intrinsic weaknesses for the few-shot object detection task. Finally, some suggestions and perspectives are formulated according to these insights.
GPU Accelerated Batch Multi-Convex Trajectory Optimization for a Rectangular Holonomic Mobile Robot
Venue: arXiv cs.RO 2021
Year: 2021

We present a batch trajectory optimizer that can simultaneously solve hundreds of different instances of the problem in real-time. We consider holonomic robots but relax the assumption of circular base footprint. Our main algorithmic contributions lie in: (i) improving the computational tractability of the underlying non-convex problem and (ii) leveraging batch computation to mitigate initialization bottlenecks and improve solution quality. We achieve both goals by deriving a multi-convex reformulation of the kinematics and collision avoidance constraints. We exploit these structures through an Alternating Minimization approach and show that the resulting batch operation reduces to computing just matrix-vector products that can be trivially accelerated over GPUs. We improve the state-of-the-art in three respects. First, we improve quality of navigation (success-rate, tracking) as compared to baseline approach that relies on computing a single locally optimal trajectory at each control loop. Second, we show that when initialized with trajectory samples from a Gaussian distribution, our batch optimizer outperforms state-of-the-art cross-entropy method in solution quality. Finally, our batch optimizer is several orders of magnitude faster than the conceptually simpler alternative of running different optimization instances in parallel CPU threads. \textbf{Codes:} \url{https://tinyurl.com/a3b99m8}
Searching for Minimal Optimal Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Large neural network models have high predictive power but may suffer from overfitting if the training set is not large enough. Therefore, it is desirable to select an appropriate size for neural networks. The destructive approach, which starts with a large architecture and then reduces the size using a Lasso-type penalty, has been used extensively for this task. Despite its popularity, there is no theoretical guarantee for this technique. Based on the notion of minimal neural networks, we posit a rigorous mathematical framework for studying the asymptotic theory of the destructive technique. We prove that Adaptive group Lasso is consistent and can reconstruct the correct number of hidden nodes of one-hidden-layer feedforward networks with high probability. To the best of our knowledge, this is the first theoretical result establishing for the destructive technique.
Accelerating LSM-Tree with the Dentry Management of File System
Venue: arXiv cs.DS 2021
Year: 2021

The log-structured merge tree (LSM-tree) gains wide popularity in building key-value (KV) stores. It employs logs to back up arriving KV pairs and maintains a few on-disk levels with exponentially increasing capacity limits, resembling a tiered tree-like structure. A level comprises SST files, each of which holds a sequence of sorted KV pairs. From time to time, LSM-tree redeploys KV pairs from a full level to the lower level by compaction, which merge-sorts and moves KV pairs among SST files, thereby incurring substantial disk I/Os. In this paper, we revisit the design of LSM-tree and find that organizing multiple KV pairs in an SST file entails the heavyweight redeployment of actual KV pairs in a compaction. Accordingly we revolutionize the organization of KV pairs by transforming an SST file of KV pairs to an SST directory, in which each KV pair makes into an independent KV file with the key and value as filename and main file contents, respectively. Moving KV pairs in a compaction converts to transferring directory entries (dentrys), which causes concretely fewer disk I/Os. This is the essence of our design named DeLSM. We build a prototype of DeLSM on LevelDB and evaluation results show that it significantly outperforms the state-of-the-art LSM-tree variants in different dimensions.
G-VOM: A GPU Accelerated Voxel Off-Road Mapping System
Venue: arXiv cs.RO 2021
Year: 2021

We present a local 3D voxel mapping framework for off-road path planning and navigation. Our method provides both hard and soft positive obstacle detection, negative obstacle detection, slope estimation, and roughness estimation. By using a 3D array lookup table data structure and by leveraging the GPU it can provide online performance. We then demonstrate the system working on three vehicles, a Clearpath Robotics Warthog, Moose, and a Polaris Ranger, and compare against a set of pre-recorded waypoints. This was done at 4.5 m/s in autonomous operation and 12 m/s in manual operation with a map update rate of 10 Hz. Finally, an open-source ROS implementation is provided. https://github.com/unmannedlab/G-VOM
Comparison of Object Detection Algorithms Using Video and Thermal Images Collected from a UAS Platform: An Application of Drones in Traffic Management
Venue: arXiv cs.CV 2021
Year: 2021

There is a rapid growth of applications of Unmanned Aerial Vehicles (UAVs) in traffic management, such as traffic surveillance, monitoring, and incident detection. However, the existing literature lacks solutions to real-time incident detection while addressing privacy issues in practice. This study explored real-time vehicle detection algorithms on both visual and infrared cameras and conducted experiments comparing their performance. Red Green Blue (RGB) videos and thermal images were collected from a UAS platform along highways in the Tampa, Florida, area. Experiments were designed to quantify the performance of a real-time background subtraction-based method in vehicle detection from a stationary camera on hovering UAVs under free-flow conditions. Several parameters were set in the experiments based on the geometry of the drone and sensor relative to the roadway. The results show that a background subtraction-based method can achieve good detection performance on RGB images (F1 scores around 0.9 for most cases), and a more varied performance is seen on thermal images with different azimuth angles. The results of these experiments will help inform the development of protocols, standards, and guidance for the use of drones to detect highway congestion and provide input for the development of incident detection algorithms.
Consistency Training of Multi-exit Architectures for Sensor Data
Venue: arXiv cs.LG 2021
Year: 2021

Deep neural networks have become larger over the years with increasing demand of computational resources for inference; incurring exacerbate costs and leaving little room for deployment on devices with limited battery and other resources for real-time applications. The multi-exit architectures are type of deep neural network that are interleaved with several output (or exit) layers at varying depths of the model. They provide a sound approach for improving computational time and energy utilization of running a model through producing predictions from early exits. In this work, we present a novel and architecture-agnostic approach for robust training of multi-exit architectures termed consistent exit training. The crux of the method lies in a consistency-based objective to enforce prediction invariance over clean and perturbed inputs. We leverage weak supervision to align model output with consistency training and jointly optimize dual-losses in a multi-task learning fashion over the exits in a network. Our technique enables exit layers to generalize better when confronted with increasing uncertainty, hence, resulting in superior quality-efficiency trade-offs. We demonstrate through extensive evaluation on challenging learning tasks involving sensor data that our approach allows examples to exit earlier with better detection rate and without executing all the layers in a deep model.
The use of multi-agent systems for modeling technological processes
Venue: arXiv cs.MA 2021
Year: 2021

The article is devoted to the issues of using discrete simulation models for modeling some basic technological processes. In the scientific work, models in the form of multi-agent systems have been investigated, which allow us to consider a continuous environment as a set of interacting elements (agents), the behavior of which obeys local functions. The authors describe the basic techniques and general methodology for the development of deterministic agent-based models. The paper considers the use of multi-agent systems for modeling thermal conductivity, taking into account the nonlinearity of the process, in homogeneity of the material and the presence of volumetric heat sources of variable power in it. The obtained scientific results are in good agreement with the generally accepted classical approaches and do not contradict the provisions adopted in the theory of thermal phenomena.
MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research
Venue: arXiv cs.LG 2021
Year: 2021

Progress in deep reinforcement learning (RL) is heavily driven by the availability of challenging benchmarks used for training agents. However, benchmarks that are widely adopted by the community are not explicitly designed for evaluating specific capabilities of RL methods. While there exist environments for assessing particular open problems in RL (such as exploration, transfer learning, unsupervised environment design, or even language-assisted RL), it is generally difficult to extend these to richer, more complex environments once research goes beyond proof-of-concept results. We present MiniHack, a powerful sandbox framework for easily designing novel RL environments. MiniHack is a one-stop shop for RL experiments with environments ranging from small rooms to complex, procedurally generated worlds. By leveraging the full set of entities and environment dynamics from NetHack, one of the richest grid-based video games, MiniHack allows designing custom RL testbeds that are fast and convenient to use. With this sandbox framework, novel environments can be designed easily, either using a human-readable description language or a simple Python interface. In addition to a variety of RL tasks and baselines, MiniHack can wrap existing RL benchmarks and provide ways to seamlessly add additional complexity.
Classification and Adversarial examples in an Overparameterized Linear Model: A Signal Processing Perspective
Venue: arXiv cs.LG 2021
Year: 2021

State-of-the-art deep learning classifiers are heavily overparameterized with respect to the amount of training examples and observed to generalize well on "clean" data, but be highly susceptible to infinitesmal adversarial perturbations. In this paper, we identify an overparameterized linear ensemble, that uses the "lifted" Fourier feature map, that demonstrates both of these behaviors. The input is one-dimensional, and the adversary is only allowed to perturb these inputs and not the non-linear features directly. We find that the learned model is susceptible to adversaries in an intermediate regime where classification generalizes but regression does not. Notably, the susceptibility arises despite the absence of model mis-specification or label noise, which are commonly cited reasons for adversarial-susceptibility. These results are extended theoretically to a random-Fourier-sum setup that exhibits double-descent behavior. In both feature-setups, the adversarial vulnerability arises because of a phenomenon we term spatial localization: the predictions of the learned model are markedly more sensitive in the vicinity of training points than elsewhere. This sensitivity is a consequence of feature lifting and is reminiscent of Gibb's and Runge's phenomena from signal processing and functional analysis. Despite the adversarial susceptibility, we find that classification with these features can be easier than the more commonly studied "independent feature" models.
DOODLER: Determining Out-Of-Distribution Likelihood from Encoder Reconstructions
Venue: arXiv cs.LG 2021
Year: 2021

Deep Learning models possess two key traits that, in combination, make their use in the real world a risky prospect. One, they do not typically generalize well outside of the distribution for which they were trained, and two, they tend to exhibit confident behavior regardless of whether or not they are producing meaningful outputs. While Deep Learning possesses immense power to solve realistic, high-dimensional problems, these traits in concert make it difficult to have confidence in their real-world applications. To overcome this difficulty, the task of Out-Of-Distribution (OOD) Detection has been defined, to determine when a model has received an input from outside of the distribution for which it is trained to operate. This paper introduces and examines a novel methodology, DOODLER, for OOD Detection, which directly leverages the traits which result in its necessity. By training a Variational Auto-Encoder (VAE) on the same data as another Deep Learning model, the VAE learns to accurately reconstruct In-Distribution (ID) inputs, but not to reconstruct OOD inputs, meaning that its failure state can be used to perform OOD Detection. Unlike other work in the area, DOODLER requires only very weak assumptions about the existence of an OOD dataset, allowing for more realistic application. DOODLER also enables pixel-wise segmentations of input images by OOD likelihood, and experimental results show that it matches or outperforms methodologies that operate under the same constraints.
Enhanced Audit Bit Based Distributed Bayesian Detection in the Presence of Strategic Attacks
Venue: arXiv cs.CR 2021
Year: 2021

This paper employs an audit bit based mechanism to mitigate the effect of Byzantine attacks. In this framework, the optimal attacking strategy for intelligent attackers is investigated for the traditional audit bit based scheme (TAS) to evaluate the robustness of the system. We show that it is possible for an intelligent attacker to degrade the performance of TAS to the system without audit bits. To enhance the robustness of the system in the presence of intelligent attackers, we propose an enhanced audit bit based scheme (EAS). The optimal fusion rule for the proposed scheme is derived and the detection performance of the system is evaluated via the probability of error for the system. Simulation results show that the proposed EAS improves the robustness and the detection performance of the system. Moreover, based on EAS, another new scheme called the reduced audit bit based scheme (RAS) is proposed which further improves system performance. We derive the new optimal fusion rule and the simulation results show that RAS outperforms EAS and TAS in terms of both robustness and detection performance of the system. Then, we extend the proposed RAS for a wide-area cluster based distributed wireless sensor networks (CWSNs). Simulation results show that the proposed RAS significantly reduces the communication overhead between the sensors and the FC, which prolongs the lifetime of the network.
Success at high peaks: a multiscale approach combining individual and expedition-wide factors
Venue: arXiv cs.SI 2021
Year: 2021

This work presents a network-based data-driven study of the combination of factors that contribute to success in mountaineering. It simultaneously examines the effects of individual factors such as age, gender, experience etc., as well as expedition-wide factors such as number of camps, ratio of sherpas to paying climbers etc. Specifically, it combines the two perspectives into a multiscale network, i.e., a network of individual climber features within each expedition at the finer scale, and an expedition similarity network on the coarser scale. The latter is represented as a multiplex network where layers encode different factors. The analysis reveals that chances of failure to summit due to fatigue, altitude or logistical problems, drastically reduce when climbing with repeat partners, especially for experienced climbers. Additionally, node-centrality indicates that individual traits of youth and oxygen use are the strongest drivers of success. Further, the learning of network projections enables computation of correlations between intra-expedition networks and corresponding expedition success rates. Of expedition-wide factors, the expedition size and length layers are found to be strongly correlated with success rate. Lastly, community detection on the expedition-similarity network reveals distinct communities where a difference in success rates naturally emerges amongst the communities.