OpenRaiser/Intern-Atlas huggingface dataset:
RWF-2000: An Open Large Scale Video Database for Violence Detection
Venue: arXiv cs.CV 2020
Year: 2020

In recent years, surveillance cameras are widely deployed in public places, and the general crime rate has been reduced significantly due to these ubiquitous devices. Usually, these cameras provide cues and evidence after crimes are conducted, while they are rarely used to prevent or stop criminal activities in time. It is both time and labor consuming to manually monitor a large amount of video data from surveillance cameras. Therefore, automatically recognizing violent behaviors from video signals becomes essential. This paper summarizes several existing video datasets for violence detection and proposes the RWF-2000 database with 2,000 videos captured by surveillance cameras in real-world scenes. Also, we present a new method that utilizes both the merits of 3D-CNNs and optical flow, namely Flow Gated Network. The proposed approach obtains an accuracy of 87.25% on the test set of our proposed database. The database and source codes are currently open to access.
Training a code-switching language model with monolingual data
Venue: arXiv cs.CL 2020
Year: 2020

A lack of code-switching data complicates the training of code-switching (CS) language models. We propose an approach to train such CS language models on monolingual data only. By constraining and normalizing the output projection matrix in RNN-based language models, we bring embeddings of different languages closer to each other. Numerical and visualization results show that the proposed approaches remarkably improve the performance of CS language models trained on monolingual data. The proposed approaches are comparable or even better than training CS language models with artificially generated CS data. We additionally use unsupervised bilingual word translation to analyze whether semantically equivalent words in different languages are mapped together.
The RaPID-OMEGA system: Room and Proctor Intelligent Decider for large scale tests programming
Venue: arXiv cs.MS 2020
Year: 2020

We present the mathematical modeling for the problem of choosing rooms and proctoring crews for massive tests, together with its implementation as the open-box system RaPID-Omega. The mathematical model is a binary integer programming problem: a combination of the 0-1 Knapsack problem and the job-assignment problem. The model makes decisions according the following criteria in order of priority: minimization of labor-hours, maximization of equity in the distribution of duties and maximization of the proctoring quality. The software is a digital solution for the aforementioned problem, which is a common need in educational institutions offering large, coordinated, lower-division courses. The system can be downloaded from \url{https://sites.google.com/a/unal.edu.co/fernando-a-morales-j/home/research/software}
Generalized Flexible Hybrid Cable-Driven Robot (HCDR): Modeling, Control, and Analysis
Venue: arXiv cs.RO 2020
Year: 2020

This paper presents a generalized flexible Hybrid Cable-Driven Robot (HCDR). For the proposed HCDR, the derivation of the equations of motion and proof provide a very effective way to find items for generalized system modeling. The proposed dynamic modeling approach avoids the drawback of traditional methods and can be easily extended to other types of hybrid robots, such as a robot arm mounted on an aircraft platform. Additionally, another goal of this paper is to develop integrated control systems to reduce vibrations and improve the accuracy and performance of the HCDR. To achieve this goal, redundancy resolution, stiffness optimization, and control strategies are studied. The proposed optimization problem and algorithm address the limitations of existing stiffness optimization approaches. Three types of control architecture are proposed, and their performances (i.e., reducing undesirable vibrations and trajectory tracking errors, especially for the end-effector) are evaluated using several well-designed case studies. Results show that the fully integrated control strategy can improve the tracking performance of the end-effector significantly.
Self-Supervised Learning of State Estimation for Manipulating Deformable Linear Objects
Venue: arXiv cs.RO 2020
Year: 2020

We demonstrate model-based, visual robot manipulation of linear deformable objects. Our approach is based on a state-space representation of the physical system that the robot aims to control. This choice has multiple advantages, including the ease of incorporating physics priors in the dynamics model and perception model, and the ease of planning manipulation actions. In addition, physical states can naturally represent object instances of different appearances. Therefore, dynamics in the state space can be learned in one setting and directly used in other visually different settings. This is in contrast to dynamics learned in pixel space or latent space, where generalization to visual differences are not guaranteed. Challenges in taking the state-space approach are the estimation of the high-dimensional state of a deformable object from raw images, where annotations are very expensive on real data, and finding a dynamics model that is both accurate, generalizable, and efficient to compute. We are the first to demonstrate self-supervised training of rope state estimation on real images, without requiring expensive annotations. This is achieved by our novel self-supervising learning objective, which is generalizable across a wide range of visual appearances. With estimated rope states, we train a fast and differentiable neural network dynamics model that encodes the physics of mass-spring systems. Our method has a higher accuracy in predicting future states compared to models that do not involve explicit state estimation and do not use any physics prior, while only using 3\% of training data. We also show that our approach achieves more efficient manipulation, both in simulation and on a real robot, when used within a model predictive controller.
Integrality of Linearizations of Polynomials over Binary Variables using Additional Monomials
Venue: arXiv cs.DM 2020
Year: 2020

Polynomial optimization problems over binary variables can be expressed as integer programs using a linearization with extra monomials in addition to those arising in the given polynomial. We characterize when such a linearization yields an integral relaxation polytope, generalizing work by Del Pia and Khajavirad (SIAM Journal on Optimization, 2018) and Buchheim, Crama and Rodríguez-Heck (European Journal of Operations Research, 2019). We also present an algorithm that finds these extra monomials for a given polynomial to yield an integral relaxation polytope or determines that no such set of extra monomials exists. In the former case, our approach yields an algorithm to solve the given polynomial optimization problem as a compact LP, and we complement this with a purely combinatorial algorithm.
Delta-stepping SSSP: from Vertices and Edges to GraphBLAS Implementations
Venue: arXiv cs.DS 2020
Year: 2020

GraphBLAS is an interface for implementing graph algorithms. Algorithms implemented using the GraphBLAS interface are cast in terms of linear algebra-like operations. However, many graph algorithms are canonically described in terms of operations on vertices and/or edges. Despite the known duality between these two representations, the differences in the way algorithms are described using the two approaches can pose considerable difficulties in the adoption of the GraphBLAS as standard interface for development. This paper investigates a systematic approach for translating a graph algorithm described in the canonical vertex and edge representation into an implementation that leverages the GraphBLAS interface. We present a two-step approach to this problem. First, we express common vertex- and edge-centric design patterns using a linear algebraic language. Second, we map this intermediate representation to the GraphBLAS interface. We illustrate our approach by translating the delta-stepping single source shortest path algorithm from its canonical description to a GraphBLAS implementation, and highlight lessons learned when implementing using GraphBLAS.
Benanza: Automatic $μ$Benchmark Generation to Compute "Lower-bound" Latency and Inform Optimizations of Deep Learning Models on GPUs
Venue: arXiv cs.LG 2020
Year: 2020

As Deep Learning (DL) models have been increasingly used in latency-sensitive applications, there has been a growing interest in improving their response time. An important venue for such improvement is to profile the execution of these models and characterize their performance to identify possible optimization opportunities. However, the current profiling tools lack the highly desired abilities to characterize ideal performance, identify sources of inefficiency, and quantify the benefits of potential optimizations. Such deficiencies have led to slow characterization/optimization cycles that cannot keep up with the fast pace at which new DL models are introduced. We propose Benanza, a sustainable and extensible benchmarking and analysis design that speeds up the characterization/optimization cycle of DL models on GPUs. Benanza consists of four major components: a model processor that parses models into an internal representation, a configurable benchmark generator that automatically generates micro-benchmarks given a set of models, a database of benchmark results, and an analyzer that computes the "lower-bound" latency of DL models using the benchmark data and informs optimizations of model execution. The "lower-bound" latency metric estimates the ideal model execution on a GPU system and serves as the basis for identifying optimization opportunities in frameworks or system libraries. We used Benanza to evaluate 30 ONNX models in MXNet, ONNX Runtime, and PyTorch on 7 GPUs ranging from Kepler to the latest Turing, and identified optimizations in parallel layer execution, cuDNN convolution algorithm selection, framework inefficiency, layer fusion, and using Tensor Cores.
Self-supervised GAN: Analysis and Improvement with Multi-class Minimax Game
Venue: arXiv cs.CV 2020
Year: 2020

Self-supervised (SS) learning is a powerful approach for representation learning using unlabeled data. Recently, it has been applied to Generative Adversarial Networks (GAN) training. Specifically, SS tasks were proposed to address the catastrophic forgetting issue in the GAN discriminator. In this work, we perform an in-depth analysis to understand how SS tasks interact with learning of generator. From the analysis, we identify issues of SS tasks which allow a severely mode-collapsed generator to excel the SS tasks. To address the issues, we propose new SS tasks based on a multi-class minimax game. The competition between our proposed SS tasks in the game encourages the generator to learn the data distribution and generate diverse samples. We provide both theoretical and empirical analysis to support that our proposed SS tasks have better convergence property. We conduct experiments to incorporate our proposed SS tasks into two different GAN baseline models. Our approach establishes state-of-the-art FID scores on CIFAR-10, CIFAR-100, STL-10, CelebA, Imagenet $32\times32$ and Stacked-MNIST datasets, outperforming existing works by considerable margins in some cases. Our unconditional GAN model approaches performance of conditional GAN without using labeled data. Our code: https://github.com/tntrung/msgan
On the Age of Information in Multi-Source Queueing Models
Venue: arXiv cs.IT 2020
Year: 2020

Freshness of status update packets is essential for enabling services where a destination needs the most recent measurements of various sensors. In this paper, we study the information freshness of single-server multi-source queueing models under a first-come first-served (FCFS) serving policy. In the considered model, each source independently generates status update packets according to a Poisson process. The information freshness of the status updates of each source is evaluated by the average age of information (AoI). We derive an exact expression for the average AoI for the case with exponentially distributed service time, i.e., for a multi-source M/M/1 queueing model. Moreover, we derive three approximate expressions for the average AoI for a multi-source M/G/1 queueing model having a general service time distribution. Simulation results are provided to validate the derived exact average AoI expression, to assess the tightness of the proposed approximations, and to demonstrate the AoI behavior for different system parameters.
Loss Aware Post-training Quantization
Venue: arXiv cs.LG 2020
Year: 2020

Neural network quantization enables the deployment of large models on resource-constrained devices. Current post-training quantization methods fall short in terms of accuracy for INT4 (or lower) but provide reasonable accuracy for INT8 (or above). In this work, we study the effect of quantization on the structure of the loss landscape. Additionally, we show that the structure is flat and separable for mild quantization, enabling straightforward post-training quantization methods to achieve good results. We show that with more aggressive quantization, the loss landscape becomes highly non-separable with steep curvature, making the selection of quantization parameters more challenging. Armed with this understanding, we design a method that quantizes the layer parameters jointly, enabling significant accuracy improvement over current post-training quantization methods. Reference implementation is available at https://github.com/ynahshan/nn-quantization-pytorch/tree/master/lapq
A Sketch-Based System for Human-Guided Constrained Object Manipulation
Venue: arXiv cs.HC 2020
Year: 2020

In this paper, we present an easy to use sketch-based interface to extract geometries and generate affordance files from 3D point clouds for robot-object interaction tasks. Using our system, even novice users can perform robot task planning by employing such sketch tools. Our focus in this paper is employing human-in-the-loop approach to assist in the generation of more accurate affordance templates and guidance of robot through the task execution process. Since we do not employ any unsupervised learning to generate affordance templates, our system performs much faster and is more versatile for template generation. Our system is based on the extraction of geometries for generalized cylindrical and cuboid shapes, after extracting the geometries, affordances are generated for objects by applying simple sketches. We evaluated our technique by asking users to define affordances by employing sketches on the 3D scenes of a door handle and a drawer handle and used the resulting extracted affordance template files to perform the tasks of turning a door handle and opening a drawer by the robot.
Provable Filter Pruning for Efficient Neural Networks
Venue: arXiv cs.LG 2020
Year: 2020

We present a provable, sampling-based approach for generating compact Convolutional Neural Networks (CNNs) by identifying and removing redundant filters from an over-parameterized network. Our algorithm uses a small batch of input data points to assign a saliency score to each filter and constructs an importance sampling distribution where filters that highly affect the output are sampled with correspondingly high probability. In contrast to existing filter pruning approaches, our method is simultaneously data-informed, exhibits provable guarantees on the size and performance of the pruned network, and is widely applicable to varying network architectures and data sets. Our analytical bounds bridge the notions of compressibility and importance of network structures, which gives rise to a fully-automated procedure for identifying and preserving filters in layers that are essential to the network's performance. Our experimental evaluations on popular architectures and data sets show that our algorithm consistently generates sparser and more efficient models than those constructed by existing filter pruning approaches.
Learning to Predict More Accurate Text Instances for Scene Text Detection
Venue: arXiv cs.CV 2020
Year: 2020

At present, multi-oriented text detection methods based on deep neural network have achieved promising performances on various benchmarks. Nevertheless, there are still some difficulties for arbitrary shape text detection, especially for a simple and proper representation of arbitrary shape text instances. In this paper, a pixel-based text detector is proposed to facilitate the representation and prediction of text instances with arbitrary shapes in a simple manner. Firstly, to alleviate the effect of the target vertex sorting and achieve the direct regression of arbitrary shape text instances, the starting-point-independent coordinates regression loss is proposed. Furthermore, to predict more accurate text instances, the text instance accuracy loss is proposed as an assistant task to refine the predicted coordinates under the guidance of IoU. To evaluate the effectiveness of our detector, extensive experiments have been carried on public benchmarks which contain arbitrary shape text instances and multi-oriented text instances. We obtain 84.8% of F-measure on Total-Text benchmark. The results show that our method can reach state-of-the-art performance.
Oriented Boxes for Accurate Instance Segmentation
Venue: arXiv cs.CV 2020
Year: 2020

State-of-the-art instance-aware semantic segmentation algorithms use axis-aligned bounding boxes as an intermediate processing step to infer the final instance mask output. This often leads to coarse and inaccurate mask proposals due to the following reasons: Axis-aligned boxes have a high background to foreground pixel-ratio, there is a strong variation of mask targets with respect to the underlying box, and neighboring instances frequently reach into the axis-aligned bounding box of the instance mask of interest. In this work, we overcome these problems by proposing to use oriented boxes as the basis to infer instance masks. We show that oriented instance segmentation improves the mask predictions, especially when objects are diagonally aligned, touching, or overlapping each other. We evaluate our model on the D2S and Screws datasets and show that we can significantly improve the mask accuracy by 10% and 12% mAP compared to instance segmentation using axis-aligned bounding boxes, respectively. On the newly introduced Pill Bags dataset we outperform the baseline using only 10% of the mask annotations.
MaskedFusion: Mask-based 6D Object Pose Estimation
Venue: arXiv cs.CV 2020
Year: 2020

MaskedFusion is a framework to estimate the 6D pose of objects using RGB-D data, with an architecture that leverages multiple sub-tasks in a pipeline to achieve accurate 6D poses. 6D pose estimation is an open challenge due to complex world objects and many possible problems when capturing data from the real world, e.g., occlusions, truncations, and noise in the data. Achieving accurate 6D poses will improve results in other open problems like robot grasping or positioning objects in augmented reality. MaskedFusion improves the state-of-the-art by using object masks to eliminate non-relevant data. With the inclusion of the masks on the neural network that estimates the 6D pose of an object we also have features that represent the object shape. MaskedFusion is a modular pipeline where each sub-task can have different methods that achieve the objective. MaskedFusion achieved 97.3% on average using the ADD metric on the LineMOD dataset and 93.3% using the ADD-S AUC metric on YCB-Video Dataset, which is an improvement, compared to the state-of-the-art methods. The code is available on GitHub (https://github.com/kroglice/MaskedFusion).
Combined Covers and Beth Definability (Extended Version)
Venue: arXiv cs.LO 2020
Year: 2020

In ESOP 2008, Gulwani and Musuvathi introduced a notion of cover and exploited it to handle infinite-state model checking problems. Motivated by applications to the verification of data-aware processes, we proved in a previous paper that covers are strictly related to model completions, a well-known topic in model theory. In this paper we investigate cover transfer to theory combinations in the disjoint signatures case. We prove that for convex theories, cover algorithms can be transferred to theory combinations under the same hypothesis (equality interpolation property aka strong amalgamation property) needed to transfer quantifier-free interpolation. In the non-convex case, we show by a counterexample that covers may not exist in the combined theories, even in case combined quantifier-free interpolants do exist. However, we exhibit a cover transfer algorithm operating also in the non-convex case for special kinds of theory combinations; these combinations (called `tame combinations') concern multi-sorted theories arising in many model-checking applications (in particular, the ones oriented to verification of data-aware processes).
Cross-modal supervised learning for better acoustic representations
Venue: arXiv cs.CV 2020
Year: 2020

Obtaining large-scale human-labeled datasets to train acoustic representation models is a very challenging task. On the contrary, we can easily collect data with machine-generated labels. In this work, we propose to exploit machine-generated labels to learn better acoustic representations, based on the synchronization between vision and audio. Firstly, we collect a large-scale video dataset with 15 million samples, which totally last 16,320 hours. Each video is 3 to 5 seconds in length and annotated automatically by publicly available visual and audio classification models. Secondly, we train various classical convolutional neural networks (CNNs) including VGGish, ResNet 50 and Mobilenet v2. We also make several improvements to VGGish and achieve better results. Finally, we transfer our models on three external standard benchmarks for audio classification task, and achieve significant performance boost over the state-of-the-art results. Models and codes are available at: https://github.com/Deeperjia/vgg-like-audio-models.
Training Set Effect on Super Resolution for Automated Target Recognition
Venue: arXiv cs.CV 2020
Year: 2020

Single Image Super Resolution (SISR) is the process of mapping a low-resolution image to a high resolution image. This inherently has applications in remote sensing as a way to increase the spatial resolution in satellite imagery. This suggests a possible improvement to automated target recognition in image classification and object detection. We explore the effect that different training sets have on SISR with the network, Super Resolution Generative Adversarial Network (SRGAN). We train 5 SRGANs on different land-use classes (e.g. agriculture, cities, ports) and test them on the same unseen dataset. We attempt to find the qualitative and quantitative differences in SISR, binary classification, and object detection performance. We find that curated training sets that contain objects in the test ontology perform better on both computer vision tasks while having a complex distribution of images allows object detection models to perform better. However, Super Resolution (SR) might not be beneficial to certain problems and will see a diminishing amount of returns for datasets that are closer to being solved.