OpenRaiser/Intern-Atlas huggingface dataset:
Deconstructing the Structure of Sparse Neural Networks
Venue: arXiv cs.LG 2020
Year: 2020

Although sparse neural networks have been studied extensively, the focus has been primarily on accuracy. In this work, we focus instead on network structure, and analyze three popular algorithms. We first measure performance when structure persists and weights are reset to a different random initialization, thereby extending experiments in Deconstructing Lottery Tickets (Zhou et al., 2019). This experiment reveals that accuracy can be derived from structure alone. Second, to measure structural robustness we investigate the sensitivity of sparse neural networks to further pruning after training, finding a stark contrast between algorithms. Finally, for a recent dynamic sparsity algorithm we investigate how early in training the structure emerges. We find that even after one epoch the structure is mostly determined, allowing us to propose a more efficient algorithm which does not require dense gradients throughout training. In looking back at algorithms for sparse neural networks and analyzing their performance from a different lens, we uncover several interesting properties and promising directions for future research.
Detect, Reject, Correct: Crossmodal Compensation of Corrupted Sensors
Venue: arXiv cs.RO 2020
Year: 2020

Using sensor data from multiple modalities presents an opportunity to encode redundant and complementary features that can be useful when one modality is corrupted or noisy. Humans do this everyday, relying on touch and proprioceptive feedback in visually-challenging environments. However, robots might not always know when their sensors are corrupted, as even broken sensors can return valid values. In this work, we introduce the Crossmodal Compensation Model (CCM), which can detect corrupted sensor modalities and compensate for them. CMM is a representation model learned with self-supervision that leverages unimodal reconstruction loss for corruption detection. CCM then discards the corrupted modality and compensates for it with information from the remaining sensors. We show that CCM learns rich state representations that can be used for contact-rich manipulation policies, even when input modalities are corrupted in ways not seen during training time.
3D Guided Weakly Supervised Semantic Segmentation
Venue: arXiv cs.CV 2020
Year: 2020

Pixel-wise clean annotation is necessary for fully-supervised semantic segmentation, which is laborious and expensive to obtain. In this paper, we propose a weakly supervised 2D semantic segmentation model by incorporating sparse bounding box labels with available 3D information, which is much easier to obtain with advanced sensors. We manually labeled a subset of the 2D-3D Semantics(2D-3D-S) dataset with bounding boxes, and introduce our 2D-3D inference module to generate accurate pixel-wise segment proposal masks. Guided by 3D information, we first generate a point cloud of objects and calculate objectness probability score for each point. Then we project the point cloud with objectness probabilities back to 2D images followed by a refinement step to obtain segment proposals, which are treated as pseudo labels to train a semantic segmentation network. Our method works in a recursive manner to gradually refine the above-mentioned segment proposals. Extensive experimental results on the 2D-3D-S dataset show that the proposed method can generate accurate segment proposals when bounding box labels are available on only a small subset of training images. Performance comparison with recent state-of-the-art methods further illustrates the effectiveness of our method.
Cross-artform performance using networked interfaces: Last Man to Die's Vital LMTD
Venue: arXiv cs.HC 2020
Year: 2020

In 2009 the cross artform group, Last Man to Die, presented a series of performances using new interfaces and networked performance to integrate the three artforms of its members (actor, Hanna Cormick, visual artist, Benjamin Forster and percussionist, Charles Martin). This paper explains our artistic motivations and design for a computer vision surface and networked heartbeat sensor as well as the experience of mounting our first major work, Vital LMTD.
A New Action Recognition Framework for Video Highlights Summarization in Sporting Events
Venue: arXiv cs.CV 2020
Year: 2020

To date, machine learning for human action recognition in video has been widely implemented in sports activities. Although some studies have been successful in the past, precision is still the most significant concern. In this study, we present a high-accuracy framework to automatically clip the sports video stream by using a three-level prediction algorithm based on two classical open-source structures, i.e., YOLO-v3 and OpenPose. It is found that by using a modest amount of sports video training data, our methodology can perform sports activity highlights clipping accurately. Comparing with the previous systems, our methodology shows some advantages in accuracy. This study may serve as a new clipping system to extend the potential applications of the video summarization in sports field, as well as facilitates the development of match analysis system.
MILP-based Imitation Learning for HVAC control
Venue: arXiv cs.LG 2020
Year: 2020

To optimize the operation of a HVAC system with advanced techniques such as artificial neural network, previous studies usually need forecast information in their method. However, the forecast information inevitably contains errors all the time, which degrade the performance of the HVAC operation. Hence, in this study, we propose MILP-based imitation learning method to control a HVAC system without using the forecast information in order to reduce energy cost and maintain thermal comfort at a given level. Our proposed controller is a deep neural network (DNN) trained by using data labeled by a MILP solver with historical data. After training, our controller is used to control the HVAC system with real-time data. For comparison, we also develop a second method named forecast-based MILP which control the HVAC system using the forecast information. The performance of the two methods is verified by using real outdoor temperatures and real day-ahead prices in Detroit city, Michigan, United States. Numerical results clearly show that the performance of the MILP-based imitation learning is better than that of the forecast-based MILP method in terms of hourly power consumption, daily energy cost, and thermal comfort. Moreover, the difference between results of the MILP-based imitation learning method and optimal results is almost negligible. These optimal results are achieved only by using the MILP solver at the end of a day when we have full information on the weather and prices for the day.
Fast and Robust Bin-picking System for Densely Piled Industrial Objects
Venue: arXiv cs.RO 2020
Year: 2020

Objects grasping, also known as the bin-picking, is one of the most common tasks faced by industrial robots. While much work has been done in related topics, grasping randomly piled objects still remains a challenge because much of the existing work either lack robustness or costs too much resource. In this paper, we develop a fast and robust bin-picking system for grasping densely piled objects adaptively and safely. The proposed system starts with point cloud segmentation using improved density-based spatial clustering of application with noise (DBSCAN) algorithm, which is improved by combining the region growing algorithm and using Octree to speed up the calculation. The system then uses principle component analysis (PCA) for coarse registration and iterative closest point (ICP) for fine registration. We propose a grasp risk score (GRS) to evaluate each object by the collision probability, the stability of the object, and the whole pile's stability. Through real tests with the Anno robot, our method is verified to be advanced in speed and robustness.
Identifying pandemic-related stress factors from social-media posts -- effects on students and young-adults
Venue: arXiv cs.SI 2020
Year: 2020

The COVID-19 pandemic has thrown natural life out of gear across the globe. Strict measures are deployed to curb the spread of the virus that is causing it, and the most effective of them have been social isolation. This has led to wide-spread gloom and depression across society but more so among the young and the elderly. There are currently more than 200 million college students in 186 countries worldwide, affected due to the pandemic. The mode of education has changed suddenly, with the rapid adaptation of e-learning, whereby teaching is undertaken remotely and on digital platforms. This study presents insights gathered from social media posts that were posted by students and young adults during the COVID times. Using statistical and NLP techniques, we analyzed the behavioral issues reported by users themselves in their posts in depression-related communities on Reddit. We present methodologies to systematically analyze content using linguistic techniques to find out the stress-inducing factors. Online education, losing jobs, isolation from friends, and abusive families emerge as key stress factors.
Ultra-low bitrate video conferencing using deep image animation
Venue: arXiv cs.CV 2020
Year: 2020

In this work we propose a novel deep learning approach for ultra-low bitrate video compression for video conferencing applications. To address the shortcomings of current video compression paradigms when the available bandwidth is extremely limited, we adopt a model-based approach that employs deep neural networks to encode motion information as keypoint displacement and reconstruct the video signal at the decoder side. The overall system is trained in an end-to-end fashion minimizing a reconstruction error on the encoder output. Objective and subjective quality evaluation experiments demonstrate that the proposed approach provides an average bitrate reduction for the same visual quality of more than 80% compared to HEVC.
Symbolic AI for XAI: Evaluating LFIT Inductive Programming for Fair and Explainable Automatic Recruitment
Venue: arXiv cs.AI 2020
Year: 2020

Machine learning methods are growing in relevance for biometrics and personal information processing in domains such as forensics, e-health, recruitment, and e-learning. In these domains, white-box (human-readable) explanations of systems built on machine learning methods can become crucial. Inductive Logic Programming (ILP) is a subfield of symbolic AI aimed to automatically learn declarative theories about the process of data. Learning from Interpretation Transition (LFIT) is an ILP technique that can learn a propositional logic theory equivalent to a given black-box system (under certain conditions). The present work takes a first step to a general methodology to incorporate accurate declarative explanations to classic machine learning by checking the viability of LFIT in a specific AI application scenario: fair recruitment based on an automatic tool generated with machine learning methods for ranking Curricula Vitae that incorporates soft biometric information (gender and ethnicity). We show the expressiveness of LFIT for this specific problem and propose a scheme that can be applicable to other domains.
Adaptive Coding and Channel Shaping Through Reconfigurable Intelligent Surfaces: An Information-Theoretic Analysis
Venue: arXiv cs.IT 2020
Year: 2020

A communication link aided by a reconfigurable intelligent surface (RIS) is studied in which the transmitter can control the state of the RIS via a finite-rate control link. Channel state information (CSI) is acquired at the receiver based on pilot-assisted channel estimation, and it may or may not be shared with the transmitter. Considering quasi-static fading channels with imperfect CSI, capacity-achieving signalling is shown to implement joint encoding of the transmitted signal and of the response of the RIS. This demonstrates the information-theoretic optimality of RIS-based modulation, or "single-RF MIMO" systems. In addition, a novel signalling strategy based on separate layered encoding that enables practical successive cancellation-type decoding at the receiver is proposed. Numerical experiments show that the conventional scheme that fixes the reflection pattern of the RIS, irrespective of the transmitted information, as to maximize the achievable rate is strictly suboptimal, and is outperformed by the proposed adaptive coding strategies at all practical signal-to-noise ratio (SNR) levels.
Minimal Solutions for Panoramic Stitching Given Gravity Prior
Venue: arXiv cs.CV 2020
Year: 2020

When capturing panoramas, people tend to align their cameras with the vertical axis, i.e., the direction of gravity. Moreover, modern devices, such as smartphones and tablets, are equipped with an IMU (Inertial Measurement Unit) that can measure the gravity vector accurately. Using this prior, the y-axes of the cameras can be aligned or assumed to be already aligned, reducing their relative orientation to 1-DOF (degree of freedom). Exploiting this assumption, we propose new minimal solutions to panoramic image stitching of images taken by cameras with coinciding optical centers, i.e., undergoing pure rotation. We consider four practical camera configurations, assuming unknown fixed or varying focal length with or without radial distortion. The solvers are tested both on synthetic scenes and on more than 500k real image pairs from the Sun360 dataset and from scenes captured by us using two smartphones equipped with IMUs. It is shown, that they outperform the state-of-the-art both in terms of accuracy and processing time.
Byzantine Eventual Consistency and the Fundamental Limits of Peer-to-Peer Databases
Venue: arXiv cs.DC 2020
Year: 2020

Sybil attacks, in which a large number of adversary-controlled nodes join a network, are a concern for many peer-to-peer database systems, necessitating expensive countermeasures such as proof-of-work. However, there is a category of database applications that are, by design, immune to Sybil attacks because they can tolerate arbitrary numbers of Byzantine-faulty nodes. In this paper, we characterize this category of applications using a consistency model we call Byzantine Eventual Consistency (BEC). We introduce an algorithm that guarantees BEC based on Byzantine causal broadcast, prove its correctness, and demonstrate near-optimal performance in a prototype implementation.
Boosting the Performance of Semi-Supervised Learning with Unsupervised Clustering
Venue: arXiv cs.CV 2020
Year: 2020

Recently, Semi-Supervised Learning (SSL) has shown much promise in leveraging unlabeled data while being provided with very few labels. In this paper, we show that ignoring the labels altogether for whole epochs intermittently during training can significantly improve performance in the small sample regime. More specifically, we propose to train a network on two tasks jointly. The primary classification task is exposed to both the unlabeled and the scarcely annotated data, whereas the secondary task seeks to cluster the data without any labels. As opposed to hand-crafted pretext tasks frequently used in self-supervision, our clustering phase utilizes the same classification network and head in an attempt to relax the primary task and propagate the information from the labels without overfitting them. On top of that, the self-supervised technique of classifying image rotations is incorporated during the unsupervised learning phase to stabilize training. We demonstrate our method's efficacy in boosting several state-of-the-art SSL algorithms, significantly improving their results and reducing running time in various standard semi-supervised benchmarks, including 92.6% accuracy on CIFAR-10 and 96.9% on SVHN, using only 4 labels per class in each task. We also notably improve the results in the extreme cases of 1,2 and 3 labels per class, and show that features learned by our model are more meaningful for separating the data.
Free-Space Optical Communication With Reconfigurable Intelligent Surfaces
Venue: arXiv cs.IT 2020
Year: 2020

Despite the promising gains, free-space optical (FSO) communication is severely influenced by atmospheric turbulence and pointing error issues, which make its practical design a bit challenging. In this paper, with the aim to increase the communication coverage and improve the system performance, reconfigurable intelligent surfaces (RISs) are considered in an FSO communication setup, in which both atmospheric turbulence and pointing errors are considered. Closed-form expressions for the outage probability, average bit error rate, and channel capacity are derived assuming large number of reflecting elements at the RIS. Specifically, according to central limit theorem (CLT), while assuming multiple reflecting elements approximate expressions are proposed. It is shown that the respective accuracies increase as the number of elements at the RIS increases. Illustrative numerical examples are shown along with insightful discussions. Finally, Monte Carlo simulations are presented to verify the correctness of the analytical results.
Iterative VAE as a predictive brain model for out-of-distribution generalization
Venue: arXiv cs.LG 2020
Year: 2020

Our ability to generalize beyond training data to novel, out-of-distribution, image degradations is a hallmark of primate vision. The predictive brain, exemplified by predictive coding networks (PCNs), has become a prominent neuroscience theory of neural computation. Motivated by the recent successes of variational autoencoders (VAEs) in machine learning, we rigorously derive a correspondence between PCNs and VAEs. This motivates us to consider iterative extensions of VAEs (iVAEs) as plausible variational extensions of the PCNs. We further demonstrate that iVAEs generalize to distributional shifts significantly better than both PCNs and VAEs. In addition, we propose a novel measure of recognizability for individual samples which can be tested against human psychophysical data. Overall, we hope this work will spur interest in iVAEs as a promising new direction for modeling in neuroscience.
Unfolded Deep Neural Network (UDNN) for High Mobility Channel Estimation
Venue: arXiv cs.IT 2020
Year: 2020

High mobility channel estimation is crucial for beyond 5G (B5G) or 6G wireless communication networks. This paper is concerned with channel estimation of high mobility OFDM communication systems. First, a two-dimensional compressed sensing problem is formulated by approximately linearizing the channel as a product of an overcomplete dictionary with a sparse vector, in which the Doppler effect caused by the high mobility channel is considered. To solve the problem that the traditional compressed sensing algorithms have too many iterations and are time consuming, we propose an unfolded deep neural network (UDNN) as the fast solver, which is inspired by the structure of iterative shrinkage-thresholding algorithm (ISTA). All the parameters in UDNN (e.g. nonlinear transforms, shrinkage thresholds, measurement matrices, etc.) are learned end-to-end, rather than being hand-crafted. Experiments demonstrate that the proposed UDNN performs better than ISTA for OFDM high mobility channel estimation, while maintaining extremely fast computational speed.
Procedure for the Safety Assessment of an Autonomous Vehicle Using Real-World Scenarios
Venue: arXiv cs.RO 2020
Year: 2020

The development of Autonomous Vehicles (AVs) has made significant progress in the last years. An important aspect in the development of AVs is the assessment of their safety. New approaches need to be worked out. Among these, real-world scenario-based assessment is widely supported by many players in the automotive field. Scenario-based assessment allows for using virtual simulation tools in addition to physical tests, such as on a test track, proving ground, or public road. We propose a procedure for real-world scenario-based road-approval assessment considering three stakeholders: the applicant, the assessor, and the (road or vehicle) authority. The challenges are as follows. Firstly, the tests need to be tailored to the operational design domain (ODD) and dynamic driving task (DDT) description of the AV. Secondly, it is assumed that the applicant does not want to disclose all of the detailed test results because of proprietary or confidential information contained in these results. Thirdly, due to the complex ODD and DDT, many test scenarios are required to obtain sufficient confidence in the assessment of the AV. Consequently, it is assumed that due to limited resources, it is infeasible for the assessor to conduct all (physical) tests. We propose a systematic approach for determining the tests that are based on the requirements set by the authority and the AV's ODD and DDT description, such that the tests are tailored to the applicable ODD and DDT. Each test comes with metrics that enables the applicant to provide a performance rating of the AV for each of the tests. By only providing a performance rating for each test, the applicant does not need to disclose the details of the test results. In our proposed procedure, the assessor only conducts a limited number of tests. The main purpose of these tests is to verify the fidelity of the results provided by the applicant.
Cyber-Attack Consequence Prediction
Venue: arXiv cs.CR 2020
Year: 2020

Cyber-physical systems posit a complex number of security challenges due to interconnection of heterogeneous devices having limited processing, communication, and power capabilities. Additionally, the conglomeration of both physical and cyber-space further makes it difficult to devise a single security plan spanning both these spaces. Cyber-security researchers are often overloaded with a variety of cyber-alerts on a daily basis many of which turn out to be false positives. In this paper, we use machine learning and natural language processing techniques to predict the consequences of cyberattacks. The idea is to enable security researchers to have tools at their disposal that makes it easier to communicate the attack consequences with various stakeholders who may have little to no cybersecurity expertise. Additionally, with the proposed approach researchers' cognitive load can be reduced by automatically predicting the consequences of attacks in case new attacks are discovered. We compare the performance through various machine learning models employing word vectors obtained using both tf-idf and Doc2Vec models. In our experiments, an accuracy of 60% was obtained using tf-idf features and 57% using Doc2Vec method for models based on LinearSVC model.
GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution
Venue: arXiv cs.CV 2020
Year: 2020

We show that pre-trained Generative Adversarial Networks (GANs), e.g., StyleGAN, can be used as a latent bank to improve the restoration quality of large-factor image super-resolution (SR). While most existing SR approaches attempt to generate realistic textures through learning with adversarial loss, our method, Generative LatEnt bANk (GLEAN), goes beyond existing practices by directly leveraging rich and diverse priors encapsulated in a pre-trained GAN. But unlike prevalent GAN inversion methods that require expensive image-specific optimization at runtime, our approach only needs a single forward pass to generate the upscaled image. GLEAN can be easily incorporated in a simple encoder-bank-decoder architecture with multi-resolution skip connections. Switching the bank allows the method to deal with images from diverse categories, e.g., cat, building, human face, and car. Images upscaled by GLEAN show clear improvements in terms of fidelity and texture faithfulness in comparison to existing methods.