OpenRaiser/Intern-Atlas huggingface dataset:
Stay with Your Community: Bridges between Clusters Trigger Expansion of COVID-19
Venue: arXiv cs.SI 2020
Year: 2020

The spreading of virus infection is here simulated over artificial human networks. The real-space urban life of people is modeled as a modified scale-free network with constraints. A scale-free network has been adopted in several studies for modeling on-line communities so far but is modified here for the aim to represent peoples' social behaviors where the generated communities are restricted reflecting the spatiotemporal constraints in the real life. Furthermore, the networks have been extended by introducing multiple cliques in the initial step of network construction and enabling people to zero-degree people as well as popular (large degree) people. As a result, four findings and a policy proposal have been obtained. First, the "second waves" occur without external influence or constraints on contacts or the releasing of the constraints. These second waves, mostly lower than the first wave, implies the bridges between infected and fresh clusters may trigger new expansions of spreading. Second, if the network changes the structure on the way of infection spreading or after its suppression, the peak of the second wave can be larger than the first. Third, the peak height in the time series depends on the difference between the upper bound of the number of people each member accepts to meet and the number of people one chooses to meet. This tendency is observed for two kinds of artificial networks and implies the impact of the bridges between communities on the virus spreading. Fourth, the release of once given constraint may trigger a second wave higher than the peak of the time series without introducing any constraint from the beginning, if the release is introduced at a time close to the peak. Thus, both governments and individuals should be careful in returning to human society with inter-community contacts.
Spatiotemporal Calibration of Camera and 3D Laser Scanner
Venue: arXiv cs.RO 2020
Year: 2020

The multi-sensory setups consisting of the laser scanners and cameras are popular as the measurements complement each other and provide necessary robustness for applications. Under dynamic conditions or when in motion, a direct transformation (spatial calibration) and time offset between sensors (temporal calibration) is needed to determine the correspondence between measurements. We propose an open-source spatiotemporal calibration framework for a camera and a 3D laser scanner. Our solution is based on commonly available chessboard markers requiring one-minute calibration before the operation that offers accurate and repeatable results. The framework is based on batch optimization of point-to-plane constraints with a time offset calibration possible by a novel continuous representation of the plane equations based on a minimal representation in the Lie algebra and the use of B-splines. The framework's properties are evaluated in simulation while correctness is verified with two distinct sensory setups with Velodyne VLP-16 and SICK MRS6124 3D laser scanners.
Multichannel CNN with Attention for Text Classification
Venue: arXiv cs.CL 2020
Year: 2020

Recent years, the approaches based on neural networks have shown remarkable potential for sentence modeling. There are two main neural network structures: recurrent neural network (RNN) and convolution neural network (CNN). RNN can capture long term dependencies and store the semantics of the previous information in a fixed-sized vector. However, RNN is a biased model and its ability to extract global semantics is restricted by the fixed-sized vector. Alternatively, CNN is able to capture n-gram features of texts by utilizing convolutional filters. But the width of convolutional filters restricts its performance. In order to combine the strengths of the two kinds of networks and alleviate their shortcomings, this paper proposes Attention-based Multichannel Convolutional Neural Network (AMCNN) for text classification. AMCNN utilizes a bi-directional long short-term memory to encode the history and future information of words into high dimensional representations, so that the information of both the front and back of the sentence can be fully expressed. Then the scalar attention and vectorial attention are applied to obtain multichannel representations. The scalar attention can calculate the word-level importance and the vectorial attention can calculate the feature-level importance. In the classification task, AMCNN uses a CNN structure to cpture word relations on the representations generated by the scalar and vectorial attention mechanism instead of calculating the weighted sums. It can effectively extract the n-gram features of the text. The experimental results on the benchmark datasets demonstrate that AMCNN achieves better performance than state-of-the-art methods. In addition, the visualization results verify the semantic richness of multichannel representations.
Unsupervised Learning Consensus Model for Dynamic Texture Videos Segmentation
Venue: arXiv cs.CV 2020
Year: 2020

Dynamic texture (DT) segmentation, and video processing in general, is currently widely dominated by methods based on deep neural networks that require the deployment of a large number of layers. Although this parametric approach has shown superior performances for the dynamic texture segmentation, all current deep learning methods suffer from a significant main weakness related to the lack of a sufficient reference annotation to train models and to make them functional. This study explores the unsupervised segmentation approach that can be used in the absence of training data to segment new videos. We present an effective unsupervised learning consensus model for the segmentation of dynamic texture (ULCM). This model is designed to merge different segmentation maps that contain multiple and weak quality regions in order to achieve a more accurate final result of segmentation. The diverse labeling fields required for the combination process are obtained by a simplified grouping scheme applied to an input video (on the basis of a three orthogonal planes: xy, yt and xt). In the proposed model, the set of values of the requantized local binary patterns (LBP) histogram around the pixel to be classified are used as features which represent both the spatial and temporal information replicated in the video. Experiments conducted on the challenging SynthDB dataset show that, contrary to current dynamic texture segmentation approaches that either require parameter estimation or a training step, ULCM is significantly faster, easier to code, simple and has limited parameters. Further qualitative experiments based on the YUP++ dataset prove the efficiently and competitively of the ULCM.
On the Privacy-Utility Tradeoff in Peer-Review Data Analysis
Venue: arXiv cs.CR 2020
Year: 2020

A major impediment to research on improving peer review is the unavailability of peer-review data, since any release of such data must grapple with the sensitivity of the peer review data in terms of protecting identities of reviewers from authors. We posit the need to develop techniques to release peer-review data in a privacy-preserving manner. Identifying this problem, in this paper we propose a framework for privacy-preserving release of certain conference peer-review data -- distributions of ratings, miscalibration, and subjectivity -- with an emphasis on the accuracy (or utility) of the released data. The crux of the framework lies in recognizing that a part of the data pertaining to the reviews is already available in public, and we use this information to post-process the data released by any privacy mechanism in a manner that improves the accuracy (utility) of the data while retaining the privacy guarantees. Our framework works with any privacy-preserving mechanism that operates via releasing perturbed data. We present several positive and negative theoretical results, including a polynomial-time algorithm for improving on the privacy-utility tradeoff.
Online Exploration and Coverage Planning in Unknown Obstacle-Cluttered Environments
Venue: arXiv cs.RO 2020
Year: 2020

Online coverage planning can be useful in applications like field monitoring and search and rescue. Without prior information of the environment, achieving resolution-complete coverage considering the non-holonomic mobility constraints in commonly-used vehicles (e.g., wheeled robots) remains a challenge. In this paper, we propose a hierarchical, hex-decomposition-based coverage planning algorithm for unknown, obstacle-cluttered environments. The proposed approach ensures resolution-complete coverage, can be tuned to achieve fast exploration, and plans smooth paths for Dubins vehicles to follow at constant velocity in real-time. Gazebo simulations and hardware experiments with a non-holonomic wheeled robot show that our approach can successfully tradeoff between coverage and exploration speed and can outperform existing online coverage algorithms in terms of total covered area or exploration speed according to how it is tuned.
Conditional GAN for timeseries generation
Venue: arXiv cs.LG 2020
Year: 2020

It is abundantly clear that time dependent data is a vital source of information in the world. The challenge has been for applications in machine learning to gain access to a considerable amount of quality data needed for algorithm development and analysis. Modeling synthetic data using a Generative Adversarial Network (GAN) has been at the heart of providing a viable solution. Our work focuses on one dimensional times series and explores the few shot approach, which is the ability of an algorithm to perform well with limited data. This work attempts to ease the frustration by proposing a new architecture, Time Series GAN (TSGAN), to model realistic time series data. We evaluate TSGAN on 70 data sets from a benchmark time series database. Our results demonstrate that TSGAN performs better than the competition both quantitatively using the Frechet Inception Score (FID) metric, and qualitatively when classification is used as the evaluation criteria.
Accelerating Reinforcement Learning Agent with EEG-based Implicit Human Feedback
Venue: arXiv cs.NE 2020
Year: 2020

Providing Reinforcement Learning (RL) agents with human feedback can dramatically improve various aspects of learning. However, previous methods require human observer to give inputs explicitly (e.g., press buttons, voice interface), burdening the human in the loop of RL agent's learning process. Further, it is sometimes difficult or impossible to obtain the explicit human advise (feedback), e.g., autonomous driving, disabled rehabilitation, etc. In this work, we investigate capturing human's intrinsic reactions as implicit (and natural) feedback through EEG in the form of error-related potentials (ErrP), providing a natural and direct way for humans to improve the RL agent learning. As such, the human intelligence can be integrated via implicit feedback with RL algorithms to accelerate the learning of RL agent. We develop three reasonably complex 2D discrete navigational games to experimentally evaluate the overall performance of the proposed work. Major contributions of our work are as follows, (i) we propose and experimentally validate the zero-shot learning of ErrPs, where the ErrPs can be learned for one game, and transferred to other unseen games, (ii) we propose a novel RL framework for integrating implicit human feedbacks via ErrPs with RL agent, improving the label efficiency and robustness to human mistakes, and (iii) compared to prior works, we scale the application of ErrPs to reasonably complex environments, and demonstrate the significance of our approach for accelerated learning through real user experiments.
Vehicle Re-ID for Surround-view Camera System
Venue: arXiv cs.CV 2020
Year: 2020

The vehicle re-identification (ReID) plays a critical role in the perception system of autonomous driving, which attracts more and more attention in recent years. However, to our best knowledge, there is no existing complete solution for the surround-view system mounted on the vehicle. In this paper, we argue two main challenges in above scenario: i) In single camera view, it is difficult to recognize the same vehicle from the past image frames due to the fisheye distortion, occlusion, truncation, etc. ii) In multi-camera view, the appearance of the same vehicle varies greatly from different camera's viewpoints. Thus, we present an integral vehicle Re-ID solution to address these problems. Specifically, we propose a novel quality evaluation mechanism to balance the effect of tracking box's drift and target's consistency. Besides, we take advantage of the Re-ID network based on attention mechanism, then combined with a spatial constraint strategy to further boost the performance between different cameras. The experiments demonstrate that our solution achieves state-of-the-art accuracy while being real-time in practice. Besides, we will release the code and annotated fisheye dataset for the benefit of community.
Uniform Priors for Data-Efficient Transfer
Venue: arXiv cs.LG 2020
Year: 2020

Deep Neural Networks have shown great promise on a variety of downstream applications; but their ability to adapt and generalize to new data and tasks remains a challenge. However, the ability to perform few or zero-shot adaptation to novel tasks is important for the scalability and deployment of machine learning models. It is therefore crucial to understand what makes for good, transfer-able features in deep networks that best allow for such adaptation. In this paper, we shed light on this by showing that features that are most transferable have high uniformity in the embedding space and propose a uniformity regularization scheme that encourages better transfer and feature reuse. We evaluate the regularization on its ability to facilitate adaptation to unseen tasks and data, for which we conduct a thorough experimental study covering four relevant, and distinct domains: few-shot Meta-Learning, Deep Metric Learning, Zero-Shot Domain Adaptation, as well as Out-of-Distribution classification. Across all experiments, we show that uniformity regularization consistently offers benefits over baseline methods and is able to achieve state-of-the-art performance in Deep Metric Learning and Meta-Learning.
Subspace approximation with outliers
Venue: arXiv cs.CG 2020
Year: 2020

The subspace approximation problem with outliers, for given $n$ points in $d$ dimensions $x_{1},\ldots, x_{n} \in R^{d}$, an integer $1 \leq k \leq d$, and an outlier parameter $0 \leq α\leq 1$, is to find a $k$-dimensional linear subspace of $R^{d}$ that minimizes the sum of squared distances to its nearest $(1-α)n$ points. More generally, the $\ell_{p}$ subspace approximation problem with outliers minimizes the sum of $p$-th powers of distances instead of the sum of squared distances. Even the case of robust PCA is non-trivial, and previous work requires additional assumptions on the input. Any multiplicative approximation algorithm for the subspace approximation problem with outliers must solve the robust subspace recovery problem, a special case in which the $(1-α)n$ inliers in the optimal solution are promised to lie exactly on a $k$-dimensional linear subspace. However, robust subspace recovery is Small Set Expansion (SSE)-hard. We show how to extend dimension reduction techniques and bi-criteria approximations based on sampling to the problem of subspace approximation with outliers. To get around the SSE-hardness of robust subspace recovery, we assume that the squared distance error of the optimal $k$-dimensional subspace summed over the optimal $(1-α)n$ inliers is at least $δ$ times its squared-error summed over all $n$ points, for some $0 < δ\leq 1 - α$. With this assumption, we give an efficient algorithm to find a subset of $poly(k/ε) \log(1/δ) \log\log(1/δ)$ points whose span contains a $k$-dimensional subspace that gives a multiplicative $(1+ε)$-approximation to the optimal solution. The running time of our algorithm is linear in $n$ and $d$. Interestingly, our results hold even when the fraction of outliers $α$ is large, as long as the obvious condition $0 < δ\leq 1 - α$ is satisfied.
On the Demystification of Knowledge Distillation: A Residual Network Perspective
Venue: arXiv cs.CV 2020
Year: 2020

Knowledge distillation (KD) is generally considered as a technique for performing model compression and learned-label smoothing. However, in this paper, we study and investigate the KD approach from a new perspective: we study its efficacy in training a deeper network without any residual connections. We find that in most of the cases, non-residual student networks perform equally or better than their residual versions trained on raw data without KD (baseline network). Surprisingly, in some cases, they surpass the accuracy of baseline networks even with the inferior teachers. After a certain depth of non-residual student network, the accuracy drop, coming from the removal of residual connections, is substantial, and training with KD boosts the accuracy of the student up to a great extent; however, it does not fully recover the accuracy drop. Furthermore, we observe that the conventional teacher-student view of KD is incomplete and does not adequately explain our findings. We propose a novel interpretation of KD with the Trainee-Mentor hypothesis, which provides a holistic view of KD. We also present two viewpoints, loss landscape, and feature reuse, to explain the interplay between residual connections and KD. We substantiate our claims through extensive experiments on residual networks.
Social Distancing 2.0 with Privacy-Preserving Contact Tracing to Avoid a Second Wave of COVID-19
Venue: arXiv cs.SI 2020
Year: 2020

How to avoid a second wave of COVID-19 after reopening the economy is a pressing question. The extremely high basic reproductive number $R_0$ (5.7 to 6.4, shown in new studies) of SARS-CoV-2 further complicates the challenge. Here we assess effects of Social distancing 2.0, i.e. proximity alert (to maintain inter-personal distance) plus privacy-preserving contact tracing. To solve the dual task, we developed an open source mobile app. The app uses a Bluetooth-based, decentralized contact tracing platform over which the anonymous user ID cannot be linked by the government or a third party. Modelling results show that a 50\% adoption rate of Social distancing 2.0, with privacy-preserving contact tracing, would suffice to decrease the $R_0$ to less than 1 and prevent the resurgence of COVID-19 epidemic.
On Dynamic Substructuring of Systems with Localised Nonlinearities
Venue: arXiv cs.CE 2020
Year: 2020

Dynamic substructuring (DS) methods encompass a range of techniques to decompose large structural systems into multiple coupled subsystems. This decomposition has the principle benefit of reducing computational time for dynamic simulation of the system. In this context, DS methods may form an essential component of hybrid simulation, wherein they can be used to couple physical and numerical substructures at reduced computational cost. Since most engineered systems are inherently nonlinear, particular potential lies in incorporating nonlinear methods in existing substructuring schemes which are largely linear methods. The most widely used and studied DS methods are classical linear techniques such as the Craig-Bampton (CB) method. However, as linear methods they naturally break down in the presence of nonlinearities. Recent advancements in substructuring have involved the development of enrichments to linear methods, which allow for some degree of nonlinearity to be captured. The use of mode shape derivatives has been shown to be able to capture geometrically non-linear effects as an extension to the CBmethod. Other candidates include the method of Finite Element Tearing and Interconnecting. In this work, a virtual hybrid simulation is presented in which a linear elastic vehicle frame supported on four nonlinear spring damper isolators is decomposed into separate domains. One domain consisting of the finite element model of the vehicle frame, which is reduced using the CB method. The second domain consists of the nonlinear isolators whose restoring forces are characterised by nonlinear spring and damper forces. Coupling between the models is carried out using a Lagrange multiplier method and time series simulations of the system are conducted and compared to the full global system with regards to simulation time and accuracy.
Extending the OpenCHK Model with Advanced Checkpoint Features
Venue: arXiv cs.DC 2020
Year: 2020

One of the major challenges in using extreme scale systems efficiently is to mitigate the impact of faults. Application-level checkpoint/restart (CR) methods provide the best trade-off between productivity, robustness, and performance. There are many solutions implementing CR at the application level. They all provide advanced I/O capabilities to minimize the overhead introduced by CR. Nevertheless, there is still room for improvement in terms of programmability and flexibility, because end-users must manually serialize and deserialize application state using low-level APIs, modify the flow of the application to consider restarts, or rewrite CR code whenever the backend library changes. In this work, we propose a set of compiler directives and clauses that allow users to specify CR operations in a simple way. Our approach supports the common CR features provided by all the CR libraries. However, it can also be extended to support advanced features that are only available in some CR libraries, such as differential checkpointing, the use of HDF5 format, and the possibility of using fault-tolerance-dedicated threads. The result of our evaluation revealed a high increase in programmability. On average, we reduced the number of lines of code by 71%, 94%, and 64% for FTI, SCR, and VeloC, respectively, and no additional overhead was perceived using our solution compared to using the backend libraries directly. Finally, portability is enhanced because our programming model allows the use of any backend library without changing any code.
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
Venue: arXiv cs.CL 2020
Year: 2020

Neural network scaling has been critical for improving the model quality in many real-world machine learning applications with vast amounts of training data and compute. Although this trend of scaling is affirmed to be a sure-fire approach for better model quality, there are challenges on the path such as the computation cost, ease of programming, and efficient implementation on parallel devices. GShard is a module composed of a set of lightweight annotation APIs and an extension to the XLA compiler. It provides an elegant way to express a wide range of parallel computation patterns with minimal changes to the existing model code. GShard enabled us to scale up multilingual neural machine translation Transformer model with Sparsely-Gated Mixture-of-Experts beyond 600 billion parameters using automatic sharding. We demonstrate that such a giant model can efficiently be trained on 2048 TPU v3 accelerators in 4 days to achieve far superior quality for translation from 100 languages to English compared to the prior art.