OpenRaiser/Intern-Atlas huggingface dataset:
Reference-Based Video Colorization with Spatiotemporal Correspondence
Venue: arXiv cs.CV 2020
Year: 2020

We propose a novel reference-based video colorization framework with spatiotemporal correspondence. Reference-based methods colorize grayscale frames referencing a user input color frame. Existing methods suffer from the color leakage between objects and the emergence of average colors, derived from non-local semantic correspondence in space. To address this issue, we warp colors only from the regions on the reference frame restricted by correspondence in time. We propagate masks as temporal correspondences, using two complementary tracking approaches: off-the-shelf instance tracking for high performance segmentation, and newly proposed dense tracking to track various types of objects. By restricting temporally-related regions for referencing colors, our approach propagates faithful colors throughout the video. Experiments demonstrate that our method outperforms state-of-the-art methods quantitatively and qualitatively.
Consistency-aware and Inconsistency-aware Graph-based Multi-view Clustering
Venue: arXiv cs.LG 2020
Year: 2020

Multi-view data analysis has gained increasing popularity because multi-view data are frequently encountered in machine learning applications. A simple but promising approach for clustering of multi-view data is multi-view clustering (MVC), which has been developed extensively to classify given subjects into some clustered groups by learning latent common features that are shared across multi-view data. Among existing approaches, graph-based multi-view clustering (GMVC) achieves state-of-the-art performance by leveraging a shared graph matrix called the unified matrix. However, existing methods including GMVC do not explicitly address inconsistent parts of input graph matrices. Consequently, they are adversely affected by unacceptable clustering performance. To this end, this paper proposes a new GMVC method that incorporates consistent and inconsistent parts lying across multiple views. This proposal is designated as CI-GMVC. Numerical evaluations of real-world datasets demonstrate the effectiveness of the proposed CI-GMVC.
Measuring Happiness Around the World Through Artificial Intelligence
Venue: arXiv cs.AI 2020
Year: 2020

In this work, we analyze the happiness levels of countries using an unbiased emotion detector, artificial intelligence (AI). To date, researchers proposed many factors that may affect happiness such as wealth, health and safety. Even though these factors all seem relevant, there is no clear consensus between sociologists on how to interpret these, and the models to estimate the cost of these utilities include some assumptions. Researchers in social sciences have been working on determination of the happiness levels in society and exploration of the factors correlated with it through polls and different statistical methods. In our work, by using artificial intelligence, we introduce a different and relatively unbiased approach to this problem. By using AI, we make no assumption about what makes a person happy, and leave the decision to AI to detect the emotions from the faces of people collected from publicly available street footages. We analyzed the happiness levels in eight different cities around the world through available footage on the Internet and found out that there is no statistically significant difference between countries in terms of happiness.
Enhanced Scene Specificity with Sparse Dynamic Value Estimation
Venue: arXiv cs.LG 2020
Year: 2020

Multi-scene reinforcement learning involves training the RL agent across multiple scenes / levels from the same task, and has become essential for many generalization applications. However, the inclusion of multiple scenes leads to an increase in sample variance for policy gradient computations, often resulting in suboptimal performance with the direct application of traditional methods (e.g. PPO, A3C). One strategy for variance reduction is to consider each scene as a distinct Markov decision process (MDP) and learn a joint value function dependent on both state (s) and MDP (M). However, this is non-trivial as the agent is usually unaware of the underlying level at train / test times in multi-scene RL. Recently, Singh et al. [1] tried to address this by proposing a dynamic value estimation approach that models the true joint value function distribution as a Gaussian mixture model (GMM). In this paper, we argue that the error between the true scene-specific value function and the predicted dynamic estimate can be further reduced by progressively enforcing sparse cluster assignments once the agent has explored most of the state space. The resulting agents not only show significant improvements in the final reward score across a range of OpenAI ProcGen environments, but also exhibit increased navigation efficiency while completing a game level.
SLARM: Simultaneous Localization and Radio Mapping for Communication-aware Connected Robot
Venue: arXiv cs.RO 2020
Year: 2020

A novel simultaneous localization and radio mapping (SLARM) framework for communication-aware connected robots in the unknown indoor environment is proposed, where the simultaneous localization and mapping (SLAM) algorithm and the global geographic map recovery (GGMR) algorithm are leveraged to simultaneously construct a geographic map and a radio map named a channel power gain map. Specifically, the geographic map contains the information of a precise layout of obstacles and passable regions, and the radio map characterizes the position-dependent maximum expected channel power gain between the access point and the connected robot. Numerical results show that: 1) The pre-defined resolution in the SLAM algorithm and the proposed GGMR algorithm significantly affect the accuracy of the constructed radio map; and 2) The accuracy of radio map constructed by the SLARM framework is more than 78.78% when the resolution value smaller than 0.15m, and the accuracy reaches 91.95% when the resolution value is pre-defined as 0.05m.
Resolving the cybersecurity Data Sharing Paradox to scale up cybersecurity via a co-production approach towards data sharing
Venue: arXiv cs.CR 2020
Year: 2020

As cybercriminals scale up their operations to increase their profits or inflict greater harm, we argue that there is an equal need to respond to their threats by scaling up cybersecurity. To achieve this goal, we have to develop a co-productive approach towards data collection and sharing by overcoming the cybersecurity data sharing paradox. This is where we all agree on the definition of the problem and end goal (improving cybersecurity and getting rid of cybercrime), but we disagree about how to achieve it and fail to work together efficiently. At the core of this paradox is the observation that public interests differ from private interests. As a result, industry and law enforcement take different approaches to the cybersecurity problem as they seek to resolve incidents in their own interests, which manifests in different data sharing practices between both and also other interested parties, such as cybersecurity researchers. The big question we ask is can these interests be reconciled to develop an interdisciplinary approach towards co-operation and sharing data. In essence, all three will have to co-own the problem in order to co-produce a solution. We argue that a few operational models with good practices exist that provide guides to a possible solution, especially multiple third-party ownership organisations which consolidate, anonymise and analyse data. To take this forward, we suggest the practical solution of organising co-productive data collection on a sectoral basis, but acknowledge that common standards for data collection will also have to be developed and agreed upon. We propose an initial set of best practices for building collaborations and sharing data and argue that these best practices need to be developed and standardised in order to mitigate the paradox.
A Secure Deep Probabilistic Dynamic Thermal Line Rating Prediction
Venue: arXiv cs.CR 2020
Year: 2020

Accurate short-term prediction of overhead line (OHL) transmission ampacity can directly affect the efficiency of power system operation and planning. Any overestimation of the dynamic thermal line rating (DTLR) can lead to lifetime degradation and failure of OHLs, safety hazards, etc. This paper presents a secure yet sharp probabilistic prediction model for the hour-ahead forecasting of the DTLR. The security of the proposed DTLR limits the frequency of DTLR prediction exceeding the actual DTLR. The model is based on an augmented deep learning architecture that makes use of a wide range of predictors, including historical climatology data and latent variables obtained during DTLR calculation. Furthermore, by introducing a customized cost function, the deep neural network is trained to consider the DTLR security based on the required probability of exceedance while minimizing deviations of the predicted DTLRs from the actual values. The proposed probabilistic DTLR is developed and verified using recorded experimental data. The simulation results validate the superiority of the proposed DTLR compared to state-of-the-art prediction models using well-known evaluation metrics.
Ranking Deep Learning Generalization using Label Variation in Latent Geometry Graphs
Venue: arXiv cs.LG 2020
Year: 2020

Measuring the generalization performance of a Deep Neural Network (DNN) without relying on a validation set is a difficult task. In this work, we propose exploiting Latent Geometry Graphs (LGGs) to represent the latent spaces of trained DNN architectures. Such graphs are obtained by connecting samples that yield similar latent representations at a given layer of the considered DNN. We then obtain a generalization score by looking at how strongly connected are samples of distinct classes in LGGs. This score allowed us to rank 3rd on the NeurIPS 2020 Predicting Generalization in Deep Learning (PGDL) competition.
Towards the Development of 3D Engine Assembly Simulation Learning Module for Senior High School
Venue: arXiv cs.CY 2020
Year: 2020

The focus of the study is to develop a 3D engine assembly simulation learning module to address the lack of equipment in one senior high school in the Philippines. The study used mixed-method to determine the considerations needed in developing an application for educational use particularly among laboratory/practical subjects like engine assembly. The study used ISO 25010 quality standards in evaluating the application(n=153 students and 3 ICT experts).Results showed that the application is moderately acceptable(overall mean = 3.52) under ISO 25010 quality standards. The study created an engine simulation learning assembly in which teachers can use to augment their lesson. The study also highlights the applicability of using 3D-related technologies for practical and laboratory subjects particularly highly technical-related subjects. Future studies may develop a similar application in the same context using mobile and other emerging technology(i.e., Virtual Reality, Augmented Reality) as well as making the content more customizable. Effectivity of the system in an actual setting is also worth pursuing. The study highlighted the potential use of 3D technology in a classroom setting.
Face recognition using PCA integrated with Delaunay triangulation
Venue: arXiv cs.CV 2020
Year: 2020

Face Recognition is most used for biometric user authentication that identifies a user based on his or her facial features. The system is in high demand, as it is used by many businesses and employed in many devices such as smartphones and surveillance cameras. However, one frequent problem that is still observed in this user-verification method is its accuracy rate. Numerous approaches and algorithms have been experimented to improve the stated flaw of the system. This research develops one such algorithm that utilizes a combination of two different approaches. Using the concepts from Linear Algebra and computational geometry, the research examines the integration of Principal Component Analysis with Delaunay Triangulation; the method triangulates a set of face landmark points and obtains eigenfaces of the provided images. It compares the algorithm with traditional PCA and discusses the inclusion of different face landmark points to deliver an effective recognition rate.
Attention-Based Learning on Molecular Ensembles
Venue: arXiv cs.LG 2020
Year: 2020

The three-dimensional shape and conformation of small-molecule ligands are critical for biomolecular recognition, yet encoding 3D geometry has not improved ligand-based virtual screening approaches. We describe an end-to-end deep learning approach that operates directly on small-molecule conformational ensembles and identifies key conformational poses of small-molecules. Our networks leverage two levels of representation learning: 1) individual conformers are first encoded as spatial graphs using a graph neural network, and 2) sampled conformational ensembles are represented as sets using an attention mechanism to aggregate over individual instances. We demonstrate the feasibility of this approach on a simple task based on bidentate coordination of biaryl ligands, and show how attention-based pooling can elucidate key conformational poses in tasks based on molecular geometry. This work illustrates how set-based learning approaches may be further developed for small molecule-based virtual screening.
Enhanced 3DMM Attribute Control via Synthetic Dataset Creation Pipeline
Venue: arXiv cs.CV 2020
Year: 2020

While facial attribute manipulation of 2D images via Generative Adversarial Networks (GANs) has become common in computer vision and graphics due to its many practical uses, research on 3D attribute manipulation is relatively undeveloped. Existing 3D attribute manipulation methods are limited because the same semantic changes are applied to every 3D face. The key challenge for developing better 3D attribute control methods is the lack of paired training data in which one attribute is changed while other attributes are held fixed -- e.g., a pair of 3D faces where one is male and the other is female but all other attributes, such as race and expression, are the same. To overcome this challenge, we design a novel pipeline for generating paired 3D faces by harnessing the power of GANs. On top of this pipeline, we then propose an enhanced non-linear 3D conditional attribute controller that increases the precision and diversity of 3D attribute control compared to existing methods. We demonstrate the validity of our dataset creation pipeline and the superior performance of our conditional attribute controller via quantitative and qualitative evaluations.
Cable Tree Wiring -- Benchmarking Solvers on a Real-World Scheduling Problem with a Variety of Precedence Constraints
Venue: arXiv cs.AI 2020
Year: 2020

Cable trees are used in industrial products to transmit energy and information between different product parts. To this date, they are mostly assembled by humans and only few automated manufacturing solutions exist using complex robotic machines. For these machines, the wiring plan has to be translated into a wiring sequence of cable plugging operations to be followed by the machine. In this paper, we study and formalize the problem of deriving the optimal wiring sequence for a given layout of a cable tree. We summarize our investigations to model this cable tree wiring Problem (CTW) as a traveling salesman problem with atomic, soft atomic, and disjunctive precedence constraints as well as tour-dependent edge costs such that it can be solved by state-of-the-art constraint programming (CP), Optimization Modulo Theories (OMT), and mixed-integer programming (MIP) solvers. It is further shown, how the CTW problem can be viewed as a soft version of the coupled tasks scheduling problem. We discuss various modeling variants for the problem, prove its NP-hardness, and empirically compare CP, OMT, and MIP solvers on a benchmark set of 278 instances. The complete benchmark set with all models and instance data is available on github and is accepted for inclusion in the MiniZinc challenge 2020.
Improving Redundancy Availability: Dynamic Subtasks Modulation for Robots with Redundancy Insufficiency
Venue: arXiv cs.RO 2020
Year: 2020

This work presents an approach for robots to suitably carry out complex applications characterized by the presence of multiple additional constraints or subtasks (e.g. obstacle and self-collision avoidance) but subject to redundancy insufficiency. The proposed approach, based on a novel subtask merging strategy, enforces all subtasks in due course by dynamically modulating a virtual secondary task, where the task status and soft priority are incorporated to improve the overall efficiency of redundancy resolution. The proposed approach greatly improves the redundancy availability by unitizing and deploying subtasks in a fine-grained and compact manner. We build up our control framework on the null space projection, which guarantees the execution of subtasks does not interfere with the primary task. Experimental results on two case studies are presented to show the performance of our approach.
Generalized Focal Loss V2: Learning Reliable Localization Quality Estimation for Dense Object Detection
Venue: arXiv cs.CV 2020
Year: 2020

Localization Quality Estimation (LQE) is crucial and popular in the recent advancement of dense object detectors since it can provide accurate ranking scores that benefit the Non-Maximum Suppression processing and improve detection performance. As a common practice, most existing methods predict LQE scores through vanilla convolutional features shared with object classification or bounding box regression. In this paper, we explore a completely novel and different perspective to perform LQE -- based on the learned distributions of the four parameters of the bounding box. The bounding box distributions are inspired and introduced as "General Distribution" in GFLV1, which describes the uncertainty of the predicted bounding boxes well. Such a property makes the distribution statistics of a bounding box highly correlated to its real localization quality. Specifically, a bounding box distribution with a sharp peak usually corresponds to high localization quality, and vice versa. By leveraging the close correlation between distribution statistics and the real localization quality, we develop a considerably lightweight Distribution-Guided Quality Predictor (DGQP) for reliable LQE based on GFLV1, thus producing GFLV2. To our best knowledge, it is the first attempt in object detection to use a highly relevant, statistical representation to facilitate LQE. Extensive experiments demonstrate the effectiveness of our method. Notably, GFLV2 (ResNet-101) achieves 46.2 AP at 14.6 FPS, surpassing the previous state-of-the-art ATSS baseline (43.6 AP at 14.6 FPS) by absolute 2.6 AP on COCO {\tt test-dev}, without sacrificing the efficiency both in training and inference. Code will be available at https://github.com/implus/GFocalV2.
Recalibration of Neural Networks for Point Cloud Analysis
Venue: arXiv cs.CV 2020
Year: 2020

Spatial and channel re-calibration have become powerful concepts in computer vision. Their ability to capture long-range dependencies is especially useful for those networks that extract local features, such as CNNs. While re-calibration has been widely studied for image analysis, it has not yet been used on shape representations. In this work, we introduce re-calibration modules on deep neural networks for 3D point clouds. We propose a set of re-calibration blocks that extend Squeeze and Excitation blocks and that can be added to any network for 3D point cloud analysis that builds a global descriptor by hierarchically combining features from multiple local neighborhoods. We run two sets of experiments to validate our approach. First, we demonstrate the benefit and versatility of our proposed modules by incorporating them into three state-of-the-art networks for 3D point cloud analysis: PointNet++, DGCNN, and RSCNN. We evaluate each network on two tasks: object classification on ModelNet40, and object part segmentation on ShapeNet. Our results show an improvement of up to 1% in accuracy for ModelNet40 compared to the baseline method. In the second set of experiments, we investigate the benefits of re-calibration blocks on Alzheimer's Disease (AD) diagnosis. Our results demonstrate that our proposed methods yield a 2% increase in accuracy for diagnosing AD and a 2.3% increase in concordance index for predicting AD onset with time-to-event analysis. Concluding, re-calibration improves the accuracy of point cloud architectures, while only minimally increasing the number of parameters.
StyleUV: Diverse and High-fidelity UV Map Generative Model
Venue: arXiv cs.GR 2020
Year: 2020

Reconstructing 3D human faces in the wild with the 3D Morphable Model (3DMM) has become popular in recent years. While most prior work focuses on estimating more robust and accurate geometry, relatively little attention has been paid to improving the quality of the texture model. Meanwhile, with the advent of Generative Adversarial Networks (GANs), there has been great progress in reconstructing realistic 2D images. Recent work demonstrates that GANs trained with abundant high-quality UV maps can produce high-fidelity textures superior to those produced by existing methods. However, acquiring such high-quality UV maps is difficult because they are expensive to acquire, requiring laborious processes to refine. In this work, we present a novel UV map generative model that learns to generate diverse and realistic synthetic UV maps without requiring high-quality UV maps for training. Our proposed framework can be trained solely with in-the-wild images (i.e., UV maps are not required) by leveraging a combination of GANs and a differentiable renderer. Both quantitative and qualitative evaluations demonstrate that our proposed texture model produces more diverse and higher fidelity textures compared to existing methods.
Assessing the Quality of Gridded Population Data for Quantifying the Population Living in Deprived Communities
Venue: arXiv cs.CY 2020
Year: 2020

Over a billion people live in slums in settlements that are often located in ecologically sensitive areas and hence highly vulnerable. This is a problem in many parts of the world, but it is more prominent in low-income countries, where in 2014 on average 65% of the urban population lived in slums. As a result, building resilient communities requires quantifying the population living in these deprived areas and improving their living conditions. However, most of the data about slums comes from census data, which is only available at aggregate levels and often excludes these settlements. Consequently, researchers have looked at alternative approaches. These approaches, however, commonly rely on expensive high-resolution satellite imagery and field-surveys, which hinders their large-scale applicability. In this paper, we investigate a cost-effective methodology to estimate the slum population by assessing the quality of gridded population data. We evaluate the accuracy of the WorldPOP and LandScan population layers against ground-truth data composed of 1,703 georeferenced polygons that were mapped as deprived areas and which had their population surveyed during the 2010 Brazilian census. While the LandScan data did not produce satisfactory results for most polygons, the WorldPOP estimates were less than 20% off for 67% of the polygons and the overall error for the totality of the studied area was only -5.9%. This small error margin demonstrates that population layers with a resolution of at least a 100m, such as WorldPOP's, can be useful tools to estimate the population living in slums.