OpenRaiser/Intern-Atlas huggingface dataset:
Fine-grained Classification via Categorical Memory Networks
Venue: arXiv cs.CV 2020
Year: 2020

Motivated by the desire to exploit patterns shared across classes, we present a simple yet effective class-specific memory module for fine-grained feature learning. The memory module stores the prototypical feature representation for each category as a moving average. We hypothesize that the combination of similarities with respect to each category is itself a useful discriminative cue. To detect these similarities, we use attention as a querying mechanism. The attention scores with respect to each class prototype are used as weights to combine prototypes via weighted sum, producing a uniquely tailored response feature representation for a given input. The original and response features are combined to produce an augmented feature for classification. We integrate our class-specific memory module into a standard convolutional neural network, yielding a Categorical Memory Network. Our memory module significantly improves accuracy over baseline CNNs, achieving competitive accuracy with state-of-the-art methods on four benchmarks, including CUB-200-2011, Stanford Cars, FGVC Aircraft, and NABirds.
Evaluation and Comparison of Diffusion Models with Motif Features
Venue: arXiv cs.SI 2020
Year: 2020

Diffusion models simulate the propagation of influence in networks. The design and evaluation of diffusion models has been subjective and empirical. When being applied to a network represented by a graph, the diffusion model generates a sequence of edges on which the influence flows, such sequence forms a temporal network. In most scenarios, the statistical properties or the characteristics of a network are inferred by analyzing the temporal networks generated by diffusion models. To analyze real temporal networks, the motif has been proposed as a reliable feature. However, it is unclear how the network topology and the diffusion model affect the motif feature of a generated temporal network. In this paper, we adopt the motif feature to evaluate the temporal graph generated by a diffusion model, thence the diffusion model itself. Two benchmarks for quantitively evaluating diffusion models with motif, stability and separability, are proposed and measured on numerous diffusion models. One motif-based metric is proposed to measure the similarity between diffusion models. The experiments suggest that the motif of a generated temporal network is dominated by the diffusion model, while the network topology is almost ignored. This result indicates that more practical and reliable diffusion models have to be designed with delicacy in order to capture the propagation patterns of real temporal networks.
Physics-Informed Machine Learning Simulator for Wildfire Propagation
Venue: arXiv cs.LG 2020
Year: 2020

The aim of this work is to evaluate the feasibility of re-implementing some key parts of the widely used Weather Research and Forecasting WRF-SFIRE simulator by replacing its core differential equations numerical solvers with state-of-the-art physics-informed machine learning techniques to solve ODEs and PDEs, in order to transform it into a real-time simulator for wildfire spread prediction. The main programming language used is Julia, a compiled language which offers better perfomance than interpreted ones, providing Just in Time (JIT) compilation with different optimization levels. Moreover, Julia is particularly well suited for numerical computation and for the solution of complex physical models, both considering the syntax and the presence of some specific libraries such as DifferentialEquations.jl and ModellingToolkit.jl.
Multi-Scale Cascading Network with Compact Feature Learning for RGB-Infrared Person Re-Identification
Venue: arXiv cs.CV 2020
Year: 2020

RGB-Infrared person re-identification (RGB-IR Re-ID) aims to match persons from heterogeneous images captured by visible and thermal cameras, which is of great significance in the surveillance system under poor light conditions. Facing great challenges in complex variances including conventional single-modality and additional inter-modality discrepancies, most of the existing RGB-IR Re-ID methods propose to impose constraints in image level, feature level or a hybrid of both. Despite the better performance of hybrid constraints, they are usually implemented with heavy network architecture. As a matter of fact, previous efforts contribute more as pioneering works in new cross-modal Re-ID area while leaving large space for improvement. This can be mainly attributed to: (1) lack of abundant person image pairs from different modalities for training, and (2) scarcity of salient modality-invariant features especially on coarse representations for effective matching. To address these issues, a novel Multi-Scale Part-Aware Cascading framework (MSPAC) is formulated by aggregating multi-scale fine-grained features from part to global in a cascading manner, which results in a unified representation containing rich and enhanced semantic features. Furthermore, a marginal exponential centre (MeCen) loss is introduced to jointly eliminate mixed variances from intra- and inter-modal examples. Cross-modality correlations can thus be efficiently explored on salient features for distinctive modality-invariant feature learning. Extensive experiments are conducted to demonstrate that the proposed method outperforms all the state-of-the-art by a large margin.
LiveChess2FEN: a Framework for Classifying Chess Pieces based on CNNs
Venue: arXiv cs.CV 2020
Year: 2020

Automatic digitization of chess games using computer vision is a significant technological challenge. This problem is of much interest for tournament organizers and amateur or professional players to broadcast their over-the-board (OTB) games online or analyze them using chess engines. Previous work has shown promising results, but the recognition accuracy and the latency of state-of-the-art techniques still need further enhancements to allow their practical and affordable deployment. We have investigated how to implement them on an Nvidia Jetson Nano single-board computer effectively. Our first contribution has been accelerating the chessboard's detection algorithm. Subsequently, we have analyzed different Convolutional Neural Networks for chess piece classification and how to map them efficiently on our embedded platform. Notably, we have implemented a functional framework that automatically digitizes a chess position from an image in less than 1 second, with 92% accuracy when classifying the pieces and 95% when detecting the board.
Revisiting "Qualitatively Characterizing Neural Network Optimization Problems"
Venue: arXiv cs.LG 2020
Year: 2020

We revisit and extend the experiments of Goodfellow et al. (2014), who showed that - for then state-of-the-art networks - "the objective function has a simple, approximately convex shape" along the linear path between initialization and the trained weights. We do not find this to be the case for modern networks on CIFAR-10 and ImageNet. Instead, although loss is roughly monotonically non-increasing along this path, it remains high until close to the optimum. In addition, training quickly becomes linearly separated from the optimum by loss barriers. We conclude that, although Goodfellow et al.'s findings describe the "relatively easy to optimize" MNIST setting, behavior is qualitatively different in modern settings.
PAIRS AutoGeo: an Automated Machine Learning Framework for Massive Geospatial Data
Venue: arXiv cs.CV 2020
Year: 2020

An automated machine learning framework for geospatial data named PAIRS AutoGeo is introduced on IBM PAIRS Geoscope big data and analytics platform. The framework simplifies the development of industrial machine learning solutions leveraging geospatial data to the extent that the user inputs are minimized to merely a text file containing labeled GPS coordinates. PAIRS AutoGeo automatically gathers required data at the location coordinates, assembles the training data, performs quality check, and trains multiple machine learning models for subsequent deployment. The framework is validated using a realistic industrial use case of tree species classification. Open-source tree species data are used as the input to train a random forest classifier and a modified ResNet model for 10-way tree species classification based on aerial imagery, which leads to an accuracy of $59.8\%$ and $81.4\%$, respectively. This use case exemplifies how PAIRS AutoGeo enables users to leverage machine learning without extensive geospatial expertise.
Characterizing People's Daily Activity Patterns in the Urban Environment: A Mobility Network Approach with Geographic Context-Aware Twitter Data
Venue: arXiv cs.SI 2020
Year: 2020

People's daily activities in the urban environment are complex and vary by individuals. Existing studies using mobile phone data revealed distinct and recurrent transitional activity patterns, known as mobility motifs, in people's daily lives. However, the limitation in using only a few inferred activity types hinders our ability to examine general patterns in detail. We proposed a mobility network approach with geographic context-aware Twitter data to investigate granular daily activity patterns in the urban environment. We first utilized publicly accessible geo-located tweets to track the movements of individuals in two major U.S. cities: Chicago and Greater Boston, where each recorded location is associated with its closest land use parcel to enrich its geographic context. A direct mobility network represents the daily location history of the selected active users, where the nodes are physical places with semantically labeled activity types, and the edges represent the transitions. Analyzing the isomorphic structure of the mobility networks uncovered 16 types of location-based motifs, which describe over 83% of the networks in both cities and are comparable to those from previous studies. With detailed and semantically labeled transitions between every two activities, we further dissected the general location-based motifs into activity-based motifs, where 16 common activity-based motifs describe more than 57% transitional behaviors in the daily activities in the two cities. The integration of geographic context from the synthesis of geo-located Twitter data with land use parcels enables us to reveal unique activity motifs that form the fundamental elements embedded in complex urban activities.
Split then Refine: Stacked Attention-guided ResUNets for Blind Single Image Visible Watermark Removal
Venue: arXiv cs.CV 2020
Year: 2020

Digital watermark is a commonly used technique to protect the copyright of medias. Simultaneously, to increase the robustness of watermark, attacking technique, such as watermark removal, also gets the attention from the community. Previous watermark removal methods require to gain the watermark location from users or train a multi-task network to recover the background indiscriminately. However, when jointly learning, the network performs better on watermark detection than recovering the texture. Inspired by this observation and to erase the visible watermarks blindly, we propose a novel two-stage framework with a stacked attention-guided ResUNets to simulate the process of detection, removal and refinement. In the first stage, we design a multi-task network called SplitNet. It learns the basis features for three sub-tasks altogether while the task-specific features separately use multiple channel attentions. Then, with the predicted mask and coarser restored image, we design RefineNet to smooth the watermarked region with a mask-guided spatial attention. Besides network structure, the proposed algorithm also combines multiple perceptual losses for better quality both visually and numerically. We extensively evaluate our algorithm over four different datasets under various settings and the experiments show that our approach outperforms other state-of-the-art methods by a large margin. The code is available at http://github.com/vinthony/deep-blind-watermark-removal.
Efficient Online Trajectory Planning for Integrator Chain Dynamics using Polynomial Elimination
Venue: arXiv cs.RO 2020
Year: 2020

Providing smooth reference trajectories can effectively increase performance and accuracy of tracking control applications while overshoot and unwanted vibrations are reduced. Trajectory planning computations can often be simplified significantly by transforming the system dynamics into decoupled integrator chains using methods such as feedback linearization, differential flatness or the controller canonical form. We present an efficient method to plan time optimal trajectories for integrator chains subject to derivative bound constraints. Therefore, an algebraic precomputation algorithm formulates the necessary conditions for time optimality in form of a set of polynomial systems, followed by a symbolic polynomial elimination using Gröbner bases. A fast online algorithm then plans the trajectories by calculating the roots of the decomposed polynomial systems. These roots describe the switching time instants of the input signal and the full trajectory simply follows by multiple integration. This method presents a systematic way to compute time optimal trajectories exactly via algebraic calculations without numerical approximation iterations. It is applied to various trajectory types with different continuity order, asymmetric derivative bounds and non-rest initial and final states.
Uncertainty Estimation in Deep Neural Networks for Point Cloud Segmentation in Factory Planning
Venue: arXiv cs.CV 2020
Year: 2020

The digital factory provides undoubtedly a great potential for future production systems in terms of efficiency and effectivity. A key aspect on the way to realize the digital copy of a real factory is the understanding of complex indoor environments on the basis of 3D data. In order to generate an accurate factory model including the major components, i.e. building parts, product assets and process details, the 3D data collected during digitalization can be processed with advanced methods of deep learning. In this work, we propose a fully Bayesian and an approximate Bayesian neural network for point cloud segmentation. This allows us to analyze how different ways of estimating uncertainty in these networks improve segmentation results on raw 3D point clouds. We achieve superior model performance for both, the Bayesian and the approximate Bayesian model compared to the frequentist one. This performance difference becomes even more striking when incorporating the networks' uncertainty in their predictions. For evaluation we use the scientific data set S3DIS as well as a data set, which was collected by the authors at a German automotive production plant. The methods proposed in this work lead to more accurate segmentation results and the incorporation of uncertainty information makes this approach especially applicable to safety critical applications.
Budgeted and Non-budgeted Causal Bandits
Venue: arXiv cs.LG 2020
Year: 2020

Learning good interventions in a causal graph can be modelled as a stochastic multi-armed bandit problem with side-information. First, we study this problem when interventions are more expensive than observations and a budget is specified. If there are no backdoor paths from an intervenable node to the reward node then we propose an algorithm to minimize simple regret that optimally trades-off observations and interventions based on the cost of intervention. We also propose an algorithm that accounts for the cost of interventions, utilizes causal side-information, and minimizes the expected cumulative regret without exceeding the budget. Our cumulative-regret minimization algorithm performs better than standard algorithms that do not take side-information into account. Finally, we study the problem of learning best interventions without budget constraint in general graphs and give an algorithm that achieves constant expected cumulative regret in terms of the instance parameters when the parent distribution of the reward variable for each intervention is known. Our results are experimentally validated and compared to the best-known bounds in the current literature.
Deliberative and Conceptual Inference in Service Robots
Venue: arXiv cs.RO 2020
Year: 2020

Service robots need to reason to support people in daily life situations. Reasoning is an expensive resource that should be used on demand whenever the expectations of the robot do not match the situation of the world and the execution of the task is broken down; in such scenarios the robot must perform the common sense daily life inference cycle consisting on diagnosing what happened, deciding what to do about it, and inducing and executing a plan, recurring in such behavior until the service task can be resumed. Here we examine two strategies to implement this cycle: (1) a pipe-line strategy involving abduction, decision-making and planning, which we call deliberative inference and (2) the use of the knowledge and preferences stored in the robot's knowledge-base, which we call conceptual inference. The former involves an explicit definition of a problem space that is explored through heuristic search, and the latter is based on conceptual knowledge including the human user preferences, and its representation requires a non-monotonic knowledge-based system. We compare the strengths and limitations of both approaches. We also describe a service robot conceptual model and architecture capable of supporting the daily life inference cycle during the execution of a robotics service task. The model is centered in the declarative specification and interpretation of robot's communication and task structure. We also show the implementation of this framework in the fully autonomous robot Golem-III. The framework is illustrated with two demonstration scenarios.
Learning Contextual Causality from Time-consecutive Images
Venue: arXiv cs.AI 2020
Year: 2020

Causality knowledge is crucial for many artificial intelligence systems. Conventional textual-based causality knowledge acquisition methods typically require laborious and expensive human annotations. As a result, their scale is often limited. Moreover, as no context is provided during the annotation, the resulting causality knowledge records (e.g., ConceptNet) typically do not take the context into consideration. To explore a more scalable way of acquiring causality knowledge, in this paper, we jump out of the textual domain and investigate the possibility of learning contextual causality from the visual signal. Compared with pure text-based approaches, learning causality from the visual signal has the following advantages: (1) Causality knowledge belongs to the commonsense knowledge, which is rarely expressed in the text but rich in videos; (2) Most events in the video are naturally time-ordered, which provides a rich resource for us to mine causality knowledge from; (3) All the objects in the video can be used as context to study the contextual property of causal relations. In detail, we first propose a high-quality dataset Vis-Causal and then conduct experiments to demonstrate that with good language and visual representation models as well as enough training signals, it is possible to automatically discover meaningful causal knowledge from the videos. Further analysis also shows that the contextual property of causal relations indeed exists, taking which into consideration might be crucial if we want to use the causality knowledge in real applications, and the visual signal could serve as a good resource for learning such contextual causality.
On Convergence of Gradient Expected Sarsa($λ$)
Venue: arXiv cs.LG 2020
Year: 2020

We study the convergence of $\mathtt{Expected~Sarsa}(λ)$ with linear function approximation. We show that applying the off-line estimate (multi-step bootstrapping) to $\mathtt{Expected~Sarsa}(λ)$ is unstable for off-policy learning. Furthermore, based on convex-concave saddle-point framework, we propose a convergent $\mathtt{Gradient~Expected~Sarsa}(λ)$ ($\mathtt{GES}(λ)$) algorithm. The theoretical analysis shows that our $\mathtt{GES}(λ)$ converges to the optimal solution at a linear convergence rate, which is comparable to extensive existing state-of-the-art gradient temporal difference learning algorithms. Furthermore, we develop a Lyapunov function technique to investigate how the step-size influences finite-time performance of $\mathtt{GES}(λ)$, such technique of Lyapunov function can be potentially generalized to other GTD algorithms. Finally, we conduct experiments to verify the effectiveness of our $\mathtt{GES}(λ)$.
Deep Optimized Priors for 3D Shape Modeling and Reconstruction
Venue: arXiv cs.CV 2020
Year: 2020

Many learning-based approaches have difficulty scaling to unseen data, as the generality of its learned prior is limited to the scale and variations of the training samples. This holds particularly true with 3D learning tasks, given the sparsity of 3D datasets available. We introduce a new learning framework for 3D modeling and reconstruction that greatly improves the generalization ability of a deep generator. Our approach strives to connect the good ends of both learning-based and optimization-based methods. In particular, unlike the common practice that fixes the pre-trained priors at test time, we propose to further optimize the learned prior and latent code according to the input physical measurements after the training. We show that the proposed strategy effectively breaks the barriers constrained by the pre-trained priors and could lead to high-quality adaptation to unseen data. We realize our framework using the implicit surface representation and validate the efficacy of our approach in a variety of challenging tasks that take highly sparse or collapsed observations as input. Experimental results show that our approach compares favorably with the state-of-the-art methods in terms of both generality and accuracy.
Federated Learning under Importance Sampling
Venue: arXiv cs.LG 2020
Year: 2020

Federated learning encapsulates distributed learning strategies that are managed by a central unit. Since it relies on using a selected number of agents at each iteration, and since each agent, in turn, taps into its local data, it is only natural to study optimal sampling policies for selecting agents and their data in federated learning implementations. Usually, only uniform sampling schemes are used. However, in this work, we examine the effect of importance sampling and devise schemes for sampling agents and data non-uniformly guided by a performance measure. We find that in schemes involving sampling without replacement, the performance of the resulting architecture is controlled by two factors related to data variability at each agent, and model variability across agents. We illustrate the theoretical findings with experiments on simulated and real data and show the improvement in performance that results from the proposed strategies.