OpenRaiser/Intern-Atlas huggingface dataset:
Leveraging Local and Global Descriptors in Parallel to Search Correspondences for Visual Localization
Venue: arXiv cs.CV 2020
Year: 2020

Visual localization to compute 6DoF camera pose from a given image has wide applications such as in robotics, virtual reality, augmented reality, etc. Two kinds of descriptors are important for the visual localization. One is global descriptors that extract the whole feature from each image. The other is local descriptors that extract the local feature from each image patch usually enclosing a key point. More and more methods of the visual localization have two stages: at first to perform image retrieval by global descriptors and then from the retrieval feedback to make 2D-3D point correspondences by local descriptors. The two stages are in serial for most of the methods. This simple combination has not achieved superiority of fusing local and global descriptors. The 3D points obtained from the retrieval feedback are as the nearest neighbor candidates of the 2D image points only by global descriptors. Each of the 2D image points is also called a query local feature when performing the 2D-3D point correspondences. In this paper, we propose a novel parallel search framework, which leverages advantages of both local and global descriptors to get nearest neighbor candidates of a query local feature. Specifically, besides using deep learning based global descriptors, we also utilize local descriptors to construct random tree structures for obtaining nearest neighbor candidates of the query local feature. We propose a new probabilistic model and a new deep learning based local descriptor when constructing the random trees. A weighted Hamming regularization term to keep discriminativeness after binarization is given in the loss function for the proposed local descriptor. The loss function co-trains both real and binary descriptors of which the results are integrated into the random trees.
Fundamental Limits of Byzantine Agreement
Venue: arXiv cs.IT 2020
Year: 2020

Byzantine agreement (BA) is a distributed consensus problem where $n$ processors want to reach agreement on an $\ell$-bit message or value, but up to $t$ of the processors are dishonest or faulty. The challenge of this BA problem lies in achieving agreement despite the presence of dishonest processors who may arbitrarily deviate from the designed protocol. The quality of a BA protocol is measured primarily by using the following three parameters: the number of processors $n$ as a function of $t$ allowed (resilience); the number of rounds (round complexity, denoted by $r$); and the total number of communication bits (communication complexity, denoted by $b$). For any error-free BA protocol, the known lower bounds on those three parameters are $n\geq 3t+1$, $r\geq t+1$ and $b\geqΩ(\max\{n\ell, nt\})$, respectively, where a protocol that is guaranteed to be correct in all executions is said to be error free. In this work by using coding theory, together with graph theory and linear algebra, we design a coded BA protocol (termed as COOL) that achieves consensus on an $\ell$-bit message with optimal resilience, asymptotically optimal round complexity, and asymptotically optimal communication complexity when $\ell \geq t\log t$, simultaneously. The proposed COOL is an error-free and deterministic BA protocol that does not rely on cryptographic technique. It is secure against computationally unbounded adversary. With the achievable performance by the proposed COOL and the known lower bounds, we characterize the optimal communication complexity exponent as \[β^*(α,δ)=\max\{1+α,1+δ\}\] for $β= \lim_{n\to\infty}\log b/\log n$, $α=\lim_{n \to \infty} \log \ell/\log n$ and $δ=\lim_{n\to\infty} \log t/\log n$. This work reveals that coding is an effective approach for achieving the fundamental limits of Byzantine agreement and its variants.
Hierarchical Affordance Discovery using Intrinsic Motivation
Venue: arXiv cs.AI 2020
Year: 2020

To be capable of lifelong learning in a real-life environment, robots have to tackle multiple challenges. Being able to relate physical properties they may observe in their environment to possible interactions they may have is one of them. This skill, named affordance learning, is strongly related to embodiment and is mastered through each person's development: each individual learns affordances differently through their own interactions with their surroundings. Current methods for affordance learning usually use either fixed actions to learn these affordances or focus on static setups involving a robotic arm to be operated. In this article, we propose an algorithm using intrinsic motivation to guide the learning of affordances for a mobile robot. This algorithm is capable to autonomously discover, learn and adapt interrelated affordances without pre-programmed actions. Once learned, these affordances may be used by the algorithm to plan sequences of actions in order to perform tasks of various difficulties. We then present one experiment and analyse our system before comparing it with other approaches from reinforcement learning and affordance learning.
Few-shot Font Generation with Localized Style Representations and Factorization
Venue: arXiv cs.CV 2020
Year: 2020

Automatic few-shot font generation is a practical and widely studied problem because manual designs are expensive and sensitive to the expertise of designers. Existing few-shot font generation methods aim to learn to disentangle the style and content element from a few reference glyphs, and mainly focus on a universal style representation for each font style. However, such approach limits the model in representing diverse local styles, and thus makes it unsuitable to the most complicated letter system, e.g., Chinese, whose characters consist of a varying number of components (often called "radical") with a highly complex structure. In this paper, we propose a novel font generation method by learning localized styles, namely component-wise style representations, instead of universal styles. The proposed style representations enable us to synthesize complex local details in text designs. However, learning component-wise styles solely from reference glyphs is infeasible in the few-shot font generation scenario, when a target script has a large number of components, e.g., over 200 for Chinese. To reduce the number of reference glyphs, we simplify component-wise styles by a product of component factor and style factor, inspired by low-rank matrix factorization. Thanks to the combination of strong representation and a compact factorization strategy, our method shows remarkably better few-shot font generation results (with only 8 reference glyph images) than other state-of-the-arts, without utilizing strong locality supervision, e.g., location of each component, skeleton, or strokes. The source code is available at https://github.com/clovaai/lffont.
Unsupervised Feature Learning for Event Data: Direct vs Inverse Problem Formulation
Venue: arXiv cs.CV 2020
Year: 2020

Event-based cameras record an asynchronous stream of per-pixel brightness changes. As such, they have numerous advantages over the standard frame-based cameras, including high temporal resolution, high dynamic range, and no motion blur. Due to the asynchronous nature, efficient learning of compact representation for event data is challenging. While it remains not explored the extent to which the spatial and temporal event "information" is useful for pattern recognition tasks. In this paper, we focus on single-layer architectures. We analyze the performance of two general problem formulations: the direct and the inverse, for unsupervised feature learning from local event data (local volumes of events described in space-time). We identify and show the main advantages of each approach. Theoretically, we analyze guarantees for an optimal solution, possibility for asynchronous, parallel parameter update, and the computational complexity. We present numerical experiments for object recognition. We evaluate the solution under the direct and the inverse problem and give a comparison with the state-of-the-art methods. Our empirical results highlight the advantages of both approaches for representation learning from event data. We show improvements of up to 9 % in the recognition accuracy compared to the state-of-the-art methods from the same class of methods.
Variant-based Equational Unification under Constructor Symbols
Venue: arXiv cs.LO 2020
Year: 2020

Equational unification of two terms consists of finding a substitution that, when applied to both terms, makes them equal modulo some equational properties. A narrowing-based equational unification algorithm relying on the concept of the variants of a term is available in the most recent version of Maude, version 3.0, which provides quite sophisticated unification features. A variant of a term t is a pair consisting of a substitution sigma and the canonical form of tsigma. Variant-based unification is decidable when the equational theory satisfies the finite variant property. However, this unification procedure does not take into account constructor symbols and, thus, may compute many more unifiers than the necessary or may not be able to stop immediately. In this paper, we integrate the notion of constructor symbol into the variant-based unification algorithm. Our experiments on positive and negative unification problems show an impressive speedup.
Optimal Resource Allocation for Multi-user OFDMA-URLLC MEC Systems
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we study resource allocation algorithm design for multi-user orthogonal frequency division multiple access (OFDMA) ultra-reliable low latency communication (URLLC) in mobile edge computing (MEC) systems. To meet the stringent end-to-end delay and reliability requirements of URLLC MEC systems, we propose joint uplink-downlink resource allocation and finite blocklength transmission. Furthermore, we employ a partial time overlap between the uplink and downlink frames to minimize the end-to-end delay, which introduces a new time causality constraint. The proposed resource allocation algorithm is formulated as an optimization problem for minimization of the total weighted power consumption of the network under a constraint on the number of URLLC user bits computed within the maximum allowable computation time, i.e., the end-to-end delay of a computation task. Despite the non-convexity of the formulated optimization problem, we develop a globally optimal solution using a branch-and-bound approach based on discrete monotonic optimization theory. The branch-and-bound algorithm minimizes an upper bound on the total power consumption until convergence to the globally optimal value. Furthermore, to strike a balance between computational complexity and performance, we propose two efficient suboptimal algorithms based on successive convex approximation and second-order cone techniques. Our simulation results reveal that the proposed resource allocation algorithm design facilitates URLLC in MEC systems, and yields significant power savings compared to three baseline schemes. Moreover, our simulation results show that the proposed suboptimal algorithms offer different trade-offs between performance and complexity and attain a close-to-optimal performance at comparatively low complexity.
Real-time Millimeter Wave Omnidirectional Channel Sounder Using Phased Array Antennas
Venue: arXiv cs.IT 2020
Year: 2020

Characterization of the millimeter wave wireless channel is needed to facilitate fully connected vehicular communication in the future. To study the multipath-rich, rapidly varying nature of the vehicular propagation environment, fast millimeter wave channel sounders are required. We present a channel sounder design capable of covering 360 degrees in azimuth and 60 degrees in elevation with 200 individual beam directions in 6.25 ms by using four phased arrays simultaneously. The channel measurements are accompanied by high resolution positioning and video data, allowing channel sounding to be conducted while either the transmitter, or the receiver, or both are moving. Channel sounding campaigns were conducted at multiple urban locations with light traffic conditions in Austin, Texas. Preliminary results show that beam selection at the receiver can lower the effective pathloss exponent to 1.6 for line-of-sight and 2.25 for non line-of-sight.
Residual Embedding Similarity-Based Network Selection for Predicting Brain Network Evolution Trajectory from a Single Observation
Venue: arXiv cs.CV 2020
Year: 2020

While existing predictive frameworks are able to handle Euclidean structured data (i.e, brain images), they might fail to generalize to geometric non-Euclidean data such as brain networks. Besides, these are rooted the sample selection step in using Euclidean or learned similarity measure between vectorized training and testing brain networks. Such sample connectomic representation might include irrelevant and redundant features that could mislead the training sample selection step. Undoubtedly, this fails to exploit and preserve the topology of the brain connectome. To overcome this major drawback, we propose Residual Embedding Similarity-Based Network selection (RESNets) for predicting brain network evolution trajectory from a single timepoint. RESNets first learns a compact geometric embedding of each training and testing sample using adversarial connectome embedding network. This nicely reduces the high-dimensionality of brain networks while preserving their topological properties via graph convolutional networks. Next, to compute the similarity between subjects, we introduce the concept of a connectional brain template (CBT), a fixed network reference, where we further represent each training and testing network as a deviation from the reference CBT in the embedding space. As such, we select the most similar training subjects to the testing subject at baseline by comparing their learned residual embeddings with respect to the pre-defined CBT. Once the best training samples are selected at baseline, we simply average their corresponding brain networks at follow-up timepoints to predict the evolution trajectory of the testing network. Our experiments on both healthy and disordered brain networks demonstrate the success of our proposed method in comparison to RESNets ablated versions and traditional approaches.
Phishing Detection Using Machine Learning Techniques
Venue: arXiv cs.CR 2020
Year: 2020

The Internet has become an indispensable part of our life, However, It also has provided opportunities to anonymously perform malicious activities like Phishing. Phishers try to deceive their victims by social engineering or creating mock-up websites to steal information such as account ID, username, password from individuals and organizations. Although many methods have been proposed to detect phishing websites, Phishers have evolved their methods to escape from these detection methods. One of the most successful methods for detecting these malicious activities is Machine Learning. This is because most Phishing attacks have some common characteristics which can be identified by machine learning methods. In this paper, we compared the results of multiple machine learning methods for predicting phishing websites.
Multiple interaction learning with question-type prior knowledge for constraining answer search space in visual question answering
Venue: arXiv cs.CV 2020
Year: 2020

Different approaches have been proposed to Visual Question Answering (VQA). However, few works are aware of the behaviors of varying joint modality methods over question type prior knowledge extracted from data in constraining answer search space, of which information gives a reliable cue to reason about answers for questions asked in input images. In this paper, we propose a novel VQA model that utilizes the question-type prior information to improve VQA by leveraging the multiple interactions between different joint modality methods based on their behaviors in answering questions from different types. The solid experiments on two benchmark datasets, i.e., VQA 2.0 and TDIUC, indicate that the proposed method yields the best performance with the most competitive approaches.
Bounds on the Spectral Sparsification of Symmetric and Off-Diagonal Nonnegative Real Matrices
Venue: arXiv cs.DS 2020
Year: 2020

We say that a square real matrix $M$ is \emph{off-diagonal nonnegative} if and only if all entries outside its diagonal are nonnegative real numbers. In this note we show that for any off-diagonal nonnegative symmetric matrix $M$, there exists a nonnegative symmetric matrix $\widehat{M}$ which is sparse and close in spectrum to $M$.
Visualization of Human Spine Biomechanics for Spinal Surgery
Venue: arXiv cs.GR 2020
Year: 2020

We propose a visualization application, designed for the exploration of human spine simulation data. Our goal is to support research in biomechanical spine simulation and advance efforts to implement simulation-backed analysis in surgical applications. Biomechanical simulation is a state-of-the-art technique for analyzing load distributions of spinal structures. Through the inclusion of patient-specific data, such simulations may facilitate personalized treatment and customized surgical interventions. Difficulties in spine modelling and simulation can be partly attributed to poor result representation, which may also be a hindrance when introducing such techniques into a clinical environment. Comparisons of measurements across multiple similar anatomical structures and the integration of temporal data make commonly available diagrams and charts insufficient for an intuitive and systematic display of results. Therefore, we facilitate methods such as multiple coordinated views, abstraction and focus and context to display simulation outcomes in a dedicated tool. By linking the result data with patient-specific anatomy, we make relevant parameters tangible for clinicians. Furthermore, we introduce new concepts to show the directions of impact force vectors, which were not accessible before. We integrated our toolset into a spine segmentation and simulation pipeline and evaluated our methods with both surgeons and biomechanical researchers. When comparing our methods against standard representations that are currently in use, we found increases in accuracy and speed in data exploration tasks. In a qualitative review, domain experts deemed the tool highly useful when dealing with simulation result data, which typically combines time-dependent patient movement and the resulting force distributions on spinal structures.
Label-Efficient Multi-Task Segmentation using Contrastive Learning
Venue: arXiv cs.CV 2020
Year: 2020

Obtaining annotations for 3D medical images is expensive and time-consuming, despite its importance for automating segmentation tasks. Although multi-task learning is considered an effective method for training segmentation models using small amounts of annotated data, a systematic understanding of various subtasks is still lacking. In this study, we propose a multi-task segmentation model with a contrastive learning based subtask and compare its performance with other multi-task models, varying the number of labeled data for training. We further extend our model so that it can utilize unlabeled data through the regularization branch in a semi-supervised manner. We experimentally show that our proposed method outperforms other multi-task methods including the state-of-the-art fully supervised model when the amount of annotated data is limited.
A Simple Yet Effective Method for Video Temporal Grounding with Cross-Modality Attention
Venue: arXiv cs.CV 2020
Year: 2020

The task of language-guided video temporal grounding is to localize the particular video clip corresponding to a query sentence in an untrimmed video. Though progress has been made continuously in this field, some issues still need to be resolved. First, most of the existing methods rely on the combination of multiple complicated modules to solve the task. Second, due to the semantic gaps between the two different modalities, aligning the information at different granularities (local and global) between the video and the language is significant, which is less addressed. Last, previous works do not consider the inevitable annotation bias due to the ambiguities of action boundaries. To address these limitations, we propose a simple two-branch Cross-Modality Attention (CMA) module with intuitive structure design, which alternatively modulates two modalities for better matching the information both locally and globally. In addition, we introduce a new task-specific regression loss function, which improves the temporal grounding accuracy by alleviating the impact of annotation bias. We conduct extensive experiments to validate our method, and the results show that just with this simple model, it can outperform the state of the arts on both Charades-STA and ActivityNet Captions datasets.
A Token-wise CNN-based Method for Sentence Compression
Venue: arXiv cs.CL 2020
Year: 2020

Sentence compression is a Natural Language Processing (NLP) task aimed at shortening original sentences and preserving their key information. Its applications can benefit many fields e.g. one can build tools for language education. However, current methods are largely based on Recurrent Neural Network (RNN) models which suffer from poor processing speed. To address this issue, in this paper, we propose a token-wise Convolutional Neural Network, a CNN-based model along with pre-trained Bidirectional Encoder Representations from Transformers (BERT) features for deletion-based sentence compression. We also compare our model with RNN-based models and fine-tuned BERT. Although one of the RNN-based models outperforms marginally other models given the same input, our CNN-based model was ten times faster than the RNN-based approach.
Augmented Convolutional LSTMs for Generation of High-Resolution Climate Change Projections
Venue: arXiv cs.CV 2020
Year: 2020

Projection of changes in extreme indices of climate variables such as temperature and precipitation are critical to assess the potential impacts of climate change on human-made and natural systems, including critical infrastructures and ecosystems. While impact assessment and adaptation planning rely on high-resolution projections (typically in the order of a few kilometers), state-of-the-art Earth System Models (ESMs) are available at spatial resolutions of few hundreds of kilometers. Current solutions to obtain high-resolution projections of ESMs include downscaling approaches that consider the information at a coarse-scale to make predictions at local scales. Complex and non-linear interdependence among local climate variables (e.g., temperature and precipitation) and large-scale predictors (e.g., pressure fields) motivate the use of neural network-based super-resolution architectures. In this work, we present auxiliary variables informed spatio-temporal neural architecture for statistical downscaling. The current study performs daily downscaling of precipitation variable from an ESM output at 1.15 degrees (~115 km) to 0.25 degrees (25 km) over the world's most climatically diversified country, India. We showcase significant improvement gain against three popular state-of-the-art baselines with a better ability to predict extreme events. To facilitate reproducible research, we make available all the codes, processed datasets, and trained models in the public domain.