OpenRaiser/Intern-Atlas huggingface dataset:
On Hallucinating Context and Background Pixels from a Face Mask using Multi-scale GANs
Venue: arXiv cs.CV 2020
Year: 2020

We propose a multi-scale GAN model to hallucinate realistic context (forehead, hair, neck, clothes) and background pixels automatically from a single input face mask. Instead of swapping a face on to an existing picture, our model directly generates realistic context and background pixels based on the features of the provided face mask. Unlike face inpainting algorithms, it can generate realistic hallucinations even for a large number of missing pixels. Our model is composed of a cascaded network of GAN blocks, each tasked with hallucination of missing pixels at a particular resolution while guiding the synthesis process of the next GAN block. The hallucinated full face image is made photo-realistic by using a combination of reconstruction, perceptual, adversarial and identity preserving losses at each block of the network. With a set of extensive experiments, we demonstrate the effectiveness of our model in hallucinating context and background pixels from face masks varying in facial pose, expression and lighting, collected from multiple datasets subject disjoint with our training data. We also compare our method with two popular face swapping and face completion methods in terms of visual quality and recognition performance. Additionally, we analyze our cascaded pipeline and compare it with the recently proposed progressive growing of GANs.
Integrating domain knowledge: using hierarchies to improve deep classifiers
Venue: arXiv cs.CV 2020
Year: 2020

One of the most prominent problems in machine learning in the age of deep learning is the availability of sufficiently large annotated datasets. For specific domains, e.g. animal species, a long-tail distribution means that some classes are observed and annotated insufficiently. Additional labels can be prohibitively expensive, e.g. because domain experts need to be involved. However, there is more information available that is to the best of our knowledge not exploited accordingly. In this paper, we propose to make use of preexisting class hierarchies like WordNet to integrate additional domain knowledge into classification. We encode the properties of such a class hierarchy into a probabilistic model. From there, we derive a novel label encoding and a corresponding loss function. On the ImageNet and NABirds datasets our method offers a relative improvement of 10.4% and 9.6% in accuracy over the baseline respectively. After less than a third of training time, it is already able to match the baseline's fine-grained recognition performance. Both results show that our suggested method is efficient and effective.
Semantic Security via Seeded Modular Coding Schemes and Ramanujan Graphs
Venue: arXiv cs.IT 2020
Year: 2020

A novel type of functions called biregular irreducible functions is introduced and applied as security components (instead of, e.g., universal hash functions) in seeded modular wiretap coding schemes, whose second component is an error-correcting code. These schemes are called modular BRI schemes. An upper bound on the semantic security information leakage of modular BRI schemes in a one-shot setting is derived which separates the effects of the biregular irreducible function on the one hand and the error-correcting code plus the channel on the other hand. The effect of the biregular irreducible function is described by the second-largest eigenvalue of an associated stochastic matrix. A characterization of biregular irreducible functions is given in terms of connected edge-disjoint biregular graphs. It allows for the construction of new biregular irreducible functions from families of edge-disjoint Ramanujan graphs, which are shown to exist. A frequently used arithmetic universal hash function can be converted into a biregular irreducible function for certain parameters. Sequences of Ramanujan biregular irreducible functions are constructed which exhibit an optimal trade-off between the size of the regularity set and the rate of decrease of the associated second-largest eigenvalue. Together with the one-shot bound on the information leakage, the existence of these sequences implies an asymptotic coding result for modular BRI schemes applied to discrete and Gaussian wiretap channels. It shows that the separation of error correction and security as done in a modular BRI scheme is secrecy capacity-achieving for discrete and Gaussian wiretap channels. The same holds for a derived construction where the seed is generated locally by the sender and reused several times. Finally, optimal sequences of biregular irreducible functions used in the above constructions must be nearly Ramanujan.
Event-based Gesture Recognition with Dynamic Background Suppression using Smartphone Computational Capabilities
Venue: arXiv cs.CV 2020
Year: 2020

This paper introduces a framework of gesture recognition operating on the output of an event based camera using the computational resources of a mobile phone. We will introduce a new development around the concept of time-surfaces modified and adapted to run on the limited computational resources of a mobile platform. We also introduce a new method to remove dynamically backgrounds that makes full use of the high temporal resolution of event-based cameras. We assess the performances of the framework by operating on several dynamic scenarios in uncontrolled lighting conditions indoors and outdoors. We also introduce a new publicly available event-based dataset for gesture recognition selected through a clinical process to allow human-machine interactions for the visually-impaired and the elderly. We finally report comparisons with prior works that tackled event-based gesture recognition reporting comparable if not superior results if taking into account the limited computational and memory constraints of the used hardware.
Visual Localization Under Appearance Change: Filtering Approaches
Venue: arXiv cs.CV 2020
Year: 2020

A major focus of current research on place recognition is visual localization for autonomous driving. In this scenario, as cameras will be operating continuously, it is realistic to expect videos as an input to visual localization algorithms, as opposed to the single-image querying approach used in other visual localization works. In this paper, we show that exploiting temporal continuity in the testing sequence significantly improves visual localization - qualitatively and quantitatively. Although intuitive, this idea has not been fully explored in recent works. To this end, we propose two filtering approaches to exploit the temporal smoothness of image sequences: i) filtering on discrete domain with Hidden Markov Model, and ii) filtering on continuous domain with Monte Carlo-based visual localization. Our approaches rely on local features with an encoding technique to represent an image as a single vector. The experimental results on synthetic and real datasets show that our proposed methods achieve better results than state of the art (i.e., deep learning-based pose regression approaches) for the task on visual localization under significant appearance change. Our synthetic dataset and source code are made publicly available at https://sites.google.com/view/g2d-software/home and https://github.com/dadung/Visual-Localization-Filtering.
Polarity Loss for Zero-shot Object Detection
Venue: arXiv cs.CV 2020
Year: 2020

Conventional object detection models require large amounts of training data. In comparison, humans can recognize previously unseen objects by merely knowing their semantic description. To mimic similar behaviour, zero-shot object detection aims to recognize and localize 'unseen' object instances by using only their semantic information. The model is first trained to learn the relationships between visual and semantic domains for seen objects, later transferring the acquired knowledge to totally unseen objects. This setting gives rise to the need for correct alignment between visual and semantic concepts, so that the unseen objects can be identified using only their semantic attributes. In this paper, we propose a novel loss function called 'Polarity loss', that promotes correct visual-semantic alignment for an improved zero-shot object detection. On one hand, it refines the noisy semantic embeddings via metric learning on a 'Semantic vocabulary' of related concepts to establish a better synergy between visual and semantic domains. On the other hand, it explicitly maximizes the gap between positive and negative predictions to achieve better discrimination between seen, unseen and background objects. Our approach is inspired by embodiment theories in cognitive science, that claim human semantic understanding to be grounded in past experiences (seen objects), related linguistic concepts (word vocabulary) and visual perception (seen/unseen object images). We conduct extensive evaluations on MS-COCO and Pascal VOC datasets, showing significant improvements over state of the art.
DHLP 1&2: Giraph based distributed label propagation algorithms on heterogeneous drug-related networks
Venue: arXiv cs.DC 2020
Year: 2020

Background and Objective: Heterogeneous complex networks are large graphs consisting of different types of nodes and edges. The knowledge extraction from these networks is complicated. Moreover, the scale of these networks is steadily increasing. Thus, scalable methods are required. Methods: In this paper, two distributed label propagation algorithms for heterogeneous networks, namely DHLP-1 and DHLP-2 have been introduced. Biological networks are one type of the heterogeneous complex networks. As a case study, we have measured the efficiency of our proposed DHLP-1 and DHLP-2 algorithms on a biological network consisting of drugs, diseases, and targets. The subject we have studied in this network is drug repositioning but our algorithms can be used as general methods for heterogeneous networks other than the biological network. Results: We compared the proposed algorithms with similar non-distributed versions of them namely MINProp and Heter-LP. The experiments revealed the good performance of the algorithms in terms of running time and accuracy.
Kleene stars of the plane, polylogarithms and symmetries
Venue: arXiv cs.SC 2020
Year: 2020

We extend the definition and construct several bases for polylogarithms Li T , where T are some series, recognizable by a finite state (multiplicity) automaton of alphabet 4 X = {x 0 , x 1 }. The kernel of this new "polylogarithmic map" Li $\bullet$ is also characterized and provides a rewriting process which terminates to a normal form. We concentrate on algebraic and analytic aspects of this extension allowing index polylogarithms at non positive multi-indices, by rational series and regularize polyzetas at non positive multi-indices.
Learning pronunciation from a foreign language in speech synthesis networks
Venue: arXiv cs.CL 2020
Year: 2020

Although there are more than 6,500 languages in the world, the pronunciations of many phonemes sound similar across the languages. When people learn a foreign language, their pronunciation often reflects their native language's characteristics. This motivates us to investigate how the speech synthesis network learns the pronunciation from datasets from different languages. In this study, we are interested in analyzing and taking advantage of multilingual speech synthesis network. First, we train the speech synthesis network bilingually in English and Korean and analyze how the network learns the relations of phoneme pronunciation between the languages. Our experimental result shows that the learned phoneme embedding vectors are located closer if their pronunciations are similar across the languages. Consequently, the trained networks can synthesize the English speakers' Korean speech and vice versa. Using this result, we propose a training framework to utilize information from a different language. To be specific, we pre-train a speech synthesis network using datasets from both high-resource language and low-resource language, then we fine-tune the network using the low-resource language dataset. Finally, we conducted more simulations on 10 different languages to show it is generally extendable to other languages.
Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE)
Venue: arXiv cs.NE 2020
Year: 2020

A growing body of work underlines striking similarities between biological neural networks and recurrent, binary neural networks. A relatively smaller body of work, however, discusses similarities between learning dynamics employed in deep artificial neural networks and synaptic plasticity in spiking neural networks. The challenge preventing this is largely caused by the discrepancy between the dynamical properties of synaptic plasticity and the requirements for gradient backpropagation. Learning algorithms that approximate gradient backpropagation using locally synthesized gradients can overcome this challenge. Here, we show that synthetic gradients enable the derivation of Deep Continuous Local Learning (DECOLLE) in spiking neural networks. DECOLLE is capable of learning deep spatio-temporal representations from spikes relying solely on local information. Synaptic plasticity rules are derived systematically from user-defined cost functions and neural dynamics by leveraging existing autodifferentiation methods of machine learning frameworks. We benchmark our approach on the MNIST and the event-based neuromorphic DvsGesture dataset, on which DECOLLE performs comparably to the state-of-the-art. DECOLLE networks provide continuously learning machines that are relevant to biology and supportive of event-based, low-power computer vision architectures matching the accuracies of conventional computers on tasks where temporal precision and speed are essential.
Shortest Path Distance in Manhattan Poisson Line Cox Process
Venue: arXiv cs.IT 2020
Year: 2020

While the Euclidean distance characteristics of the Poisson line Cox process (PLCP) have been investigated in the literature, the analytical characterization of the path distances is still an open problem. In this paper, we solve this problem for the stationary Manhattan Poisson line Cox process (MPLCP), which is a variant of the PLCP. Specifically, we derive the exact cumulative distribution function (CDF) for the length of the shortest path to the nearest point of the MPLCP in the sense of path distance measured from two reference points: (i) the typical intersection of the Manhattan Poisson line process (MPLP), and (ii) the typical point of the MPLCP. We also discuss the application of these results in infrastructure planning, wireless communication, and transportation networks.
CCNet: Criss-Cross Attention for Semantic Segmentation
Venue: arXiv cs.CV 2020
Year: 2020

Contextual information is vital in visual understanding problems, such as semantic segmentation and object detection. We propose a Criss-Cross Network (CCNet) for obtaining full-image contextual information in a very effective and efficient way. Concretely, for each pixel, a novel criss-cross attention module harvests the contextual information of all the pixels on its criss-cross path. By taking a further recurrent operation, each pixel can finally capture the full-image dependencies. Besides, a category consistent loss is proposed to enforce the criss-cross attention module to produce more discriminative features. Overall, CCNet is with the following merits: 1) GPU memory friendly. Compared with the non-local block, the proposed recurrent criss-cross attention module requires 11x less GPU memory usage. 2) High computational efficiency. The recurrent criss-cross attention significantly reduces FLOPs by about 85% of the non-local block. 3) The state-of-the-art performance. We conduct extensive experiments on semantic segmentation benchmarks including Cityscapes, ADE20K, human parsing benchmark LIP, instance segmentation benchmark COCO, video segmentation benchmark CamVid. In particular, our CCNet achieves the mIoU scores of 81.9%, 45.76% and 55.47% on the Cityscapes test set, the ADE20K validation set and the LIP validation set respectively, which are the new state-of-the-art results. The source codes are available at \url{https://github.com/speedinghzl/CCNet}.
Scaling up Probabilistic Inference in Linear and Non-Linear Hybrid Domains by Leveraging Knowledge Compilation
Venue: arXiv cs.AI 2020
Year: 2020

Weighted model integration (WMI) extends weighted model counting (WMC) in providing a computational abstraction for probabilistic inference in mixed discrete-continuous domains. WMC has emerged as an assembly language for state-of-the-art reasoning in Bayesian networks, factor graphs, probabilistic programs and probabilistic databases. In this regard, WMI shows immense promise to be much more widely applicable, especially as many real-world applications involve attribute and feature spaces that are continuous and mixed. Nonetheless, state-of-the-art tools for WMI are limited and less mature than their propositional counterparts. In this work, we propose a new implementation regime that leverages propositional knowledge compilation for scaling up inference. In particular, we use sentential decision diagrams, a tractable representation of Boolean functions, as the underlying model counting and model enumeration scheme. Our regime performs competitively to state-of-the-art WMI systems but is also shown to handle a specific class of non-linear constraints over non-linear potentials.
Chiller: Contention-centric Transaction Execution and Data Partitioning for Modern Networks
Venue: arXiv cs.DB 2020
Year: 2020

Distributed transactions on high-overhead TCP/IP-based networks were conventionally considered to be prohibitively expensive and thus were avoided at all costs. To that end, the primary goal of almost any existing partitioning scheme is to minimize the number of cross-partition transactions. However, with the new generation of fast RDMA-enabled networks, this assumption is no longer valid. In fact, recent work has shown that distributed databases can scale even when the majority of transactions are cross-partition. In this paper, we first make the case that the new bottleneck which hinders truly scalable transaction processing in modern RDMA-enabled databases is data contention, and that optimizing for data contention leads to different partitioning layouts than optimizing for the number of distributed transactions. We then present Chiller, a new approach to data partitioning and transaction execution, which aims to minimize data contention for both local and distributed transactions. Finally, we evaluate Chiller using various workloads, and show that our partitioning and execution strategy outperforms traditional partitioning techniques which try to avoid distributed transactions, by up to a factor of 2.
Electrosense+: Crowdsourcing Radio Spectrum Decoding using IoT Receivers
Venue: arXiv cs.NI 2020
Year: 2020

Web spectrum monitoring systems based on crowdsourcing have recently gained popularity. These systems are however limited to applications of interest for governamental organizationsor telecom providers, and only provide aggregated information about spectrum statistics. Theresult is that there is a lack of interest for layman users to participate, which limits its widespreaddeployment. We present Electrosense+ which addresses this challenge and creates a general-purpose and open platform for spectrum monitoring using low-cost, embedded, and software-defined spectrum IoT sensors. Electrosense+ allows users to remotely decode specific parts ofthe radio spectrum. It builds on the centralized architecture of its predecessor, Electrosense, forcontrolling and monitoring the spectrum IoT sensors, but implements a real-time and peer-to-peercommunication system for scalable spectrum data decoding. We propose different mechanismsto incentivize the participation of users for deploying new sensors and keep them operational inthe Electrosense network. As a reward for the user, we propose an incentive accounting systembased on virtual tokens to encourage the participants to host IoT sensors. We present the newElectrosense+ system architecture and evaluate its performance at decoding various wireless sig-nals, including FM radio, AM radio, ADS-B, AIS, LTE, and ACARS.
$Ψ$ec: A Local Spectral Exterior Calculus
Venue: arXiv cs.NA 2020
Year: 2020

We introduce $Ψ\mathrm{ec}$, a discretization of Cartan's exterior calculus of differential forms using wavelets. Our construction consists of differential $r$-form wavelets with flexible directional localization that provide tight frames for the spaces $Ω^r(\mathbb{R}^n)$ of forms in $\mathbb{R}^2$ and $\mathbb{R}^3$. By construction, the wavelets satisfy the de Rahm co-chain complex, the Hodge decomposition, and that the $k$-dimensional integral of an $r$-form is an $(r-k)$-form. They also verify Stokes' theorem for differential forms, with the most efficient finite dimensional approximation attained using directionally localized, curvelet- or ridgelet-like forms. The construction of $Ψ\mathrm{ec}$ builds on the geometric simplicity of the exterior calculus in the Fourier domain. We establish this structure by extending existing results on the Fourier transform of differential forms to a frequency description of the exterior calculus, including, for example, a Plancherel theorem for forms and a description of the symbols of all important operators.
Sequential Effect Systems with Control Operators
Venue: arXiv cs.PL 2020
Year: 2020

Sequential effect systems are a class of effect system that exploits information about program order, rather than discarding it as traditional commutative effect systems do. This extra expressive power allows effect systems to reason about behavior over time, capturing properties such as atomicity, unstructured lock ownership, or even general safety properties. While we now understand the essential denotational (categorical) models fairly well, application of these ideas to real software is hampered by the sheer variety of source level control flow constructs in real languages. Denotational approaches are general enough to accommodate any particular control flow construct, but provide no guidance on the details, let alone applications. We address this new problem by appeal to a classic idea: macro-expression of commonly-used programming constructs in terms of control operators. We give an effect system for a subset of Racket's tagged delimited control operators, as a lifting of an effect system for a language without direct control operators. This gives the first account of sequential effects in the presence of general control operators. Using this system, we also re-derive the sequential effect system rules for control flow constructs previously shown sound directly, and derive sequential effect rules for new constructs not previously studied in the context of source-level sequential effect systems. This offers a way to directly extend source-level support for sequential effect systems to real programming languages.
Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environments
Venue: arXiv cs.CV 2020
Year: 2020

We study the problem of jointly reasoning about language and vision through a navigation and spatial reasoning task. We introduce the Touchdown task and dataset, where an agent must first follow navigation instructions in a real-life visual urban environment, and then identify a location described in natural language to find a hidden object at the goal position. The data contains 9,326 examples of English instructions and spatial descriptions paired with demonstrations. Empirical analysis shows the data presents an open challenge to existing methods, and qualitative linguistic analysis shows that the data displays richer use of spatial reasoning compared to related resources.