OpenRaiser/Intern-Atlas huggingface dataset:
Semantic Segmentation from Remote Sensor Data and the Exploitation of Latent Learning for Classification of Auxiliary Tasks
Venue: arXiv cs.CV 2019
Year: 2019

In this paper we address three different aspects of semantic segmentation from remote sensor data using deep neural networks. Firstly, we focus on the semantic segmentation of buildings from remote sensor data and propose ICT-Net. The proposed network has been tested on the INRIA and AIRS benchmark datasets and is shown to outperform all other state of the art by more than 1.5% and 1.8% on the Jaccard index, respectively. Secondly, as the building classification is typically the first step of the reconstruction process, we investigate the relationship of the classification accuracy to the reconstruction accuracy. Finally, we present the simple yet compelling concept of latent learning and the implications it carries within the context of deep learning. We posit that a network trained on a primary task (i.e. building classification) is unintentionally learning about auxiliary tasks (e.g. the classification of road, tree, etc) which are complementary to the primary task. We extensively tested the proposed technique on the ISPRS benchmark dataset which contains multi-label ground truth, and report an average classification accuracy (F1 score) of 54.29% (SD=17.03) for roads, 10.15% (SD=2.54) for cars, 24.11% (SD=5.25) for trees, 42.74% (SD=6.62) for low vegetation, and 18.30% (SD=16.08) for clutter. The source code and supplemental material is publicly available at http://www.theICTlab.org/lp/2019ICT-Net/.
Scalable Resilience Against Node Failures for Communication-Hiding Preconditioned Conjugate Gradient and Conjugate Residual Methods
Venue: arXiv cs.DC 2019
Year: 2019

The observed and expected continued growth in the number of nodes in large-scale parallel computers gives rise to two major challenges: global communication operations are becoming major bottlenecks due to their limited scalability, and the likelihood of node failures is increasing. We study an approach for addressing these challenges in the context of solving large sparse linear systems. In particular, we focus on the pipelined preconditioned conjugate gradient (PPCG) method, which has been shown to successfully deal with the first of these challenges. In this paper, we address the second challenge. We present extensions to the PPCG solver and two of its variants which make them resilient against the failure of a compute node while fully preserving their communication-hiding properties and thus their scalability. The basic idea is to efficiently communicate a few redundant copies of local vector elements to neighboring nodes with very little overhead. In case a node fails, these redundant copies are gathered at a replacement node, which can then accurately reconstruct the lost parts of the solver's state. After that, the parallel solver can continue as in the failure-free scenario. Experimental evaluations of our approach illustrate on average very low runtime overheads compared to the standard non-resilient algorithms. This shows that scalable algorithmic resilience can be achieved at low extra cost.
Piecewise Linear Valued CSPs Solvable by Linear Programming Relaxation
Venue: arXiv cs.LO 2019
Year: 2019

Valued constraint satisfaction problems (VCSPs) are a large class of combinatorial optimisation problems. The computational complexity of VCSPs depends on the set of allowed cost functions in the input. Recently, the computational complexity of all VCSPs for finite sets of cost functions over finite domains has been classified. Many natural optimisation problems, however, cannot be formulated as VCSPs over a finite domain. We initiate the systematic investigation of infinite-domain VCSPs by studying the complexity of VCSPs for piecewise linear homogeneous cost functions. Such VCSPs can be solved in polynomial time if the cost functions are improved by fully symmetric fractional operations of all arities. We show this by reducing the problem to a finite-domain VCSP which can be solved using the basic linear program relaxation. It follows that VCSPs for submodular PLH cost functions can be solved in polynomial time; in fact, we show that submodular PLH functions form a maximally tractable class of PLH cost functions.
Tangent Space Separability in Feedforward Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

Hierarchical neural networks are exponentially more efficient than their corresponding "shallow" counterpart with the same expressive power, but involve huge number of parameters and require tedious amounts of training. By approximating the tangent subspace, we suggest a sparse representation that enables switching to shallow networks, GradNet after a very early training stage. Our experiments show that the proposed approximation of the metric improves and sometimes even surpasses the achievable performance of the original network significantly even after a few epochs of training the original feedforward network.
Data Science through the looking glass and what we found there
Venue: arXiv cs.LG 2019
Year: 2019

The recent success of machine learning (ML) has led to an explosive growth both in terms of new systems and algorithms built in industry and academia, and new applications built by an ever-growing community of data science (DS) practitioners. This quickly shifting panorama of technologies and applications is challenging for builders and practitioners alike to follow. In this paper, we set out to capture this panorama through a wide-angle lens, by performing the largest analysis of DS projects to date, focusing on questions that can help determine investments on either side. Specifically, we download and analyze: (a) over 6M Python notebooks publicly available on GITHUB, (b) over 2M enterprise DS pipelines developed within COMPANYX, and (c) the source code and metadata of over 900 releases from 12 important DS libraries. The analysis we perform ranges from coarse-grained statistical characterizations to analysis of library imports, pipelines, and comparative studies across datasets and time. We report a large number of measurements for our readers to interpret, and dare to draw a few (actionable, yet subjective) conclusions on (a) what systems builders should focus on to better serve practitioners, and (b) what technologies should practitioners bet on given current trends. We plan to automate this analysis and release associated tools and results periodically.
Deep Exemplar Networks for VQA and VQG
Venue: arXiv cs.CV 2019
Year: 2019

In this paper, we consider the problem of solving semantic tasks such as `Visual Question Answering' (VQA), where one aims to answers related to an image and `Visual Question Generation' (VQG), where one aims to generate a natural question pertaining to an image. Solutions for VQA and VQG tasks have been proposed using variants of encoder-decoder deep learning based frameworks that have shown impressive performance. Humans however often show generalization by relying on exemplar based approaches. For instance, the work by Tversky and Kahneman suggests that humans use exemplars when making categorizations and decisions. In this work, we propose the incorporation of exemplar based approaches towards solving these problems. Specifically, we incorporate exemplar based approaches and show that an exemplar based module can be incorporated in almost any of the deep learning architectures proposed in the literature and the addition of such a block results in improved performance for solving these tasks. Thus, just as the incorporation of attention is now considered de facto useful for solving these tasks, similarly, incorporating exemplars also can be considered to improve any proposed architecture for solving this task. We provide extensive empirical analysis for the same through various architectures, ablations, and state of the art comparisons.
Measuring the intelligence of an idealized mechanical knowing agent
Venue: arXiv cs.AI 2019
Year: 2019

We define a notion of the intelligence level of an idealized mechanical knowing agent. This is motivated by efforts within artificial intelligence research to define real-number intelligence levels of complicated intelligent systems. Our agents are more idealized, which allows us to define a much simpler measure of intelligence level for them. In short, we define the intelligence level of a mechanical knowing agent to be the supremum of the computable ordinals that have codes the agent knows to be codes of computable ordinals. We prove that if one agent knows certain things about another agent, then the former necessarily has a higher intelligence level than the latter. This allows our intelligence notion to serve as a stepping stone to obtain results which, by themselves, are not stated in terms of our intelligence notion (results of potential interest even to readers totally skeptical that our notion correctly captures intelligence). As an application, we argue that these results comprise evidence against the possibility of intelligence explosion (that is, the notion that sufficiently intelligent machines will eventually be capable of designing even more intelligent machines, which can then design even more intelligent machines, and so on).
Graph Convolutional Networks: analysis, improvements and results
Venue: arXiv cs.LG 2019
Year: 2019

In the current era of neural networks and big data, higher dimensional data is processed for automation of different application areas. Graphs represent a complex data organization in which dependencies between more than one object or activity occur. Due to the high dimensionality, this data creates challenges for machine learning algorithms. Graph convolutional networks were introduced to utilize the convolutional models concepts that shows good results. In this context, we enhanced two of the existing Graph convolutional network models by proposing four enhancements. These changes includes: hyper parameters optimization, convex combination of activation functions, topological information enrichment through clustering coefficients measure, and structural redesign of the network through addition of dense layers. We present extensive results on four state-of-art benchmark datasets. The performance is notable not only in terms of lesser computational cost compared to competitors, but also achieved competitive results for three of the datasets and state-of-the-art for the fourth dataset.
Exploiting the potential of deep reinforcement learning for classification tasks in high-dimensional and unstructured data
Venue: arXiv cs.LG 2019
Year: 2019

This paper presents a framework for efficiently learning feature selection policies which use less features to reach a high classification precision on large unstructured data. It uses a Deep Convolutional Autoencoder (DCAE) for learning compact feature spaces, in combination with recently-proposed Reinforcement Learning (RL) algorithms as Double DQN and Retrace.
Comparing Hierarchical Data Structures for Sparse Volume Rendering with Empty Space Skipping
Venue: arXiv cs.GR 2019
Year: 2019

Empty space skipping can be efficiently implemented with hierarchical data structures such as k-d trees and bounding volume hierarchies. This paper compares several recently published hierarchical data structures with regard to construction and rendering performance. The papers that form our prior work have primarily focused on interactively building the data structures and only showed that rendering performance is superior to using simple acceleration data structures such as uniform grids with macro cells. In the area of surface ray tracing, there exists a trade-off between construction and rendering performance of hierarchical data structures. In this paper we present performance comparisons for several empty space skipping data structures in order to determine if such a trade-off also exists for volume rendering with uniform data topologies.
An $\tilde{O}(\log^2 n)$-approximation algorithm for $2$-edge-connected dominating set
Venue: arXiv cs.DS 2019
Year: 2019

In the Connected Dominating Set problem we are given a graph $G=(V,E)$ and seek a minimum size dominating set $S \subseteq V$ such that the subgraph $G[S]$ of $G$ induced by $S$ is connected. In the $2$-Edge-Connected Dominating Set problem $G[S]$ should be $2$-edge-connected. We give the first non-trivial approximation algorithm for this problem, with expected approximation ratio $\tilde{O}(\log^2n)$.
Distributed Online Optimization with Long-Term Constraints
Venue: arXiv cs.LG 2019
Year: 2019

We consider distributed online convex optimization problems, where the distributed system consists of various computing units connected through a time-varying communication graph. In each time step, each computing unit selects a constrained vector, experiences a loss equal to an arbitrary convex function evaluated at this vector, and may communicate to its neighbors in the graph. The objective is to minimize the system-wide loss accumulated over time. We propose a decentralized algorithm with regret and cumulative constraint violation in $\mathcal{O}(T^{\max\{c,1-c\} })$ and $\mathcal{O}(T^{1-c/2})$, respectively, for any $c\in (0,1)$, where $T$ is the time horizon. When the loss functions are strongly convex, we establish improved regret and constraint violation upper bounds in $\mathcal{O}(\log(T))$ and $\mathcal{O}(\sqrt{T\log(T)})$. These regret scalings match those obtained by state-of-the-art algorithms and fundamental limits in the corresponding centralized online optimization problem (for both convex and strongly convex loss functions). In the case of bandit feedback, the proposed algorithms achieve a regret and constraint violation in $\mathcal{O}(T^{\max\{c,1-c/3 \} })$ and $\mathcal{O}(T^{1-c/2})$ for any $c\in (0,1)$. We numerically illustrate the performance of our algorithms for the particular case of distributed online regularized linear regression problems.
Reverse Fingerprinting
Venue: arXiv cs.CR 2019
Year: 2019

Software connected to the Internet is an attractive target for attackers: as soon as a security flaw is known, services may be taken under attack. In contrast, software developers release updates to add further features and fix flaws in order to increase its security. Consequently, a user of the software wants to have the latest secure version running. However, if the software is provided as a service, e.g., as part of the cloud, the user relies on the service provider (SP) to perform such updates. But when asking for the software version, the user has to trust the output of SP or his software. Latter may be malformed, since updating software costs time and money, i.e., in comparison to changing a (false) version string. Now the question rises how a software service's client can provably determine the real software version of the running service at the SP, also known as Remote Software Identification (RSI). While existing tools provide an answer, they can be tricked by the service to output any forged string because they rely on the information handed directly by the SP. We solve the problem of RSI by introducing Reverse Fingerprinting (RFP), a novel challenge-response scheme which employs the evaluation of inherit functions of software versions depending on certain inputs. That is, RFP does not rely on version number APIs but employs a database consisting of function inputs and according outputs and combines them with a strategy and a randomness source to provably determine the version number. We also provide a theoretical framework for RSI and RFP, and describe how to create databases and strategies. Additionally, RFP can be securely outsourced to a third party, called the auditor, to take away the burden of the user while respecting liability. We also provide an implementation and API to perform RFP in practice, showing that most of the providers have installed the latest versions.
A new construction of nonlinear codes via rational function fields
Venue: arXiv cs.IT 2019
Year: 2019

It is well known that constructing codes with good parameters is one of the most important and fundamental problems in coding theory. Though a great many of good codes have been produced, most of them are defined over alphabets of sizes equal to prime powers. In this paper, we provide a new explicit construction of (q+1)-ary nonlinear codes via rational function fields, where q is a prime power. Our codes are constructed by evaluations of rational functions at all the rational places (including the place of "infinity") of the rational function field. Compared to the rational algebraic geometry codes, the main difference is that we allow rational functions to be evaluated at pole places. After evaluating rational functions from a union of Riemann-Roch spaces, we obtain a family of nonlinear codes with length q+1 over the alphabet $\mathbb{F}_{q}\cup \{\infty\}$. As a result, our codes have reasonable parameters as they are very close to the Singleton bound. Furthermore, our codes have better parameters than those obtained from MDS codes via code alphabet restriction or extension.
Analysis of Video Feature Learning in Two-Stream CNNs on the Example of Zebrafish Swim Bout Classification
Venue: arXiv cs.CV 2019
Year: 2019

Semmelhack et al. (2014) have achieved high classification accuracy in distinguishing swim bouts of zebrafish using a Support Vector Machine (SVM). Convolutional Neural Networks (CNNs) have reached superior performance in various image recognition tasks over SVMs, but these powerful networks remain a black box. Reaching better transparency helps to build trust in their classifications and makes learned features interpretable to experts. Using a recently developed technique called Deep Taylor Decomposition, we generated heatmaps to highlight input regions of high relevance for predictions. We find that our CNN makes predictions by analyzing the steadiness of the tail's trunk, which markedly differs from the manually extracted features used by Semmelhack et al. (2014). We further uncovered that the network paid attention to experimental artifacts. Removing these artifacts ensured the validity of predictions. After correction, our best CNN beats the SVM by 6.12%, achieving a classification accuracy of 96.32%. Our work thus demonstrates the utility of AI explainability for CNNs.
Subpacketization-Beamformer Interaction in Multi-Antenna Coded Caching
Venue: arXiv cs.IT 2019
Year: 2019

We study the joint effect of beamformer structure and subpacketization value on the achievable rate of cache-enabled multi-antenna communications at low-SNR. A mathematical approach with low-SNR approximations is used, to show that using simplistic beamformer structures, increasing subpacketization degrades the achievable rate; in contrast to what has been shown in the literature for more complex, optimized beamformer structures. The results suggest that for improving the low-SNR rate, subpacketization and beamformer complexity should be jointly increased.
Advanced Variations of Two-Dimensional Principal Component Analysis for Face Recognition
Venue: arXiv cs.CV 2019
Year: 2019

The two-dimensional principal component analysis (2DPCA) has become one of the most powerful tools of artificial intelligent algorithms. In this paper, we review 2DPCA and its variations, and propose a general ridge regression model to extract features from both row and column directions. To enhance the generalization ability of extracted features, a novel relaxed 2DPCA (R2DPCA) is proposed with a new ridge regression model. R2DPCA generates a weighting vector with utilizing the label information, and maximizes a relaxed criterion with applying an optimal algorithm to get the essential features. The R2DPCA-based approaches for face recognition and image reconstruction are also proposed and the selected principle components are weighted to enhance the role of main components. Numerical experiments on well-known standard databases indicate that R2DPCA has high generalization ability and can achieve a higher recognition rate than the state-of-the-art methods, including in the deep learning methods such as CNNs, DBNs, and DNNs.
Polynomial Neural Networks and Taylor maps for Dynamical Systems Simulation and Learning
Venue: arXiv cs.NE 2019
Year: 2019

The connection of Taylor maps and polynomial neural networks (PNN) to solve ordinary differential equations (ODEs) numerically is considered. Having the system of ODEs, it is possible to calculate weights of PNN that simulates the dynamics of these equations. It is shown that proposed PNN architecture can provide better accuracy with less computational time in comparison with traditional numerical solvers. Moreover, neural network derived from the ODEs can be used for simulation of system dynamics with different initial conditions, but without training procedure. On the other hand, if the equations are unknown, the weights of the PNN can be fitted in a data-driven way. In the paper we describe the connection of PNN with differential equations in a theoretical way along with the examples for both dynamics simulation and learning with data.
Busca de melhor caminho entre múltiplas origens e múltiplos destinos em redes complexas que representam cidades
Venue: arXiv cs.AI 2019
Year: 2019

Was investigated in this paper the use of a search strategy in the problem of finding the best path among multiple origins and multiple destinations. In this kind of problem, it must be decided within a lot of combinations which is the best origin and the best destination, and also the best path between these two regions. One remarkable difficulty to answer this sort of problem is to perform the search in a reduced time. This monography is a extension of previous research in which the problem described here was studied only in a bus network in the city of Fortaleza. This extension consisted of an exploration of the search strategy in graphs that represent public ways in cities like Fortaleza, Mumbai and Tokyo. Using this strategy with a heuristic algorithm, Haversine distance, was noticed that is possible to reduce substantially the time of the search, but introducing an error because of the loss of the admissible characteristic of the heuristic function applied.
Saliency Based Fire Detection Using Texture and Color Features
Venue: arXiv cs.CV 2019
Year: 2019

Due to industry deployment and extension of urban areas, early warning systems have an essential role in giving emergency. Fire is an event that can rapidly spread and cause injury, death, and damage. Early detection of fire could significantly reduce these injuries. Video-based fire detection is a low cost and fast method in comparison with conventional fire detectors. Most available fire detection methods have a high false-positive rate and low accuracy. In this paper, we increase accuracy by using spatial and temporal features. Captured video sequences are divided into Spatio-temporal blocks. Then a saliency map and combination of color and texture features are used for detecting fire regions. We use the HSV color model as a spatial feature and LBP-TOP for temporal processing of fire texture. Fire detection tests on publicly available datasets have shown the accuracy and robustness of the algorithm.
TentacleNet: A Pseudo-Ensemble Template for Accurate Binary Convolutional Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

Binarization is an attractive strategy for implementing lightweight Deep Convolutional Neural Networks (CNNs). Despite the unquestionable savings offered, memory footprint above all, it may induce an excessive accuracy loss that prevents a widespread use. This work elaborates on this aspect introducing TentacleNet, a new template designed to improve the predictive performance of binarized CNNs via parallelization. Inspired by the ensemble learning theory, it consists of a compact topology that is end-to-end trainable and organized to minimize memory utilization. Experimental results collected over three realistic benchmarks show TentacleNet fills the gap left by classical binary models, ensuring substantial memory savings w.r.t. state-of-the-art binary ensemble methods.