OpenRaiser/Intern-Atlas huggingface dataset:
A Generalization of the Łoś-Tarski Preservation Theorem - Dissertation Summary
Venue: arXiv cs.LO 2018
Year: 2018

This article gives a summary of the author's Ph.D. dissertation (arXiv:1609.06297). In addition to an overview of notions and results, it also provides sketches of various proofs and simplified presentations of certain abstract results of the dissertation, that concern tree representations of structures. Further, some extensions of the dissertation results are presented. These include the connections of the model-theoretic notions introduced in the thesis with fixed parameter tractability and notions in the structure theory of sparse graph classes. The constructive aspects of the proofs of the model-theoretic results of the dissertation are used to obtain (algorithmic) meta-kernels for various dense graphs such as graphs of bounded clique-width and subclasses of these like $m$-partite cographs and graph classes of bounded shrub-depth. Finally, the article presents updated definitions and results concerning the notion of logical fractals which is a generalization of the Equivalent Bounded Substructure Property from the dissertation. In particular, our results show that (natural finitary adaptations of) both the upward and downward versions of the Löwenheim-Skolem theorem from classical model theory can be recovered in a variety of algorithmically interesting settings, and further in most cases, in effective form and even for logics beyond first order logic.
An Adaptive Pruning Algorithm for Spoofing Localisation Based on Tropical Geometry
Venue: arXiv cs.CR 2018
Year: 2018

The problem of spoofing attacks is increasingly relevant as digital systems are becoming more ubiquitous. Thus the detection of such attacks and the localisation of attackers have been objects of recent study. After an attack has been detected, various algorithms have been proposed in order to localise the attacker. In this work we propose a new adaptive pruning algorithm inspired by the tropical and geometrical analysis of the traditional Viterbi pruning algorithm to solve the localisation problem. In particular, the proposed algorithm tries to localise the attacker by adapting the leniency parameter based on estimates about the state of the solution space. These estimates stem from the enclosed volume and the entropy of the solution space, as they were introduced in our previous works.
Scalable Deep $k$-Subspace Clustering
Venue: arXiv cs.CV 2018
Year: 2018

Subspace clustering algorithms are notorious for their scalability issues because building and processing large affinity matrices are demanding. In this paper, we introduce a method that simultaneously learns an embedding space along subspaces within it to minimize a notion of reconstruction error, thus addressing the problem of subspace clustering in an end-to-end learning paradigm. To achieve our goal, we propose a scheme to update subspaces within a deep neural network. This in turn frees us from the need of having an affinity matrix to perform clustering. Unlike previous attempts, our method can easily scale up to large datasets, making it unique in the context of unsupervised learning with deep architectures. Our experiments show that our method significantly improves the clustering accuracy while enjoying cheaper memory footprints.
The estimation of bias and variance in clustering coefficient streaming algorithms
Venue: arXiv cs.SI 2018
Year: 2018

Clustering coefficient is one of the most important metrics to understand the complex structure of networks. This paper addresses the estimation of clustering coefficient in network streams. There have been substantial work in this area, most of conducting empirical comparisons of various algorithms. The variance and the bias of the estimators have not been quantified. Starting with a simple yet powerful streaming algorithm, we derived the variance and bias for the estimator, and the estimators for the variances and bias. More importantly, we simplify the estimators so that it can be used in practice. The variance and bias estimators are verified extensively on 49 real networks.
CAAD 2018: Powerful None-Access Black-Box Attack Based on Adversarial Transformation Network
Venue: arXiv cs.LG 2018
Year: 2018

In this paper, we propose an improvement of Adversarial Transformation Networks(ATN) to generate adversarial examples, which can fool white-box models and black-box models with a state of the art performance and won the 2rd place in the non-target task in CAAD 2018.
Optimizations of the Eigensolvers in the ELPA Library
Venue: arXiv cs.MS 2018
Year: 2018

The solution of (generalized) eigenvalue problems for symmetric or Hermitian matrices is a common subtask of many numerical calculations in electronic structure theory or materials science. Solving the eigenvalue problem can easily amount to a sizeable fraction of the whole numerical calculation. For researchers in the field of computational materials science, an efficient and scalable solution of the eigenvalue problem is thus of major importance. The ELPA-library is a well-established dense direct eigenvalue solver library, which has proven to be very efficient and scalable up to very large core counts. In this paper, we describe the latest optimizations of the ELPA-library for new HPC architectures of the Intel Skylake processor family with an AVX-512 SIMD instruction set, or for HPC systems accelerated with recent GPUs. We also describe a complete redesign of the API in a modern modular way, which, apart from a much simpler and more flexible usability, leads to a new path to access system-specific performance optimizations. In order to ensure optimal performance for a particular scientific setting or a specific HPC system, the new API allows the user to influence in straightforward way the internal details of the algorithms and of performance-critical parameters used in the ELPA-library. On top of that, we introduced an autotuning functionality, which allows for finding the best settings in a self-contained automated way. In situations where many eigenvalue problems with similar settings have to be solved consecutively, the autotuning process of the ELPA-library can be done "on-the-fly". Practical applications from materials science which rely on so-called self-consistency iterations can profit from the autotuning. On some examples of scientific interest, simulated with the FHI-aims application, the advantages of the latest optimizations of the ELPA-library are demonstrated.
Nonlinear Collaborative Scheme for Deep Neural Networks
Venue: arXiv cs.LG 2018
Year: 2018

Conventional research attributes the improvements of generalization ability of deep neural networks either to powerful optimizers or the new network design. Different from them, in this paper, we aim to link the generalization ability of a deep network to optimizing a new objective function. To this end, we propose a \textit{nonlinear collaborative scheme} for deep network training, with the key technique as combining different loss functions in a nonlinear manner. We find that after adaptively tuning the weights of different loss functions, the proposed objective function can efficiently guide the optimization process. What is more, we demonstrate that, from the mathematical perspective, the nonlinear collaborative scheme can lead to (i) smaller KL divergence with respect to optimal solutions; (ii) data-driven stochastic gradient descent; (iii) tighter PAC-Bayes bound. We also prove that its advantage can be strengthened by nonlinearity increasing. To some extent, we bridge the gap between learning (i.e., minimizing the new objective function) and generalization (i.e., minimizing a PAC-Bayes bound) in the new scheme. We also interpret our findings through the experiments on Residual Networks and DenseNet, showing that our new scheme performs superior to single-loss and multi-loss schemes no matter with randomization or not.
Improving GAN with neighbors embedding and gradient matching
Venue: arXiv cs.CV 2018
Year: 2018

We propose two new techniques for training Generative Adversarial Networks (GANs). Our objectives are to alleviate mode collapse in GAN and improve the quality of the generated samples. First, we propose neighbor embedding, a manifold learning-based regularization to explicitly retain local structures of latent samples in the generated samples. This prevents generator from producing nearly identical data samples from different latent samples, and reduces mode collapse. We propose an inverse t-SNE regularizer to achieve this. Second, we propose a new technique, gradient matching, to align the distributions of the generated samples and the real samples. As it is challenging to work with high-dimensional sample distributions, we propose to align these distributions through the scalar discriminator scores. We constrain the difference between the discriminator scores of the real samples and generated ones. We further constrain the difference between the gradients of these discriminator scores. We derive these constraints from Taylor approximations of the discriminator function. We perform experiments to demonstrate that our proposed techniques are computationally simple and easy to be incorporated in existing systems. When Gradient matching and Neighbour embedding are applied together, our GN-GAN achieves outstanding results on 1D/2D synthetic, CIFAR-10 and STL-10 datasets, e.g. FID score of $30.80$ for the STL-10 dataset. Our code is available at: https://github.com/tntrung/gan
DeepKey: Towards End-to-End Physical Key Replication From a Single Photograph
Venue: arXiv cs.CV 2018
Year: 2018

This paper describes DeepKey, an end-to-end deep neural architecture capable of taking a digital RGB image of an 'everyday' scene containing a pin tumbler key (e.g. lying on a table or carpet) and fully automatically inferring a printable 3D key model. We report on the key detection performance and describe how candidates can be transformed into physical prints. We show an example opening a real-world lock. Our system is described in detail, providing a breakdown of all components including key detection, pose normalisation, bitting segmentation and 3D model inference. We provide an in-depth evaluation and conclude by reflecting on limitations, applications, potential security risks and societal impact. We contribute the DeepKey Datasets of 5, 300+ images covering a few test keys with bounding boxes, pose and unaligned mask data.
FAdeML: Understanding the Impact of Pre-Processing Noise Filtering on Adversarial Machine Learning
Venue: arXiv cs.LG 2018
Year: 2018

Deep neural networks (DNN)-based machine learning (ML) algorithms have recently emerged as the leading ML paradigm particularly for the task of classification due to their superior capability of learning efficiently from large datasets. The discovery of a number of well-known attacks such as dataset poisoning, adversarial examples, and network manipulation (through the addition of malicious nodes) has, however, put the spotlight squarely on the lack of security in DNN-based ML systems. In particular, malicious actors can use these well-known attacks to cause random/targeted misclassification, or cause a change in the prediction confidence, by only slightly but systematically manipulating the environmental parameters, inference data, or the data acquisition block. Most of the prior adversarial attacks have, however, not accounted for the pre-processing noise filters commonly integrated with the ML-inference module. Our contribution in this work is to show that this is a major omission since these noise filters can render ineffective the majority of the existing attacks, which rely essentially on introducing adversarial noise. Apart from this, we also extend the state of the art by proposing a novel pre-processing noise Filter-aware Adversarial ML attack called FAdeML. To demonstrate the effectiveness of the proposed methodology, we generate an adversarial attack image by exploiting the "VGGNet" DNN trained for the "German Traffic Sign Recognition Benchmarks (GTSRB" dataset, which despite having no visual noise, can cause a classifier to misclassify even in the presence of pre-processing noise filters.
RePair in Compressed Space and Time
Venue: arXiv cs.DS 2018
Year: 2018

Given a string $T$ of length $N$, the goal of grammar compression is to construct a small context-free grammar generating only $T$. Among existing grammar compression methods, RePair (recursive paring) [Larsson and Moffat, 1999] is notable for achieving good compression ratios in practice. Although the original paper already achieved a time-optimal algorithm to compute the RePair grammar RePair($T$) in expected $O(N)$ time, the study to reduce its working space is still active so that it is applicable to large-scale data. In this paper, we propose the first RePair algorithm working in compressed space, i.e., potentially $o(N)$ space for highly compressible texts. The key idea is to give a new way to restructure an arbitrary grammar $S$ for $T$ into RePair($T$) in compressed space and time. Based on the recompression technique, we propose an algorithm for RePair($T$) in $O(\min(N, nm \log N))$ space and expected $O(\min(N, nm \log N) m)$ time or $O(\min(N, nm \log N) \log \log N)$ time, where $n$ is the size of $S$ and $m$ is the number of variables in RePair($T$). We implemented our algorithm running in $O(\min(N, nm \log N) m)$ time and show it can actually run in compressed space. We also present a new approach to reduce the peak memory usage of existing RePair algorithms combining with our algorithms, and show that the new approach outperforms, both in computation time and space, the most space efficient linear-time RePair implementation to date.
FairMod - Making Predictive Models Discrimination Aware
Venue: arXiv cs.AI 2018
Year: 2018

Predictive models such as decision trees and neural networks may produce discrimination in their predictions. This paper proposes a method to post-process the predictions of a predictive model to make the processed predictions non-discriminatory. The method considers multiple protected variables together. Multiple protected variables make the problem more challenging than a simple protected variable. The method uses a well-cited discrimination metric and adapts it to allow the specification of explanatory variables, such as position, profession, education, that describe the contexts of the applications. It models the post-processing of predictions problem as a nonlinear optimization problem to find best adjustments to the predictions so that the discrimination constraints of all protected variables are all met at the same time. The proposed method is independent of classification methods. It can handle the cases that existing methods cannot handle: satisfying multiple protected attributes at the same time, allowing multiple explanatory attributes, and being independent of classification model types. An evaluation using four real world data sets shows that the proposed method is as effectively as existing methods, in addition to its extra power.
Quantifying the Reality Gap in Robotic Manipulation Tasks
Venue: arXiv cs.RO 2018
Year: 2018

We quantify the accuracy of various simulators compared to a real world robotic reaching and interaction task. Simulators are used in robotics to design solutions for real world hardware without the need for physical access. The `reality gap' prevents solutions developed or learnt in simulation from performing well, or at at all, when transferred to real-world hardware. Making use of a Kinova robotic manipulator and a motion capture system, we record a ground truth enabling comparisons with various simulators, and present quantitative data for various manipulation-oriented robotic tasks. We show the relative strengths and weaknesses of numerous contemporary simulators, highlighting areas of significant discrepancy, and assisting researchers in the field in their selection of appropriate simulators for their use cases. All code and parameter listings are publicly available from: https://bitbucket.csiro.au/scm/~col549/quantifying-the-reality-gap-in-robotic-manipulation-tasks.git .
Unsupervised RGBD Video Object Segmentation Using GANs
Venue: arXiv cs.CV 2018
Year: 2018

Video object segmentation is a fundamental step in many advanced vision applications. Most existing algorithms are based on handcrafted features such as HOG, super-pixel segmentation or texture-based techniques, while recently deep features have been found to be more efficient. Existing algorithms observe performance degradation in the presence of challenges such as illumination variations, shadows, and color camouflage. To handle these challenges we propose a fusion based moving object segmentation algorithm which exploits color as well as depth information using GAN to achieve more accuracy. Our goal is to segment moving objects in the presence of challenging background scenes, in real environments. To address this problem, GAN is trained in an unsupervised manner on color and depth information independently with challenging video sequences. During testing, the trained GAN generates backgrounds similar to that in the test sample. The generated background samples are then compared with the test sample to segment moving objects. The final result is computed by fusion of object boundaries in both modalities, RGB and the depth. The comparison of our proposed algorithm with five state-of-the-art methods on publicly available dataset has shown the strength of our algorithm for moving object segmentation in videos in the presence of challenging real scenarios.
Transfer learning for time series classification
Venue: arXiv cs.LG 2018
Year: 2018

Transfer learning for deep neural networks is the process of first training a base network on a source dataset, and then transferring the learned features (the network's weights) to a second network to be trained on a target dataset. This idea has been shown to improve deep neural network's generalization capabilities in many computer vision tasks such as image recognition and object localization. Apart from these applications, deep Convolutional Neural Networks (CNNs) have also recently gained popularity in the Time Series Classification (TSC) community. However, unlike for image recognition problems, transfer learning techniques have not yet been investigated thoroughly for the TSC task. This is surprising as the accuracy of deep learning models for TSC could potentially be improved if the model is fine-tuned from a pre-trained neural network instead of training it from scratch. In this paper, we fill this gap by investigating how to transfer deep CNNs for the TSC task. To evaluate the potential of transfer learning, we performed extensive experiments using the UCR archive which is the largest publicly available TSC benchmark containing 85 datasets. For each dataset in the archive, we pre-trained a model and then fine-tuned it on the other datasets resulting in 7140 different deep neural networks. These experiments revealed that transfer learning can improve or degrade the model's predictions depending on the dataset used for transfer. Therefore, in an effort to predict the best source dataset for a given target dataset, we propose a new method relying on Dynamic Time Warping to measure inter-datasets similarities. We describe how our method can guide the transfer to choose the best source dataset leading to an improvement in accuracy on 71 out of 85 datasets.
Cocktail BPSK: Energy Reused Scheme for High Achievable Data Rates
Venue: arXiv cs.IT 2018
Year: 2018

The present paper proposes a novel transmission strategy, referred to as cocktail BPSK, whereat two independent BPSKs are superposed with the non-orthogonal basis in a parallel transmission. In contrast to the conventional signal superpositions, the proposed scheme avoids the interference between the two symbols, allows the symbol-energy-reuse of each other and gains the extra energy. Based on the formulation of the mutual informations, the theoretical analysis shows that the cocktail BPSK scheme can achieve high data rate beyond the channel capacity at very low SNR, and the numerical results confirm this approach eventually.
Continual Occlusions and Optical Flow Estimation
Venue: arXiv cs.CV 2018
Year: 2018

Two optical flow estimation problems are addressed: i) occlusion estimation and handling, and ii) estimation from image sequences longer than two frames. The proposed ContinualFlow method estimates occlusions before flow, avoiding the use of flow corrupted by occlusions for their estimation. We show that providing occlusion masks as an additional input to flow estimation improves the standard performance metric by more than 25\% on both KITTI and Sintel. As a second contribution, a novel method for incorporating information from past frames into flow estimation is introduced. The previous frame flow serves as an input to occlusion estimation and as a prior in occluded regions, i.e. those without visual correspondences. By continually using the previous frame flow, ContinualFlow performance improves further by 18\% on KITTI and 7\% on Sintel, achieving top performance on KITTI and Sintel.
Electrical Energy Consumption Control in Buildings Using Wireless Sensor Networks
Venue: arXiv cs.NI 2018
Year: 2018

Energy consumption in residential and commercial buildings has increased dramatically worldwide in the last decade, due to the constant population and economic growth, the proliferation of electronic and consumer appliances. This has dramatic footprint on the environment in terms of carbon emission, in addition to the economic impact. Green and smart building strategies will play a pivotal role to reduce this footprint and maximize economic and environmental performance. These strategies can be integrated into buildings at any stage, from design and construction, to maintenance and renovation. The use of modern Information and Communication Technologies (ICT), notably IoT solutions, for building control is one of the promising strategies for the future. The aim of this project was to explore this domain, and as a first step to develop a wireless sensor networks based solution for monitoring and energy management in offices. A prototype has been targeted as a proof of concept where sensors monitor physical parameters in CERSIT offices (presence of people, ambient light, etc.), and accordingly actuate lighting, air conditioning, etc. This report is a short summery of the different parts developed in this project.
On the Transferability of Adversarial Examples Against CNN-Based Image Forensics
Venue: arXiv cs.CR 2018
Year: 2018

Recent studies have shown that Convolutional Neural Networks (CNN) are relatively easy to attack through the generation of so-called adversarial examples. Such vulnerability also affects CNN-based image forensic tools. Research in deep learning has shown that adversarial examples exhibit a certain degree of transferability, i.e., they maintain part of their effectiveness even against CNN models other than the one targeted by the attack. This is a very strong property undermining the usability of CNN's in security-oriented applications. In this paper, we investigate if attack transferability also holds in image forensics applications. With specific reference to the case of manipulation detection, we analyse the results of several experiments considering different sources of mismatch between the CNN used to build the adversarial examples and the one adopted by the forensic analyst. The analysis ranges from cases in which the mismatch involves only the training dataset, to cases in which the attacker and the forensic analyst adopt different architectures. The results of our experiments show that, in the majority of the cases, the attacks are not transferable, thus easing the design of proper countermeasures at least when the attacker does not have a perfect knowledge of the target detector.