OpenRaiser/Intern-Atlas huggingface dataset:
Multi-Representational Learning for Offline Signature Verification using Multi-Loss Snapshot Ensemble of CNNs
Venue: arXiv cs.CV 2019
Year: 2019

Offline Signature Verification (OSV) is a challenging pattern recognition task, especially in presence of skilled forgeries that are not available during training. This study aims to tackle its challenges and meet the substantial need for generalization for OSV by examining different loss functions for Convolutional Neural Network (CNN). We adopt our new approach to OSV by asking two questions: 1. which classification loss provides more generalization for feature learning in OSV? , and 2. How integration of different losses into a unified multi-loss function lead to an improved learning framework? These questions are studied based on analysis of three loss functions, including cross entropy, Cauchy-Schwarz divergence, and hinge loss. According to complementary features of these losses, we combine them into a dynamic multi-loss function and propose a novel ensemble framework for simultaneous use of them in CNN. Our proposed Multi-Loss Snapshot Ensemble (MLSE) consists of several sequential trials. In each trial, a dominant loss function is selected from the multi-loss set, and the remaining losses act as a regularizer. Different trials learn diverse representations for each input based on signature identification task. This multi-representation set is then employed for the verification task. An ensemble of SVMs is trained on these representations, and their decisions are finally combined according to the selection of most generalizable SVM for each user. We conducted two sets of experiments based on two different protocols of OSV, i.e., writer-dependent and writer-independent on three signature datasets: GPDS-Synthetic, MCYT, and UT-SIG. Based on the writer-dependent OSV protocol, we achieved substantial improvements over the best EERs in the literature. The results of the second set of experiments also confirmed the robustness to the arrival of new users enrolled in the OSV system.
Replication-Aware Linearizability
Venue: arXiv cs.PL 2019
Year: 2019

Geo-distributed systems often replicate data at multiple locations to achieve availability and performance despite network partitions. These systems must accept updates at any replica and propagate these updates asynchronously to every other replica. Conflict-Free Replicated Data Types (CRDTs) provide a principled approach to the problem of ensuring that replicas are eventually consistent despite the asynchronous delivery of updates. We address the problem of specifying and verifying CRDTs, introducing a new correctness criterion called Replication-Aware Linearizability. This criterion is inspired by linearizability, the de-facto correctness criterion for (shared-memory) concurrent data structures. We argue that this criterion is both simple to understand, and it fits most known implementations of CRDTs. We provide a proof methodology to show that a CRDT satisfies replication-aware linearizability which we apply on a wide range of implementations. Finally, we show that our criterion can be leveraged to reason modularly about the composition of CRDTs.
Multi-label Cloud Segmentation Using a Deep Network
Venue: arXiv cs.CV 2019
Year: 2019

Different empirical models have been developed for cloud detection. There is a growing interest in using the ground-based sky/cloud images for this purpose. Several methods exist that perform binary segmentation of clouds. In this paper, we propose to use a deep learning architecture (U-Net) to perform multi-label sky/cloud image segmentation. The proposed approach outperforms recent literature by a large margin.
On Certifying Non-uniform Bound against Adversarial Attacks
Venue: arXiv cs.LG 2019
Year: 2019

This work studies the robustness certification problem of neural network models, which aims to find certified adversary-free regions as large as possible around data points. In contrast to the existing approaches that seek regions bounded uniformly along all input features, we consider non-uniform bounds and use it to study the decision boundary of neural network models. We formulate our target as an optimization problem with nonlinear constraints. Then, a framework applicable for general feedforward neural networks is proposed to bound the output logits so that the relaxed problem can be solved by the augmented Lagrangian method. Our experiments show the non-uniform bounds have larger volumes than uniform ones and the geometric similarity of the non-uniform bounds gives a quantitative, data-agnostic metric of input features' robustness. Further, compared with normal models, the robust models have even larger non-uniform bounds and better interpretability.
Analyzing digital politics: Challenges and experiments in a dual perspective
Venue: arXiv cs.SI 2019
Year: 2019

Social networks have become in the last decade central to political life. However, to those interested in analysing the communication strategies of parties and candidates at election time, the introduction of the Internet into the political sphere has proved a mixed blessing. Indeed, while retrieving, consulting, and archiving original documents pertaining to a specific campaign have become easier, faster, and achievable on a larger scale, thus opening up a promising El Dorado for research in this area, studying online campaigns has also inevitably introduced new technical, methodological and legal challenges which have turned out to be increasingly complex for academics in the humanities and social sciences to solve on their own.This paper therefore proposes to provide feedback on experience and experimental validation from a multidisciplinary project called POLIWEB devoted to the comparative analysis of political campaigns on social media in the run up to the 2014 elections to the European Parliament in France and in the United Kingdom. Together with observations from a humanities' perspective on issues related to such a project, this paper also presents experimental results concerning three of the data collection life cycle phases: collection, cleaning, and storage. The outcome is a data collection ready to be analysed for various purposes meant to address the political science topic under consideration.
GestureKeeper: Gesture Recognition for Controlling Devices in IoT Environments
Venue: arXiv cs.HC 2019
Year: 2019

This paper introduces and evaluates the GestureKeeper, a robust hand-gesture recognition system based on a wearable inertial measurements unit (IMU). The identification of the time windows where the gestures occur, without relying on an explicit user action or a special gesture marker, is a very challenging task. To address this problem, GestureKeeper identifies the start of a gesture by exploiting the underlying dynamics of the associated time series using a recurrence quantification analysis (RQA). RQA is a powerful method for nonlinear time-series analysis, which enables the detection of critical transitions in the system's dynamical behavior. Most importantly, it does not make any assumption about the underlying distribution or model that governs the data. Having estimated the gesture window, a support vector machine is employed to recognize the specific gesture. Our proposed method is evaluated by means of a small-scale pilot study at FORTH and demonstrated that GestureKeeper can identify correctly the start of a gesture with a 87\% mean balanced accuracy and classify correctly the specific hand-gesture with a mean accuracy of over 96\%. To the best of our knowledge, GestureKeeper is the first automatic hand-gesture identification system based only on accelerometer. The performance analysis reveals the predictive power of the features and the system's robustness in the presence of additive noise. We also performed a sensitivity analysis to examine the impact of various parameters and a comparative analysis of different classifiers (SVM, random forests). Most importantly, the system can be extended to incorporate a large dictionary of gestures and operate without further calibration for a new user.
Extending the Web3D: Design of Conventional GUI Libraries in X3D
Venue: arXiv cs.HC 2019
Year: 2019

Extensible 3D (X3D) modeling language is one of the leading Web3D technologies. Despite the rich functionality, the language does not currently provide tools for rapid development of conventional graphical user interfaces (GUIs). Every X3D author is responsible for building from primitives a purpose specific set of required interface components, often for a single use. We address the challenge of creating consistent, efficient, interactive, and visually appealing GUIs by proposing the X3D User Interface (X3DUI) library. This library includes a wide range of cross-compatible X3D widgets, equipped with configurable appearance and behavior. With this library, we attempt to standardize the GUI construction across various X3D-driven projects, and improve the usability, compatibility, adaptability, readability, and flexibility of many existing applications.
Improving Strong-Scaling of CNN Training by Exploiting Finer-Grained Parallelism
Venue: arXiv cs.DC 2019
Year: 2019

Scaling CNN training is necessary to keep up with growing datasets and reduce training time. We also see an emerging need to handle datasets with very large samples, where memory requirements for training are large. Existing training frameworks use a data-parallel approach that partitions samples within a mini-batch, but limits to scaling the mini-batch size and memory consumption makes this untenable for large samples. We describe and implement new approaches to convolution, which parallelize using spatial decomposition or a combination of sample and spatial decomposition. This introduces many performance knobs for a network, so we develop a performance model for CNNs and present a method for using it to automatically determine efficient parallelization strategies. We evaluate our algorithms with microbenchmarks and image classification with ResNet-50. Our algorithms allow us to prototype a model for a mesh-tangling dataset, where sample sizes are very large. We show that our parallelization achieves excellent strong and weak scaling and enables training for previously unreachable datasets.
Wasserstein Distance based Deep Adversarial Transfer Learning for Intelligent Fault Diagnosis
Venue: arXiv cs.LG 2019
Year: 2019

The demand of artificial intelligent adoption for condition-based maintenance strategy is astonishingly increased over the past few years. Intelligent fault diagnosis is one critical topic of maintenance solution for mechanical systems. Deep learning models, such as convolutional neural networks (CNNs), have been successfully applied to fault diagnosis tasks for mechanical systems and achieved promising results. However, for diverse working conditions in the industry, deep learning suffers two difficulties: one is that the well-defined (source domain) and new (target domain) datasets are with different feature distributions; another one is the fact that insufficient or no labelled data in target domain significantly reduce the accuracy of fault diagnosis. As a novel idea, deep transfer learning (DTL) is created to perform learning in the target domain by leveraging information from the relevant source domain. Inspired by Wasserstein distance of optimal transport, in this paper, we propose a novel DTL approach to intelligent fault diagnosis, namely Wasserstein Distance based Deep Transfer Learning (WD-DTL), to learn domain feature representations (generated by a CNN based feature extractor) and to minimize the distributions between the source and target domains through adversarial training. The effectiveness of the proposed WD-DTL is verified through 3 transfer scenarios and 16 transfer fault diagnosis experiments of both unsupervised and supervised (with insufficient labelled data) learning. We also provide a comprehensive analysis of the network visualization of those transfer tasks.
Day-Ahead Hourly Forecasting of Power Generation from Photovoltaic Plants
Venue: arXiv cs.LG 2019
Year: 2019

The ability to accurately forecast power generation from renewable sources is nowadays recognised as a fundamental skill to improve the operation of power systems. Despite the general interest of the power community in this topic, it is not always simple to compare different forecasting methodologies, and infer the impact of single components in providing accurate predictions. In this paper we extensively compare simple forecasting methodologies with more sophisticated ones over 32 photovoltaic plants of different size and technology over a whole year. Also, we try to evaluate the impact of weather conditions and weather forecasts on the prediction of PV power generation.
On the classification and false alarm of invalid prefixes in RPKI based BGP route origin validation
Venue: arXiv cs.NI 2019
Year: 2019

BGP is the default inter-domain routing protocol in today's Internet, but has serious security vulnerabilities\cite{murphy2005bgp}. One of them is (sub)prefix hijacking. IETF standardizes RPKI to validate the AS origin but RPKI has a lot of problems\cite{heilman2014consent}\cite{cooper2013risk}\cite{gilad2017we}\cite{gilad2017maxlength}, among which is potential false alarm. Although some previous work\cite{gilad2017we}\cite{heilman2014consent} points it out explicitly or implicitly, further measurement and analysis remain to be done. Our work measures and analyzes the invalid prefixes systematically. We first classify the invalid prefixes into six different types and then analyze their stability. We show that a large proportion of the invalid prefixes very likely result from traffic engineering, IP address transfer and failing to aggregate rather than real hijackings.
swCaffe: a Parallel Framework for Accelerating Deep Learning Applications on Sunway TaihuLight
Venue: arXiv cs.DC 2019
Year: 2019

This paper reports our efforts on swCaffe, a highly efficient parallel framework for accelerating deep neural networks (DNNs) training on Sunway TaihuLight, the current fastest supercomputer in the world that adopts a unique many-core heterogeneous architecture, with 40,960 SW26010 processors connected through a customized communication network. First, we point out some insightful principles to fully exploit the performance of the innovative many-core architecture. Second, we propose a set of optimization strategies for redesigning a variety of neural network layers based on Caffe. Third, we put forward a topology-aware parameter synchronization scheme to scale the synchronous Stochastic Gradient Descent (SGD) method to multiple processors efficiently. We evaluate our framework by training a variety of widely used neural networks with the ImageNet dataset. On a single node, swCaffe can achieve 23\%\~{}119\% overall performance compared with Caffe running on K40m GPU. As compared with the Caffe on CPU, swCaffe runs 3.04\~{}7.84x faster on all the networks. Finally, we present the scalability of swCaffe for the training of ResNet-50 and AlexNet on the scale of 1024 nodes.
Prescribed-time convergence with input constraints: A control Lyapunov function based approach
Venue: arXiv cs.SY 2019
Year: 2019

In this paper, we present a control framework for a general class of control-affine nonlinear systems under spatiotemporal and input constraints. Specifically, the proposed control architecture addresses the problem of reaching a given final set $S$ in a prescribed (user-defined) time with bounded control inputs. To this end, a time transformation technique is utilized to transform the system subject to temporal constraints into an equivalent form without temporal constraints. The transformation is defined so that asymptotic convergence in the transformed time scale results into prescribed-time convergence in the original time scale. To incorporate input constraints, we characterize a set of initial conditions $D_M$ such that starting from this set, the closed-loop trajectories reach the set $S$ within the prescribed time. We further show that starting from outside the set $D_M$, the system trajectories reach the set $D_M$ in a finite time that depends upon the initial conditions and the control input bounds. We use a novel parameter $μ$ in the controller, that controls the convergence-rate of the closed-loop trajectories and dictates the size of the set $D_M$. Finally, we present a numerical example to showcase the efficacy of our proposed method.
A Principal-Agent Model of Systems Engineering Processes with Application to Satellite Design
Venue: arXiv cs.MA 2019
Year: 2019

We present a principal-agent model of a one-shot, shallow, systems engineering process. The process is one-shot in the sense that decisions are made during one time step and that they are final. The term shallow refers to a one-layer hierarchy of the process. Specifically, we assume that the systems engineer has already decomposed the problem in subsystems, and that each subsystem is assigned to a different subsystem engineer. Each subsystem engineer works independently to maximize their own expected payoff. The goal of the systems engineer is to maximize the system-level payoff by incentivizing the subsystem engineers. We restrict our attention to requirement-based system-level payoffs, i.e., the systems engineer makes a profit only if all the design requirements are met. We illustrate the model using the design of an Earth-orbiting satellite system where the systems engineer determines the optimum incentive structures and requirements for two subsystems: the propulsion subsystem and the power subsystem. The model enables the analysis of a systems engineer's decisions about optimal passed-down requirements and incentives for sub-system engineers under different levels of task difficulty and associated costs. Sample results, for the case of risk-neutral systems and subsystems engineers, show that it is not always in the best interest of the systems engineer to pass down the true requirements. As expected, the model predicts that for small to moderate task uncertainties the optimal requirements are higher than the true ones, effectively eliminating the probability of failure for the systems engineer. In contrast, the model predicts that for large task uncertainties the optimal requirements should be smaller than the true ones in order to lure the subsystem engineers into participation.
SLSGD: Secure and Efficient Distributed On-device Machine Learning
Venue: arXiv cs.LG 2019
Year: 2019

We consider distributed on-device learning with limited communication and security requirements. We propose a new robust distributed optimization algorithm with efficient communication and attack tolerance. The proposed algorithm has provable convergence and robustness under non-IID settings. Empirical results show that the proposed algorithm stabilizes the convergence and tolerates data poisoning on a small number of workers.
Reconstructing neuronal anatomy from whole-brain images
Venue: arXiv cs.CV 2019
Year: 2019

Reconstructing multiple molecularly defined neurons from individual brains and across multiple brain regions can reveal organizational principles of the nervous system. However, high resolution imaging of the whole brain is a technically challenging and slow process. Recently, oblique light sheet microscopy has emerged as a rapid imaging method that can provide whole brain fluorescence microscopy at a voxel size of 0.4 by 0.4 by 2.5 cubic microns. On the other hand, complex image artifacts due to whole-brain coverage produce apparent discontinuities in neuronal arbors. Here, we present connectivity-preserving methods and data augmentation strategies for supervised learning of neuroanatomy from light microscopy using neural networks. We quantify the merit of our approach by implementing an end-to-end automated tracing pipeline. Lastly, we demonstrate a scalable, distributed implementation that can reconstruct the large datasets that sub-micron whole-brain images produce.
Time Synchronization Attack and Countermeasure for Multi-System Scheduling in Remote Estimation
Venue: arXiv cs.SY 2019
Year: 2019

We consider time synchronization attack against multi-system scheduling in a remote state estimation scenario where a number of sensors monitor different linear dynamical processes and schedule their transmissions through a shared collision channel. We show that by randomly injecting relative time offsets on the sensors, the malicious attacker is able to make the expected estimation error covariance of the overall system diverge without any system knowledge. For the case that the attacker has full system information, we propose an efficient algorithm to calculate the optimal attack, which spoofs the least number of sensors and leads to unbounded average estimation error covariance. To mitigate the attack consequence, we further propose a countermeasure by constructing shift invariant transmission policies and characterize the lower and upper bounds for system estimation performance. Simulation examples are provided to illustrate the obtained results.
Shining a light on Spotlight: Leveraging Apple's desktop search utility to recover deleted file metadata on macOS
Venue: arXiv cs.CR 2019
Year: 2019

Spotlight is a proprietary desktop search technology released by Apple in 2004 for its Macintosh operating system Mac OS X 10.4 (Tiger) and remains as a feature in current releases of macOS. Spotlight allows users to search for files or information by querying databases populated with filesystem attributes, metadata, and indexed textual content. Existing forensic research into Spotlight has provided an understanding of the metadata attributes stored within the metadata store database. Current approaches in the literature have also enabled the extraction of metadata records for extant files, but not for deleted files. The objective of this paper is to research the persistence of records for deleted files within Spotlight's metadata store, identify if deleted database pages are recoverable from unallocated space on the volume, and to present a strategy for the processing of discovered records. In this paper, the structure of the metadata store database is outlined, and experimentation reveals that records persist for a period of time within the database but once deleted, are no longer recoverable. The experimentation also demonstrates that deleted pages from the database (containing metadata records) are recoverable from unused space on the filesystem.
Adversarial Attacks on Deep Neural Networks for Time Series Classification
Venue: arXiv cs.LG 2019
Year: 2019

Time Series Classification (TSC) problems are encountered in many real life data mining tasks ranging from medicine and security to human activity recognition and food safety. With the recent success of deep neural networks in various domains such as computer vision and natural language processing, researchers started adopting these techniques for solving time series data mining problems. However, to the best of our knowledge, no previous work has considered the vulnerability of deep learning models to adversarial time series examples, which could potentially make them unreliable in situations where the decision taken by the classifier is crucial such as in medicine and security. For computer vision problems, such attacks have been shown to be very easy to perform by altering the image and adding an imperceptible amount of noise to trick the network into wrongly classifying the input image. Following this line of work, we propose to leverage existing adversarial attack mechanisms to add a special noise to the input time series in order to decrease the network's confidence when classifying instances at test time. Our results reveal that current state-of-the-art deep learning time series classifiers are vulnerable to adversarial attacks which can have major consequences in multiple domains such as food safety and quality assurance.