OpenRaiser/Intern-Atlas huggingface dataset:
Robust Attacks against Multiple Classifiers
Venue: arXiv cs.LG 2019
Year: 2019

We address the challenge of designing optimal adversarial noise algorithms for settings where a learner has access to multiple classifiers. We demonstrate how this problem can be framed as finding strategies at equilibrium in a two-player, zero-sum game between a learner and an adversary. In doing so, we illustrate the need for randomization in adversarial attacks. In order to compute Nash equilibrium, our main technical focus is on the design of best response oracles that can then be implemented within a Multiplicative Weights Update framework to boost deterministic perturbations against a set of models into optimal mixed strategies. We demonstrate the practical effectiveness of our approach on a series of image classification tasks using both linear classifiers and deep neural networks.
A Note on Lower Digits Extraction Polynomial for Bootstrapping
Venue: arXiv cs.CR 2019
Year: 2019

Bootstrapping is a crucial but computationally expensive step for realizing Fully Homomorphic Encryption (FHE). Recently, Chen and Han (Eurocrypt 2018) introduced a family of low-degree polynomials to extract the lowest digit with respect to a certain congruence, which helps improve the bootstrapping for both FV and BGV schemes. In this note, we present the following relevant findings about the work of Chen and Han (referred to as CH18): 1. We provide a simpler construction of the low-degree polynomials that serve the same purpose and match the asymptotic bound achieved in CH18; 2. We show the optimality and limit of our approach by solving a minimal polynomial degree problem; 3. We consider the problem of extracting other low-order digits using polynomials, and provide negative results.
Please, no more scientific journals! The strategy of the scientific publication system
Venue: arXiv cs.DL 2019
Year: 2019

In the same way ecosystems tend to increase maturity by decreasing the flow of energy per unit biomass, we should move towards a more mature science by publishing less but high-quality papers and getting away from joining large teams in small roles. That is, we should decrease our scientific productivity for good.
Learning Representations of Graph Data -- A Survey
Venue: arXiv cs.LG 2019
Year: 2019

Deep Neural Networks have shown tremendous success in the area of object recognition, image classification and natural language processing. However, designing optimal Neural Network architectures that can learn and output arbitrary graphs is an ongoing research problem. The objective of this survey is to summarize and discuss the latest advances in methods to Learn Representations of Graph Data. We start by identifying commonly used types of graph data and review basics of graph theory. This is followed by a discussion of the relationships between graph kernel methods and neural networks. Next we identify the major approaches used for learning representations of graph data namely: Kernel approaches, Convolutional approaches, Graph neural networks approaches, Graph embedding approaches and Probabilistic approaches. A variety of methods under each of the approaches are discussed and the survey is concluded with a brief discussion of the future of learning representation of graph data.
Argus: Smartphone-enabled Human Cooperation via Multi-Agent Reinforcement Learning for Disaster Situational Awareness
Venue: arXiv cs.CY 2019
Year: 2019

Argus exploits a Multi-Agent Reinforcement Learning (MARL) framework to create a 3D mapping of the disaster scene using agents present around the incident zone to facilitate the rescue operations. The agents can be both human bystanders at the disaster scene as well as drones or robots that can assist the humans. The agents are involved in capturing the images of the scene using their smartphones (or on-board cameras in case of drones) as directed by the MARL algorithm. These images are used to build real time a 3D map of the disaster scene. Via both simulations and real experiments, an evaluation of the framework in terms of effectiveness in tracking random dynamicity of the environment is presented.
Coherent Point Drift Networks: Unsupervised Learning of Non-Rigid Point Set Registration
Venue: arXiv cs.GR 2019
Year: 2019

Given new pairs of source and target point sets, standard point set registration methods often repeatedly conduct the independent iterative search of desired geometric transformation to align the source point set with the target one. This limits their use in applications to handle the real-time point set registration with large volume dataset. This paper presents a novel method, named coherent point drift networks (CPD-Net), for the unsupervised learning of geometric transformation towards real-time non-rigid point set registration. In contrast to previous efforts (e.g. coherent point drift), CPD-Net can learn displacement field function to estimate geometric transformation from a training dataset, consequently, to predict the desired geometric transformation for the alignment of previously unseen pairs without any additional iterative optimization process. Furthermore, CPD-Net leverages the power of deep neural networks to fit an arbitrary function, that adaptively accommodates different levels of complexity of the desired geometric transformation. Particularly, CPD-Net is proved with a theoretical guarantee to learn a continuous displacement vector function that could further avoid imposing additional parametric smoothness constraint as in previous works. Our experiments verify the impressive performance of CPD-Net for non-rigid point set registration on various 2D/3D datasets, even in the presence of significant displacement noise, outliers, and missing points. Our code will be available at https://github.com/nyummvc/CPD-Net.
Comparison of block-based and hybrid-based environments in transferring programming skills to text-based environments
Venue: arXiv cs.CY 2019
Year: 2019

Teachers face several challenges when presenting the fundamental concepts of programming in the classroom. Several tools are introduced to give a visual dimension to support the learning process. These tools rely on code blocks, easily manipulated in a plug and play fashion, to build a program. These block-based tools intend to familiarize students with programming logic, before diving into text-based programming languages such as Java, Python, etc. However; when transitioning from block-based to text-based programming, students often encounter a gap in their learning. The student may not be able to apply block-based foundations in a text-based environment. To bridge the gap between both environments, we developed a hybrid-based learning approach. We found that on average a hybrid-based approach increases the students understanding of programming foundations, memorization, and ease of transition by more than 30% when compared to a block-based to text-based learning approach. Finally, we provide the community with an open source, hybrid-based learning tool that can be used by students when learning programming concepts or for future studies.
Investigating Writing Style Development in High School
Venue: arXiv cs.CY 2019
Year: 2019

In this paper we do the first large scale analysis of writing style development among Danish high school students. More than 10K students with more than 100K essays are analyzed. Writing style itself is often studied in the natural language processing community, but usually with the goal of verifying authorship, assessing quality or popularity, or other kinds of predictions. In this work, we analyze writing style changes over time, with the goal of detecting global development trends among students, and identifying at-risk students. We train a Siamese neural network to compute the similarity between two texts. Using this similarity measure, a student's newer essays are compared to their first essays, and a writing style development profile is constructed for the student. We cluster these student profiles and analyze the resulting clusters in order to detect general development patterns. We evaluate clusters with respect to writing style quality indicators, and identify optimal clusters, showing significant improvement in writing style, while also observing suboptimal clusters, exhibiting periods of limited development and even setbacks. Furthermore, we identify general development trends between high school students, showing that as students progress through high school, their writing style deviates, leaving students less similar when they finish high school, than when they start.
QoS Categories Activeness-Aware Adaptive EDCA Algorithm for Dense IoT Networks
Venue: arXiv cs.NI 2019
Year: 2019

IEEE 802.11 networks have a great role to play in supporting and deploying of the Internet of Things (IoT). The realization of IoT depends on the ability of the network to handle a massive number of stations and transmissions, and to support Quality of Service (QoS). IEEE 802.11 networks enable the QoS by applying the Enhanced Distributed Channel Access (EDCA) with static parameters regardless of existing network capacity or which Access Category (AC) of QoS is already active. Our objective in this paper is to improve the efficiency of the uplink access in 802.11 networks; therefore we proposed an algorithm called QoS Categories Activeness-Aware Adaptive EDCA Algorithm (QCAAAE) which adapts Contention Window (CW) size, and Arbitration Inter-Frame Space Number (AIFSN) values depending on the number of associated Stations (STAs) and considering the presence of each AC. For different traffic scenarios, the simulation results confirm the outperformance of the proposed algorithm in terms of throughput (increased on average 23%) and retransmission attempts rate (decreased on average 47%) considering acceptable delay for sensitive delay services.
Multi-modal Active Learning From Human Data: A Deep Reinforcement Learning Approach
Venue: arXiv cs.LG 2019
Year: 2019

Human behavior expression and experience are inherently multi-modal, and characterized by vast individual and contextual heterogeneity. To achieve meaningful human-computer and human-robot interactions, multi-modal models of the users states (e.g., engagement) are therefore needed. Most of the existing works that try to build classifiers for the users states assume that the data to train the models are fully labeled. Nevertheless, data labeling is costly and tedious, and also prone to subjective interpretations by the human coders. This is even more pronounced when the data are multi-modal (e.g., some users are more expressive with their facial expressions, some with their voice). Thus, building models that can accurately estimate the users states during an interaction is challenging. To tackle this, we propose a novel multi-modal active learning (AL) approach that uses the notion of deep reinforcement learning (RL) to find an optimal policy for active selection of the users data, needed to train the target (modality-specific) models. We investigate different strategies for multi-modal data fusion, and show that the proposed model-level fusion coupled with RL outperforms the feature-level and modality-specific models, and the naive AL strategies such as random sampling, and the standard heuristics such as uncertainty sampling. We show the benefits of this approach on the task of engagement estimation from real-world child-robot interactions during an autism therapy. Importantly, we show that the proposed multi-modal AL approach can be used to efficiently personalize the engagement classifiers to the target user using a small amount of actively selected users data.
Identifying Operational Data-paths in Software Defined Networking Driven Data-planes
Venue: arXiv cs.NI 2019
Year: 2019

In this paper, we propose an approach that relies on distributed traffic generation and monitoring to identify the operational data-paths in a given Software Defined Networking (SDN) driven data-plane. We show that under certain assumptions, there exist necessary and sufficient conditions for formally guaranteeing that all operational data-paths are discovered using our approach. In order to provide reliable communication within the SDN driven data-planes, assuring that the implemented data-paths are the requested (and expected) ones is necessary. This requires discovering the actual operational (running) data-paths in the data-plane. In SDN, different applications may configure different coexisting data-paths, the resulting data-paths a specific network flow traverses may not be the intended ones. Furthermore, the SDN components may be defected or compromised. We focus on discovering the operational data-paths on SDN driven data-planes. However, the proposed approach is applicable to any data-plane where the operational data-paths must be verified and / or certified. A data-path discovery toolkit has been implemented. We describe the corresponding set of tools, and showcase the obtained experimental results that reveal inconsistencies in well-known SDN applications.
Importance Weighted Adversarial Variational Autoencoders for Spike Inference from Calcium Imaging Data
Venue: arXiv cs.LG 2019
Year: 2019

The Importance Weighted Auto Encoder (IWAE) objective has been shown to improve the training of generative models over the standard Variational Auto Encoder (VAE) objective. Here, we derive importance weighted extensions to AVB and AAE. These latent variable models use implicitly defined inference networks whose approximate posterior density q_ϕ(z|x) cannot be directly evaluated, an essential ingredient for importance weighting. We show improved training and inference in latent variable models with our adversarially trained importance weighting method, and derive new theoretical connections between adversarial generative model training criteria and marginal likelihood based methods. We apply these methods to the important problem of inferring spiking neural activity from calcium imaging data, a challenging posterior inference problem in neuroscience, and show that posterior samples from the adversarial methods outperform factorized posteriors used in VAEs.
Representing and Using Knowledge with the Contextual Evaluation Model
Venue: arXiv cs.AI 2019
Year: 2019

This paper introduces the Contextual Evaluation Model (CEM), a novel method for knowledge representation and manipulation. The CEM differs from existing models in that it integrates facts, patterns and sequences into a single contextual framework. V5, an implementation of the model is presented and demonstrated with multiple annotated examples. The paper includes simulations demonstrating how the model reacts to pleasure/pain stimuli. The 'thought' is defined within the model and examples are given converting thoughts to language, converting language to thoughts and how 'meaning' arises from thoughts. A pattern learning algorithm is described. The algorithm is applied to multiple problems ranging from recognizing a voice to the autonomous learning of a simplified natural language.
Transwarp Conduit: Interoperable Blockchain Application Framework
Venue: arXiv cs.SE 2019
Year: 2019

Transwarp-Conduit (TWC) is a protocol for message transfers between two smart-contract enabled blockchains. Furthermore, we specify an application framework (leveraging the TWC protocol) that enables developers to define arbitrarily complex cross-blockchain applications, simply by deploying framework-compliant smart contracts and hosting a TWC node (daemon process).The TWC protocol is implementable without additional effort on part of the base blockchain protocol.
Visual-Inertial Odometry of Aerial Robots
Venue: arXiv cs.RO 2019
Year: 2019

Visual-Inertial odometry (VIO) is the process of estimating the state (pose and velocity) of an agent (e.g., an aerial robot) by using only the input of one or more cameras plus one or more Inertial Measurement Units (IMUs) attached to it. VIO is the only viable alternative to GPS and lidar-based odometry to achieve accurate state estimation. Since both cameras and IMUs are very cheap, these sensor types are ubiquitous in all today's aerial robots.
Token-Curated Registry with Citation Graph
Venue: arXiv cs.DL 2019
Year: 2019

In this study, we aim to incorporate the expertise of anonymous curators into a token-curated registry (TCR), a decentralized recommender system for collecting a list of high-quality content. This registry is important, because previous studies on TCRs have not specifically focused on technical content, such as academic papers and patents, whose effective curation requires expertise in relevant fields. To measure expertise, curation in our model focuses on both the content and its citation relationships, for which curator assignment uses the Personalized PageRank (PPR) algorithm while reward computation uses a multi-task peer-prediction mechanism. Our proposed CitedTCR bridges the literature on network-based and token-based recommender systems and contributes to the autonomous development of an evolving citation graph for high-quality content. Moreover, we experimentally confirm the incentive for registration and curation in CitedTCR using the simplification of a one-to-one correspondence between users and content (nodes).
Do Authors Deposit on Time? Tracking Open Access Policy Compliance
Venue: arXiv cs.DL 2019
Year: 2019

Recent years have seen fast growth in the number of policies mandating Open Access (OA) to research outputs. We conduct a large-scale analysis of over 800 thousand papers from repositories around the world published over a period of 5 years to investigate: a) if the time lag between the date of publication and date of deposit in a repository can be effectively tracked across thousands of repositories globally, and b) if introducing deposit deadlines is associated with a reduction of time from acceptance to public availability of research outputs. We show that after the introduction of the UK REF 2021 OA policy, this time lag has decreased significantly in the UK and that the policy introduction might have accelerated the UK's move towards immediate OA compared to other countries. This supports the argument for the inclusion of a time-limited deposit requirement in OA policies.
When Unseen Domain Generalization is Unnecessary? Rethinking Data Augmentation
Venue: arXiv cs.CV 2019
Year: 2019

Recent advances in deep learning for medical image segmentation demonstrate expert-level accuracy. However, in clinically realistic environments, such methods have marginal performance due to differences in image domains, including different imaging protocols, device vendors and patient populations. Here we consider the problem of domain generalization, when a model is trained once, and its performance generalizes to unseen domains. Intuitively, within a specific medical imaging modality the domain differences are smaller relative to natural images domain variability. We rethink data augmentation for medical 3D images and propose a deep stacked transformations (DST) approach for domain generalization. Specifically, a series of n stacked transformations are applied to each image in each mini-batch during network training to account for the contribution of domain-specific shifts in medical images. We comprehensively evaluate our method on three tasks: segmentation of whole prostate from 3D MRI, left atrial from 3D MRI, and left ventricle from 3D ultrasound. We demonstrate that when trained on a small source dataset, (i) on average, DST models on unseen datasets degrade only by 11% (Dice score change), compared to the conventional augmentation (degrading 39%) and CycleGAN-based domain adaptation method (degrading 25%); (ii) when evaluation on the same domain, DST is also better albeit only marginally. (iii) When training on large-sized data, DST on unseen domains reaches performance of state-of-the-art fully supervised models. These findings establish a strong benchmark for the study of domain generalization in medical imaging, and can be generalized to the design of robust deep segmentation models for clinical deployment.
A Two-Step Graph Convolutional Decoder for Molecule Generation
Venue: arXiv cs.LG 2019
Year: 2019

We propose a simple auto-encoder framework for molecule generation. The molecular graph is first encoded into a continuous latent representation $z$, which is then decoded back to a molecule. The encoding process is easy, but the decoding process remains challenging. In this work, we introduce a simple two-step decoding process. In a first step, a fully connected neural network uses the latent vector $z$ to produce a molecular formula, for example CO$_2$ (one carbon and two oxygen atoms). In a second step, a graph convolutional neural network uses the same latent vector $z$ to place bonds between the atoms that were produced in the first step (for example a double bond will be placed between the carbon and each of the oxygens). This two-step process, in which a bag of atoms is first generated, and then assembled, provides a simple framework that allows us to develop an efficient molecule auto-encoder. Numerical experiments on basic tasks such as novelty, uniqueness, validity and optimized chemical property for the 250k ZINC molecules demonstrate the performances of the proposed system. Particularly, we achieve the highest reconstruction rate of 90.5\%, improving the previous rate of 76.7\%. We also report the best property improvement results when optimization is constrained by the molecular distance between the original and generated molecules.
News Labeling as Early as Possible: Real or Fake?
Venue: arXiv cs.SI 2019
Year: 2019

Making disguise between real and fake news propagation through online social networks is an important issue in many applications. The time gap between the news release time and detection of its label is a significant step towards broadcasting the real information and avoiding the fake. Therefore, one of the challenging tasks in this area is to identify fake and real news in early stages of propagation. However, there is a trade-off between minimizing the time gap and maximizing accuracy. Despite recent efforts in detection of fake news, there has been no significant work that explicitly incorporates early detection in its model. In this paper, we focus on accurate early labeling of news, and propose a model by considering earliness both in modeling and prediction. The proposed method utilizes recurrent neural networks with a novel loss function, and a new stopping rule. Given the context of news, we first embed it with a class-specific text representation. Then, we utilize the available public profile of users, and speed of news diffusion, for early labeling of the news. Experiments on real datasets demonstrate the effectiveness of our model both in terms of early labelling and accuracy, compared to the state of the art baseline and models.
Sensitivity of Deep Convolutional Networks to Gabor Noise
Venue: arXiv cs.LG 2019
Year: 2019

Deep Convolutional Networks (DCNs) have been shown to be sensitive to Universal Adversarial Perturbations (UAPs): input-agnostic perturbations that fool a model on large portions of a dataset. These UAPs exhibit interesting visual patterns, but this phenomena is, as yet, poorly understood. Our work shows that visually similar procedural noise patterns also act as UAPs. In particular, we demonstrate that different DCN architectures are sensitive to Gabor noise patterns. This behaviour, its causes, and implications deserve further in-depth study.