OpenRaiser/Intern-Atlas huggingface dataset:
Asynchronous Distributed Learning from Constraints
Venue: arXiv cs.LG 2019
Year: 2019

In this paper, the extension of the framework of Learning from Constraints (LfC) to a distributed setting where multiple parties, connected over the network, contribute to the learning process is studied. LfC relies on the generic notion of "constraint" to inject knowledge into the learning problem and, due to its generality, it deals with possibly nonconvex constraints, enforced either in a hard or soft way. Motivated by recent progresses in the field of distributed and constrained nonconvex optimization, we apply the (distributed) Asynchronous Method of Multipliers (ASYMM) to LfC. The study shows that such a method allows us to support scenarios where selected constraints (i.e., knowledge), data, and outcomes of the learning process can be locally stored in each computational node without being shared with the rest of the network, opening the road to further investigations into privacy-preserving LfC. Constraints act as a bridge between what is shared over the net and what is private to each node and no central authority is required. We demonstrate the applicability of these ideas in two distributed real-world settings in the context of digit recognition and document classification.
Generation and Classification of Activity Sequences for Spatiotemporal Modeling of Human Populations
Venue: arXiv cs.SI 2019
Year: 2019

Human activity encompasses a series of complex spatiotemporal processes that are difficult to model, but represents an essential component of human exposure assessment. A significant empirical data source like the American Time Use Survey (ATUS) can be leveraged to model human activity, but tractable models require a better stratification of activity data to inform about different, but classifiable groups of individuals that exhibit similar activities and mobility patterns. We have developed a simple unsupervised classification and sequence generation method from existing machine learning algorithms that is capable of generating coherent and stochastic sequences of activity from the data in the ATUS. This classification, when combined with any spatiotemporal exposure profile, allows the development of stochastic models of exposure patterns for groups of individuals exhibiting similar activity behaviors.
Cost-efficient segmentation of electron microscopy images using active learning
Venue: arXiv cs.CV 2019
Year: 2019

Over the last decade, electron microscopy has improved up to a point that generating high quality gigavoxel sized datasets only requires a few hours. Automated image analysis, particularly image segmentation, however, has not evolved at the same pace. Even though state-of-the-art methods such as U-Net and DeepLab have improved segmentation performance substantially, the required amount of labels remains too expensive. Active learning is the subfield in machine learning that aims to mitigate this burden by selecting the samples that require labeling in a smart way. Many techniques have been proposed, particularly for image classification, to increase the steepness of learning curves. In this work, we extend these techniques to deep CNN based image segmentation. Our experiments on three different electron microscopy datasets show that active learning can improve segmentation quality by 10 to 15% in terms of Jaccard score compared to standard randomized sampling.
Interaction with Ubiquitous Robots and Autonomous IoT
Venue: arXiv cs.HC 2019
Year: 2019

Robotics have been slowly permeating Internet of Things (IoT) where the previously ubiquitous but static sensors are now given the power to actively navigate the environment and even interact with users. Emergence of these ubiquitous swarms of robots not only opens up the range of possible applications, but also increases the number of elements to study and design for. We do not yet understand how, when, and where these robots should move, manipulate, and touch around people. Through user-centered studies, we aim to better understand how to best design for interaction with Autonomous IoT or a swarm of ubiquitous robots.
Exploiting Satellite Broadcast despite HTTPS
Venue: arXiv cs.CR 2019
Year: 2019

HTTPS enhances end-user privacy and is often preferred or enforced by over-the-top content providers, but renders inoperable all intermediate network functions operating above the transport layer, including caching, content/protocol optimization, and security filtering tools. These functions are crucial for the optimization of integrated satellite-terrestrial networks. Additionally, due to the use of end-to-end and per-session encryption keys, the advantages of a satellite's wide-area broadcasting capabilities are limited or even negated completely. This paper investigates two solutions for authorized TLS interception that involve TLS splitting. We present how these solutions can be incorporated into integrated satellite-terrestrial networks and we discuss their trade-offs in terms of deployment, performance, and privacy. Furthermore, we design a solution that leverages satellite broadcast transmission even in the presence of TLS (i.e. with the use of HTTPS) by exploiting application layer encryption in the path between the satellite terminal and the TLS server. Our findings indicate that even if no other operation than TLS splitting is performed, TLS handshake time, which involves roundtrips through possibly a Geosynchronous satellite, can be reduced by up to 94%. Moreover, by combining an application layer encryption solution with TLS splitting, broadcast transmissions can be exploited
ANOVA Gaussian process modeling for high-dimensional stochastic computational models
Venue: arXiv cs.CE 2019
Year: 2019

In this paper we present a novel analysis of variance Gaussian process (ANOVA-GP) emulator for models governed by partial differential equations (PDEs) with high-dimensional random inputs. Gaussian process (GP) is a widely used surrogate modeling strategy, but it can become invalid when the inputs are high-dimensional. In this new ANOVA-GP strategy, high-dimensional inputs are decomposed into unions of local low-dimensional inputs, and principal component analysis (PCA) is applied to provide dimension reduction for each ANOVA term. We then systematically build local GP models for PCA coefficients based on ANOVA decomposition to provide an emulator for the overall high-dimensional problem. We present a general mathematical framework of ANOVA-GP, validate its accuracy and demonstrate its efficiency with numerical experiments.
Wavelets to the Rescue: Improving Sample Quality of Latent Variable Deep Generative Models
Venue: arXiv cs.CV 2019
Year: 2019

Variational Autoencoders (VAE) are probabilistic deep generative models underpinned by elegant theory, stable training processes, and meaningful manifold representations. However, they produce blurry images due to a lack of explicit emphasis over high-frequency textural details of the images, and the difficulty to directly model the complex joint probability distribution over the high-dimensional image space. In this work, we approach these two challenges with a novel wavelet space VAE that uses the decoder to model the images in the wavelet coefficient space. This enables the VAE to emphasize over high-frequency components within an image obtained via wavelet decomposition. Additionally, by decomposing the complex function of generating high-dimensional images into inverse wavelet transformation and generation of wavelet coefficients, the latter becomes simpler to model by the VAE. We empirically validate that deep generative models operating in the wavelet space can generate images of higher quality than the image (RGB) space counterparts. Quantitatively, on benchmark natural image datasets, we achieve consistently better FID scores than VAE based architectures and competitive FID scores with a variety of GAN models for the same architectural and experimental setup. Furthermore, the proposed wavelet-based generative model retains desirable attributes like disentangled and informative latent representation without losing the quality in the generated samples.
A Brief Review on Some Architectures Providing Support for DIFT
Venue: arXiv cs.CR 2019
Year: 2019

Dynamic Information Flow Tracking (DIFT) is a technique to track potential security vulnerabilities in software and hardware systems at run time. The last fifteen years have seen a lot of research work on DIFT, including both hardware-based and software-based implementations for different types of processor architectures. This survey briefly reviews some hardware architectures that provide DIFT support. Starting from introducing different approaches for hardware based DIFT, this survey focuses on integrated/in-core architectures. Protection schemes, including tagging system, tag propagation, and tag checking for each architecture will be discussed. The survey is organized in such a way that it illustrates the evolution of integrated DIFT architectures, each architecture tries to improve the precious proposed architectures generality/versatility weaknesses. However, improving security while providing generality and versatility is kind of trade-offs. This survey compares the architectures from different aspects to show the trade-offs clearer.
Modeling patterns of smartphone usage and their relationship to cognitive health
Venue: arXiv cs.LG 2019
Year: 2019

The ubiquity of smartphone usage in many people's lives make it a rich source of information about a person's mental and cognitive state. In this work we analyze 12 weeks of phone usage data from 113 older adults, 31 with diagnosed cognitive impairment and 82 without. We develop structured models of users' smartphone interactions to reveal differences in phone usage patterns between people with and without cognitive impairment. In particular, we focus on inferring specific types of phone usage sessions that are predictive of cognitive impairment. Our model achieves an AUROC of 0.79 when discriminating between healthy and symptomatic subjects, and its interpretability enables novel insights into which aspects of phone usage strongly relate with cognitive health in our dataset.
Anomaly Detection for Industrial Control Networks using Machine Learning with the help from the Inter-Arrival Curves
Venue: arXiv cs.CR 2019
Year: 2019

Industrial Control Networks (ICN) such as Supervisory Control and Data Acquisition (SCADA) systems are widely used in industries for monitoring and controlling physical processes. These industries include power generation and supply, gas and oil production and delivery, water and waste management, telecommunication and transport facilities. The integration of internet exposes these systems to cyber threats. The consequences of compromised ICN are determine for a country economic and functional sustainability. Therefore, enforcing security and ensuring correctness operation became one of the biggest concerns for Industrial Control Systems (ICS), and need to be addressed. In this paper, we propose an anomaly detection approach for ICN using the physical properties of the system. We have developed operational baseline of electricity generation process and reduced the feature set using greedy and genetic feature selection algorithms. The classification is done based on Support Vector Machine (SVM), k-Nearest Neighbor (k-NN), and C4.5 decision tree with the help from the inter-arrival curves. The results show that the proposed approach successfully detects anomalies with a high degree of accuracy. In addition, they proved that SVM and C4.5 produces accurate results even for high sensitivity attacks when they used with the inter-arrival curves. As compared to this, k-NN is unable to produce good results for low and medium sensitivity attacks test cases.
On the Relativized Alon Eigenvalue Conjecture II: Asymptotic Expansion Theorems for Walks
Venue: arXiv cs.DM 2019
Year: 2019

This is the second in a series of articles devoted to showing that a typical covering map of large degree to a fixed, regular graph has its new adjacency eigenvalues within the bound conjectured by Alon for random regular graphs. The first main result in this article concerns the function $f(k,n)$ defined as the number of SNBC (strictly non-backtracking closed) walks of length $k$ of a given homotopy type in a random covering graph of degree $n$ of a fixed graph. We prove the existence of asymptotic expansions in powers of $1/n$ for $f(k,n)$, where the coefficients---functions of $k$---are proven to have some desirable properties; namely, these coefficients are approximately a sum of polynomials times exponential functions. The second main result is a generalization of the first, where the number of SNBC walks of length $k$ is multiplied by an indicator function that the covering graph contains a certain type of {\em tangle}; the second result requires more terminology, although its proof uses the same basic tools used to prove the first result. % The motivation for the second main result will be clear in % the third article in this series of articles. The results in this article are mostly straightforward generalizations of methods used in previous works. However, this article (1) "factors" these methods into a number of short, conceptually simple, and independent parts, (2) writes each independent part in more general terms, and (3) significantly simplifies of one of the previous computations. As such we expect that this article will make it easier to apply trace methods to related models of random graphs.
Mark my Word: A Sequence-to-Sequence Approach to Definition Modeling
Venue: arXiv cs.CL 2019
Year: 2019

Defining words in a textual context is a useful task both for practical purposes and for gaining insight into distributed word representations. Building on the distributional hypothesis, we argue here that the most natural formalization of definition modeling is to treat it as a sequence-to-sequence task, rather than a word-to-sequence task: given an input sequence with a highlighted word, generate a contextually appropriate definition for it. We implement this approach in a Transformer-based sequence-to-sequence model. Our proposal allows to train contextualization and definition generation in an end-to-end fashion, which is a conceptual improvement over earlier works. We achieve state-of-the-art results both in contextual and non-contextual definition modeling.
Atomic Services: sustainable ecosystem of smart city services through pan-European collaboration
Venue: arXiv cs.CY 2019
Year: 2019

In a world with an ever increasing urbanization, governance is investigating innovative solutions to sustain the society evolution. Internet-of-Things promises huge benefits for cities and the proliferation of smart city deployments demonstrates the common acceptance of IoT as basis for many solutions. The city pilots developments occurred in parallel and with different designs thus creating fragmentation of IoT. The European project SynchroniCity aims to synchronize 8 smart cities to establish a shared environment fostering a self-sustained business growth. In this article we present the collaborative methodology and shared efforts spent towards the creation of a common ecosystem for the development of smart city services. Our design evolves around the concept of "atomic services" that implements a single functional block to be composed for full-fledged smart city services. This creates opportunities for diverse stakeholders to participate to a global smart cities market. The methodology and outcome of our efforts will be followed by 10 new cities globally, thus expanding the market range for IoT stakeholders
Maximizing the Partial Decode-and-Forward Rate in the Gaussian MIMO Relay Channel
Venue: arXiv cs.IT 2019
Year: 2019

It is known that circularly symmetric Gaussian signals are the optimal input signals for the partial decode-and-forward (PDF) coding scheme in the Gaussian multiple-input multiple-output (MIMO) relay channel, but there is currently no method to find the optimal covariance matrices nor to compute the optimal achievable PDF rate since the optimization is a non-convex problem in its original formulation. In this paper, we show that it is possible to find a convex reformulation of the problem by means of an approximation and a primal decomposition. We derive an explicit solution for the inner problems as well as an explicit gradient for the outer problem, so that the efficient cutting-plane method can be applied for solving the outer problem. As the accuracy of this provably convergent algorithm might be impaired by the previous approximation, we additionally propose a modified algorithm, for which we cannot give a converge guarantee, but which provides rigorous upper and lower bounds to the optimum of the original rate maximization. In numerical simulations, these bounds become tight in all considered instances of the problem, showing that the proposed method manages to find the global optimum in all these instances.
A Root-Free Splitting-Lemma for Systems of Linear Differential Equations
Venue: arXiv cs.SC 2019
Year: 2019

We consider the formal reduction of a system of linear differential equations and show that, if the system can be block-diagonalised through transformation with a ramified Shearing-transformation and following application of the Splitting Lemma, and if the spectra of the leading block matrices of the ramified system satisfy a symmetry condition, this block-diagonalisation can also be achieved through an unramified transformation. Combined with classical results by Turritin and Wasow as well as work by Balser, this yields a constructive and simple proof of the existence of an unramified block-diagonal form from which formal invariants such as the Newton polygon can be read directly. Our result is particularly useful for designing efficient algorithms for the formal reduction of the system.
Federated and Differentially Private Learning for Electronic Health Records
Venue: arXiv cs.LG 2019
Year: 2019

The use of collaborative and decentralized machine learning techniques such as federated learning have the potential to enable the development and deployment of clinical risk predictions models in low-resource settings without requiring sensitive data be shared or stored in a central repository. This process necessitates communication of model weights or updates between collaborating entities, but it is unclear to what extent patient privacy is compromised as a result. To gain insight into this question, we study the efficacy of centralized versus federated learning in both private and non-private settings. The clinical prediction tasks we consider are the prediction of prolonged length of stay and in-hospital mortality across thirty one hospitals in the eICU Collaborative Research Database. We find that while it is straightforward to apply differentially private stochastic gradient descent to achieve strong privacy bounds when training in a centralized setting, it is considerably more difficult to do so in the federated setting.
Witnessing Secure Compilation
Venue: arXiv cs.FL 2019
Year: 2019

Compiler optimizations are designed to improve run-time performance while preserving input-output behavior. Correctness in this sense does not necessarily preserve security: it is known that standard optimizations may break or weaken security properties that hold of the source program. This work develops a translation validation method for secure compilation. Security (hyper-)properties are expressed using automata operating over a bundle of program traces. A flexible, automaton-based refinement scheme, generalizing existing refinement methods, guarantees that the associated security property is preserved by a program transformation. In practice, the refinement relations ("security witnesses") can be generated during compilation and validated independently with a refinement checker. This process is illustrated for common optimizations. Crucially, it is not necessary to verify the compiler implementation itself, which is infeasible in practice for production compilers.
Partial-Order, Partially-Seen Observations of Fluents or Actions for Plan Recognition as Planning
Venue: arXiv cs.AI 2019
Year: 2019

This work aims to make plan recognition as planning more ready for real-world scenarios by adapting previous compilations to work with partial-order, half-seen observations of both fluents and actions. We first redefine what observations can be and what it means to satisfy each kind. We then provide a compilation from plan recognition problem to classical planning problem, similar to original work by Ramirez and Geffner, but accommodating these more complex observation types. This compilation can be adapted towards other planning-based plan recognition techniques. Lastly we evaluate this method against an "ignore complexity" strategy that uses the original method by Ramirez and Geffner. Our experimental results suggest that, while slower, our method is equally or more accurate than baseline methods; our technique sometimes significantly reduces the size of the solution to the plan recognition problem, i.e, the size of the optimal goal set. We discuss these findings in the context of plan recognition problem difficulty and present an avenue for future work.
Image-Based Feature Representation for Insider Threat Classification
Venue: arXiv cs.CR 2019
Year: 2019

Insiders are the trusted entities in the organization, but poses threat to the with access to sensitive information network and resources. The insider threat detection is a well studied problem in security analytics. Identifying the features from data sources and using them with the right data analytics algorithms makes various kinds of threat analysis possible. The insider threat analysis is mainly done using the frequency based attributes extracted from the raw data available from data sources. In this paper, we propose an image-based feature representation of the daily resource usage pattern of users in the organization. The features extracted from the audit files of the organization are represented as gray scale images. Hence, these images are used to represent the resource access patterns and thereby the behavior of users. Classification models are applied to the representative images to detect anomalous behavior of insiders. The images are classified to malicious and non-malicious. The effectiveness of the proposed representation is evaluated using the CMU CERT data V4.2, and state-of-art image classification models like Mobilenet, VGG and ResNet. The experimental results showed improved accuracy. The comparison with existing works show a performance improvement in terms of high recall and precision values.
Coincidence, Categorization, and Consolidation: Learning to Recognize Sounds with Minimal Supervision
Venue: arXiv cs.SD 2019
Year: 2019

Humans do not acquire perceptual abilities in the way we train machines. While machine learning algorithms typically operate on large collections of randomly-chosen, explicitly-labeled examples, human acquisition relies more heavily on multimodal unsupervised learning (as infants) and active learning (as children). With this motivation, we present a learning framework for sound representation and recognition that combines (i) a self-supervised objective based on a general notion of unimodal and cross-modal coincidence, (ii) a clustering objective that reflects our need to impose categorical structure on our experiences, and (iii) a cluster-based active learning procedure that solicits targeted weak supervision to consolidate categories into relevant semantic classes. By training a combined sound embedding/clustering/classification network according to these criteria, we achieve a new state-of-the-art unsupervised audio representation and demonstrate up to a 20-fold reduction in the number of labels required to reach a desired classification performance.