OpenRaiser/Intern-Atlas huggingface dataset:
Privacy against Statistical Matching: Inter-User Correlation
Venue: arXiv cs.IT 2018
Year: 2018

Modern applications significantly enhance user experience by adapting to each user's individual condition and/or preferences. While this adaptation can greatly improve utility or be essential for the application to work (e.g., for ride-sharing applications), the exposure of user data to the application presents a significant privacy threat to the users, even when the traces are anonymized, since the statistical matching of an anonymized trace to prior user behavior can identify a user and their habits. Because of the current and growing algorithmic and computational capabilities of adversaries, provable privacy guarantees as a function of the degree of anonymization and obfuscation of the traces are necessary. Our previous work has established the requirements on anonymization and obfuscation in the case that data traces are independent between users. However, the data traces of different users will be dependent in many applications, and an adversary can potentially exploit such. In this paper, we consider the impact of correlation between user traces on their privacy. First, we demonstrate that the adversary can readily identify the association graph, revealing which user data traces are correlated. Next, we demonstrate that the adversary can use this association graph to break user privacy with significantly shorter traces than in the case when traces are independent between users, and that independent obfuscation of the data traces is often insufficient to remedy such. Finally, we discuss how the users can employ dependency in their obfuscation to improve their privacy.
Evaluation of CNN-based Single-Image Depth Estimation Methods
Venue: arXiv cs.CV 2018
Year: 2018

While an increasing interest in deep models for single-image depth estimation methods can be observed, established schemes for their evaluation are still limited. We propose a set of novel quality criteria, allowing for a more detailed analysis by focusing on specific characteristics of depth maps. In particular, we address the preservation of edges and planar regions, depth consistency, and absolute distance accuracy. In order to employ these metrics to evaluate and compare state-of-the-art single-image depth estimation approaches, we provide a new high-quality RGB-D dataset. We used a DSLR camera together with a laser scanner to acquire high-resolution images and highly accurate depth maps. Experimental results show the validity of our proposed evaluation protocol.
Research on the Brain-inspired Cross-modal Neural Cognitive Computing Framework
Venue: arXiv cs.NE 2018
Year: 2018

To address modeling problems of brain-inspired intelligence, this thesis is focused on researching in the semantic-oriented framework design for multimedia and multimodal information. The Multimedia Neural Cognitive Computing (MNCC) model was designed based on the nervous mechanism and cognitive architecture. Furthermore, the semantic-oriented hierarchical Cross-modal Neural Cognitive Computing (CNCC) framework was proposed based on MNCC model, and formal description and analysis for CNCC framework was given. It would effectively improve the performance of semantic processing for multimedia and cross-modal information, and has far-reaching significance for exploration and realization brain-inspired computing.
Convexity of mutual information along the Ornstein-Uhlenbeck flow
Venue: arXiv cs.IT 2018
Year: 2018

We study the convexity of mutual information as a function of time along the flow of the Ornstein-Uhlenbeck process. We prove that if the initial distribution is strongly log-concave, then mutual information is eventually convex, i.e., convex for all large time. In particular, if the initial distribution is sufficiently strongly log-concave compared to the target Gaussian measure, then mutual information is always a convex function of time. We also prove that if the initial distribution is either bounded or has finite fourth moment and Fisher information, then mutual information is eventually convex. Finally, we provide counterexamples to show that mutual information can be nonconvex at small time.
The Fine Line between Linguistic Generalization and Failure in Seq2Seq-Attention Models
Venue: arXiv cs.CL 2018
Year: 2018

Seq2Seq based neural architectures have become the go-to architecture to apply to sequence to sequence language tasks. Despite their excellent performance on these tasks, recent work has noted that these models usually do not fully capture the linguistic structure required to generalize beyond the dense sections of the data distribution \cite{ettinger2017towards}, and as such, are likely to fail on samples from the tail end of the distribution (such as inputs that are noisy \citep{belkinovnmtbreak} or of different lengths \citep{bentivoglinmtlength}). In this paper, we look at a model's ability to generalize on a simple symbol rewriting task with a clearly defined structure. We find that the model's ability to generalize this structure beyond the training distribution depends greatly on the chosen random seed, even when performance on the standard test set remains the same. This suggests that a model's ability to capture generalizable structure is highly sensitive. Moreover, this sensitivity may not be apparent when evaluating it on standard test sets.
Lifted Neural Networks
Venue: arXiv cs.LG 2018
Year: 2018

We describe a novel family of models of multi- layer feedforward neural networks in which the activation functions are encoded via penalties in the training problem. Our approach is based on representing a non-decreasing activation function as the argmin of an appropriate convex optimiza- tion problem. The new framework allows for algo- rithms such as block-coordinate descent methods to be applied, in which each step is composed of a simple (no hidden layer) supervised learning problem that is parallelizable across data points and/or layers. Experiments indicate that the pro- posed models provide excellent initial guesses for weights for standard neural networks. In addi- tion, the model provides avenues for interesting extensions, such as robustness against noisy in- puts and optimizing over parameters in activation functions.
Pixel-wise Attentional Gating for Parsimonious Pixel Labeling
Venue: arXiv cs.CV 2018
Year: 2018

To achieve parsimonious inference in per-pixel labeling tasks with a limited computational budget, we propose a \emph{Pixel-wise Attentional Gating} unit (\emph{PAG}) that learns to selectively process a subset of spatial locations at each layer of a deep convolutional network. PAG is a generic, architecture-independent, problem-agnostic mechanism that can be readily "plugged in" to an existing model with fine-tuning. We utilize PAG in two ways: 1) learning spatially varying pooling fields that improve model performance without the extra computation cost associated with multi-scale pooling, and 2) learning a dynamic computation policy for each pixel to decrease total computation while maintaining accuracy. We extensively evaluate PAG on a variety of per-pixel labeling tasks, including semantic segmentation, boundary detection, monocular depth and surface normal estimation. We demonstrate that PAG allows competitive or state-of-the-art performance on these tasks. Our experiments show that PAG learns dynamic spatial allocation of computation over the input image which provides better performance trade-offs compared to related approaches (e.g., truncating deep models or dynamically skipping whole layers). Generally, we observe PAG can reduce computation by $10\%$ without noticeable loss in accuracy and performance degrades gracefully when imposing stronger computational constraints.
IBBE-SGX: Cryptographic Group Access Control using Trusted Execution Environments
Venue: arXiv cs.CR 2018
Year: 2018

While many cloud storage systems allow users to protect their data by making use of encryption, only few support collaborative editing on that data. A major challenge for enabling such collaboration is the need to enforce cryptographic access control policies in a secure and efficient manner. In this paper, we introduce IBBE-SGX, a new cryptographic access control extension that is efficient both in terms of computation and storage even when processing large and dynamic workloads of membership operations, while at the same time offering zero knowledge guarantees. IBBE-SGX builds upon Identity-Based Broadcasting Encryption (IBBE). We address IBBE's impracticality for cloud deployments by exploiting Intel Software Guard Extensions (SGX) to derive cuts in the computational complexity. Moreover, we propose a group partitioning mechanism such that the computational cost of membership update is bound to a fixed constant partition size rather than the size of the whole group. We have implemented and evaluated our new access control extension. Results highlight that IBBE-SGX performs membership changes 1.2 orders of magnitude faster than the traditional approach of Hybrid Encryption (HE), producing group metadata that are 6 orders of magnitude smaller than HE, while at the same time offering zero knowledge guarantees.
Recent Progress on Graph Partitioning Problems Using Evolutionary Computation
Venue: arXiv cs.NE 2018
Year: 2018

The graph partitioning problem (GPP) is a representative combinatorial optimization problem which is NP-hard. Currently, various approaches to solve GPP have been introduced. Among these, the GPP solution using evolutionary computation (EC) is an effective approach. There has not been any survey on the research applying EC to GPP since 2011. In this survey, we introduce various attempts to apply EC to GPP made in the recent seven years.
A Distributed Routing Algorithm for Internet-wide Geocast
Venue: arXiv cs.NI 2018
Year: 2018

Geocast is the concept of sending data packets to nodes in a specified geographical area instead of nodes with a specific address. To route geocast messages to their destination we need a geographic routing algorithm that can route packets efficiently to the devices inside the destination area. Our goal is to design an algorithm that can deliver shortest path tree like forwarding while relying purely on distributed data without central knowledge. In this paper, we present two algorithms for geographic routing. One based purely on distance vector data, and one more complicated algorithm based on path data. In our evaluation, we show that our purely distance vector based algorithm can come close to shortest path tree performance when a small number of routers are present in the destination area. We also show that our path based algorithm can come close to the performance of a shortest path tree in almost all geocast situations.
A Convex Approximation of the Relaxed Binaural Beamforming Optimization Problem
Venue: arXiv cs.SD 2018
Year: 2018

The recently proposed relaxed binaural beamforming (RBB) optimization problem provides a flexible trade-off between noise suppression and binaural-cue preservation of the sound sources in the acoustic scene. It minimizes the output noise power, under the constraints which guarantee that the target remains unchanged after processing and the binaural-cue distortions of the acoustic sources will be less than a user-defined threshold. However, the RBB problem is a computationally demanding non-convex optimization problem. The only existing suboptimal method which approximately solves the RBB is a successive convex optimization (SCO) method which, typically, requires to solve multiple convex optimization problems per frequency bin, in order to converge. Convergence is achieved when all constraints of the RBB optimization problem are satisfied. In this paper, we propose a semi-definite convex relaxation (SDCR) of the RBB optimization problem. The proposed suboptimal SDCR method solves a single convex optimization problem per frequency bin, resulting in a much lower computational complexity than the SCO method. Unlike the SCO method, the SDCR method does not guarantee user-controlled upper-bounded binaural-cue distortions. To tackle this problem we also propose a suboptimal hybrid method which combines the SDCR and SCO methods. Instrumental measures combined with a listening test show that the SDCR and hybrid methods achieve significantly lower computational complexity than the SCO method, and in most cases better trade-off between predicted intelligibility and binaural-cue preservation than the SCO method.
Dynamic Control Flow in Large-Scale Machine Learning
Venue: arXiv cs.DC 2018
Year: 2018

Many recent machine learning models rely on fine-grained dynamic control flow for training and inference. In particular, models based on recurrent neural networks and on reinforcement learning depend on recurrence relations, data-dependent conditional execution, and other features that call for dynamic control flow. These applications benefit from the ability to make rapid control-flow decisions across a set of computing devices in a distributed system. For performance, scalability, and expressiveness, a machine learning system must support dynamic control flow in distributed and heterogeneous environments. This paper presents a programming model for distributed machine learning that supports dynamic control flow. We describe the design of the programming model, and its implementation in TensorFlow, a distributed machine learning system. Our approach extends the use of dataflow graphs to represent machine learning models, offering several distinctive features. First, the branches of conditionals and bodies of loops can be partitioned across many machines to run on a set of heterogeneous devices, including CPUs, GPUs, and custom ASICs. Second, programs written in our model support automatic differentiation and distributed gradient computations, which are necessary for training machine learning models that use control flow. Third, our choice of non-strict semantics enables multiple loop iterations to execute in parallel across machines, and to overlap compute and I/O operations. We have done our work in the context of TensorFlow, and it has been used extensively in research and production. We evaluate it using several real-world applications, and demonstrate its performance and scalability.
SecureCloud: Secure Big Data Processing in Untrusted Clouds
Venue: arXiv cs.DC 2018
Year: 2018

We present the SecureCloud EU Horizon 2020 project, whose goal is to enable new big data applications that use sensitive data in the cloud without compromising data security and privacy. For this, SecureCloud designs and develops a layered architecture that allows for (i) the secure creation and deployment of secure micro-services; (ii) the secure integration of individual micro-services to full-fledged big data applications; and (iii) the secure execution of these applications within untrusted cloud environments. To provide security guarantees, SecureCloud leverages novel security mechanisms present in recent commodity CPUs, in particular, Intel's Software Guard Extensions (SGX). SecureCloud applies this architecture to big data applications in the context of smart grids. We describe the SecureCloud approach, initial results, and considered use cases.
Failure Prediction for Autonomous Driving
Venue: arXiv cs.CV 2018
Year: 2018

The primary focus of autonomous driving research is to improve driving accuracy. While great progress has been made, state-of-the-art algorithms still fail at times. Such failures may have catastrophic consequences. It therefore is important that automated cars foresee problems ahead as early as possible. This is also of paramount importance if the driver will be asked to take over. We conjecture that failures do not occur randomly. For instance, driving models may fail more likely at places with heavy traffic, at complex intersections, and/or under adverse weather/illumination conditions. This work presents a method to learn to predict the occurrence of these failures, i.e. to assess how difficult a scene is to a given driving model and to possibly give the human driver an early headsup. A camera-based driving model is developed and trained over real driving datasets. The discrepancies between the model's predictions and the human `ground-truth' maneuvers were then recorded, to yield the `failure' scores. Experimental results show that the failure score can indeed be learned and predicted. Thus, our prediction method is able to improve the overall safety of an automated driving model by alerting the human driver timely, leading to better human-vehicle collaborative driving.
Regular Expressions and Transducers over Alphabet-invariant and User-defined Labels
Venue: arXiv cs.FL 2018
Year: 2018

We are interested in regular expressions and transducers that represent word relations in an alphabet-invariant way---for example, the set of all word pairs u,v where v is a prefix of u independently of what the alphabet is. Current software systems of formal language objects do not have a mechanism to define such objects. We define transducers in which transition labels involve what we call set specifications, some of which are alphabet invariant. In fact, we give a more broad definition of automata-type objects, called labelled graphs, where each transition label can be any string, as long as that string represents a subset of a certain monoid. Then, the behaviour of the labelled graph is a subset of that monoid. We do the same for regular expressions. We obtain extensions of a few classic algorithmic constructions on ordinary regular expressions and transducers at the broad level of labelled graphs and in such a way that the computational efficiency of the extended constructions is not sacrificed. For regular expressions with set specs we obtain the corresponding partial derivative automata. For transducers with set specs we obtain further algorithms that can be applied to questions about independent regular languages, in particular the witness version of the independent property satisfaction question.
Positioning of High-speed Trains using 5G New Radio Synchronization Signals
Venue: arXiv cs.IT 2018
Year: 2018

We study positioning of high-speed trains in 5G new radio (NR) networks by utilizing specific NR synchronization signals. The studies are based on simulations with 3GPP-specified radio channel models including path loss, shadowing and fast fading effects. The considered positioning approach exploits measurement of Time-Of-Arrival (TOA) and Angle-Of-Departure (AOD), which are estimated from beamformed NR synchronization signals. Based on the given measurements and the assumed train movement model, the train position is tracked by using an Extended Kalman Filter (EKF), which is able to handle the non-linear relationship between the TOA and AOD measurements, and the estimated train position parameters. It is shown that in the considered scenario the TOA measurements are able to achieve better accuracy compared to the AOD measurements. However, as shown by the results, the best tracking performance is achieved, when both of the measurements are considered. In this case, a very high, sub-meter, tracking accuracy can be achieved for most (>75%) of the tracking time, thus achieving the positioning accuracy requirements envisioned for the 5G NR. The pursued high-accuracy and high-availability positioning technology is considered to be in a key role in several envisioned HST use cases, such as mission-critical autonomous train systems.
RMDL: Random Multimodel Deep Learning for Classification
Venue: arXiv cs.LG 2018
Year: 2018

The continually increasing number of complex datasets each year necessitates ever improving machine learning methods for robust and accurate categorization of these data. This paper introduces Random Multimodel Deep Learning (RMDL): a new ensemble, deep learning approach for classification. Deep learning models have achieved state-of-the-art results across many domains. RMDL solves the problem of finding the best deep learning structure and architecture while simultaneously improving robustness and accuracy through ensembles of deep learning architectures. RDML can accept as input a variety data to include text, video, images, and symbolic. This paper describes RMDL and shows test results for image and text data including MNIST, CIFAR-10, WOS, Reuters, IMDB, and 20newsgroup. These test results show that RDML produces consistently better performance than standard methods over a broad range of data types and classification problems.
Reliability Map Estimation For CNN-Based Camera Model Attribution
Venue: arXiv cs.CV 2018
Year: 2018

Among the image forensic issues investigated in the last few years, great attention has been devoted to blind camera model attribution. This refers to the problem of detecting which camera model has been used to acquire an image by only exploiting pixel information. Solving this problem has great impact on image integrity assessment as well as on authenticity verification. Recent advancements that use convolutional neural networks (CNNs) in the media forensic field have enabled camera model attribution methods to work well even on small image patches. These improvements are also important for determining forgery localization. Some patches of an image may not contain enough information related to the camera model (e.g., saturated patches). In this paper, we propose a CNN-based solution to estimate the camera model attribution reliability of a given image patch. We show that we can estimate a reliability-map indicating which portions of the image contain reliable camera traces. Testing using a well known dataset confirms that by using this information, it is possible to increase small patch camera model attribution accuracy by more than 8% on a single patch.
Lossy Transmission of Correlated Sources over Two-Way Channels
Venue: arXiv cs.IT 2018
Year: 2018

Achievability and converse results for the lossy transmission of correlated sources over Shannon's two-way channels (TWCs) are presented. A joint source-channel coding theorem for independent sources and TWCs for which adaptation cannot enlarge the capacity region is also established. We further investigate the optimality of scalar coding for TWCs with discrete modulo additive noise as well as additive white Gaussian noise. Comparing the distortion of scalar coding with the derived bounds, we observe that scalar coding achieves the minimum distortion over both families of TWCs for independent and uniformly distributed sources and independent Gaussian sources.
Energy-delay-cost Tradeoff for Task Offloading in Imbalanced Edge Cloud Based Computing
Venue: arXiv cs.NI 2018
Year: 2018

In this paper, the imbalance edge cloud based computing offloading for multiple mobile users (MUs) with multiple tasks per MU is studied. In which, several edge cloud servers (ECSs) are shared and accessed by multiple wireless access points (APs) with the backhaul links, and each MU has multiple computing intensive and latency critical tasks should be offloaded to execute, which involves both the AP association and ECS selection, with the objective of minimizing the offloading cost. Distinguished with existing research, besides the transmission delay and energy consumption from MU to AP, the ECS access-cost which characterizes the ECS access delay and (or) resource using cost is introduced, thus finally formulates the delay-energy-cost tradeoff based offloading cost criteria, for the MUs' offloading decision and resource allocation problems. In specific, in our system, the ECS access-cost depends on both the AP and ECS, which reflects the effects of different backhaul techniques used by these APs, and their negotiated payments of ECS resource using under different service level agreements. Both problems of minimizing the sum offloading costs for all MUs (efficiency-based) and minimizing the maximal offloading cost per MU (fairness-based) are discussed. Since these problems are all NP-hard. Therefore, some centralized and distributed heuristic algorithms are proposed to find the suboptimal solutions. Further analysis and numerical results are presented at last to demonstrate the performance of these algorithms from several aspects, such as efficiency, complexity and fairness.