OpenRaiser/Intern-Atlas huggingface dataset:
Understanding the Related-Key Security of Feistel Ciphers from a Provable Perspective
Venue: arXiv cs.CR 2019
Year: 2019

We initiate the provable related-key security treatment for models of practical Feistel ciphers. In detail, we consider Feistel networks with four whitening keys $w_i(k)$ ($i=0,1,2,3$) and round-functions of the form $f(γ_i(k)\oplus X)$, where $k$ is the main-key, $w_i$ and $γ_i$ are efficient transformations, and $f$ is a public ideal function or permutation that the adversary is allowed to query. We investigate conditions on the key-schedules that are sufficient for security against XOR-induced related-key attacks up to $2^{n/2}$ adversarial queries. When the key-schedules are non-linear, we prove security for 4 rounds. When only affine key-schedules are used, we prove security for 6 rounds. These also imply secure tweakable Feistel ciphers in the Random Oracle model. By shuffling the key-schedules, our model unifies both the DES-like structure (known as Feistel-2 scheme in the cryptanalytic community, a.k.a. key-alternating Feistel due to Lampe and Seurin, FSE 2014) and the Lucifer-like model (previously analyzed by Guo and Lin, TCC 2015). This allows us to derive concrete implications on these two (more common) models, and helps understanding their differences---and further understanding the related-key security of Feistel ciphers.
Does the public discuss other topics on climate change than researchers? A comparison of explorative networks based on author keywords and hashtags
Venue: arXiv cs.DL 2019
Year: 2019

Twitter accounts have already been used in many scientometric studies, but the meaningfulness of the data for societal impact measurements in research evaluation has been questioned. Earlier research focused on social media counts and neglected the interactive nature of the data. We explore a new network approach based on Twitter data in which we compare author keywords to hashtags as indicators of topics. We analyze the topics of tweeted publications and compare them with the topics of all publications (tweeted and not tweeted). Our exploratory study is based on a comprehensive publication set of climate change research. We are interested in whether Twitter data are able to reveal topics of public discussions which can be separated from research-focused topics. We find that the most tweeted topics regarding climate change research focus on the consequences of climate change for humans. Twitter users are interested in climate change publications which forecast effects of a changing climate on the environment and to adaptation, mitigation and management issues rather than in the methodology of climate-change research and causes of climate change. Our results indicate that publications using scientific jargon are less likely to be tweeted than publications using more general keywords. Twitter networks seem to be able to visualize public discussions about specific topics.
Implementation and Analysis of QUIC for MQTT
Venue: arXiv cs.NI 2019
Year: 2019

Transport and security protocols are essential to ensure reliable and secure communication between two parties. For IoT applications, these protocols must be lightweight, since IoT devices are usually resource constrained. Unfortunately, the existing transport and security protocols -- namely TCP/TLS and UDP/DTLS -- fall short in terms of connection overhead, latency, and connection migration when used in IoT applications. In this paper, after studying the root causes of these shortcomings, we show how utilizing QUIC in IoT scenarios results in a higher performance. Based on these observations, and given the popularity of MQTT as an IoT application layer protocol, we integrate MQTT with QUIC. By presenting the main APIs and functions developed, we explain how connection establishment and message exchange functionalities work. We evaluate the performance of MQTTw/QUIC versus MQTTw/TCP using wired, wireless, and long-distance testbeds. Our results show that MQTTw/QUIC reduces connection overhead in terms of the number of packets exchanged with the broker by up to 56%. In addition, by eliminating half-open connections, MQTTw/QUIC reduces processor and memory usage by up to 83% and 50%, respectively. Furthermore, by removing the head-of-line blocking problem, delivery latency is reduced by up to 55%. We also show that the throughput drops experienced by MQTTw/QUIC when a connection migration happens is considerably lower than that of MQTTw/TCP.
Multi-Agent Fully Decentralized Value Function Learning with Linear Convergence Rates
Venue: arXiv cs.LG 2019
Year: 2019

This work develops a fully decentralized multi-agent algorithm for policy evaluation. The proposed scheme can be applied to two distinct scenarios. In the first scenario, a collection of agents have distinct datasets gathered following different behavior policies (none of which is required to explore the full state space) in different instances of the same environment and they all collaborate to evaluate a common target policy. The network approach allows for efficient exploration of the state space and allows all agents to converge to the optimal solution even in situations where neither agent can converge on its own without cooperation. The second scenario is that of multi-agent games, in which the state is global and rewards are local. In this scenario, agents collaborate to estimate the value function of a target team policy. The proposed algorithm combines off-policy learning, eligibility traces and linear function approximation. The proposed algorithm is of the variance-reduced kind and achieves linear convergence with $O(1)$ memory requirements. The linear convergence of the algorithm is established analytically, and simulations are used to illustrate the effectiveness of the method.
The Effects of Using Taxi-Hailing Application on Driving Performance
Venue: arXiv cs.HC 2019
Year: 2019

Driver distraction has become a major threat to the road safety, and the globally booming taxi-hailing application introduces new source of distraction to drivers. Although various in-vehicle information systems (IVIS) have been studied extensively, no documentation exists objectively measuring the extent to which interacting with taxi-hailing application during driving impacts drivers' behavior. To fill this gap, a simulator-based study was conducted to synthetically compare the effects that different output modalities (visual, audio, combined visual-audio) and input modalities (baseline, manual, speech) imposed on the driving performance. The results show that the visual output introduced more negative effects on driving performance compared to audio output. In the combined output, visual component dominated the effects imposed on the longitudinal control and hazard detection; audio component only exacerbated the negative effects of visual component on the lateral control. Speech input modality was overall less detrimental to driving performance than manual input modality, especially reflected in the drivers' quicker reaction to hazard events. The visual-manual interaction modality most severely impaired the hazard detecting ability, while also led to strong compensative behaviors. The audio-speech and visual-speech modality associated with more smooth lateral control and faster response to hazard events respectively compared to other modality. These results could be applied to improve the design of not only the taxi-hailing application, but also other input-output balanced IVIS.
Deep Learning for Encrypted Traffic Classification: An Overview
Venue: arXiv cs.NI 2019
Year: 2019

Traffic classification has been studied for two decades and applied to a wide range of applications from QoS provisioning and billing in ISPs to security-related applications in firewalls and intrusion detection systems. Port-based, data packet inspection, and classical machine learning methods have been used extensively in the past, but their accuracy have been declined due to the dramatic changes in the Internet traffic, particularly the increase in encrypted traffic. With the proliferation of deep learning methods, researchers have recently investigated these methods for traffic classification task and reported high accuracy. In this article, we introduce a general framework for deep-learning-based traffic classification. We present commonly used deep learning methods and their application in traffic classification tasks. Then, we discuss open problems and their challenges, as well as opportunities for traffic classification.
Don't Unroll Adjoint: Differentiating SSA-Form Programs
Venue: arXiv cs.PL 2019
Year: 2019

This paper presents reverse-mode algorithmic differentiation (AD) based on source code transformation, in particular of the Static Single Assignment (SSA) form used by modern compilers. The approach can support control flow, nesting, mutation, recursion, data structures, higher-order functions, and other language constructs, and the output is given to an existing compiler to produce highly efficient differentiated code. Our implementation is a new AD tool for the Julia language, called Zygote, which presents high-level dynamic semantics while transparently compiling adjoint code under the hood. We discuss the benefits of this approach to both the usability and performance of AD tools.
Aeronautical Data Aggregation and Field Estimation in IoT Networks: Hovering & Traveling Time Dilemma of UAVs
Venue: arXiv cs.NI 2019
Year: 2019

The next era of information revolution will rely on aggregating big data from massive numbers of devices that are widely scattered in our environment. Most of these devices are expected to be of low-complexity, low-cost, and limited power supply, which impose stringent constraints on the network operation. In this regard, this paper investigates aerial data aggregation and field estimation from a finite spatial field via an unmanned aerial vehicle (UAV). Instead of fusing, relaying, and routing the data across the wireless nodes to fixed locations access points, a UAV flies over the field and collects the required data for two prominent missions; data aggregation and field estimation. To accomplish these tasks, the field of interest is divided into several subregions over which the UAV hovers to collect samples from the underlying nodes. To this end, we formulate and solve an optimization problem to minimize total hovering and traveling time of each mission. While the former requires the collection of a prescribed average number of samples from the field, the latter ensures for a given field spatial correlation model that the average mean-squared estimation error of the field value is no more than a predetermined threshold at any point. These goals are fulfilled by optimizing the number of subregions, the area of each subregion, the hovering locations, the hovering time at each location, and the trajectory traversed between hovering locations. The proposed formulation is shown to be np-hard mixed integer problem, and hence, a decoupled heuristic solution is proposed. The results show that there exists an optimal number of subregions that balance the tradeoff between hovering and traveling times such that the total time for collecting the required samples is minimized.
Implicit Dual-domain Convolutional Network for Robust Color Image Compression Artifact Reduction
Venue: arXiv cs.CV 2019
Year: 2019

Several dual-domain convolutional neural network-based methods show outstanding performance in reducing image compression artifacts. However, they suffer from handling color images because the compression processes for gray-scale and color images are completely different. Moreover, these methods train a specific model for each compression quality and require multiple models to achieve different compression qualities. To address these problems, we proposed an implicit dual-domain convolutional network (IDCN) with the pixel position labeling map and the quantization tables as inputs. Specifically, we proposed an extractor-corrector framework-based dual-domain correction unit (DCU) as the basic component to formulate the IDCN. A dense block was introduced to improve the performance of extractor in DRU. The implicit dual-domain translation allows the IDCN to handle color images with the discrete cosine transform (DCT)-domain priors. A flexible version of IDCN (IDCN-f) was developed to handle a wide range of compression qualities. Experiments for both objective and subjective evaluations on benchmark datasets show that IDCN is superior to the state-of-the-art methods and IDCN-f exhibits excellent abilities to handle a wide range of compression qualities with little performance sacrifice and demonstrates great potential for practical applications.
Adaptive Communication Strategies to Achieve the Best Error-Runtime Trade-off in Local-Update SGD
Venue: arXiv cs.LG 2019
Year: 2019

Large-scale machine learning training, in particular distributed stochastic gradient descent, needs to be robust to inherent system variability such as node straggling and random communication delays. This work considers a distributed training framework where each worker node is allowed to perform local model updates and the resulting models are averaged periodically. We analyze the true speed of error convergence with respect to wall-clock time (instead of the number of iterations), and analyze how it is affected by the frequency of averaging. The main contribution is the design of AdaComm, an adaptive communication strategy that starts with infrequent averaging to save communication delay and improve convergence speed, and then increases the communication frequency in order to achieve a low error floor. Rigorous experiments on training deep neural networks show that AdaComm can take $3 \times$ less time than fully synchronous SGD, and still reach the same final training loss.
Multitask Learning for Large-scale Semantic Change Detection
Venue: arXiv cs.CV 2019
Year: 2019

Change detection is one of the main problems in remote sensing, and is essential to the accurate processing and understanding of the large scale Earth observation data available through programs such as Sentinel and Landsat. Most of the recently proposed change detection methods bring deep learning to this context, but openly available change detection datasets are still very scarce, which limits the methods that can be proposed and tested. In this paper we present the first large scale high resolution semantic change detection (HRSCD) dataset, which enables the usage of deep learning methods for semantic change detection. The dataset contains coregistered RGB image pairs, pixel-wise change information and land cover information. We then propose several methods using fully convolutional neural networks to perform semantic change detection. Most notably, we present a network architecture that performs change detection and land cover mapping simultaneously, while using the predicted land cover information to help to predict changes. We also describe a sequential training scheme that allows this network to be trained without setting a hyperparameter that balances different loss functions and achieves the best overall results.
A Modern Take on the Bias-Variance Tradeoff in Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

The bias-variance tradeoff tells us that as model complexity increases, bias falls and variances increases, leading to a U-shaped test error curve. However, recent empirical results with over-parameterized neural networks are marked by a striking absence of the classic U-shaped test error curve: test error keeps decreasing in wider networks. This suggests that there might not be a bias-variance tradeoff in neural networks with respect to network width, unlike was originally claimed by, e.g., Geman et al. (1992). Motivated by the shaky evidence used to support this claim in neural networks, we measure bias and variance in the modern setting. We find that both bias and variance can decrease as the number of parameters grows. To better understand this, we introduce a new decomposition of the variance to disentangle the effects of optimization and data sampling. We also provide theoretical analysis in a simplified setting that is consistent with our empirical findings.
Optimization of Molecules via Deep Reinforcement Learning
Venue: arXiv cs.LG 2019
Year: 2019

We present a framework, which we call Molecule Deep $Q$-Networks (MolDQN), for molecule optimization by combining domain knowledge of chemistry and state-of-the-art reinforcement learning techniques (double $Q$-learning and randomized value functions). We directly define modifications on molecules, thereby ensuring 100\% chemical validity. Further, we operate without pre-training on any dataset to avoid possible bias from the choice of that set. Inspired by problems faced during medicinal chemistry lead optimization, we extend our model with multi-objective reinforcement learning, which maximizes drug-likeness while maintaining similarity to the original molecule. We further show the path through chemical space to achieve optimization for a molecule to understand how the model works.
A Regressive Convolution Neural network and Support Vector Regression Model for Electricity Consumption Forecasting
Venue: arXiv cs.CV 2019
Year: 2019

Electricity consumption forecasting has important implications for the mineral companies on guiding quarterly work, normal power system operation, and the management. However, electricity consumption prediction for the mineral company is different from traditional electricity load prediction since mineral company electricity consumption can be affected by various factors (e.g., ore grade, processing quantity of the crude ore, ball milling fill rate). The problem is non-trivial due to three major challenges for traditional methods: insufficient training data, high computational cost and low prediction accu-racy. To tackle these challenges, we firstly propose a Regressive Convolution Neural Network (RCNN) to predict the electricity consumption. While RCNN still suffers from high computation overhead, we utilize RCNN to extract features from the history data and Regressive Support Vector Machine (SVR) trained with the features to predict the electricity consumption. The experimental results show that the proposed RCNN-SVR model achieves higher accuracy than using the traditional RNN or SVM alone. The MSE, MAPE, and CV-RMSE of RCNN-SVR model are 0.8564, 1.975%, and 0.0687% respectively, which illustrates the low predicting error rate of the proposed model.
Decoding of Non-Binary LDPC Codes Using the Information Bottleneck Method
Venue: arXiv cs.IT 2019
Year: 2019

Recently, a novel lookup table based decoding method for binary low-density parity-check codes has attracted considerable attention. In this approach, mutual-information maximizing lookup tables replace the conventional operations of the variable nodes and the check nodes in message passing decoding. Moreover, the exchanged messages are represented by integers with very small bit width. A machine learning framework termed the information bottleneck method is used to design the corresponding lookup tables. In this paper, we extend this decoding principle from binary to non-binary codes. This is not a straightforward extension, but requires a more sophisticated lookup table design to cope with the arithmetic in higher order Galois fields. Provided bit error rate simulations show that our proposed scheme outperforms the log-max decoding algorithm and operates close to sum-product decoding.
Teaching Inverse Reinforcement Learners via Features and Demonstrations
Venue: arXiv cs.LG 2019
Year: 2019

Learning near-optimal behaviour from an expert's demonstrations typically relies on the assumption that the learner knows the features that the true reward function depends on. In this paper, we study the problem of learning from demonstrations in the setting where this is not the case, i.e., where there is a mismatch between the worldviews of the learner and the expert. We introduce a natural quantity, the teaching risk, which measures the potential suboptimality of policies that look optimal to the learner in this setting. We show that bounds on the teaching risk guarantee that the learner is able to find a near-optimal policy using standard algorithms based on inverse reinforcement learning. Based on these findings, we suggest a teaching scheme in which the expert can decrease the teaching risk by updating the learner's worldview, and thus ultimately enable her to find a near-optimal policy.
Runtime Concurrency Control and Operation Scheduling for High Performance Neural Network Training
Venue: arXiv cs.DC 2019
Year: 2019

Training neural network often uses a machine learning framework such as TensorFlow and Caffe2. These frameworks employ a dataflow model where the NN training is modeled as a directed graph composed of a set of nodes. Operations in neural network training are typically implemented by the frameworks as primitives and represented as nodes in the dataflow graph. Training NN models in a dataflow-based machine learning framework involves a large number of fine-grained operations. Those operations have diverse memory access patterns and computation intensity. How to manage and schedule those operations is challenging, because we have to decide the number of threads to run each operation (concurrency control) and schedule those operations for good hardware utilization and system throughput. In this paper, we extend an existing runtime system (the TensorFlow runtime) to enable automatic concurrency control and scheduling of operations. We explore performance modeling to predict the performance of operations with various thread-level parallelism. Our performance model is highly accurate and lightweight. Leveraging the performance model, our runtime system employs a set of scheduling strategies that co-run operations to improve hardware utilization and system throughput. Our runtime system demonstrates a big performance benefit. Comparing with using the recommended configurations for concurrency control and operation scheduling in TensorFlow, our approach achieves 33% performance (execution time) improvement on average (up to 49%) for three neural network models, and achieves high performance closing to the optimal one manually obtained by the user.
On the Conditional Smooth Renyi Entropy and its Applications in Guessing and Source Coding
Venue: arXiv cs.IT 2019
Year: 2019

A novel definition of the conditional smooth Renyi entropy, which is different from that of Renner and Wolf, is introduced. It is shown that our definition of the conditional smooth Renyi entropy is appropriate to give lower and upper bounds on the optimal guessing moment in a guessing problem where the guesser is allowed to stop guessing and declare an error. Further a general formula for the optimal guessing exponent is given. In particular, a single-letterized formula for mixture of i.i.d. sources is obtained. Another application in the problem of source coding with the common side-information available at the encoder and decoder is also demonstrated.