OpenRaiser/Intern-Atlas huggingface dataset:
Generalized Batch Normalization: Towards Accelerating Deep Neural Networks
Venue: arXiv cs.LG 2018
Year: 2018

Utilizing recently introduced concepts from statistics and quantitative risk management, we present a general variant of Batch Normalization (BN) that offers accelerated convergence of Neural Network training compared to conventional BN. In general, we show that mean and standard deviation are not always the most appropriate choice for the centering and scaling procedure within the BN transformation, particularly if ReLU follows the normalization step. We present a Generalized Batch Normalization (GBN) transformation, which can utilize a variety of alternative deviation measures for scaling and statistics for centering, choices which naturally arise from the theory of generalized deviation measures and risk theory in general. When used in conjunction with the ReLU non-linearity, the underlying risk theory suggests natural, arguably optimal choices for the deviation measure and statistic. Utilizing the suggested deviation measure and statistic, we show experimentally that training is accelerated more so than with conventional BN, often with improved error rate as well. Overall, we propose a more flexible BN transformation supported by a complimentary theoretical framework that can potentially guide design choices.
Achieving Data Truthfulness and Privacy Preservation in Data Markets
Venue: arXiv cs.DB 2018
Year: 2018

As a significant business paradigm, many online information platforms have emerged to satisfy society's needs for person-specific data, where a service provider collects raw data from data contributors, and then offers value-added data services to data consumers. However, in the data trading layer, the data consumers face a pressing problem, i.e., how to verify whether the service provider has truthfully collected and processed data? Furthermore, the data contributors are usually unwilling to reveal their sensitive personal data and real identities to the data consumers. In this paper, we propose TPDM, which efficiently integrates data Truthfulness and Privacy preservation in Data Markets. TPDM is structured internally in an Encrypt-then-Sign fashion, using partially homomorphic encryption and identity-based signature. It simultaneously facilitates batch verification, data processing, and outcome verification, while maintaining identity preservation and data confidentiality. We also instantiate TPDM with a profile matching service and a distribution fitting service, and extensively evaluate their performances on Yahoo! Music ratings dataset and 2009 RECS dataset, respectively. Our analysis and evaluation results reveal that TPDM achieves several desirable properties, while incurring low computation and communication overheads when supporting large-scale data markets.
Impact of MQTT Based Sensor Network Architecture on Delivery Delay Time
Venue: arXiv cs.NI 2018
Year: 2018

The purpose of this study is to present two new architectures of the complexes of embedded systems in the context of the Internet of Things. The proposed new schemes use the MQTT protocol, but provide both an autonomous operation and an online mode. This is achieved by adding one additional layer, which consists of its own MQTT-broker and critical messages handler, so that in case of the Internet access disappearance, the system could continue to respond to changes. The research of message delivery delays has been carried out for the proposed models. The comparative analysis was conducted to identify the advantages and disadvantages of both solutions.
A Verified Timsort C Implementation in Isabelle/HOL
Venue: arXiv cs.SE 2018
Year: 2018

Formal verification of traditional algorithms are of great significance due to their wide application in state-of-the-art software. Timsort is a complicated and hybrid stable sorting algorithm, derived from merge sort and insertion sort. Although Timsort implementation in OpenJDK has been formally verified, there is still not a standard and formally verified Timsort implementation in C programming language. This paper studies Timsort implementation and its formal verification using a generic imperative language - Simpl in Isabelle/HOL. Then, we manually generate an C implementation of Timsort from the verified Simpl specification. Due to the C-like concrete syntax of Simpl, the code generation is straightforward. The C implementation has also been tested by a set of random test cases.
Memristors for the Curious Outsiders
Venue: arXiv cs.ET 2018
Year: 2018

We present both an overview and a perspective of recent experimental advances and proposed new approaches to performing computation using memristors. A memristor is a 2-terminal passive component with a dynamic resistance depending on an internal parameter. We provide an brief historical introduction, as well as an overview over the physical mechanism that lead to memristive behavior. This review is meant to guide nonpractitioners in the field of memristive circuits and their connection to machine learning and neural computation.
Learning Graph Representation via Formal Concept Analysis
Venue: arXiv cs.LG 2018
Year: 2018

We present a novel method that can learn a graph representation from multivariate data. In our representation, each node represents a cluster of data points and each edge represents the subset-superset relationship between clusters, which can be mutually overlapped. The key to our method is to use formal concept analysis (FCA), which can extract hierarchical relationships between clusters based on the algebraic closedness property. We empirically show that our method can effectively extract hierarchical structures of clusters compared to the baseline method.
A global sensitivity analysis and reduced order models for hydraulically-fractured horizontal wells
Venue: arXiv cs.NA 2018
Year: 2018

We present a systematic global sensitivity analysis using the Sobol method which can be utilized to rank the variables that affect two quantity of interests -- pore pressure depletion and stress change -- around a hydraulically-fractured horizontal well based on their degree of importance. These variables include rock properties and stimulation design variables. A fully-coupled poroelastic hydraulic fracture model is used to account for pore pressure and stress changes due to production. To ease the computational cost of a simulator, we also provide reduced order models (ROMs), which can be used to replace the complex numerical model with a rather simple analytical model, for calculating the pore pressure and stresses at different locations around hydraulic fractures. The main findings of this research are: (i) mobility, production pressure, and fracture half-length are the main contributors to the changes in the quantities of interest. The percentage of the contribution of each parameter depends on the location with respect to pre-existing hydraulic fractures and the quantity of interest. (ii) As the time progresses, the effect of mobility decreases and the effect of production pressure increases. (iii) These two variables are also dominant for horizontal stresses at large distances from hydraulic fractures. (iv) At zones close to hydraulic fracture tips or inside the spacing area, other parameters such as fracture spacing and half-length are the dominant factors that affect the minimum horizontal stress. The results of this study will provide useful guidelines for the stimulation design of legacy wells and secondary operations such as refracturing and infill drilling.
Real-Time Referring Expression Comprehension by Single-Stage Grounding Network
Venue: arXiv cs.CV 2018
Year: 2018

In this paper, we propose a novel end-to-end model, namely Single-Stage Grounding network (SSG), to localize the referent given a referring expression within an image. Different from previous multi-stage models which rely on object proposals or detected regions, our proposed model aims to comprehend a referring expression through one single stage without resorting to region proposals as well as the subsequent region-wise feature extraction. Specifically, a multimodal interactor is proposed to summarize the local region features regarding the referring expression attentively. Subsequently, a grounder is proposed to localize the referring expression within the given image directly. For further improving the localization accuracy, a guided attention mechanism is proposed to enforce the grounder to focus on the central region of the referent. Moreover, by exploiting and predicting visual attribute information, the grounder can further distinguish the referent objects within an image and thereby improve the model performance. Experiments on RefCOCO, RefCOCO+, and RefCOCOg datasets demonstrate that our proposed SSG without relying on any region proposals can achieve comparable performance with other advanced models. Furthermore, our SSG outperforms the previous models and achieves the state-of-art performance on the ReferItGame dataset. More importantly, our SSG is time efficient and can ground a referring expression in a 416*416 image from the RefCOCO dataset in 25ms (40 referents per second) on average with a Nvidia Tesla P40, accomplishing more than 9* speedups over the existing multi-stage models.
Dialogue Generation: From Imitation Learning to Inverse Reinforcement Learning
Venue: arXiv cs.CL 2018
Year: 2018

The performance of adversarial dialogue generation models relies on the quality of the reward signal produced by the discriminator. The reward signal from a poor discriminator can be very sparse and unstable, which may lead the generator to fall into a local optimum or to produce nonsense replies. To alleviate the first problem, we first extend a recently proposed adversarial dialogue generation method to an adversarial imitation learning solution. Then, in the framework of adversarial inverse reinforcement learning, we propose a new reward model for dialogue generation that can provide a more accurate and precise reward signal for generator training. We evaluate the performance of the resulting model with automatic metrics and human evaluations in two annotation settings. Our experimental results demonstrate that our model can generate more high-quality responses and achieve higher overall performance than the state-of-the-art.
How to Increase the Achievable Information Rate by Per-Channel Dispersion Compensation
Venue: arXiv cs.IT 2018
Year: 2018

Deploying periodic inline chromatic dispersion compensation enables reducing the complexity of the digital back propagation (DBP) algorithm. However, compared with nondispersion-managed (NDM) links, dispersion-managed (DM) ones suffer a stronger cross-phase modulation (XPM). Utilizing per-channel dispersion-managed (CDM) links (e.g., using fiber Bragg grating) allows for a complexity reduction of DBP, while abating XPM compared to DM links. In this paper, we show for the first time that CDM links enable also a more effective XPM compensation compared to NDM ones, allowing a higher achievable information rate (AIR). This is explained by resorting to the frequency-resolved logarithmic perturbation model and showing that per-channel dispersion compensation increases the frequency correlation of the distortions induced by XPM over the channel bandwidth, making them more similar to a conventional phase noise. We compare the performance (in terms of the AIR) of a DM, an NDM, and a CDM link, considering two types of mismatched receivers: one neglects the XPM phase distortion and the other compensates for it. With the former, the CDM link is inferior to the NDM one due to an increased in-band signal--noise interaction. However, with the latter, a higher AIR is obtained with the CDM link than with the NDM one owing to a higher XPM frequency correlation. The DM link has the lowest AIR for both receivers because of a stronger XPM.
Distributed Task Management in Cyber-Physical Systems: How to Cooperate under Uncertainty?
Venue: arXiv cs.MA 2018
Year: 2018

We consider the problem of task allocation in a network of cyber-physical systems (CPSs). The network can have different states, and the tasks are of different types. The task arrival is stochastic and state-dependent. Every CPS is capable of performing each type of task with some specific state-dependent efficiency. The CPSs have to agree on task allocation prior to knowing about the realized network's state and/or the arrived tasks. We model the problem as a multi-state stochastic cooperative game with state uncertainty. We then use the concept of deterministic equivalence and sequential core to solve the problem. We establish the non-emptiness of the strong sequential core in our designed task allocation game and investigate its characteristics including uniqueness and optimality. Moreover, we prove that in the task allocation game, the strong sequential core is equivalent to Walrasian equilibrium under state uncertainty; consequently, it can be implemented by using the Walras' tatonnement process.
Learning Style Compatibility for Furniture
Venue: arXiv cs.CV 2018
Year: 2018

When judging style, a key question that often arises is whether or not a pair of objects are compatible with each other. In this paper we investigate how Siamese networks can be used efficiently for assessing the style compatibility between images of furniture items. We show that the middle layers of pretrained CNNs can capture essential information about furniture style, which allows for efficient applications of such networks for this task. We also use a joint image-text embedding method that allows for the querying of stylistically compatible furniture items, along with additional attribute constraints based on text. To evaluate our methods, we collect and present a large scale dataset of images of furniture of different style categories accompanied by text attributes.
Practical Optional Types for Clojure
Venue: arXiv cs.PL 2018
Year: 2018

Typed Clojure is an optional type system for Clojure, a dynamic language in the Lisp family that targets the JVM. Typed Clojure enables Clojure programmers to gain greater confidence in the correctness of their code via static type checking while remaining in the Clojure world, and has acquired significant adoption in the Clojure community. Typed Clojure repurposes Typed Racket's occurrence typing, an approach to statically reasoning about predicate tests, and also includes several new type system features to handle existing Clojure idioms. In this paper, we describe Typed Clojure and present these type system extensions, focusing on three features widely used in Clojure. First, multimethods provide extensible operations, and their Clojure semantics turns out to have a surprising synergy with the underlying occurrence typing framework. Second, Java interoperability is central to Clojure's mission but introduces challenges such as ubiquitous null; Typed Clojure handles Java interoperability while ensuring the absence of null-pointer exceptions in typed programs. Third, Clojure programmers idiomatically use immutable dictionaries for data structures; Typed Clojure handles this with multiple forms of heterogeneous dictionary types. We provide a formal model of the Typed Clojure type system incorporating these and other features, with a proof of soundness. Additionally, Typed Clojure is now in use by numerous corporations and developers working with Clojure, and we present a quantitative analysis on the use of type system features in two substantial code bases.
Study of Puncturing Techniques for Polar Codes in 5G Cellular and IoT Networks
Venue: arXiv cs.IT 2018
Year: 2018

This paper presents a puncturing technique based on the channel polarization index for the design of rate-compatible polar codes in the fifth generation (5G) of wireless systems. The proposed strategy consists of two steps: we first generate the codeword message; and then we reduce the length of the codeword based on the channel polarization index where channels with the smallest channel polarization indices are punctured. We consider the proposed punctured polar codes with the successive cancellation (SC) decoder and the Cyclic Redundancy Check (CRC) aided SC list (CA-SCL) decoder and punctured bits known to both the encoder and the decoder. The Polar Spectra (PS) are then used to performance analysis the puncturing technique. Simulations for 5G scenarios show that the proposed polar codes perform comparable to Low-Density Parity-Check (LDPC) codes.
Near-optimal Smooth Path Planning for Multisection Continuum Arms
Venue: arXiv cs.RO 2018
Year: 2018

We study the path planning problem for continuum-arm robots, in which we are given a starting and an end point, and we need to compute a path for the tip of the continuum arm between the two points. We consider both cases where obstacles are present and where they are not. We demonstrate how to leverage the continuum arm features to introduce a new model that enables a path planning approach based on the configurations graph, for a continuum arm consisting of three sections, each consisting of three muscle actuators. The algorithm we apply to the configurations graph allows us to exploit parallelism in the computation to obtain efficient implementation. We conducted extensive tests, and the obtained results show the completeness of the proposed algorithm under the considered discretizations, in both cases where obstacles are present and where they are not. We compared our approach to the standard inverse kinematics approach. While the inverse kinematics approach is much faster when successful, our algorithm always succeeds in finding a path or reporting that no path exists, compared to a roughly 70% success rate of the inverse kinematics approach (when a path exists).
Efficient Training Management for Mobile Crowd-Machine Learning: A Deep Reinforcement Learning Approach
Venue: arXiv cs.NI 2018
Year: 2018

In this letter, we consider the concept of Mobile Crowd-Machine Learning (MCML) for a federated learning model. The MCML enables mobile devices in a mobile network to collaboratively train neural network models required by a server while keeping data on the mobile devices. The MCML thus addresses data privacy issues of traditional machine learning. However, the mobile devices are constrained by energy, CPU, and wireless bandwidth. Thus, to minimize the energy consumption, training time and communication cost, the server needs to determine proper amounts of data and energy that the mobile devices use for training. However, under the dynamics and uncertainty of the mobile environment, it is challenging for the server to determine the optimal decisions on mobile device resource management. In this letter, we propose to adopt a deep- Q learning algorithm that allows the server to learn and find optimal decisions without any a priori knowledge of network dynamics. Simulation results show that the proposed algorithm outperforms the static algorithms in terms of energy consumption and training latency.
Taxi Demand-Supply Forecasting: Impact of Spatial Partitioning on the Performance of Neural Networks
Venue: arXiv cs.LG 2018
Year: 2018

In this paper, we investigate the significance of choosing an appropriate tessellation strategy for a spatio-temporal taxi demand-supply modeling framework. Our study compares (i) the variable-sized polygon based Voronoi tessellation, and (ii) the fixed-sized grid based Geohash tessellation, using taxi demand-supply GPS data for the cities of Bengaluru, India and New York, USA. Long Short-Term Memory (LSTM) networks are used for modeling and incorporating information from spatial neighbors into the model. We find that the LSTM model based on input features extracted from a variable-sized polygon tessellation yields superior performance over the LSTM model based on fixed-sized grid tessellation. Our study highlights the need to explore multiple spatial partitioning techniques for improving the prediction performance in neural network models.
Ramp-based Twin Support Vector Clustering
Venue: arXiv cs.LG 2018
Year: 2018

Traditional plane-based clustering methods measure the cost of within-cluster and between-cluster by quadratic, linear or some other unbounded functions, which may amplify the impact of cost. This letter introduces a ramp cost function into the plane-based clustering to propose a new clustering method, called ramp-based twin support vector clustering (RampTWSVC). RampTWSVC is more robust because of its boundness, and thus it is more easier to find the intrinsic clusters than other plane-based clustering methods. The non-convex programming problem in RampTWSVC is solved efficiently through an alternating iteration algorithm, and its local solution can be obtained in a finite number of iterations theoretically. In addition, the nonlinear manifold-based formation of RampTWSVC is also proposed by kernel trick. Experimental results on several benchmark datasets show the better performance of our RampTWSVC compared with other plane-based clustering methods.
Functional Design of Computation Graph
Venue: arXiv cs.MS 2018
Year: 2018

Representing the control flow of a computer program as a computation graph can bring many benefits in a broad variety of domains where performance is critical. This technique is a core component of most major numerical libraries (TensorFlow, PyTorch, Theano, MXNet,...) and is successfully used to speed up and optimise many computationally-intensive tasks. However, different design choices in each of these libraries lead to noticeable differences in efficiency and in the way an end user writes efficient code. In this report, we detail the implementation and features of the computation graph support in OCaml's numerical library Owl, a recent entry in the world of scientific computing.
Learning to Drive from Simulation without Real World Labels
Venue: arXiv cs.CV 2018
Year: 2018

Simulation can be a powerful tool for understanding machine learning systems and designing methods to solve real-world problems. Training and evaluating methods purely in simulation is often "doomed to succeed" at the desired task in a simulated environment, but the resulting models are incapable of operation in the real world. Here we present and evaluate a method for transferring a vision-based lane following driving policy from simulation to operation on a rural road without any real-world labels. Our approach leverages recent advances in image-to-image translation to achieve domain transfer while jointly learning a single-camera control policy from simulation control labels. We assess the driving performance of this method using both open-loop regression metrics, and closed-loop performance operating an autonomous vehicle on rural and urban roads.
The particle track reconstruction based on deep learning neural networks
Venue: arXiv cs.LG 2018
Year: 2018

One of the most important problems of data processing in high energy and nuclear physics is the event reconstruction. Its main part is the track reconstruction procedure which consists in looking for all tracks that elementary particles leave when they pass through a detector among a huge number of points, so-called hits, produced when flying particles fire detector coordinate planes. Unfortunately, the tracking is seriously impeded by the famous shortcoming of multiwired, strip in GEM detectors due to the appearance in them a lot of fake hits caused by extra spurious crossings of fired strips. Since the number of those fakes is several orders of magnitude greater than for true hits, one faces with the quite serious difficulty to unravel possible track-candidates via true hits ignoring fakes. On the basis of our previous two-stage approach based on hits preprocessing using directed K-d tree search followed by a deep neural classifier we introduce here two new tracking algorithms. Both algorithms combine those two stages in one while using different types of deep neural nets. We show that both proposed deep networks do not require any special preprocessing stage, are more accurate, faster and can be easier parallelized. Preliminary results of our new approaches for simulated events are presented.