OpenRaiser/Intern-Atlas huggingface dataset:
High Five: Improving Gesture Recognition by Embracing Uncertainty
Venue: arXiv cs.CV 2017
Year: 2017

Sensors on mobile devices---accelerometers, gyroscopes, pressure meters, and GPS---invite new applications in gesture recognition, gaming, and fitness tracking. However, programming them remains challenging because human gestures captured by sensors are noisy. This paper illustrates that noisy gestures degrade training and classification accuracy for gesture recognition in state-of-the-art deterministic Hidden Markov Models (HMM). We introduce a new statistical quantization approach that mitigates these problems by (1) during training, producing gesture-specific codebooks, HMMs, and error models for gesture sequences; and (2) during classification, exploiting the error model to explore multiple feasible HMM state sequences. We implement classification in Uncertain<t>, a probabilistic programming system that encapsulates HMMs and error models and then automates sampling and inference in the runtime. Uncertain<T> developers directly express a choice of application-specific trade-off between recall and precision at gesture recognition time, rather than at training time. We demonstrate benefits in configurability, precision, recall, and recognition on two data sets with 25 gestures from 28 people and 4200 total gestures. Incorporating gesture error more accurately in modeling improves the average recognition rate of 20 gestures from 34\% in prior work to 62\%. Incorporating the error model during classification further improves the average gesture recognition rate to 71\%. As far as we are aware, no prior work shows how to generate an HMM error model during training and use it to improve classification rates.
Millimeter Wave MIMO Prototype: Measurements and Experimental Results
Venue: arXiv cs.IT 2017
Year: 2017

Millimeter-wave multi-input multi-output (mm-Wave MIMO) systems are one of the candidate schemes for 5G wireless standardization efforts. In this context, the main contributions of this article are three-fold. 1) We describe parallel sets of measurements at identical transmit-receive location pairs with 2.9, 29 and 61 GHz carrier frequencies in indoor office, shopping mall, and outdoor settings. These measurements provide insights on propagation, blockage and material penetration losses, and the key elements necessary in system design to make mm-Wave systems viable in practice. 2) One of these elements is hybrid beamforming necessary for better link margins by reaping the array gain with large antenna dimensions. From the class of fully-flexible hybrid beamformers, we describe a robust class of directional beamformers towards meeting the high data-rate requirements of mm-Wave systems. 3) Leveraging these design insights, we then describe an experimental prototype system at 28 GHz that realizes high data-rates on both the downlink and uplink and robustly maintains these rates in outdoor and indoor mobility scenarios. In addition to maintaining large signal constellation sizes in spite of radio frequency challenges, this prototype leverages the directional nature of the mm-Wave channel to perform seamless beam switching and handover across mm-Wave base-stations thereby overcoming the path losses in non-line-of-sight links and blockages encountered at mm-Wave frequencies.
Context-Aware Generative Adversarial Privacy
Venue: arXiv cs.LG 2017
Year: 2017

Preserving the utility of published datasets while simultaneously providing provable privacy guarantees is a well-known challenge. On the one hand, context-free privacy solutions, such as differential privacy, provide strong privacy guarantees, but often lead to a significant reduction in utility. On the other hand, context-aware privacy solutions, such as information theoretic privacy, achieve an improved privacy-utility tradeoff, but assume that the data holder has access to dataset statistics. We circumvent these limitations by introducing a novel context-aware privacy framework called generative adversarial privacy (GAP). GAP leverages recent advancements in generative adversarial networks (GANs) to allow the data holder to learn privatization schemes from the dataset itself. Under GAP, learning the privacy mechanism is formulated as a constrained minimax game between two players: a privatizer that sanitizes the dataset in a way that limits the risk of inference attacks on the individuals' private variables, and an adversary that tries to infer the private variables from the sanitized dataset. To evaluate GAP's performance, we investigate two simple (yet canonical) statistical dataset models: (a) the binary data model, and (b) the binary Gaussian mixture model. For both models, we derive game-theoretically optimal minimax privacy mechanisms, and show that the privacy mechanisms learned from data (in a generative adversarial fashion) match the theoretically optimal ones. This demonstrates that our framework can be easily applied in practice, even in the absence of dataset statistics.
SRE: Semantic Rules Engine For the Industrial Internet-Of-Things Gateways
Venue: arXiv cs.AI 2017
Year: 2017

The Advent of the Internet-of-Things (IoT) paradigm has brought opportunities to solve many real-world problems. Energy management, for example, has attracted huge interest from academia, industries, governments and regulatory bodies. It involves collecting energy usage data, analyzing it, and optimizing the energy consumption by applying control strategies. However, in industrial environments, performing such optimization is not trivial. The changes in business rules, process control, and customer requirements make it much more challenging. In this paper, a Semantic Rules Engine (SRE) for industrial gateways is presented that allows implementing dynamic and flexible rule-based control strategies. It is simple, expressive, and allows managing rules on-the-fly without causing any service interruption. Additionally, it can handle semantic queries and provide results by inferring additional knowledge from previously defined concepts in ontologies. SRE has been validated and tested on different hardware platforms and in commercial products. Performance evaluations are also presented to validate its conformance to the customer requirements.
Segment Parameter Labelling in MCMC Mean-Shift Change Detection
Venue: arXiv cs.LG 2017
Year: 2017

This work addresses the problem of segmentation in time series data with respect to a statistical parameter of interest in Bayesian models. It is common to assume that the parameters are distinct within each segment. As such, many Bayesian change point detection models do not exploit the segment parameter patterns, which can improve performance. This work proposes a Bayesian mean-shift change point detection algorithm that makes use of repetition in segment parameters, by introducing segment class labels that utilise a Dirichlet process prior. The performance of the proposed approach was assessed on both synthetic and real world data, highlighting the enhanced performance when using parameter labelling.
Evaluating Password Advice
Venue: arXiv cs.CR 2017
Year: 2017

Password advice is constantly circulated by standards agencies, companies, websites and specialists. But there appears to be great diversity in terms of the advice that is given. Users have noticed that different websites are enforcing different restrictions. For example, requiring different combinations of uppercase and lowercase letters, numbers and special characters. We collected password advice and found that the advice distributed by one organization can directly contradict advice given by another. Our paper aims to illuminate interesting characteristics for a sample of the password advice distributed. We also create a framework for identifying the costs associated with implementing password advice. In doing so we identify a reason for why password advice is often both derided and ignored.
Minimum Circuit Size, Graph Isomorphism, and Related Problems
Venue: arXiv cs.CC 2017
Year: 2017

We study the computational power of deciding whether a given truth-table can be described by a circuit of a given size (the Minimum Circuit Size Problem, or MCSP for short), and of the variant denoted as MKTP where circuit size is replaced by a polynomially-related Kolmogorov measure. All prior reductions from supposedly-intractable problems to MCSP / MKTP hinged on the power of MCSP / MKTP to distinguish random distributions from distributions produced by hardness-based pseudorandom generator constructions. We develop a fundamentally different approach inspired by the well-known interactive proof system for the complement of Graph Isomorphism (GI). It yields a randomized reduction with zero-sided error from GI to MKTP. We generalize the result and show that GI can be replaced by any isomorphism problem for which the underlying group satisfies some elementary properties. Instantiations include Linear Code Equivalence, Permutation Group Conjugacy, and Matrix Subspace Conjugacy. Along the way we develop encodings of isomorphism classes that are efficiently decodable and achieve compression that is at or near the information-theoretic optimum; those encodings may be of independent interest.
Exploiting Commutativity For Practical Fast Replication
Venue: arXiv cs.DC 2017
Year: 2017

Traditional approaches to replication require client requests to be ordered before making them durable by copying them to replicas. As a result, clients must wait for two round-trip times (RTTs) before updates complete. In this paper, we show that this entanglement of ordering and durability is unnecessary for strong consistency. Consistent Unordered Replication Protocol (CURP) allows clients to replicate requests that have not yet been ordered, as long as they are commutative. This strategy allows most operations to complete in 1 RTT (the same as an unreplicated system). We implemented CURP in the Redis and RAMCloud storage systems. In RAMCloud, CURP improved write latency by ~2x (13.8 us -> 7.3 us) and write throughput by 4x. Compared to unreplicated RAMCloud, CURP's latency overhead for 3-way replication is just 0.4 us (6.9 us vs 7.3 us). CURP transformed a non-durable Redis cache into a consistent and durable storage system with only a small performance overhead.
Improving Deep Learning by Inverse Square Root Linear Units (ISRLUs)
Venue: arXiv cs.LG 2017
Year: 2017

We introduce the "inverse square root linear unit" (ISRLU) to speed up learning in deep neural networks. ISRLU has better performance than ELU but has many of the same benefits. ISRLU and ELU have similar curves and characteristics. Both have negative values, allowing them to push mean unit activation closer to zero, and bring the normal gradient closer to the unit natural gradient, ensuring a noise-robust deactivation state, lessening the over fitting risk. The significant performance advantage of ISRLU on traditional CPUs also carry over to more efficient HW implementations on HW/SW codesign for CNNs/RNNs. In experiments with TensorFlow, ISRLU leads to faster learning and better generalization than ReLU on CNNs. This work also suggests a computationally efficient variant called the "inverse square root unit" (ISRU) which can be used for RNNs. Many RNNs use either long short-term memory (LSTM) and gated recurrent units (GRU) which are implemented with tanh and sigmoid activation functions. ISRU has less com- putational complexity but still has a similar curve to tanh and sigmoid.
An efficient SAT formulation for learning multiple criteria non-compensatory sorting rules from examples
Venue: arXiv cs.AI 2017
Year: 2017

The literature on Multiple Criteria Decision Analysis (MCDA) proposes several methods in order to sort alternatives evaluated on several attributes into ordered classes. Non Compensatory Sorting models (NCS) assign alternatives to classes based on the way they compare to multicriteria profiles separating the consecutive classes. Previous works have proposed approaches to learn the parameters of a NCS model based on a learning set. Exact approaches based on mixed integer linear programming ensures that the learning set is best restored, but can only handle datasets of limited size. Heuristic approaches can handle large learning sets, but do not provide any guarantee about the inferred model. In this paper, we propose an alternative formulation to learn a NCS model. This formulation, based on a SAT problem, guarantees to find a model fully consistent with the learning set (whenever it exists), and is computationally much more efficient than existing exact MIP approaches.
RRT-CoLearn: towards kinodynamic planning without numerical trajectory optimization
Venue: arXiv cs.RO 2017
Year: 2017

Sampling-based kinodynamic planners, such as Rapidly-exploring Random Trees (RRTs), pose two fundamental challenges: computing a reliable (pseudo-)metric for the distance between two randomly sampled nodes, and computing a steering input to connect the nodes. The core of these challenges is a Two Point Boundary Value Problem, which is known to be NP-hard. Recently, the distance metric has been approximated using supervised learning, reducing computation time drastically. The previous work on such learning RRTs use direct optimal control to generate the data for supervised learning. This paper proposes to use indirect optimal control instead, because it provides two benefits: it reduces the computational effort to generate the data, and it provides a low dimensional parametrization of the action space. The latter allows us to learn both the distance metric and the steering input to connect two nodes. This eliminates the need for a local planner in learning RRTs. Experimental results on a pendulum swing up show 10-fold speed-up in both the offline data generation and the online planning time, leading to at least a 10-fold speed-up in the overall planning time.
A Self-Training Method for Semi-Supervised GANs
Venue: arXiv cs.LG 2017
Year: 2017

Since the creation of Generative Adversarial Networks (GANs), much work has been done to improve their training stability, their generated image quality, their range of application but nearly none of them explored their self-training potential. Self-training has been used before the advent of deep learning in order to allow training on limited labelled training data and has shown impressive results in semi-supervised learning. In this work, we combine these two ideas and make GANs self-trainable for semi-supervised learning tasks by exploiting their infinite data generation potential. Results show that using even the simplest form of self-training yields an improvement. We also show results for a more complex self-training scheme that performs at least as well as the basic self-training scheme but with significantly less data augmentation.
A Range-Doppler-Angle Estimation Method for Passive Bistatic Radar
Venue: arXiv cs.IT 2017
Year: 2017

In this paper, an effective target detection and localization method is proposed for a passive bistatic radar (PBR) system. The PBR system consists of a commercial FM radio station, which is a non-cooperative illuminator of opportunity (IO), referred to as the transmitter antenna and multiple surveillance antennas that form an antenna array, e.g., uniform linear array (ULA). Unlike other literatures where the reference signal is received by a directional antenna, here, the reference signal (direct path) is estimated by beamforming method. Then a modified extensive cancellation algorithm (MECA) based on (least squares) LS method is proposed to solve the disturbance cancellation. After cancelling the disturbance, the matched filter (MF) and LS methods are used for range-Doppler estimation of targets, and then the angles of targets are estimated based on beamforming method. The proposed method is suitable for an antenna array. Simulation results are presented to illustrate the superiority of the proposed MECA disturbance cancellation method and parameter estimation method.
Total-Text: A Comprehensive Dataset for Scene Text Detection and Recognition
Venue: arXiv cs.CV 2017
Year: 2017

Text in curve orientation, despite being one of the common text orientations in real world environment, has close to zero existence in well received scene text datasets such as ICDAR2013 and MSRA-TD500. The main motivation of Total-Text is to fill this gap and facilitate a new research direction for the scene text community. On top of the conventional horizontal and multi-oriented texts, it features curved-oriented text. Total-Text is highly diversified in orientations, more than half of its images have a combination of more than two orientations. Recently, a new breed of solutions that casted text detection as a segmentation problem has demonstrated their effectiveness against multi-oriented text. In order to evaluate its robustness against curved text, we fine-tuned DeconvNet and benchmark it on Total-Text. Total-Text with its annotation is available at https://github.com/cs-chan/Total-Text-Dataset
Trainable back-propagated functional transfer matrices
Venue: arXiv cs.LG 2017
Year: 2017

Connections between nodes of fully connected neural networks are usually represented by weight matrices. In this article, functional transfer matrices are introduced as alternatives to the weight matrices: Instead of using real weights, a functional transfer matrix uses real functions with trainable parameters to represent connections between nodes. Multiple functional transfer matrices are then stacked together with bias vectors and activations to form deep functional transfer neural networks. These neural networks can be trained within the framework of back-propagation, based on a revision of the delta rules and the error transmission rule for functional connections. In experiments, it is demonstrated that the revised rules can be used to train a range of functional connections: 20 different functions are applied to neural networks with up to 10 hidden layers, and most of them gain high test accuracies on the MNIST database. It is also demonstrated that a functional transfer matrix with a memory function can roughly memorise a non-cyclical sequence of 400 digits.
Wasserstein Identity Testing
Venue: arXiv cs.LG 2017
Year: 2017

Uniformity testing and the more general identity testing are well studied problems in distributional property testing. Most previous work focuses on testing under $L_1$-distance. However, when the support is very large or even continuous, testing under $L_1$-distance may require a huge (even infinite) number of samples. Motivated by such issues, we consider the identity testing in Wasserstein distance (a.k.a. transportation distance and earthmover distance) on a metric space (discrete or continuous). In this paper, we propose the Wasserstein identity testing problem (Identity Testing in Wasserstein distance). We obtain nearly optimal worst-case sample complexity for the problem. Moreover, for a large class of probability distributions satisfying the so-called "Doubling Condition", we provide nearly instance-optimal sample complexity.
Partial Knowledge In Embeddings
Venue: arXiv cs.AI 2017
Year: 2017

Representing domain knowledge is crucial for any task. There has been a wide range of techniques developed to represent this knowledge, from older logic based approaches to the more recent deep learning based techniques (i.e. embeddings). In this paper, we discuss some of these methods, focusing on the representational expressiveness tradeoffs that are often made. In particular, we focus on the the ability of various techniques to encode `partial knowledge' - a key component of successful knowledge systems. We introduce and describe the concepts of `ensembles of embeddings' and `aggregate embeddings' and demonstrate how they allow for partial knowledge.
Complexity Analysis Approach for Prefabricated Construction Products Using Uncertain Data Clustering
Venue: arXiv cs.DB 2017
Year: 2017

This paper proposes an uncertain data clustering approach to quantitatively analyze the complexity of prefabricated construction components through the integration of quality performance-based measures with associated engineering design information. The proposed model is constructed in three steps, which (1) measure prefabricated construction product complexity (hereafter referred to as product complexity) by introducing a Bayesian-based nonconforming quality performance indicator; (2) score each type of product complexity by developing a Hellinger distance-based distribution similarity measurement; and (3) cluster products into homogeneous complexity groups by using the agglomerative hierarchical clustering technique. An illustrative example is provided to demonstrate the proposed approach, and a case study of an industrial company in Edmonton, Canada, is conducted to validate the feasibility and applicability of the proposed model. This research inventively defines and investigates product complexity from the perspective of product quality performance with design information associated. The research outcomes provide simplified, interpretable, and informative insights for practitioners to better analyze and manage product complexity. In addition to this practical contribution, a novel hierarchical clustering technique is devised. This technique is capable of clustering uncertain data (i.e., beta distributions) with lower computational complexity and has the potential to be generalized to cluster all types of uncertain data.
Finding Dominant User Utterances And System Responses in Conversations
Venue: arXiv cs.CL 2017
Year: 2017

There are several dialog frameworks which allow manual specification of intents and rule based dialog flow. The rule based framework provides good control to dialog designers at the expense of being more time consuming and laborious. The job of a dialog designer can be reduced if we could identify pairs of user intents and corresponding responses automatically from prior conversations between users and agents. In this paper we propose an approach to find these frequent user utterances (which serve as examples for intents) and corresponding agent responses. We propose a novel SimCluster algorithm that extends standard K-means algorithm to simultaneously cluster user utterances and agent utterances by taking their adjacency information into account. The method also aligns these clusters to provide pairs of intents and response groups. We compare our results with those produced by using simple Kmeans clustering on a real dataset and observe upto 10% absolute improvement in F1-scores. Through our experiments on synthetic dataset, we show that our algorithm gains more advantage over K-means algorithm when the data has large variance.