OpenRaiser/Intern-Atlas huggingface dataset:
Underwater Anchor-AUV Localization Geometries with an Isogradient Sound Speed Profile: A CRLB-Based Optimality Analysis
Venue: arXiv cs.IT 2018
Year: 2018

Existing works have explored the anchor deployment for autonomous underwater vehicles (AUVs) localization under the assumption that the sound propagates straightly underwater at a constant speed. Considering that the underwater acoustic waves propagate along bent curves at varying speeds in practice, it becomes much more challenging to determine a proper anchor deployment configuration. In this paper, taking the practical variability of underwater sound speed into account, we investigate the anchor-AUV geometry problem in a 3-D time-of-flight (ToF) based underwater scenario from the perspective of localization accuracy. To address this problem, we first rigorously derive the Jacobian matrix of measurement errors to quantify the Cramer-Rao lower bound (CRLB) with a widely-adopted isogradient sound speed profile (SSP). We then formulate an optimization problem that minimizes the trace of the CRLB subject to the angle and range constraints to figure out the anchor-AUV geometry, which is multivariate and nonlinear and thus generally hard to handle. For mathematical tractability, by adopting tools from the estimation theory, we interestingly find that this problem can be equivalently transformed into a more explicit univariate optimization problem. By this, we obtain an easy-to-implement anchor-AUV geometry that yields satisfactory localization performance, referred to as the uniform sea-surface circumference (USC) deployment. Extensive simulation results validate our theoretical analysis and show that our proposed USC scheme outperforms both the cube and the random deployment schemes in terms of localization accuracy under the same parameter settings.
Phonology-Augmented Statistical Framework for Machine Transliteration using Limited Linguistic Resources
Venue: arXiv cs.CL 2018
Year: 2018

Transliteration converts words in a source language (e.g., English) into words in a target language (e.g., Vietnamese). This conversion considers the phonological structure of the target language, as the transliterated output needs to be pronounceable in the target language. For example, a word in Vietnamese that begins with a consonant cluster is phonologically invalid and thus would be an incorrect output of a transliteration system. Most statistical transliteration approaches, albeit being widely adopted, do not explicitly model the target language's phonology, which often results in invalid outputs. The problem is compounded by the limited linguistic resources available when converting foreign words to transliterated words in the target language. In this work, we present a phonology-augmented statistical framework suitable for transliteration, especially when only limited linguistic resources are available. We propose the concept of pseudo-syllables as structures representing how segments of a foreign word are organized according to the syllables of the target language's phonology. We performed transliteration experiments on Vietnamese and Cantonese. We show that the proposed framework outperforms the statistical baseline by up to 44.68% relative, when there are limited training examples (587 entries).
Reinforcement Evolutionary Learning Method for self-learning
Venue: arXiv cs.LG 2018
Year: 2018

In statistical modelling the biggest threat is concept drift which makes the model gradually showing deteriorating performance over time. There are state of the art methodologies to detect the impact of concept drift, however general strategy considered to overcome the issue in performance is to rebuild or re-calibrate the model periodically as the variable patterns for the model changes significantly due to market change or consumer behavior change etc. Quantitative research is the most widely spread application of data science in Marketing or financial domain where applicability of state of the art reinforcement learning for auto-learning is less explored paradigm. Reinforcement learning is heavily dependent on having a simulated environment which is majorly available for gaming or online systems, to learn from the live feedback. However, there are some research happened on the area of online advertisement, pricing etc where due to the nature of the online learning environment scope of reinforcement learning is explored. Our proposed solution is a reinforcement learning based, true self-learning algorithm which can adapt to the data change or concept drift and auto learn and self-calibrate for the new patterns of the data solving the problem of concept drift. Keywords - Reinforcement learning, Genetic Algorithm, Q-learning, Classification modelling, CMA-ES, NES, Multi objective optimization, Concept drift, Population stability index, Incremental learning, F1-measure, Predictive Modelling, Self-learning, MCTS, AlphaGo, AlphaZero
A Comprehensive Study on Load Balancers for VNF chains Horizontal Scaling
Venue: arXiv cs.NI 2018
Year: 2018

We present an architectural design and a reference implementation for horizontal scaling of virtual network function chains. Our solution does not require any changes to network functions and is able to handle stateful network functions for which states may depend on both directions of the traffic. We use connection-aware traffic load balancers based on hashing function to maintain mappings between connections and the dynamically changing network function chains. Our references implementation uses OpenFlow switches to route traffic to the assigned network function instances according to the load balancer decisions. We conducted extensive simulations to test the feasibility of the architecture and evaluate the performance of our implementation.
Guiding Intelligent Surveillance System by learning-by-synthesis gaze estimation
Venue: arXiv cs.CV 2018
Year: 2018

We describe a novel learning-by-synthesis method for estimating gaze direction of an automated intelligent surveillance system. Recently, progress in learning-by-synthesis has proposed training models on synthetic images, which can effectively reduce the cost of manpower and material resources. However, learning from synthetic images still fails to achieve the desired performance compared to naturalistic images due to the different distribution of synthetic images. In an attempt to address this issue, previous method is to improve the realism of synthetic images by learning a model. However, the disadvantage of the method is that the distortion has not been improved and the authenticity level is unstable. To solve this problem, we put forward a new structure to improve synthetic images, via the reference to the idea of style transformation, through which we can efficiently reduce the distortion of pictures and minimize the need of real data annotation. We estimate that this enables generation of highly realistic images, which we demonstrate both qualitatively and with a user study. We quantitatively evaluate the generated images by training models for gaze estimation. We show a significant improvement over using synthetic images, and achieve state-of-the-art results on various datasets including MPIIGaze dataset.
Detecting Memorization in ReLU Networks
Venue: arXiv cs.LG 2018
Year: 2018

We propose a new notion of `non-linearity' of a network layer with respect to an input batch that is based on its proximity to a linear system, which is reflected in the non-negative rank of the activation matrix. We measure this non-linearity by applying non-negative factorization to the activation matrix. Considering batches of similar samples, we find that high non-linearity in deep layers is indicative of memorization. Furthermore, by applying our approach layer-by-layer, we find that the mechanism for memorization consists of distinct phases. We perform experiments on fully-connected and convolutional neural networks trained on several image and audio datasets. Our results demonstrate that as an indicator for memorization, our technique can be used to perform early stopping.
Decentralized Multi-Antenna Coded Caching with Cyclic Exchanges
Venue: arXiv cs.IT 2018
Year: 2018

This paper considers a single cell multi-antenna base station delivering content to multiple cache enabled single-antenna users. Coding strategies are developed that allow for decentralized placement in the wireless setting. Three different cases namely, max-min multicasting, linear combinations in the complex field, and linear combinations in the finite field, are considered and closed-form rate expressions are provided that hold with high probability. For the case of max-min fair multicasting delivery, we propose a new coding scheme that is capable of working with only two-user broadcasts. A cyclic-exchange protocol for efficient content delivery is proposed and shown to perform almost as well as the original multi-user broadcast scheme.
Consistent Query Answering for Primary Keys in Logspace
Venue: arXiv cs.DB 2018
Year: 2018

We study the complexity of consistent query answering on databases that may violate primary key constraints. A repair of such a database is any consistent database that can be obtained by deleting a minimal set of tuples. For every Boolean query q, CERTAINTY(q) is the problem that takes a database as input and asks whether q evaluates to true on every repair. In [KW17], the authors show that for every self-join-free Boolean conjunctive query q, the problem CERTAINTY(q) is either in P or coNP-complete, and it is decidable which of the two cases applies. In this paper, we sharpen this result by showing that for every self-join-free Boolean conjunctive query q, the problem CERTAINTY(q) is either expressible in symmetric stratified Datalog or coNP-complete. Since symmetric stratified Datalog is in L, we thus obtain a complexity-theoretic dichotomy between L and coNP-complete. Another new finding of practical importance is that CERTAINTY(q) is on the logspace side of the dichotomy for queries q where all join conditions express foreign-to-primary key matches, which is undoubtedly the most common type of join condition.
Dense Multimodal Fusion for Hierarchically Joint Representation
Venue: arXiv cs.CV 2018
Year: 2018

Multiple modalities can provide more valuable information than single one by describing the same contents in various ways. Hence, it is highly expected to learn effective joint representation by fusing the features of different modalities. However, previous methods mainly focus on fusing the shallow features or high-level representations generated by unimodal deep networks, which only capture part of the hierarchical correlations across modalities. In this paper, we propose to densely integrate the representations by greedily stacking multiple shared layers between different modality-specific networks, which is named as Dense Multimodal Fusion (DMF). The joint representations in different shared layers can capture the correlations in different levels, and the connection between shared layers also provides an efficient way to learn the dependence among hierarchical correlations. These two properties jointly contribute to the multiple learning paths in DMF, which results in faster convergence, lower training loss, and better performance. We evaluate our model on three typical multimodal learning tasks, including audiovisual speech recognition, cross-modal retrieval, and multimodal classification. The noticeable performance in the experiments demonstrates that our model can learn more effective joint representation.
Zero-calibration cVEP BCI using word prediction: a proof of concept
Venue: arXiv cs.HC 2018
Year: 2018

Brain Computer Interfaces (BCIs) based on visual evoked potentials (VEP) allow for spelling from a keyboard of flashing characters. Among VEP BCIs, code-modulated visual evoked potentials (c-VEPs) are designed for high-speed communication . In c-VEPs, all characters flash simultaneously. In particular, each character flashes according to a predefined 63-bit binary sequence (m-sequence), circular-shifted by a different time lag. For a given character, the m-sequence evokes a VEP in the electroencephalogram (EEG) of the subject, which can be used as a template. This template is obtained during a calibration phase at the beginning of each session. Then, the system outputs the desired character after a predefined number of repetitions by estimating its time lag with respect to the template. Our work avoids the calibration phase, by extracting from the VEP relative lags between successive characters, and predicting the full word using a dictionary.
Building a language evolution tree based on word vector combination model
Venue: arXiv cs.CL 2018
Year: 2018

In this paper, we try to explore the evolution of language through case calculations. First, we chose the novels of eleven British writers from 1400 to 2005 and found the corresponding works; Then, we use the natural language processing tool to construct the corresponding eleven corpora, and calculate the respective word vectors of 100 high-frequency words in eleven corpora; Next, for each corpus, we concatenate the 100 word vectors from beginning to end into one; Finally, we use the similarity comparison and hierarchical clustering method to generate the relationship tree between the combined eleven word vectors. This tree represents the relationship between eleven corpora. We found that in the tree generated by clustering, the distance between the corpus and the year corresponding to the corpus are basically the same. This means that we have discovered a specific language evolution tree. To verify the stability and versatility of this method, we add three other themes: Dickens's eight works, the 19th century poets' works, and art criticism of recent 60 years. For these four themes, we tested different parameters such as the time span of the corpus, the time interval between the corpora, the dimension of the word vector, and the number of high-frequency public words. The results show that this is fairly stable and versatile.
Fundamental Limits of Covert Bit Insertion in Packets
Venue: arXiv cs.NI 2018
Year: 2018

Covert communication is necessary when revealing the mere existence of a message leaks sensitive information to an attacker. Consider a network link where an authorized transmitter Jack sends packets to an authorized receiver Steve, and the packets visit Alice, Willie, and Bob, respectively, before they reach Steve. Covert transmitter Alice wishes to alter the packet stream in some way to send information to covert receiver Bob without watchful and capable adversary Willie being able to detect the presence of the message. In our previous works, we addressed two techniques for such covert transmission from Alice to Bob: packet insertion and packet timing. In this paper, we consider covert communication via bit insertion in packets with available space (e.g., with size less than the maximum transmission unit). We consider three scenarios: 1) packet sizes are independent and identically distributed (i.i.d.) with a probability mass function (pmf) whose support is a set of one bit spaced values; 2) packet sizes are i.i.d. with a pmf whose support is arbitrary; 3) packet sizes may be dependent. For the first and second assumptions, we show that Alice can covertly insert $\mathcal{O}(\sqrt{n})$ bits of information in a flow of $n$ packets; conversely, if she inserts $ω(\sqrt{n})$ bits of information, Willie can detect her with arbitrarily small error probability. For the third assumption, we prove Alice can covertly insert on average $\mathcal{O}(c(n)/\sqrt{n})$ bits in a sequence of $n$ packets, where $c(n)$ is the average number of conditional pmf of packet sizes given the history, with a support of at least size two.
CHOPT : Automated Hyperparameter Optimization Framework for Cloud-Based Machine Learning Platforms
Venue: arXiv cs.LG 2018
Year: 2018

Many hyperparameter optimization (HyperOpt) methods assume restricted computing resources and mainly focus on enhancing performance. Here we propose a novel cloud-based HyperOpt (CHOPT) framework which can efficiently utilize shared computing resources while supporting various HyperOpt algorithms. We incorporate convenient web-based user interfaces, visualization, and analysis tools, enabling users to easily control optimization procedures and build up valuable insights with an iterative analysis procedure. Furthermore, our framework can be incorporated with any cloud platform, thus complementarily increasing the efficiency of conventional deep learning frameworks. We demonstrate applications of CHOPT with tasks such as image recognition and question-answering, showing that our framework can find hyperparameter configurations competitive with previous work. We also show CHOPT is capable of providing interesting observations through its analysing tools
Combinatorial Attacks on Binarized Neural Networks
Venue: arXiv cs.LG 2018
Year: 2018

Binarized Neural Networks (BNNs) have recently attracted significant interest due to their computational efficiency. Concurrently, it has been shown that neural networks may be overly sensitive to "attacks" - tiny adversarial changes in the input - which may be detrimental to their use in safety-critical domains. Designing attack algorithms that effectively fool trained models is a key step towards learning robust neural networks. The discrete, non-differentiable nature of BNNs, which distinguishes them from their full-precision counterparts, poses a challenge to gradient-based attacks. In this work, we study the problem of attacking a BNN through the lens of combinatorial and integer optimization. We propose a Mixed Integer Linear Programming (MILP) formulation of the problem. While exact and flexible, the MILP quickly becomes intractable as the network and perturbation space grow. To address this issue, we propose IProp, a decomposition-based algorithm that solves a sequence of much smaller MILP problems. Experimentally, we evaluate both proposed methods against the standard gradient-based attack (FGSM) on MNIST and Fashion-MNIST, and show that IProp performs favorably compared to FGSM, while scaling beyond the limits of the MILP.
End-to-End Text Classification via Image-based Embedding using Character-level Networks
Venue: arXiv cs.CL 2018
Year: 2018

For analysing and/or understanding languages having no word boundaries based on morphological analysis such as Japanese, Chinese, and Thai, it is desirable to perform appropriate word segmentation before word embeddings. But it is inherently difficult in these languages. In recent years, various language models based on deep learning have made remarkable progress, and some of these methodologies utilizing character-level features have successfully avoided such a difficult problem. However, when a model is fed character-level features of the above languages, it often causes overfitting due to a large number of character types. In this paper, we propose a CE-CLCNN, character-level convolutional neural networks using a character encoder to tackle these problems. The proposed CE-CLCNN is an end-to-end learning model and has an image-based character encoder, i.e. the CE-CLCNN handles each character in the target document as an image. Through various experiments, we found and confirmed that our CE-CLCNN captured closely embedded features for visually and semantically similar characters and achieves state-of-the-art results on several open document classification tasks. In this paper we report the performance of our CE-CLCNN with the Wikipedia title estimation task and analyse the internal behaviour.
Joint Unsupervised Learning of Optical Flow and Depth by Watching Stereo Videos
Venue: arXiv cs.CV 2018
Year: 2018

Learning depth and optical flow via deep neural networks by watching videos has made significant progress recently. In this paper, we jointly solve the two tasks by exploiting the underlying geometric rules within stereo videos. Specifically, given two consecutive stereo image pairs from a video, we first estimate depth, camera ego-motion and optical flow from three neural networks. Then the whole scene is decomposed into moving foreground and static background by compar- ing the estimated optical flow and rigid flow derived from the depth and ego-motion. We propose a novel consistency loss to let the optical flow learn from the more accurate rigid flow in static regions. We also design a rigid alignment module which helps refine ego-motion estimation by using the estimated depth and optical flow. Experiments on the KITTI dataset show that our results significantly outperform other state-of- the-art algorithms. Source codes can be found at https: //github.com/baidu-research/UnDepthflow
A Summary of the 4th International Workshop on Recovering 6D Object Pose
Venue: arXiv cs.CV 2018
Year: 2018

This document summarizes the 4th International Workshop on Recovering 6D Object Pose which was organized in conjunction with ECCV 2018 in Munich. The workshop featured four invited talks, oral and poster presentations of accepted workshop papers, and an introduction of the BOP benchmark for 6D object pose estimation. The workshop was attended by 100+ people working on relevant topics in both academia and industry who shared up-to-date advances and discussed open problems.
The Outer Product Structure of Neural Network Derivatives
Venue: arXiv cs.LG 2018
Year: 2018

In this paper, we show that feedforward and recurrent neural networks exhibit an outer product derivative structure but that convolutional neural networks do not. This structure makes it possible to use higher-order information without needing approximations or infeasibly large amounts of memory, and it may also provide insights into the geometry of neural network optima. The ability to easily access these derivatives also suggests a new, geometric approach to regularization. We then discuss how this structure could be used to improve training methods, increase network robustness and generalizability, and inform network compression methods.
A categorisation and implementation of digital pen features for behaviour characterisation
Venue: arXiv cs.CV 2018
Year: 2018

In this paper we provide a categorisation and implementation of digital ink features for behaviour characterisation. Based on four feature sets taken from literature, we provide a categorisation in different classes of syntactic and semantic features. We implemented a publicly available framework to calculate these features and show its deployment in the use case of analysing cognitive assessments performed using a digital pen.