OpenRaiser/Intern-Atlas huggingface dataset:
End to End Recognition System for Recognizing Offline Unconstrained Vietnamese Handwriting
Venue: arXiv cs.CV 2019
Year: 2019

Inspired by recent successes in neural machine translation and image caption generation, we present an attention based encoder decoder model (AED) to recognize Vietnamese Handwritten Text. The model composes of two parts: a DenseNet for extracting invariant features, and a Long Short-Term Memory network (LSTM) with an attention model incorporated for generating output text (LSTM decoder), which are connected from the CNN part to the attention model. The input of the CNN part is a handwritten text image and the target of the LSTM decoder is the corresponding text of the input image. Our model is trained end-to-end to predict the text from a given input image since all the parts are differential components. In the experiment section, we evaluate our proposed AED model on the VNOnDB-Word and VNOnDB-Line datasets to verify its efficiency. The experiential results show that our model achieves 12.30% of word error rate without using any language model. This result is competitive with the handwriting recognition system provided by Google in the Vietnamese Online Handwritten Text Recognition competition.
Stochastic Gradient Coding for Straggler Mitigation in Distributed Learning
Venue: arXiv cs.IT 2019
Year: 2019

We consider distributed gradient descent in the presence of stragglers. Recent work on \em gradient coding \em and \em approximate gradient coding \em have shown how to add redundancy in distributed gradient descent to guarantee convergence even if some workers are \em stragglers\em---that is, slow or non-responsive. In this work we propose an approximate gradient coding scheme called \em Stochastic Gradient Coding \em (SGC), which works when the stragglers are random. SGC distributes data points redundantly to workers according to a pair-wise balanced design, and then simply ignores the stragglers. We prove that the convergence rate of SGC mirrors that of batched Stochastic Gradient Descent (SGD) for the $\ell_2$ loss function, and show how the convergence rate can improve with the redundancy. We also provide bounds for more general convex loss functions. We show empirically that SGC requires a small amount of redundancy to handle a large number of stragglers and that it can outperform existing approximate gradient codes when the number of stragglers is large.
On the number of k-skip-n-grams
Venue: arXiv cs.CL 2019
Year: 2019

The paper proves that the number of k-skip-n-grams for a corpus of size $L$ is $$\frac{Ln + n + k' - n^2 - nk'}{n} \cdot \binom{n-1+k'}{n-1}$$ where $k' = \min(L - n + 1, k)$.
Parallel genetic algorithm for planning safe and optimal route for ship
Venue: arXiv cs.NE 2019
Year: 2019

The paper represents an algorithm for planning safe and optimal routes for transport facilities with unrestricted movement direction that travel within areas with obstacles. Paper explains the algorithm using a ship as an example of such a transport facility. This paper also provides a survey of several existing solutions for the problem. The method employs an evolutionary algorithm to plan several locally optimal routes and a parallel genetic algorithm to create the final route by optimising the abovementioned set of routes. The routes are optimized against the arrival time, assuming that the optimal route is the route with the lowermost arrival time. It is also possible to apply additional restriction to the routes.
American Sign Language Alphabet Recognition using Deep Learning
Venue: arXiv cs.CV 2019
Year: 2019

Tremendous headway has been made in the field of 3D hand pose estimation but the 3D depth cameras are usually inaccessible. We propose a model to recognize American Sign Language alphabet from RGB images. Images for the training were resized and pre-processed before training the Deep Neural Network. The model was trained on a squeezenet architecture to make it capable of running on mobile devices with an accuracy of 83.29%.
A (co)algebraic theory of succinct automata
Venue: arXiv cs.FL 2019
Year: 2019

The classical subset construction for non-deterministic automata can be generalized to other side-effects captured by a monad. The key insight is that both the state space of the determinized automaton and its semantics---languages over an alphabet---have a common algebraic structure: they are Eilenberg-Moore algebras for the powerset monad. In this paper we study the reverse question to determinization. We will present a construction to associate succinct automata to languages based on different algebraic structures. For instance, for classical regular languages the construction will transform a deterministic automaton into a non-deterministic one, where the states represent the join-irreducibles of the language accepted by a (potentially) larger deterministic automaton. Other examples will yield alternating automata, automata with symmetries, CABA-structured automata, and weighted automata.
A Deep Spatio-Temporal Fuzzy Neural Network for Passenger Demand Prediction
Venue: arXiv cs.LG 2019
Year: 2019

In spite of its importance, passenger demand prediction is a highly challenging problem, because the demand is simultaneously influenced by the complex interactions among many spatial and temporal factors and other external factors such as weather. To address this problem, we propose a Spatio-TEmporal Fuzzy neural Network (STEF-Net) to accurately predict passenger demands incorporating the complex interactions of all known important factors. We design an end-to-end learning framework with different neural networks modeling different factors. Specifically, we propose to capture spatio-temporal feature interactions via a convolutional long short-term memory network and model external factors via a fuzzy neural network that handles data uncertainty significantly better than deterministic methods. To keep the temporal relations when fusing two networks and emphasize discriminative spatio-temporal feature interactions, we employ a novel feature fusion method with a convolution operation and an attention layer. As far as we know, our work is the first to fuse a deep recurrent neural network and a fuzzy neural network to model complex spatial-temporal feature interactions with additional uncertain input features for predictive learning. Experiments on a large-scale real-world dataset show that our model achieves more than 10% improvement over the state-of-the-art approaches.
Using Delay Tolerant Networks as a Backbone for Low-cost Smart Cities
Venue: arXiv cs.NI 2019
Year: 2019

Rapid urbanization burdens city infrastructure and creates the need for local governments to maximize the usage of resources to serve its citizens. Smart city projects aim to alleviate the urbanization problem by deploying a vast amount of Internet-of-things (IoT) devices to monitor and manage environmental conditions and infrastructure. However, smart city projects can be extremely expensive to deploy and manage. A significant portion of the expense is a result of providing Internet connectivity via 5G or WiFi to IoT devices. This paper proposes the use of delay tolerant networks (DTNs) as a backbone for smart city communication; enabling developing communities to become smart cities at a fraction of the cost. A model is introduced to aid policy makers in designing and evaluating the expected performance of such networks. Preliminary results are presented based on a public transit network data-set from Chapel Hill, North Carolina. Finally, innovative ways of improving network performance in a low-cost smart city is discussed.
A Qualitative Post-Experience Method for Evaluating Changes in VR Presence Experience Over Time
Venue: arXiv cs.HC 2019
Year: 2019

A particular measure to evaluate a head-mounted display (HMD) based experience is the state of feeling present in virtual reality. Interruptions of a presence experience - break in presence (BIP) - appearing over time, need to be detected to assess and improve an application. Existing methods either lack in taking these BIPs into account - questionnaires - or are complex in their application and evaluation - physiological and behavioral measures -. To provide a practical approach, we propose a post-experience method in which the users reflect on their experience by drawing a line, indicating their experienced state of presence, in a paper-based drawing template. The amplitude of the drawn line represents the variation of their presence experience over time. We propose a descriptive model that describes temporal variations in the drawings by the definition of relevant points over time - e.g., putting on the HMD -, phases of the experience - e.g., transition into VR - and parameters - e.g., the transition time -. The descriptive model enables us to objectively evaluate user drawings and represent the course of the drawings by a defined set of parameters. An exploratory user study (N=30) showed that the drawings are very consistent, the method can detect all BIPs and shows good indications for representing the intensity of a BIP. With our method practitioners and researchers can accelerate the evaluation and optimization of experiences by evaluating BIPs. The possibility to store objective parameters paves the way for automated evaluation methods and big data approaches.
LASER: Lightweight And SEcure Remote keyless entry protocol (Extended version)
Venue: arXiv cs.CR 2019
Year: 2019

Since Remote Keyless Entry (RKE) systems started to be widely used, several vulnerabilities in their protocols have been found. Attacks such as jamming-and-replay attacks and relay attacks are still effective against most recent RKE systems, even when many secure schemes have been designed. Although they are interesting from a theoretical point of view, the complexity of these solutions is excessive to implement them into a fob. This paper presents a lightweight and general solution based on a one message protocol, which guarantees the integrity and validity of the authentication in RKE systems, protecting the communication against the well-known jamming-and-replay and relay attacks, without using complex cryptographic schemes. Moreover, we also adapt our protocol for passive RKE (PRKE) systems. Our solution also includes a novel frequency-hopping-based approach which mitigates deny-of-service attacks. Finally, a prototype has been implemented using non-expensive hardware. Obtained results assure scalability, effectiveness and robustness.
A logical-based corpus for cross-lingual evaluation
Venue: arXiv cs.CL 2019
Year: 2019

At present, different deep learning models are presenting high accuracy on popular inference datasets such as SNLI, MNLI, and SciTail. However, there are different indicators that those datasets can be exploited by using some simple linguistic patterns. This fact poses difficulties to our understanding of the actual capacity of machine learning models to solve the complex task of textual inference. We propose a new set of syntactic tasks focused on contradiction detection that require specific capacities over linguistic logical forms such as: Boolean coordination, quantifiers, definite description, and counting operators. We evaluate two kinds of deep learning models that implicitly exploit language structure: recurrent models and the Transformer network BERT. We show that although BERT is clearly more efficient to generalize over most logical forms, there is space for improvement when dealing with counting operators. Since the syntactic tasks can be implemented in different languages, we show a successful case of cross-lingual transfer learning between English and Portuguese.
Software Engineering for Fairness: A Case Study with Hyperparameter Optimization
Venue: arXiv cs.SE 2019
Year: 2019

We assert that it is the ethical duty of software engineers to strive to reduce software discrimination. This paper discusses how that might be done. This is an important topic since machine learning software is increasingly being used to make decisions that affect people's lives. Potentially, the application of that software will result in fairer decisions because (unlike humans) machine learning software is not biased. However, recent results show that the software within many data mining packages exhibits "group discrimination"; i.e. their decisions are inappropriately affected by "protected attributes"(e.g., race, gender, age, etc.). There has been much prior work on validating the fairness of machine-learning models (by recognizing when such software discrimination exists). But after detection, comes mitigation. What steps can ethical software engineers take to reduce discrimination in the software they produce? This paper shows that making \textit{fairness} as a goal during hyperparameter optimization can (a) preserve the predictive power of a model learned from a data miner while also (b) generates fairer results. To the best of our knowledge, this is the first application of hyperparameter optimization as a tool for software engineers to generate fairer software.
Preservation of normality by non-oblivious group selection
Venue: arXiv cs.FL 2019
Year: 2019

We give two different proofs of the fact that non-oblivious selection via regular group sets preserves normality. Non-oblivious here means that whether or not a symbol is selected can depend on the symbol itself. One proof relies on the incompressibility of normal sequences, the other on the use of augmented dynamical systems.
Learning Policies from Self-Play with Policy Gradients and MCTS Value Estimates
Venue: arXiv cs.LG 2019
Year: 2019

In recent years, state-of-the-art game-playing agents often involve policies that are trained in self-playing processes where Monte Carlo tree search (MCTS) algorithms and trained policies iteratively improve each other. The strongest results have been obtained when policies are trained to mimic the search behaviour of MCTS by minimising a cross-entropy loss. Because MCTS, by design, includes an element of exploration, policies trained in this manner are also likely to exhibit a similar extent of exploration. In this paper, we are interested in learning policies for a project with future goals including the extraction of interpretable strategies, rather than state-of-the-art game-playing performance. For these goals, we argue that such an extent of exploration is undesirable, and we propose a novel objective function for training policies that are not exploratory. We derive a policy gradient expression for maximising this objective function, which can be estimated using MCTS value estimates, rather than MCTS visit counts. We empirically evaluate various properties of resulting policies, in a variety of board games.
Reconstruction-Aware Imaging System Ranking by use of a Sparsity-Driven Numerical Observer Enabled by Variational Bayesian Inference
Venue: arXiv cs.CV 2019
Year: 2019

It is widely accepted that optimization of imaging system performance should be guided by task-based measures of image quality (IQ). It has been advocated that imaging hardware or data-acquisition designs should be optimized by use of an ideal observer (IO) that exploits full statistical knowledge of the measurement noise and class of objects to be imaged, without consideration of the reconstruction method. In practice, accurate and tractable models of the complete object statistics are often difficult to determine. Moreover, in imaging systems that employ compressive sensing concepts, imaging hardware and sparse image reconstruction are innately coupled technologies. In this work, a sparsity-driven observer (SDO) that can be employed to optimize hardware by use of a stochastic object model describing object sparsity is described and investigated. The SDO and sparse reconstruction method can therefore be "matched" in the sense that they both utilize the same statistical information regarding the class of objects to be imaged. To efficiently compute the SDO test statistic, computational tools developed recently for variational Bayesian inference with sparse linear models are adopted. The use of the SDO to rank data-acquisition designs in a stylized example as motivated by magnetic resonance imaging (MRI) is demonstrated. This study reveals that the SDO can produce rankings that are consistent with visual assessments of the reconstructed images but different from those produced by use of the traditionally employed Hotelling observer (HO).
Performance Analysis of Non-DC-Biased OFDM
Venue: arXiv cs.IT 2019
Year: 2019

The performance analysis of a novel optical modulation scheme is presented in this paper. The basic concept is to transmit signs of modulated optical orthogonal frequency division multiplexing (O-OFDM) symbols and absolute values of the symbols separately by two information carrying units: 1) indices of two light emitting diodes (LED) transmitters that represent positive and negative signs separately; and 2) optical intensity symbols that carry the absolute values of signals. The new approach, referred as to non-DC-biased OFDM (NDC-OFDM), uses the optical spatial modulation (OSM) technique to eliminate the effect of the clipping distortion in DC-biased optical OFDM (DCO-OFDM). In addition, it can achieve similar advantages as the conventional unipolar modulation scheme, asymmetrically clipped optical OFDM (ACO-OFDM), without using additional subcarriers. In this paper, the analytical BER performance is compared with the Monte Carlo result in order to prove the reliability of the new method. Moreover, the practical BER performance of NDC-OFDM with DCO-OFDM and ACO-OFDM is compared for different constellation sizes to verify the improvement of NDC-OFDM on the spectral and power efficiencies.
Supervised Learning of the Next-Best-View for 3D Object Reconstruction
Venue: arXiv cs.CV 2019
Year: 2019

Motivated by the advances in 3D sensing technology and the spreading of low-cost robotic platforms, 3D object reconstruction has become a common task in many areas. Nevertheless, the selection of the optimal sensor pose that maximizes the reconstructed surface is a problem that remains open. It is known in the literature as the next-best-view planning problem. In this paper, we propose a novel next-best-view planning scheme based on supervised deep learning. The scheme contains an algorithm for automatic generation of datasets and an original three-dimensional convolutional neural network (3D-CNN) used to learn the next-best-view. Unlike previous work where the problem is addressed as a search, the trained 3D-CNN directly predicts the sensor pose. We present a comparison of the proposed network against a similar net, and we present several experiments of the reconstruction of unknown objects validating the effectiveness of the proposed scheme.
Consensus-based Interpretable Deep Neural Networks with Application to Mortality Prediction
Venue: arXiv cs.LG 2019
Year: 2019

Deep neural networks have achieved remarkable success in various challenging tasks. However, the black-box nature of such networks is not acceptable to critical applications, such as healthcare. In particular, the existence of adversarial examples and their overgeneralization to irrelevant, out-of-distribution inputs with high confidence makes it difficult, if not impossible, to explain decisions by such networks. In this paper, we analyze the underlying mechanism of generalization of deep neural networks and propose an ($n$, $k$) consensus algorithm which is insensitive to adversarial examples and can reliably reject out-of-distribution samples. Furthermore, the consensus algorithm is able to improve classification accuracy by using multiple trained deep neural networks. To handle the complexity of deep neural networks, we cluster linear approximations of individual models and identify highly correlated clusters among different models to capture feature importance robustly, resulting in improved interpretability. Motivated by the importance of building accurate and interpretable prediction models for healthcare, our experimental results on an ICU dataset show the effectiveness of our algorithm in enhancing both the prediction accuracy and the interpretability of deep neural network models on one-year patient mortality prediction. In particular, while the proposed method maintains similar interpretability as conventional shallow models such as logistic regression, it improves the prediction accuracy significantly.
Connectivity-Aware UAV Path Planning with Aerial Coverage Maps
Venue: arXiv cs.NI 2019
Year: 2019

Cellular networks are promising to support effective wireless communications for unmanned aerial vehicles (UAVs), which will help to enable various long-range UAV applications. However, these networks are optimized for terrestrial users, and thus do not guarantee seamless aerial coverage. In this paper, we propose to overcome this difficulty by exploiting controllable mobility of UAVs, and investigate connectivity-aware UAV path planning. To explicitly impose communication requirements on UAV path planning, we introduce two new metrics to quantify the cellular connectivity quality of a UAV path. Moreover, aerial coverage maps are used to provide accurate locations of scattered coverage holes in the complicated propagation environment. We formulate the UAV path planning problem as finding the shortest path subject to connectivity constraints. Based on graph search methods, a novel connectivity-aware path planning algorithm with low complexity is proposed. The effectiveness and superiority of our proposed algorithm are demonstrated using the aerial coverage map of an urban section in Virginia, which is built by ray tracing. Simulation results also illustrate a tradeoff between the path length and connectivity quality of UAVs.
Rethinking the Usage of Batch Normalization and Dropout in the Training of Deep Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

In this work, we propose a novel technique to boost training efficiency of a neural network. Our work is based on an excellent idea that whitening the inputs of neural networks can achieve a fast convergence speed. Given the well-known fact that independent components must be whitened, we introduce a novel Independent-Component (IC) layer before each weight layer, whose inputs would be made more independent. However, determining independent components is a computationally intensive task. To overcome this challenge, we propose to implement an IC layer by combining two popular techniques, Batch Normalization and Dropout, in a new manner that we can rigorously prove that Dropout can quadratically reduce the mutual information and linearly reduce the correlation between any pair of neurons with respect to the dropout layer parameter $p$. As demonstrated experimentally, the IC layer consistently outperforms the baseline approaches with more stable training process, faster convergence speed and better convergence limit on CIFAR10/100 and ILSVRC2012 datasets. The implementation of our IC layer makes us rethink the common practices in the design of neural networks. For example, we should not place Batch Normalization before ReLU since the non-negative responses of ReLU will make the weight layer updated in a suboptimal way, and we can achieve better performance by combining Batch Normalization and Dropout together as an IC layer.
Explorations and Lessons Learned in Building an Autonomous Formula SAE Car from Simulations
Venue: arXiv cs.RO 2019
Year: 2019

This paper describes the exploration and learnings during the process of developing a self-driving algorithm in simulation, followed by deployment on a real car. We specifically concentrate on the Formula Student Driverless competition. In such competitions, a formula race car, designed and built by students, is challenged to drive through previously unseen tracks that are marked by traffic cones. We explore and highlight the challenges associated with training a deep neural network that uses a single camera as input for inferring car steering angles in real-time. The paper explores in-depth creation of simulation, usage of simulations to train and validate the software stack and then finally the engineering challenges associated with the deployment of the system in real-world.