Reviewerly/exHarmony huggingface dataset:
VML-MOC: Segmenting a Multiply Oriented and Curved Handwritten Text Line Dataset

This paper publishes a natural and very complicated dataset of handwritten documents with multiply oriented and curved text lines, namely VML-MOC dataset. These text lines were written as remarks on the page margins by different writers over the years. They appear at different locations within the orientations that range between 0° and 180° or as curvilinear forms. We evaluate a multi-oriented Gaussian based method to segment these handwritten text lines that are skewed or curved in any orientation. It achieves a mean pixel Intersection over Union score of 80.96% on the test documents. The results are compared with the results of a single-oriented Gaussian based text line segmentation method.
Covariance Pre-Integration for Delayed Measurements in Multi-Sensor Fusion

Delay compensation in filter based sensor fusion frameworks for multiple sensors with varying delays and different rates quickly results in large computational overhead should the delayed measurements be incorporated in a statistically meaningful way. Even more so if high rate propagation sensors (e.g. IMU) are used. This work presents an approach to implement such frameworks with significant complexity reduction compared to standard implementations. We set particular focus on the state covariance propagation as this chain of re-computations (i.e. FPF <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">T</sup> + Q per propagation step) upon a delayed update is the dominant bottleneck. We draw our inspiration from the scattering theory and propose a method which projects the idea of wave propagation to an efficient concatenation of covariance propagation steps between filter updates. Through this approach, we reach a speed-up of more than a factor of 10 for the covariance propagation and render the computational complexity independent of the number of propagation steps between filter updates. We evaluated our method in simulation and with real data.
Self-trained eXtreme Gradient Boosting Trees

Semi-Supervised Learning (SSL) is an ever-growing research area offering a powerful set of methods, either single or multi-view, for exploiting both labeled and unlabeled instances in the most effective manner. Self-training is a representative SSL algorithm which has been efficiently implemented for solving several classification problems in a wide range of scientific fields. Moreover, self-training has served as the base for the development of several self-labeled methods. In addition, gradient boosting is an advanced machine learning technique, a boosting algorithm for both classification and regression problems, which produces a predictive model in the form of decision trees. In this context, the principal objective of this paper is to put forward an improved self-training algorithm for classification tasks utilizing the efficacy of eXtreme Gradient Boosting (XGBoost) trees in a self-labeled scheme in order to build a highly accurate and robust classification model. A number of experiments on benchmark datasets were executed demonstrating the superiority of the proposed method over representative semi-supervised methods, as statistically verified by the Friedman non-parametric test.
Nickel to Lego

Many companies offer automatic speech recognition or Speech-to-Text APIs for use in diverse applications. However, audio classification algorithms trained with deep neural networks (DNNs) can sometimes misclassify adversarial examples, posing a significant threat to critical applications. In this paper, we present a novel way to create adversarial audio examples using a genetic algorithm. Our algorithm creates adversarial examples by iteratively adding perturbations to the original audio signal. Unlike most white-box adversarial example generations, our approach does not require knowledge about the target DNN's model and parameters (black-box) and heavy computational power of GPU resources. To show the feasibility of the proposed idea, we implement a tool called, Foolgle, using a genetic algorithm that performs untargeted attacks to create adversarial audio examples and evaluate those with the state-of-the-art Google Cloud Speech-to-Text API. Our preliminary experiment results show that Foolgle deceives the API with a success probability of 86%.
Hardware implementation of a convolutional neural network using calculations in the residue number system

Modern convolutional neural networks architectures are very resource intensive which limits the possibilities for their wide practical application. We propose a convolutional neural network architecture in which the neural network is divided into hardware and software parts to increase performance and reduce the cost of implementation resources. We also propose to use the residue number system in the hardware part to implement the convolutional layer of the neural network for resource costs reducing. A numerical method for quantizing the filters coefficients of a convolutional network layer is proposed to minimize the influence of quantization noise on the calculation result in the residue number system and determine the bit-width of the filters coefficients. This method is based on scaling the coefficients by a fixed number of bits and rounding up and down. The operations used make it possible to reduce resources in hardware implementation due to the simplifying of their execution. All calculations in the convolutional layer are performed on numbers in a fixed-point format. Software simulations using Matlab 2017b showed that convolutional neural network with a minimum number of layers can be quickly and successfully trained. Hardware implementation using the field-programmable gate array Kintex7 xc7k70tfbg484-2 showed that the use of residue number system in the convolutional layer of the neural network reduces the hardware costs by 32.6% compared with the traditional approach based on the two’s complement representation. The research results can be applied to create effective video surveillance systems, for recognizing handwriting, individuals, objects and terrain.
Nested Relation Extraction with Iterative Neural Network

Natural language is used to describe objective facts, including simple relations like ""Jobs was the CEO of Apple"", and complex relations like ""the GDP of the United States in 2018 grew 2.9% compared with 2017". For the latter example, the growth rate relation is between two other relations. Due to the complex nature of language, this kind of nested relations is expressed frequently, especially in professional documents in fields like economics, finance, and biomedicine. But extracting nested relations is challenging, and research on this problem is almost vacant. In this paper, we formally formulate the nested relation extraction problem, and come up with a solution using Iterative Neural Network. Specifically, we observe that the nested relation structures can be expressed as a Directed Acyclic Graph (DAG), and propose the model to simultaneously consider the word sequence of natural language in the horizontal direction and the DAG structure in the vertical direction. Based on two nested relation extraction tasks, namely semantic causality relation extraction and formula extraction, we show that the proposed model works well on them. Moreover, we speed up the DAG-LSTM training significantly by a simple parallelization solution.
Learning to Match Ground Camera Image and UAV 3-D Model-Rendered Image Based on Siamese Network With Attention Mechanism

Different domain image sensors or imaging mechanisms provide cross-domain images when sensing the same scene. There is a domain shift between cross-domain images so that the image gap between different domains is the major challenge for measuring the similarity of the feature descriptors extracted from different domain images. Specifically, matching ground camera images and unmanned aerial vehicle (UAV) 3-D model-rendered images, which are two kinds of extremely challenging cross-domain images, is a way to establish indirectly the spatial relationship between 2-D and 3-D spaces. This provides a solution for the virtual-real registration of augmented reality (AR) in outdoor environments. However, during matching, handcrafted descriptors and existing learning-based feature descriptors limit the rendered images. In this letter, first, to learn robust and invariant 128-D local feature descriptors for ground camera and rendered images, we present a novel network structure, SiamAM-Net, which embeds the autoencoders with an attention mechanism into the Siamese network. Then, to narrow the gap between the cross-domain images during the optimizing of SiamAM-Net, we design an adaptive margin for the loss function. Finally, we match the ground camera-rendered images by using the learned local feature descriptors and explore the outdoor AR virtual-real registration. Experiments show that the local feature descriptors, learned by SiamAM-Net, are robust and achieve state-of-the-art retrieval performance on the cross-domain image data set of ground camera and rendered images. In addition, several outdoor AR applications also demonstrate the usefulness of the proposed outdoor AR virtual-real registration.
Improving Collaborative Filtering’s Rating Prediction Coverage in Sparse Datasets through the Introduction of Virtual Near Neighbors

Collaborative filtering creates personalized recommendations by considering ratings entered by users. Collaborative filtering algorithms initially detect users whose likings are alike, by exploring the similarity between ratings that have insofar been submitted. Users having a high degree of similarity regarding their ratings are termed near neighbors, and in order to formulate a recommendation for a user, her near neighbors' ratings are extracted and form the basis for the recommendation. Collaborative filtering algorithms however exhibit the problem commonly referred to as “gray sheep this pertains to the case where for some users no near neighbors can be identified, and hence no personalized recommendations can be computed. The “gray sheep” problem is more severe in sparse datasets, i.e. datasets where the number of ratings is small, compared to the number of items and users. In this paper, we address the “gray sheep” problem by introducing the concept of virtual near neighbors and a related algorithm for their creation on the basis of the existing ones. We evaluate the proposed algorithm, which is termed as CFVNN, using eight widely used datasets and considering two correlation metrics which are widely used in Collaborative Filtering research, namely the Pearson Correlation Coefficient and the Cosine Similarity. The results show that the proposed algorithm considerably leverages the capability of a Collaborative Filtering system to compute personalized recommendations in the context of sparse datasets, tackling thus efficiently the “gray sheep” problem. In parallel, the CFVNN algorithm achieves improvements in rating prediction quality, as this is expressed through the Mean Absolute Error and the Root Mean Square Error metrics.
A blockchain technology for protection and probative value preservation of vehicle driver data

The number of sensors in cars and other vehicles has significantly increased in the last few years. Such sensors are used to monitor vehicle telemetry as well as driver's biometric and physiological data, with the aim of increasing the safety of people and decreasing the number of accidents. Data monitored by these sensors may also be used by insurance companies or police for accident, or more in general, event reconstruction. Whatever is the use, this data should be protected against cybersecurity attacks in order to guarantee the privacy of the drivers, and to prevent malicious modifications performed with the aim of cheating the insurance companies or police. The goal of this paper is to propose a framework, based on a permissioned blockchain, that allows to both guarantee driver data protection and evidential property of data. The scenario explored considers data recorded inside the car, encrypted and transferred to a cloud storage service. The encrypted version of the data is also hashed and hash is stored in the blockchain, to guarantee data integrity and inalterability. At the same time, the blockchain can be used as an official ledger for preserving the probative value of data.
An Efficient Privacy and Integrity Preserving Data Aggregation Scheme for Multiple Applications in Wireless Sensor Networks

For wireless sensor networks (WSNs), data aggregation is an important method to reduce the energy consumption. Privacy-integrity preserving data aggregation is one of typical WSNs applications. Many secure aggregation schemes based on homomorphic encryption have been proposed in the past years. However, most of them cannot aggregate hybrid sensors' data into one in some real WSN applications, accordingly, they don't satisfy multi-application environments and heterogeneous sensors. To address this challenge, we present an efficient Privacy and Integrity preserving data aggregation scheme for Multiple Applications in WSNs (abbreviated as PIMA). The proposed PIMA employs the homomorphic Paillier encryption and the homomorphic MAC to not only protect data privacy but also check the aggregation data integrity. For multi-application environments, PIMA can efficiently aggregate hybrid sensors' data into one, and extract application-specific data from aggregated ciphertexts by Chinese Remainder Theorem (CRT) at the base station (BS). In additional, the proposed PIMA can early filter injected false data in aggregator, and achieve fault-tolerant in case that some sensors is malfunctioning or data losing. Detailed security analysis and performance evaluations show PIMA is privacy-enhanced, secure, robust and efficient. To the best of our knowledge, this is the first work that provides both integrity and privacy for multi-application in WSNs.
Semantic-aware short path adversarial training for cross-domain semantic segmentation

Recently, many methods have been proposed to deal with the problem of cross-domain semantic segmentation. Most of them choose to conduct domain adversarial training either on the high-level convolutional neural network (CNN) features or on the output segmentation maps. Typically, a relatively small weight is given to the adversarial training loss to avoid the problem of mode collapse. However, one potential weakness of these methods is that low-level CNN layers may receive little gradients for domain adaptation, especially when the network is deep. To address this problem, we propose to conduct an auxiliary adversarial training on the fused multi-level CNN features. Gradients for domain adaptation can thus flow into low-level CNN layers more easily along a shorter path. Experiments are conducted on the dataset of Cityscapes with using the source datasets of GTA5 and SYNTHIA, respectively. Quantitative and qualitative results certify the efficacy of the proposed method.
SID4VAM: A Benchmark Dataset With Synthetic Images for Visual Attention Modeling

A benchmark of saliency models performance with a synthetic image dataset is provided. Model performance is evaluated through saliency metrics as well as the influence of model inspiration and consistency with human psychophysics. SID4VAM is composed of 230 synthetic images, with known salient regions. Images were generated with 15 distinct types of low-level features (e.g. orientation, brightness, color, size...) with a target-distractor pop-out type of synthetic patterns. We have used Free-Viewing and Visual Search task instructions and 7 feature contrasts for each feature category. Our study reveals that state-of-the-art Deep Learning saliency models do not perform well with synthetic pattern images, instead, models with Spectral/Fourier inspiration outperform others in saliency metrics and are more consistent with human psychophysical experimentation. This study proposes a new way to evaluate saliency models in the forthcoming literature, accounting for synthetic images with uniquely low-level feature contexts, distinct from previous eye tracking image datasets.
Combining Active Learning with Self-train algorithm for classification of multimodal problems

In real-world cases, handling of both labeled and unlabeled data has raised the interest of several data scientists and Machine Learning engineers, leading to several demonstrations that apply data augmenting approaches to achieve an effective learning behavior. Although the majority of them propose either the exploitation of Semi-supervised or Active Learning approaches, individually, their combination has not been widely used. The ambition of this strategy is the efficient utilization of the available human knowledge relying along with the decisions driven by automated methods under a common framework. Thus, we conduct an empirical evaluation of such a combinatory approach over three problems, related to multimodal data operating under the pool-based scenario: Gender Identification, Recognition of Offensive Language and Emotion Detection. Into the proposed learning framework, which exploits initially labeled instances with small cardinality, our results prove the benefits of adopting such kind of semi-automated approaches regarding both the achieved predictive correctness and the reduced consumption of time and cost resources, as well as the smoothness of the learning convergence, mainly using ensemble classifiers.
Learning Fine-Grained Image Representations for Mathematical Expression Recognition

Optical character recognition is a key step towards automatically converting printed documents into electronic form. In this work, we consider the specialized task of mathematical expression recognition, which is characterized by a highly structured format and strict syntax rules, where the slightest mistake can lead to a very different meaning of the formula. To tackle this problem, we present a neural architecture based on a convolutional neural network focused specifically on fine-grained structures in the image. The obtained visual representations are used as an input to an encoder and an attention-based decoder module, trained jointly in an end-to-end manner. Given an input image, our model generates the underlying LaTeX markup that is able to perfectly describe the target mathematical formula. We conduct a thorough analysis of our model by examining the performance for different formula lengths and visualizing the attention maps of prediction examples. We demonstrate the effectiveness of our approach on the large-scale IM2LATEX-100K benchmark for mathematical expression recognition, where our model is able to outperform state-of-the-art methods, surpassing them by over 4% in image absolute accuracy.
Private Set Intersection with Linear Communication from General Assumptions

This work presents a hashing-based algorithm for Private Set Intersection (PSI) in the honest-but-curious setting. The protocol is generic, modular and provides both asymptotic and concrete efficiency improvements over existing PSI protocols. If each player has m elements, our scheme requires only $O(m \secpar)$ communication between the parties, where $\secpar$ is a security parameter. Our protocol builds on the hashing-based PSI protocol of Pinkas et al. (USENIX 2014, USENIX 2015), but we replace one of the sub-protocols (handling the cuckoo "stash'') with a special-purpose PSI protocol that is optimized for comparing sets of unbalanced size. This brings the asymptotic communication complexity of the overall protocol down from $ømega(m \secpar)$ to $O(m\secpar)$, and provides concrete performance improvements (10-15% reduction in communication costs) over Kolesnikov et al. (CCS 2016) under real-world parameter choices. Our protocol is simple, generic and benefits from the permutation-hashing optimizations of Pinkas et al. (USENIX 2015) and the Batched, Relaxed Oblivious Pseudo Random Functions of Kolesnikov et al. (CCS 2016).
Overview of the NASA Solar Irradiance Science Team (SIST) Program Special Section

Abstract Solar irradiance represents the dominant energy source heating the Earth's atmosphere and climate. Both total solar irradiance and spectral solar irradiance vary over the 11‐year solar cycle. Characterizing these variations with sufficient accuracy for climate studies over multidecadal timescales requires a combination of multiple observational data sets, solar activity proxies, and irradiance models. NASA established the Solar Irradiance Science Team (SIST) program in 2015 to pursue this goal using a range of technical approaches. This paper summarizes those investigations, whose results are reported in separate papers in a special section of this journal.
Face Recognition using Wavelets based Feature Extraction and PCA-L1 norm

Principal Component Analysis (PCA) is one of the key methods for solving data analysis problems with a large number of dimensions like face recognition. Nevertheless, classical PCA is based on L2-Norm which is very sensitive to noises. Recently, a new robust PCA approach has been proposed by replacing L2-Norm with L1-Norm (PCA-L1). However, PCA-L1 requires a lot of time to calculate the projection bases. To solve this problem, we propose to use a wavelets feature extraction method as pre-processing step to face recognition. Extensive experiments on two well-known face image datasets namely ORL and Georgia Tech Face-Database(GTFD), show that the proposed approach minimizes execution time and has a recognition rate up to 96.7% for ORL and 85% for GTFD.
Cluster validation in clustering‐based one‐class classification

Abstract Reconstruction‐based one‐class classification has shown to be very effective in a number of domains. This approach works by attempting to capture the underlying structure of the normal class, typically, by means of clusters of objects. It has the main disadvantage, however, that one has to indicate the number of clusters in advance, for this yields an efficient way of computing a clustering. In this paper, we introduce a new algorithm, OCKRA++, which achieves a better performance, by enhancing a clustering‐based one‐class ensemble classifier (OCKRA) with a cluster validity index that is used to set the best number of clusters during the classifier's training process. We have thoroughly tested OCKRA++ in a particular domain, namely masquerade detection. For this purpose, we have used the Windows‐Users and ‐Intruder simulation Logs data set repository, which contains 70 different masquerade data sets. We have found that OCKRA++ is currently the algorithm that achieves the best area under the curve, with a significant difference, in masquerade detection using the file system navigation approach.
Machine Reading Comprehension

In this paper, we study the machine reading comprehension of temporal order in text. Given a document of instruction sequences, a model aims to find out the most coherent sequences of activities matching the document among all answer candidates. To tackle the task, we proposeOrdMatch model, which is able to match each activity in a sequence to the corresponding instruction in the document and regularizes the partial order of activities to match the order of instructions. We evaluate the task using the RecipeQA dataset, which includes step-by-step instructions of cooking recipes. Our model outperforms the state-of-the-art models with a wide margin. The experimental results demonstrate the effectiveness of our novel ordering regularizer. Our code will be made available at \hrefhttps://github.com/Aolius/OrdMatch https://github.com/Aolius/OrdMatch.