Reviewerly/exHarmony huggingface dataset:
Minimal random code learning : Getting bits back from compressed model parameters

© 7th International Conference on Learning Representations, ICLR 2019. All Rights Reserved. While deep neural networks are a highly successful model class, their large memory footprint puts considerable strain on energy consumption, communication bandwidth, and storage requirements. Consequently, model size reduction has become an utmost goal in deep learning. A typical approach is to train a set of deterministic weights, while applying certain techniques such as pruning and quantization, in order that the empirical weight distribution becomes amenable to Shannon-style coding schemes. However, as shown in this paper, relaxing weight determinism and using a full variational distribution over weights allows for more efficient coding schemes and consequently higher compression rates. In particular, following the classical bits-back argument, we encode the network weights using a random sample, requiring only a number of bits corresponding to the Kullback-Leibler divergence between the sampled variational distribution and the encoding distribution. By imposing a constraint on the Kullback-Leibler divergence, we are able to explicitly control the compression rate, while optimizing the expected loss on the training set. The employed encoding scheme can be shown to be close to the optimal information-theoretical lower bound, with respect to the employed variational family. Our method sets new state-of-the-art in neural network compression, as it strictly dominates previous approaches in a Pareto sense: On the benchmarks LeNet-5/MNIST and VGG-16/CIFAR-10, our approach yields the best test performance for a fixed memory budget, and vice versa, it achieves the highest compression rates for a fixed test performance.
A Bayesian Conjugate Gradient Method (with Discussion)

A fundamental task in numerical computation is the solution of large linear systems.The conjugate gradient method is an iterative method which offers rapid convergence to the solution, particularly when an effective preconditioner is employed.However, for more challenging systems a substantial error can be present even after many iterations have been performed.The estimates obtained in this case are of little value unless further information can be provided about, for example, the magnitude of the error.In this paper we propose a novel statistical model for this error, set in a Bayesian framework.Our approach is a strict generalisation of the conjugate gradient method, which is recovered as the posterior mean for a particular choice of prior.The estimates obtained are analysed with Krylov subspace methods and a contraction result for the posterior is presented.The method is then analysed in a simulation study as well as being applied to a challenging problem in medical imaging.
Elasticutor

Elasticity is highly desirable for stream systems to guarantee low latency against workload dynamics, such as surges in arrival rate and fluctuations in data distribution. Existing systems achieve elasticity using a resource-centric approach that repartitions keys across the parallel instances, i.e., executors, to balance the workload and scale operators. However, such operator-level repartitioning requires global synchronization and prohibits rapid elasticity. We propose an executor-centric approach that avoids operator-level key repartitioning and implements executors as the building blocks of elasticity. By this new approach, we design the Elasticutor framework with two level of optimizations: i) a novel implementation of executors, i.e., elastic executors, that perform elastic multi-core execution via efficient intra-executor load balancing and executor scaling and ii) a global model-based scheduler that dynamically allocates CPU cores to executors based on the instantaneous workloads. We implemented a prototype of Elasticutor and conducted extensive experiments. We show that Elasticutor doubles the throughput and achieves up to two orders of magnitude lower latency than previous methods for dynamic workloads of real-world applications.
Playgol: Learning Programs Through Play

Children learn though play. We introduce the analogous idea of learning programs through play. In this approach, a program induction system (the learner) is given a set of user-supplied build tasks and initial background knowledge (BK). Before solving the build tasks, the learner enters an unsupervised playing stage where it creates its own play tasks to solve, tries to solve them, and saves any solutions (programs) to the BK. After the playing stage is finished, the learner enters the supervised building stage where it tries to solve the build tasks and can reuse solutions learnt whilst playing. The idea is that playing allows the learner to discover reusable general programs on its own which can then help solve the build tasks. We claim that playing can improve learning performance. We show that playing can reduce the textual complexity of target concepts which in turn reduces the sample complexity of a learner. We implement our idea in Playgol, a new inductive logic programming system. We experimentally test our claim on two domains: robot planning and real-world string transformations. Our experimental results suggest that playing can substantially improve learning performance.
Developing multi-sensor drones for geological mapping and mineral exploration: setup and first results from the MULSEDRO project

The use of Unmanned Aerial Systems (UAS), also known as drones, is becoming increasingly important for geological applications. Thanks to lower operational costs and ease of use, UAS offer an alternative approach to aircraft-based and ground-based geoscientific measurements (Colomina &amp; Molina 2014). Magnetic and hyperspectral UAS surveys hold particular promise for mineral exploration, and several groups have recently published studies of magnetic data collected by UAS for such applications (Malehmir et al. 2017; Cunningham et al. 2018), although equivalent studies using hyperspectral data are still rare (Kirsch et al. 2018). Combining both techniques is particularly useful. Magnetic measurements play an important role in mineral exploration, since magnetisation in rocks is mainly associated with magnetite and other iron minerals, which can be used in mapping and targeting of mineral deposits (Dentith &amp; Mudge 2014). Hyperspectral imaging (HSI) is a powerful exploration and mapping technique in areas where the rock surface is well-exposed, and where geological units and mineral compositions can be estimated from spectral features of the electromagnetic spectrum in the visual and infrared range.
Large-Scale Object Mining for Object Discovery from Unlabeled Video

This paper addresses the problem of object discovery from unlabeled driving videos captured in a realistic automotive setting. Identifying recurring object categories in such raw video streams is a very challenging problem. Not only do object candidates first have to be localized in the input images, but many interesting object categories occur relatively infrequently. Object discovery will therefore have to deal with the difficulties of operating in the long tail of the object distribution. We demonstrate the feasibility of performing fully automatic object discovery in such a setting by mining object tracks using a generic object tracker. In order to facilitate further research in objet discovery, we release a collection of more than 360,000 automatically mined object tracks from 10 + hours of video data (560,000 frames). We use this dataset to evaluate the suitability of different feature representations and clustering strategies for object discovery.
An ADS-B Message Authentication Method Based on Certificateless Short Signature

The automatic dependent surveillance—broadcast (ADS-B) system adopts an open communication mode, and the lack of designed-in security measures in the ADS-B system makes it vulnerable to various types of attacks (jamming, spoofing, etc.). In view of the low-bandwidth and less-data-bit features of the ADS-B, this paper studies the integrity and authenticity of information by signing messages and proposes an ADS-B message authentication method based on certificateless short signature. This method uses short signature and does not require certificate management and has efficient performance. Compared with the existing approach, the computation costs of the proposed method in the signature phase are reduced by 1/2, and the signature length is reduced by 3/4. Additionally, we used the extended NS2 simulation platform to simulate 1090ES data link in different scenarios of the network; the simulation results show that our solution is suitable for minimum operational performance standard of ADS-B.
A Reinforcement Learning-Based QAM/PSK Symbol Synchronizer

Machine Learning (ML) based on supervised and unsupervised learning models has been recently applied in the telecommunication field.However, such techniques rely on application-specific large datasets and the performance deteriorates if the statistics of the inference data changes over time.Reinforcement Learning (RL) is a solution to these issues because it is able to adapt its behavior to the changing statistics of the input data.In this work, we propose the design of an RL Agent able to learn the behavior of a Timing Recovery Loop (TRL) through the Q-Learning algorithm.The Agent is compatible with popular PSK and QAM formats.We validated the RL synchronizer by comparing it to the Mueller and Müller TRL in terms of Modulation Error Ratio (MER) in a noisy channel scenario.The results show a good trade-off in terms of MER performance.The RL based synchronizer loses less than 1 dB of MER with respect to the conventional one but it is able to adapt its behavior to different modulation formats without the need of any tuning for the system parameters.
Generalization of Reinforcement Learners with Working and Episodic Memory

Memory is an important aspect of intelligence and plays a role in many deep reinforcement learning models. However, little progress has been made in understanding when specific memory systems help more than others and how well they generalize. The field also has yet to see a prevalent consistent and rigorous approach for evaluating agent performance on holdout data. In this paper, we aim to develop a comprehensive methodology to test different kinds of memory in an agent and assess how well the agent can apply what it learns in training to a holdout set that differs from the training set along dimensions that we suggest are relevant for evaluating memory-specific generalization. To that end, we first construct a diverse set of memory tasks that allow us to evaluate test-time generalization across multiple dimensions. Second, we develop and perform multiple ablations on an agent architecture that combines multiple memory systems, observe its baseline models, and investigate its performance against the task suite.
A Blind CNN Denoising Model for Random-Valued Impulse Noise

Denoising convolutional neural networks (DnCNNs), initially developed for Gaussian noise removal, are powerful nonlinear mapping models in image processing. After changes in training data, they can be used for suppression of random-valued impulse noise (RVIN) with excellent results. To achieve favorable denoising performance, however, it is necessary to have an accurate perception of the noise ratio so that the most suitable DnCNN can be chosen for denoising. Thus, this model is severely limited in flexibility. To address this problem, we propose a blind CNN model for RVIN denoising with a flexible noise ratio predictor (NRP) as an indicator. Some patches are randomly selected from the RVIN-corrupted test image, and feature vectors that indicate whether the center pixel is contaminated or not are extracted by the predictor. These feature vectors are composed of multiple statistics, namely, the multiple rank-ordered absolute differences (ROADs), the clean pixel median deviation (CPMD), and the edge pixel difference (EPD). They are rapidly mapped to noise/clean (1 for noise, 0 for clean) labels by the pre-trained noise detector (the key component of our NRP). According to the ratio of the obtained noisy labels to the total number of selected patches, the predictor provides the noise ratio of the whole image. From the output of the NRP, i.e., the predicted noise ratio, the most appropriate DnCNN specifically trained for this noise ratio is exploited for denoising. Under the guidance of the NRP, the proposed method has the ability to handle unknown noise ratios. Simulation results indicate that our blind denoising CNN model achieves state-of-the-art performance in terms of both execution efficiency and restoration results.
Cross-Lingual Lemmatization and Morphology Tagging with Two-Stage Multilingual BERT Fine-Tuning

We present our CHARLES-SAARLAND system for the SIGMORPHON 2019 Shared Task on Crosslinguality and Context in Morphology, in task 2, Morphological Analysis and Lemmatization in Context.We leverage the multilingual BERT model and apply several fine-tuning strategies introduced by UDify demonstrating exceptional evaluation performance on morpho-syntactic tasks.Our results show that fine-tuning multilingual BERT on the concatenation of all available treebanks allows the model to learn cross-lingual information that is able to boost lemmatization and morphology tagging accuracy over fine-tuning it purely monolingually.Unlike UDify, however, we show that when paired with additional character-level and word-level LSTM layers, a second stage of fine-tuning on each treebank individually can improve evaluation even further.Out of all submissions for this shared task, our system achieves the highest average accuracy and f1 score in morphology tagging and places second in average lemmatization accuracy.
Rigorous engineering for hardware security: Formal modelling and proof in the CHERI design and implementation process

The root causes of many security vulnerabilities include a pernicious combination of two problems, often regarded as inescapable aspects of computing. First, the protection mechanisms provided by the mainstream processor architecture and C/C++ language abstractions, dating back to the 1970s and before, provide only coarse-grain virtual-memory-based protection. Second, mainstream system engineering relies almost exclusively on test-and-debug methods, with (at best) prose specifications. These methods have historically sufficed commercially for much of the computer industry, but they fail to prevent large numbers of exploitable bugs, and the security problems that this causes are becoming ever more acute.In this paper we show how more rigorous engineering methods can be applied to the development of a new security-enhanced processor architecture, with its accompanying hardware implementation and software stack. We use formal models of the complete instruction-set architecture (ISA) at the heart of the design and engineering process, both in lightweight ways that support and improve normal engineering practice - as documentation, in emulators used as a test oracle for hardware and for running software, and for test generation - and for formal verification. We formalise key intended security properties of the design, and establish that these hold with mechanised proof. This is for the same complete ISA models (complete enough to boot operating systems), without idealisation.We do this for CHERI, an architecture with hardware capabilities that supports fine-grained memory protection and scalable secure compartmentalisation, while offering a smooth adoption path for existing software. CHERI is a maturing research architecture, developed since 2010, with work now underway on an Arm industrial prototype to explore its possible adoption in mass-market commercial processors. The rigorous engineering work described here has been an integral part of its development to date, enabling more rapid and confident experimentation, and boosting confidence in the design.
A real-time gesture recognition system using near-infrared imagery

Visual hand gesture recognition systems are promising technologies for Human Computer Interaction, as they allow a more immersive and intuitive interaction. Most of these systems are based on the analysis of skeleton information, which is in turn inferred from color, depth, or near-infrared imagery. However, the robust extraction of skeleton information from images is only possible for a subset of hand poses, which restricts the range of gestures that can be recognized. In this paper, a real-time hand gesture recognition system based on a near-infrared device is presented, which directly analyzes the infrared imagery to infer static and dynamic gestures, without using skeleton information. Thus, a much wider range of hand gestures can be recognized in comparison with skeleton-based approaches. To validate the proposed system, a new dataset of near-infrared imagery has been created, from which good results that outperform other state-of-the-art strategies have been obtained.
Enhancing timeliness of drug overdose mortality surveillance: A machine learning approach

Timely data is key to effective public health responses to epidemics. Drug overdose deaths are identified in surveillance systems through ICD-10 codes present on death certificates. ICD-10 coding takes time, but free-text information is available on death certificates prior to ICD-10 coding. The objective of this study was to develop a machine learning method to classify free-text death certificates as drug overdoses to provide faster drug overdose mortality surveillance.Using 2017-2018 Kentucky death certificate data, free-text fields were tokenized and features were created from these tokens using natural language processing (NLP). Word, bigram, and trigram features were created as well as features indicating the part-of-speech of each word. These features were then used to train machine learning classifiers on 2017 data. The resulting models were tested on 2018 Kentucky data and compared to a simple rule-based classification approach. Documented code for this method is available for reuse and extensions: https://github.com/pjward5656/dcnlp.The top scoring machine learning model achieved 0.96 positive predictive value (PPV) and 0.98 sensitivity for an F-score of 0.97 in identification of fatal drug overdoses on test data. This machine learning model achieved significantly higher performance for sensitivity (p<0.001) than the rule-based approach. Additional feature engineering may improve the model's prediction. This model can be deployed on death certificates as soon as the free-text is available, eliminating the time needed to code the death certificates.Machine learning using natural language processing is a relatively new approach in the context of surveillance of health conditions. This method presents an accessible application of machine learning that improves the timeliness of drug overdose mortality surveillance. As such, it can be employed to inform public health responses to the drug overdose epidemic in near-real time as opposed to several weeks following events.
Profiling analysis of DISC personality traits based on Twitter posts in Bahasa Indonesia

Data in the timeline of social media users consists of data in the form of text, images, audio, and video. Large and unstructured data in social media can be processed using various techniques such as text processing or image processing. In this study, the processed text data is used to classify Twitter users’ personality based on the DISC framework. Out of the initial collected 292 users, we semi-automatically filtered them for only personal accounts with Indonesian language posts. For being able to observe and assess a user’s personality out of their tweets choice of words, we made relevant keyword vocabularies corresponding to DISC framework and theory. There are four experiment scenarios done in this study, with variations on whether the keywords and text data are stemmed or not, and the keywords frequency calculation being weighted or not. Weighting the keywords using the current number in calculation based on their level does not show positive results, neither does stemming as the best results are shown by the not stemmed and not weighted scenario. This study is a preliminary research for an automatic profiling system which employs a combination of Natural Language Processing and Machine Learning approaches.
Introducing phonetic information to speaker embedding for speaker verification

Abstract Phonetic information is one of the most essential components of a speech signal, playing an important role for many speech processing tasks. However, it is difficult to integrate phonetic information into speaker verification systems since it occurs primarily at the frame level while speaker characteristics typically reside at the segment level. In deep neural network-based speaker verification, existing methods only apply phonetic information to the frame-wise trained speaker embeddings. To improve this weakness, this paper proposes phonetic adaptation and hybrid multi-task learning and further combines these into c-vector and simplified c-vector architectures. Experiments on National Institute of Standards and Technology (NIST) speaker recognition evaluation (SRE) 2010 show that the four proposed speaker embeddings achieve better performance than the baseline. The c-vector system performs the best, providing over 30% and 15% relative improvements in equal error rate (EER) for the core-extended and 10 s–10 s conditions, respectively. On the NIST SRE 2016, 2018, and VoxCeleb datasets, the proposed c-vector approach improves the performance even when there is a language mismatch within the training sets or between the training and evaluation sets. Extensive experimental results demonstrate the effectiveness and robustness of the proposed methods.
The risk of block chain financial market based on particle swarm optimization

In order to effectively measure the risk of China's block-chain financial market, in this study, combined with KMV model, which is widely used in the world, the risk prediction model suitable for China's financial market has been found under the revision of particle swarm optimization algorithm. Based on the analysis of the data of more than 2,900 A-share companies in Beijing, Shanghai, and Guangzhou, the optimal short-term liabilities and the optimal long-term liabilities coefficients of 3.68 and 1.72 for China's financial market have been found. Through empirical analysis and verification, the ROC curve shows that the short-term debt and long-term debt coefficients 1 and 0.5 of the traditional KMV model cannot distinguish ST shares from non-ST shares in China's financial market, while 3.68 and 1.72 can distinguish very well. It is proved that the modified KMV model based on particle swarm optimization algorithm can identify the credit risk of Listed Companies in China's block chain financial market very well.