OpenRaiser/Intern-Atlas huggingface dataset:
Constrained deep neural network architecture search for IoT devices accounting hardware calibration
Venue: arXiv cs.LG 2019
Year: 2019

Deep neural networks achieve outstanding results in challenging image classification tasks. However, the design of network topologies is a complex task and the research community makes a constant effort in discovering top-accuracy topologies, either manually or employing expensive architecture searches. In this work, we propose a unique narrow-space architecture search that focuses on delivering low-cost and fast executing networks that respect strict memory and time requirements typical of Internet-of-Things (IoT) near-sensor computing platforms. Our approach provides solutions with classification latencies below 10ms running on a $35 device with 1GB RAM and 5.6GFLOPS peak performance. The narrow-space search of floating-point models improves the accuracy on CIFAR10 of an established IoT model from 70.64% to 74.87% respecting the same memory constraints. We further improve the accuracy to 82.07% by including 16-bit half types and we obtain the best accuracy of 83.45% by extending the search with model optimized IEEE 754 reduced types. To the best of our knowledge, we are the first that empirically demonstrate on over 3000 trained models that running with reduced precision pushes the Pareto optimal front by a wide margin. Under a given memory constraint, accuracy is improved by over 7% points for half and over 1% points further for running with the best model individual format.
Software architecture for YOLO, a creativity-stimulating robot
Venue: arXiv cs.RO 2019
Year: 2019

YOLO is a social robot designed and developed to stimulate creativity in children through storytelling activities. Children use it as a character in their stories. This article details the artificial intelligence software developed for YOLO. The implemented software schedules through several Creativity Behaviors to find the ones that stimulate creativity more effectively. YOLO can choose between convergent and divergent thinking techniques, two important processes of creative thought. These techniques were developed based on the psychological theories of creativity development and on research from creativity experts who work with children. Additionally, this software allows the creation of Social Behaviors that enable the robot to behave as a believable character. On top of our framework, we built 3 main social behavior parameters: Exuberant, Aloof, and Harmonious. These behaviors are meant to ease immersive play and the process of character creation. The 3 social behaviors were based on psychological theories of personality and developed using children's input during co-design studies. Overall, this work presents an attempt to design, develop, and deploy social robots that nurture intrinsic human abilities, such as the ability to be creative.
A simpler O(m log n) algorithm for branching bisimilarity on labelled transition systems
Venue: arXiv cs.LO 2019
Year: 2019

Branching bisimilarity is a behavioural equivalence relation on labelled transition systems that takes internal actions into account. It has the traditional advantage that algorithms for branching bisimilarity are more efficient than all algorithms for other weak behavioural equivalences, especially weak bisimilarity. With $m$ the number of transitions and $n$ the number of states, the classic $O(m n)$ algorithm was recently replaced by an $O(m (\log \lvert \mathit{Act}\rvert + \log n))$ algorithm, which is unfortunately rather complex. This paper combines its ideas with the ideas from Valmari. This results in a simpler algorithm with complexity $O(m \log n)$. Benchmarks show that this new algorithm is also faster and often far more memory efficient than its predecessors. This makes it the best option for branching bisimulation minimisation and preprocessing for weak bisimulation of LTSs.
On the Design and Analysis of a Biometric Authentication System using Keystroke Dynamics
Venue: arXiv cs.CR 2019
Year: 2019

This paper proposes a portable hardware token for user authentication, it is based on the use of keystroke dynamics to verify users in a bio-metric manner. The proposed approach allows for a multifactor authentication scheme in which users are not allowed access unless they provide the correct password and their unique bio-metric signature. The proposed system is implemented in hardware and its security is evaluated.
Fundamental Limits of Spectrum Sharing for NOMA-Based Cooperative Relaying Under a Peak Interference Constraint
Venue: arXiv cs.IT 2019
Year: 2019

Non-orthogonal multiple access (NOMA) and spectrum sharing (SS) are two emerging multiple access technologies for efficient spectrum utilization in future wireless communications standards. In this paper, we present the performance analysis of a NOMA-based cooperative relaying system (CRS) in an underlay spectrum sharing scenario, considering a peak interference constraint (PIC), where the peak interference inflicted by the secondary (unlicensed) network on the primary-user (licensed) receiver (PU-Rx) should be less than a predetermined threshold. In the proposed system the relay and the secondary-user receiver (SU-Rx) are equipped with multiple receive antennas and apply selection combining (SC), where the antenna with highest instantaneous signal-to-noise ratio (SNR) is selected, and maximal-ratio combining (MRC), for signal reception. Closed-form expressions are derived for the average achievable rate and outage probabilities for SS-based CRS-NOMA. These results show that for large values of peak interference power, the SS-based CRS-NOMA outperforms the CRS with conventional orthogonal multiple access (OMA) in terms of spectral efficiency. The effect of the interference channel on the system performance is also discussed, and in particular, it is shown that the interference channel between the secondary-user transmitter (SU-Tx) and the PU-Rx has a more severe effect on the average achievable rate as compared to that between the relay and the PU-Rx. A close agreement between the analytical and numerical results confirm the correctness of our rate and outage analysis.
Reservoir Topology in Deep Echo State Networks
Venue: arXiv cs.LG 2019
Year: 2019

Deep Echo State Networks (DeepESNs) recently extended the applicability of Reservoir Computing (RC) methods towards the field of deep learning. In this paper we study the impact of constrained reservoir topologies in the architectural design of deep reservoirs, through numerical experiments on several RC benchmarks. The major outcome of our investigation is to show the remarkable effect, in terms of predictive performance gain, achieved by the synergy between a deep reservoir construction and a structured organization of the recurrent units in each layer. Our results also indicate that a particularly advantageous architectural setting is obtained in correspondence of DeepESNs where reservoir units are structured according to a permutation recurrent matrix.
Paying Attention to Function Words
Venue: arXiv cs.CL 2019
Year: 2019

All natural languages exhibit a distinction between content words (like nouns and adjectives) and function words (like determiners, auxiliaries, prepositions). Yet surprisingly little has been said about the emergence of this universal architectural feature of natural languages. Why have human languages evolved to exhibit this division of labor between content and function words? How could such a distinction have emerged in the first place? This paper takes steps towards answering these questions by showing how the distinction can emerge through reinforcement learning in agents playing a signaling game across contexts which contain multiple objects that possess multiple perceptually salient gradable properties.
An Exploratory Study of How Specialists Deal with Testing in Data Stream Processing Applications
Venue: arXiv cs.SE 2019
Year: 2019

[Background] Nowadays, there is a massive growth of data volume and speed in many types of systems. It introduces new needs for infrastructure and applications that have to handle streams of data with low latency and high throughput. Testing applications that process such data streams has become a significant challenge for engineers. Companies are adopting different approaches to dealing with this issue. Some have developed their own solutions for testing, while others have adopted a combination of existing testing techniques. There is no consensus about how or in which contexts such solutions can be implemented. [Aims] To the best of our knowledge, there is no consolidated literature on that topic. The present paper is an attempt to fill this gap by conducting an exploratory study with practitioners. [Method] We used qualitative methods in this research, in particular interviews and survey. We interviewed 12 professionals who work in projects related to data streams, and also administered a questionnaire with other 105 professionals. The interviews went through a transcription and coding process, and the questionnaires were analysed to reinforce findings. [Results] This study presents current practices around software testing in data stream processing applications. These practices involve methodologies, techniques, and tools. [Conclusions] Our main contribution is a compendium of alternatives for many of the challenges that arise when testing streaming applications from a state-of-the-practice perspective.
On speeding up an asymptotic-analysis-based homogenisation scheme for designing gradient porous structured materials using a zoning strategy
Venue: arXiv cs.CE 2019
Year: 2019

Gradient porous structured materials possess significant potential of being applied in many engineering fields. To accelerate the design process of infill graded microstructures, a novel asymptotic homogenisation topology optimisation method was proposed by Zhu et al.(2019), aiming for 1) significantly enriching the pool of representable graded microstructures; 2) deriving an homogenised formulation for stress analysis in consistency with fine-scale results. But the work is severely confined from being widely applied, mainly due to the following two reasons. Firstly, to circumvent the macroscopically pointwise computation for solving various microscopic cell problems, linearisation had to be adopted for its numerical implementation, and this significantly reduces the design freedom. Secondly, lacking of sensitive analysis, genetic algorithm was chosen for optimisation, inevitably decreasing the computational efficiency. To address these bottleneck challenging issues, a zoning scheme empowered by computational parallelism is introduced, and the sensitivity analysis associated with the new asymptotic framework is conducted. Through comparisons with fine-scale simulation results, the proposed algorithm is shown to be an effective tool for evaluating the mechanical behaviour of graded microstructures. As an optimisation tool, the mapping function takes a concise and explicit form. But its parameterisation still needs further investigation, so as to improve the solution optimality of the present approach, especially in comparison with another recently proposed method by Groen et al (2018). The high computational efficiency shown by the proposed scheme is also demonstrated with optimisation results for three-dimensional graded microstructures, which are not frequently discussed in literature, possibly because of the high computational cost generated.
Atalaya at TASS 2019: Data Augmentation and Robust Embeddings for Sentiment Analysis
Venue: arXiv cs.CL 2019
Year: 2019

In this article we describe our participation in TASS 2019, a shared task aimed at the detection of sentiment polarity of Spanish tweets. We combined different representations such as bag-of-words, bag-of-characters, and tweet embeddings. In particular, we trained robust subword-aware word embeddings and computed tweet representations using a weighted-averaging strategy. We also used two data augmentation techniques to deal with data scarcity: two-way translation augmentation, and instance crossover augmentation, a novel technique that generates new instances by combining halves of tweets. In experiments, we trained linear classifiers and ensemble models, obtaining highly competitive results despite the simplicity of our approaches.
Guided Attention Network for Object Detection and Counting on Drones
Venue: arXiv cs.CV 2019
Year: 2019

Object detection and counting are related but challenging problems, especially for drone based scenes with small objects and cluttered background. In this paper, we propose a new Guided Attention Network (GANet) to deal with both object detection and counting tasks based on the feature pyramid. Different from the previous methods relying on unsupervised attention modules, we fuse different scales of feature maps by using the proposed weakly-supervised Background Attention (BA) between the background and objects for more semantic feature representation. Then, the Foreground Attention (FA) module is developed to consider both global and local appearance of the object to facilitate accurate localization. Moreover, the new data argumentation strategy is designed to train a robust model in various complex scenes. Extensive experiments on three challenging benchmarks (i.e., UAVDT, CARPK and PUCPR+) show the state-of-the-art detection and counting performance of the proposed method compared with existing methods.
Experimental Evaluation of Algorithms for Computing Quasiperiods
Venue: arXiv cs.DS 2019
Year: 2019

Quasiperiodicity is a generalization of periodicity that was introduced in the early 1990s. Since then, dozens of algorithms for computing various types of quasiperiodicity were proposed. Our work is a step towards answering the question: "Which algorithm for computing quasiperiods to choose in practice?". The central notions of quasiperiodicity are covers and seeds. We implement algorithms for computing covers and seeds in the original and in new simplified versions and compare their efficiency on various types of data. We also discuss other known types of quasiperiodicity, distinguish partial covers as currently the most promising for large real-world data, and check their effectiveness using real-world data.
Understanding the Twitter Usage of Science Citation Index (SCI) Journals
Venue: arXiv cs.DL 2019
Year: 2019

This paper investigates the Twitter interaction patterns of journals from the Science Citation Index (SCI) of Master Journal List (MJL). A total of 953,253 tweets extracted from 857 journal accounts, were analyzed in this study. Findings indicate that SCI journals interacted more with each other but much less with journals from other citation indices. The network structure of the communication graph resembled a tight crowd network, with Nature journals playing a major part. Information sources such as news portals and scientific organizations were mentioned more in tweets, than academic journal Twitter accounts. Journals with high journal impact factors (JIFs) were found to be prominent hubs in the communication graph. Differences were found between the Twitter usage of SCI journals with Humanities and Social Sciences (HSS) journals.
Usefulness of Instructor Annotations on Flipped Learning Preparation Video System
Venue: arXiv cs.CY 2019
Year: 2019

Flipped learning is a method that flips in/out class activities to make lectures learner-centered. In flipped learning, comments from learners on preparation material are useful information for instructors to consider before deciding in-class topics. Thus, we arrive at the notion that receiving comments from instructors will be effective for learners watching the video. By including annotations from instructors, we propose to improve the quality of content for learners and thus enhance learners' motivation and study satisfaction. To achieve this, we introduced "Steering Mark," a tool that enables learners to easily grasp the overall structure of a video, to the video learning system. We examined the effectiveness and influence of Steering Mark through an experiment with 34 undergraduate learners. As a result, Steering Mark was found to be useful in improving the quality of video content for learners.
Iterative division in the Distributive Full Non-associative Lambek Calculus
Venue: arXiv cs.LO 2019
Year: 2019

We study an extension of the Distributive Full Non-associative Lambek Calculus with iterative division operators. The iterative operators can be seen as representing iterative composition of linguistic resources or of actions. A complete axiomatization of the logic is provided and decidability is established via a proof of the finite model property.
HumanGAN: generative adversarial network with human-based discriminator and its evaluation in speech perception modeling
Venue: arXiv cs.SD 2019
Year: 2019

We propose the HumanGAN, a generative adversarial network (GAN) incorporating human perception as a discriminator. A basic GAN trains a generator to represent a real-data distribution by fooling the discriminator that distinguishes real and generated data. Therefore, the basic GAN cannot represent the outside of a real-data distribution. In the case of speech perception, humans can recognize not only human voices but also processed (i.e., a non-existent human) voices as human voice. Such a human-acceptable distribution is typically wider than a real-data one and cannot be modeled by the basic GAN. To model the human-acceptable distribution, we formulate a backpropagation-based generator training algorithm by regarding human perception as a black-boxed discriminator. The training efficiently iterates generator training by using a computer and discrimination by crowdsourcing. We evaluate our HumanGAN in speech naturalness modeling and demonstrate that it can represent a human-acceptable distribution that is wider than a real-data distribution.
A Survey of Binary Code Similarity
Venue: arXiv cs.CR 2019
Year: 2019

Binary code similarity approaches compare two or more pieces of binary code to identify their similarities and differences. The ability to compare binary code enables many real-world applications on scenarios where source code may not be available such as patch analysis, bug search, and malware detection and analysis. Over the past 20 years numerous binary code similarity approaches have been proposed, but the research area has not yet been systematically analyzed. This paper presents a first survey of binary code similarity. It analyzes 61 binary code similarity approaches, which are systematized on four aspects: (1) the applications they enable, (2) their approach characteristics, (3) how the approaches are implemented, and (4) the benchmarks and methodologies used to evaluate them. In addition, the survey discusses the scope and origins of the area, its evolution over the past two decades, and the challenges that lie ahead.
Breaking the Data Barrier: Towards Robust Speech Translation via Adversarial Stability Training
Venue: arXiv cs.CL 2019
Year: 2019

In a pipeline speech translation system, automatic speech recognition (ASR) system will transmit errors in recognition to the downstream machine translation (MT) system. A standard machine translation system is usually trained on parallel corpus composed of clean text and will perform poorly on text with recognition noise, a gap well known in speech translation community. In this paper, we propose a training architecture which aims at making a neural machine translation model more robust against speech recognition errors. Our approach addresses the encoder and the decoder simultaneously using adversarial learning and data augmentation, respectively. Experimental results on IWSLT2018 speech translation task show that our approach can bridge the gap between the ASR output and the MT input, outperforms the baseline by up to 2.83 BLEU on noisy ASR output, while maintaining close performance on clean text.
EEG-Based Driver Drowsiness Estimation Using Feature Weighted Episodic Training
Venue: arXiv cs.HC 2019
Year: 2019

Drowsy driving is pervasive, and also a major cause of traffic accidents. Estimating a driver's drowsiness level by monitoring the electroencephalogram (EEG) signal and taking preventative actions accordingly may improve driving safety. However, individual differences among different drivers make this task very challenging. A calibration session is usually required to collect some subject-specific data and tune the model parameters before applying it to a new subject, which is very inconvenient and not user-friendly. Many approaches have been proposed to reduce the calibration effort, but few can completely eliminate it. This paper proposes a novel approach, feature weighted episodic training (FWET), to completely eliminate the calibration requirement. It integrates two techniques: feature weighting to learn the importance of different features, and episodic training for domain generalization. Experiments on EEG-based driver drowsiness estimation demonstrated that both feature weighting and episodic training are effective, and their integration can further improve the generalization performance. FWET does not need any labelled or unlabelled calibration data from the new subject, and hence could be very useful in plug-and-play brain-computer interfaces.
Message Scheduling for Performant, Many-Core Belief Propagation
Venue: arXiv cs.DC 2019
Year: 2019

Belief Propagation (BP) is a message-passing algorithm for approximate inference over Probabilistic Graphical Models (PGMs), finding many applications such as computer vision, error-correcting codes, and protein-folding. While general, the convergence and speed of the algorithm has limited its practical use on difficult inference problems. As an algorithm that is highly amenable to parallelization, many-core Graphical Processing Units (GPUs) could significantly improve BP performance. Improving BP through many-core systems is non-trivial: the scheduling of messages in the algorithm strongly affects performance. We present a study of message scheduling for BP on GPUs. We demonstrate that BP exhibits a tradeoff between speed and convergence based on parallelism and show that existing message schedulings are not able to utilize this tradeoff. To this end, we present a novel randomized message scheduling approach, Randomized BP (RnBP), which outperforms existing methods on the GPU.
Wider Networks Learn Better Features
Venue: arXiv cs.LG 2019
Year: 2019

Transferability of learned features between tasks can massively reduce the cost of training a neural network on a novel task. We investigate the effect of network width on learned features using activation atlases --- a visualization technique that captures features the entire hidden state responds to, as opposed to individual neurons alone. We find that, while individual neurons do not learn interpretable features in wide networks, groups of neurons do. In addition, the hidden state of a wide network contains more information about the inputs than that of a narrow network trained to the same test accuracy. Inspired by this observation, we show that when fine-tuning the last layer of a network on a new task, performance improves significantly as the width of the network is increased, even though test accuracy on the original task is independent of width.