Query: Enabling Technologies for Smart City Services and Applications

Relevant passages:
Smart cities in Europe <=SEP=> Urban performance currently depends not only on the city’s endowment of hard infrastructure (‘physical capital’), but also, and increasingly so, on the availability and quality of knowledge communication and social infrastructure (‘human and social capital’). The latter form of capital is decisive for urban competitiveness. Against this background, the concept of the ‘smart city’ has recently been introduced as a strategic device to encompass modern urban production factors in a common framework and, in particular, to highlight the importance of Information and Communication Technologies (ICTs) in the last 20 years for enhancing the competitive profile of a city.

The Mobile Sensing Platform: An Embedded Activity Recognition System <=SEP=> Activity-aware systems have inspired novel user interfaces and new applications in smart environments, surveillance, emergency response, and military missions. Systems that recognize human activities from body-worn sensors can further open the door to a world of healthcare applications, such as fitness monitoring, eldercare support, long-term preventive and chronic care, and cognitive assistance. Wearable systems have the advantage of being with the user continuously. So, for example, a fitness application could use real-time activity information to encourage users to perform opportunistic activities. Furthermore, the general public is more likely to accept such activity recognition systems because they are usually easy to turn off or remove.

Activity sensing in the wild: a field trial of ubifit garden <=SEP=> Recent advances in small inexpensive sensors, low-power processing, and activity modeling have enabled applications that use on-body sensing and machine learning to infer people's activities throughout everyday life. To address the growing rate of sedentary lifestyles, we have developed a system, UbiFit Garden, which uses these technologies and a personal, mobile display to encourage physical activity. We conducted a 3-week field trial in which 12 participants used the system and report findings focusing on their experiences with the sensing and activity inference. We discuss key implications for systems that use on-body sensing and activity inference to encourage physical activity.

The Internet of Things: A survey <=SEP=> This paper addresses the Internet of Things. Main enabling factor of this promising paradigm is the integration of several technologies and communications solutions. Identification and tracking technologies, wired and wireless sensor and actuator networks, enhanced communication protocols (shared with the Next Generation Internet), and distributed intelligence for smart objects are just the most relevant. As one can easily imagine, any serious contribution to the advance of the Internet of Things must necessarily be the result of synergetic activities conducted in different fields of knowledge, such as telecommunications, informatics, electronics and social science. In such a complex scenario, this survey is directed to those who want to approach this complex discipline and contribute to its development. Different visions of this Internet of Things paradigm are reported and enabling technologies reviewed. What emerges is that still major issues shall be faced by the research community. The most relevant among them are addressed in details. 2010 Elsevier B.V. All rights reserved.

Semantic Space: an infrastructure for smart spaces <=SEP=> Semantic Space is a pervasive computing infrastructure that exploits semantic Web technologies to support explicit representation, expressive querying, and flexible reasoning of contexts in smart spaces.

Comparison of Feedback Control Techniques for Torque-Vectoring Control of Fully Electric Vehicles <=SEP=> Fully electric vehicles (FEVs) with individually controlled powertrains can significantly enhance vehicle response to steering-wheel inputs in both steady-state and transient conditions, thereby improving vehicle handling and, thus, active safety and the fun-to-drive element. This paper presents a comparison between different torque-vectoring control structures for the yaw moment control of FEVs. Two second-order sliding-mode controllers are evaluated against a feedforward controller combined with either a conventional or an adaptive proportional-integral-derivative (PID) controller. Furthermore, the potential performance and robustness benefits arising from the integration of a body sideslip controller with the yaw rate feedback control system are assessed. The results show that all the evaluated controllers are able to significantly change the understeer behavior with respect to the baseline vehicle. The PID-based controllers achieve very good vehicle performance in steady-state and transient conditions, whereas the controllers based on the sliding-mode approach demonstrate a high level of robustness against variations in the vehicle parameters. The integrated sideslip controller effectively maintains the sideslip angle within acceptable limits in the case of an erroneous estimation of the tire-road friction coefficient.

Dyscalculia : Characteristics , Causes , and Treatments <=SEP=> Developmental Dyscalculia (DD) is a learning disorder affecting the ability to acquire school-level arithmetic skills, affecting approximately 3-6% of individuals. Progress in understanding the root causes of DD and how best to treat it have been impeded by lack of widespread research and variation in characterizations of the disorder across studies. However, recent years have witnessed significant growth in the field, and a growing body of behavioral and neuroimaging evidence now points to an underlying deficit in the representation and processing of numerical magnitude information as a potential core deficit in DD. An additional product of the recent progress in understanding DD is the resurgence of a distinction between ‘primary’ and ‘secondary’ developmental dyscalculia. The first appears related to impaired development of brain mechanisms for processing numerical magnitude information, while the latter refers to mathematical deficits stemming from external factors such as poor teaching, low socio-economic status, and behavioral attention problems or domain-general cognitive deficits. Increased awareness of this distinction going forward, in combination with longitudinal empirical research, offers great potential for deepening our understanding of the disorder and developing effective educational interventions.

Acquiring Hyponymy Relations from Web Documents <=SEP=> This paper describes an automatic method for acquiring hyponymy relations from HTML documents on the WWW. Hyponymy relations can play a crucial role in various natural language processing systems. Most existing acquisition methods for hyponymy relations rely on particular linguistic patterns, such as “ NP such as NP”. Our method, however, does not use such linguistic patterns, and we expect that our procedure can be applied to a wide range of expressions for which existing methods cannot be used. Our acquisition algorithm uses clues such as itemization or listing in HTML documents and statistical measures such as document frequencies and verb-noun co-occurrences.

Gender differences in narcissism: a meta-analytic review. <=SEP=> Despite the widely held belief that men are more narcissistic than women, there has been no systematic review to establish the magnitude, variability across measures and settings, and stability over time of this gender difference. Drawing on the biosocial approach to social role theory, a meta-analysis performed for Study 1 found that men tended to be more narcissistic than women (d = .26; k = 355 studies; N = 470,846). This gender difference remained stable in U.S. college student cohorts over time (from 1990 to 2013) and across different age groups. Study 1 also investigated gender differences in three facets of the Narcissistic Personality Inventory (NPI) to reveal that the narcissism gender difference is driven by the Exploitative/Entitlement facet (d = .29; k = 44 studies; N = 44,108) and Leadership/Authority facet (d = .20; k = 40 studies; N = 44,739); whereas the gender difference in Grandiose/Exhibitionism (d = .04; k = 39 studies; N = 42,460) was much smaller. We further investigated a less-studied form of narcissism called vulnerable narcissism-which is marked by low self-esteem, neuroticism, and introversion-to find that (in contrast to the more commonly studied form of narcissism found in the DSM and the NPI) men and women did not differ on vulnerable narcissism (d = -.04; k = 42 studies; N = 46,735). Study 2 used item response theory to rule out the possibility that measurement bias accounts for observed gender differences in the three facets of the NPI (N = 19,001). Results revealed that observed gender differences were not explained by measurement bias and thus can be interpreted as true sex differences. Discussion focuses on the implications for the biosocial construction model of gender differences, for the etiology of narcissism, for clinical applications, and for the role of narcissism in helping to explain gender differences in leadership and aggressive behavior. Readers are warned against overapplying small effect sizes to perpetuate gender stereotypes.

Linggle: a Web-scale Linguistic Search Engine for Words in Context <=SEP=> In this paper, we introduce a Web-scale linguistics search engine, Linggle, that retrieves lexical bundles in response to a given query. The query might contain keywords, wildcards, wild parts of speech (PoS), synonyms, and additional regular expression (RE) operators. In our approach, we incorporate inverted file indexing, PoS information from BNC, and semantic indexing based on Latent Dirichlet Allocation with Google Web 1T. The method involves parsing the query to transforming it into several keyword retrieval commands. Word chunks are retrieved with counts, further filtering the chunks with the query as a RE, and finally displaying the results according to the counts, similarities, and topics. Clusters of synonyms or conceptually related words are also provided. In addition, Linggle provides example sentences from The New York Times on demand. The current implementation of Linggle is the most functionally comprehensive, and is in principle language and dataset independent. We plan to extend Linggle to provide fast and convenient access to a wealth of linguistic information embodied in Web scale datasets including Google Web 1T and Google Books Ngram for many major languages in the world.

Noninvasive diagnosis of fetal aneuploidy by shotgun sequencing DNA from maternal blood. <=SEP=> We directly sequenced cell-free DNA with high-throughput shotgun sequencing technology from plasma of pregnant women, obtaining, on average, 5 million sequence tags per patient sample. This enabled us to measure the over- and underrepresentation of chromosomes from an aneuploid fetus. The sequencing approach is polymorphism-independent and therefore universally applicable for the noninvasive detection of fetal aneuploidy. Using this method, we successfully identified all nine cases of trisomy 21 (Down syndrome), two cases of trisomy 18 (Edward syndrome), and one case of trisomy 13 (Patau syndrome) in a cohort of 18 normal and aneuploid pregnancies; trisomy was detected at gestational ages as early as the 14th week. Direct sequencing also allowed us to study the characteristics of cell-free plasma DNA, and we found evidence that this DNA is enriched for sequences from nucleosomes.

Dense 3D surface acquisition by structured light using off-the-shelf components <=SEP=> This paper describes the implementation of a LCD stripe projection system, based on a new processing scheme called line shift processing. One advantage of the method is, that the projection device can, but is not required to be stable over time nor does it have to be calibrated. The new method therefore allows us to use an off-the-shelf multimedia projector to acquire dense 3-D data (several 100K Triangles) of large objects in a matter of seconds. Since we are able to control our source of illumination, we can also acquire registered color information using standard monochrome cameras. Besides the description of the system some practical examples are given. The calibration of the system is also addressed.

Document Language Models, Query Models, and Risk Minimization for Information Retrieval <=SEP=> We present a framework for information retrieval that combines document models and query models using a probabilistic ranking function based on Bayesian decision theory. The framework suggests an operational retrieval model that extends recent developments in the language modeling approach to information retrieval. A language model for each document is estimated, as well as a language model for each query, and the retrieval problem is cast in terms of risk minimization. The query language model can be exploited to model user preferences, the context of a query, synonomy and word senses. While recent work has incorporated word translation models for this purpose, we introduce a new method using Markov chains defined on a set of documents to estimate the query models. The Markov chain method has connections to algorithms from link analysis and social networks. The new approach is evaluated on TREC collections and compared to the basic language modeling approach and vector space models together with query expansion using Rocchio. Significant improvements are obtained over standard query expansion methods for strong baseline TF-IDF systems, with the greatest improvements attained for short queries on Web data.

Top-N Recommendation with Novel Rank Approximation <=SEP=> The importance of accurate recommender systems has been widely recognized by academia and industry. However, the recommendation quality is still rather low. Recently, a linear sparse and low-rank representation of the user-item matrix has been applied to produce Top-N recommendations. This approach uses the nuclear norm as a convex relaxation for the rank function and has achieved better recommendation accuracy than the state-of-the-art methods. In the past several years, solving rank minimization problems by leveraging nonconvex relaxations has received increasing attention. Some empirical results demonstrate that it can provide a better approximation to original problems than convex relaxation. In this paper, we propose a novel rank approximation to enhance the performance of Top-N recommendation systems, where the approximation error is controllable. Experimental results on real data show that the proposed rank approximation improves the Top-N recommendation accuracy substantially.

Cloud RAN for Mobile Networks—A Technology Overview <=SEP=> Cloud Radio Access Network (C-RAN) is a novel mobile network architecture which can address a number of challenges the operators face while trying to support growing end-user's needs. The main idea behind C-RAN is to pool the Baseband Units (BBUs) from multiple base stations into centralized BBU Pool for statistical multiplexing gain, while shifting the burden to the high-speed wireline transmission of In-phase and Quadrature (IQ) data. C-RAN enables energy efficient network operation and possible cost savings on baseband resources. Furthermore, it improves network capacity by performing load balancing and cooperative processing of signals originating from several base stations. This paper surveys the state-of-the-art literature on C-RAN. It can serve as a starting point for anyone willing to understand C-RAN architecture and advance the research on C-RAN.

Therapeutic Uses of Active Videogames: A Systematic Review. <=SEP=> BACKGROUND
Active videogames (AVGs) may be useful for promoting physical activity for therapeutic uses, including for balance, rehabilitation, and management of illness or disease. The literature from 64 peer-reviewed publications that assessed health outcomes of AVGs for therapeutic purposes was synthesized.


MATERIALS AND METHODS
PubMed, Medline, and PyschInfo were queried for original studies related to the use of AVGs to improve physical outcomes in patients who were ill or undergoing rehabilitation related to balance, burn treatment, cancer, cerebral palsy, Down's syndrome, extremity dysfunction or amputation, hospitalization, lupus, Parkinson's disease, spinal injury, or stroke. The following inclusion criteria were used: (1) human subjects; (2) English language; (3) not duplicates; (4) new empirical data; and (5) tests an AVG, including commercially available or custom-designed. Studies were included regardless of participants' age or the study design.


RESULTS AND LIMITATIONS
Overall, the vast majority of studies demonstrated promising results for improved health outcomes related to therapy, including significantly greater or comparable effects of AVG play versus usual care. However, many studies were pilot trials with small, homogeneous samples, and many studies lacked a control or comparison group. Some trials tested multiweek or multimonth interventions, although many used a single bout of gameplay, and few included follow-up assessments to test sustainability of improved health.


CONCLUSIONS AND IMPLICATIONS
AVGs were acceptable and enjoyable to the populations examined and appear as a promising tool for balance, rehabilitation, and illness management. Future research directions and implications for clinicians are discussed.

Logic BIST for large industrial designs: real issues and case studies <=SEP=> to Abstract This paper discusses practical issues involved in app ing logic built-in self-test (BIST) to four large industria designs. These multi-clock designs, ranging in size fr 200K to 800K gates, pose significant challenges to log BIST methodology, flow, and tools. The paper presents process of generating a BIST-compliant core along with t logic BIST controller for at-speed testing. Comparativ data on fault grades and area overhead between autom test pattern generation (ATPG) and logic BIST a reported. The experimental results demonstrate that w automation of the proposed solutions, logic BIST c achieve test quality approaching that of ATPG with min mal area overhead and few changes to the design flow.

Supervised and Unsupervised Intrusion Detection Based on CAN Message Frequencies for In-vehicle Network <=SEP=> Modern vehicles are equipped with Electronic Control Units (ECUs) and external communication devices. The Controller Area Network (CAN), a widely used communication protocol for ECUs, does not have a security mechanism to detect improper packets; if attackers exploit the vulnerability of an ECU and manage to inject a malicious message, they are able to control other ECUs to cause improper operation of the vehicle. With the increasing popularity of connected cars, it has become an urgent matter to protect in-vehicle networks against security threats. In this paper, we study the applicability of statistical anomaly detection methods for identifying malicious CAN messages in invehicle networks. We focus on intrusion attacks of malicious messages. Because the occurrence of an intrusion attack certainly influences the message traffic, we focus on the number of messages observed in a fixed time window to detect intrusion attacks. We formalize features to represent a message sequence that incorporates the number of messages associated with each receiver ID. We collected CAN message data from an actual vehicle and conducted a quantitative analysis of the methods and the features in practical situations. The results of our experiments demonstrated our proposed methods provide fast and accurate detection in various cases.

PCN: Point Completion Network <=SEP=> Shape completion, the problem of estimating the complete geometry of objects from partial observations, lies at the core of many vision and robotics applications. In this work, we propose Point Completion Network (PCN), a novel learning-based approach for shape completion. Unlike existing shape completion methods, PCN directly operates on raw point clouds without any structural assumption (e.g. symmetry) or annotation (e.g. semantic class) about the underlying shape. It features a decoder design that enables the generation of fine-grained completions while maintaining a small number of parameters. Our experiments show that PCN produces dense, complete point clouds with realistic structures in the missing regions on inputs with various levels of incompleteness and noise, including cars from LiDAR scans in the KITTI dataset.

Identification of extremist videos in online video sharing sites <=SEP=> Web 2.0 has become an effective grassroots communication platform for extremists to promote their ideas, share resources, and communicate among each other. As an important component of Web 2.0, online video sharing sites such as YouTube and Google video have also been utilized by extremist groups to distribute videos. This study presented a framework for identifying extremist videos in online video sharing sites by using user-generated text content such as comments, video descriptions, and titles without downloading the videos. Text features including lexical features, syntactic features and content specific features were first extracted. Then Information Gain was used for feature selection, and Support Vector Machine was deployed for classification. The exploratory experiment showed that our proposed framework is effective for identifying online extremist videos, with the F-measure as high as 82%.

A Hybrid Approach for Arabic Text Summarization Using Domain Knowledge and Genetic Algorithms <=SEP=> Text summarization is the process of producing a shorter version of a specific text. Automatic summarization techniques have been applied to various domains such as medical, political, news, and legal domains proving that adapting domain-relevant features could improve the summarization performance. Despite the existence of plenty of research work in the domain-based summarization in English and other languages, there is a lack of such work in Arabic due to the shortage of existing knowledge bases. In this paper, a hybrid, single-document text summarization approach (abbreviated as (ASDKGA)) is presented. The approach incorporates domain knowledge, statistical features, and genetic algorithms to extract important points of Arabic political documents. The ASDKGA approach is tested on two corpora KALIMAT corpus and Essex Arabic Summaries Corpus (EASC). The Recall-Oriented Understudy for Gisting Evaluation (ROUGE) framework was used to compare the automatically generated summaries by the ASDKGA approach with summaries generated by humans. Also, the approach is compared against three other Arabic text summarization approaches. The (ASDKGA) approach demonstrated promising results when summarizing Arabic political documents with average F-measure of 0.605 at the compression ratio of 40%.

Ring-LWE Ciphertext Compression and Error Correction: Tools for Lightweight Post-Quantum Cryptography <=SEP=> Some lattice-based public key cryptosystems allow one to transform ciphertext from one lattice or ring representation to another efficiently and without knowledge of public and private keys. In this work we explore this lattice transformation property from cryptographic engineering viewpoint. We apply ciphertext transformation to compress Ring-LWE ciphertexts and to enable efficient decryption on an ultra-lightweight implementation targets such as Internet of Things, Smart Cards, and RFID applications. Significantly, this can be done without modifying the original encryption procedure or its security parameters. Such flexibility is unique to lattice-based cryptography and may find additional, unique real-life applications.
 Ciphertext compression can significantly increase the probability of decryption errors. We show that the frequency of such errors can be analyzed, measured and used to derive precise failure bounds for n-bit error correction. We introduce XECC, a fast multi-error correcting code that allows constant time implementation in software.
 We use these tools to construct and explore TRUNC8, a concrete Ring-LWE encryption and authentication system. We analyze its implementation, security, and performance. We show that our lattice compression technique reduces ciphertext size by more than 40% at equivalent security level, while also enabling public key cryptography on previously unreachable ultra-lightweight platforms.
 The experimental public key encryption and authentication system has been implemented on an 8-bit AVR target, where it easily outperforms elliptic curve and RSA-based proposals at similar security level. Similar results have been obtained with a Cortex M0 implementation. The new decryption code requires only a fraction of the software footprint of previous Ring-LWE implementations with the same encryption parameters, and is well suited for hardware implementation.

2005: Tutorial on Agent-Based Modeling and Simulation <=SEP=> Agent-based modeling and simulation (ABMS) is a new approach to modeling systems comprised of autonomous, interacting agents. ABMS promises to have far-reaching effects on the way that businesses use computers to support decision-making and researchers use electronic laboratories to support their research. Some have gone so far as to contend that ABMS is a third way of doing science besides deductive and inductive reasoning. Computational advances have made possible a growing number of agentbased applications in a variety of fields. Applications range from modeling agent behavior in the stock market and supply chains, to predicting the spread of epidemics and the threat of bio-warfare, from modeling consumer behavior to understanding the fall of ancient civilizations, to name a few. This tutorial describes the theoretical and practical foundations of ABMS, identifies toolkits and methods for developing ABMS models, and provides some thoughts on the relationship between ABMS and traditional modeling techniques.

LEARNING ENGLISH IS FUN VIA KAHOOT : STUDENTS ’ ATTITUDE , MOTIVATION AND PERCEPTIONS <=SEP=> This study is aimed to investigate primary level students’ attitude and motivation and perceptions in learning English using Kahoot game. According to current development in Malaysian education field, 21st century learning style is more focused and given importance. Thus, Kahoot, game-based learning platform has been used to stimulate students’ from a rural area in southern Malaysia to learn English as their second language more effectively, actively and interestingly. The research design for this study is action research. Nine students were chosen as the target group for this study. The Kahoot game was conducted for three different topics in English after they completed the lesson each day. The data was collected using Attitude and Motivation Test Battery questionnaires consisting of 10 items, interview session with 5 semi-structured questions for three selected students and results of each Kahoot game played by the students. The data was analysed using descriptive analysis. The results of the questionnaires, interview session and results of the game were shown in figures and tables. Findings of the study show that all the nine students were able to engage actively in the game and they were able to master the target language effectively. They enjoy learning English using games and wish to have more games in future.

Thompson Sampling for Dynamic Pricing <=SEP=> In this paper we apply active learning algorithms for dynamic pricing in a prominent e-commerce website. Dynamic pricing involves changing the price of items on a regular basis, and uses the feedback from the pricing decisions to update prices of the items. Most popular approaches to dynamic pricing use a passive learning approach, where the algorithm uses historical data to learn various parameters of the pricing problem, and uses the updated parameters to generate a new set of prices. We show that one can use active learning algorithms such as Thompson sampling to more efficiently learn the underlying parameters in a pricing problem. We apply our algorithms to a real e-commerce system and show that the algorithms indeed improve revenue compared to pricing algorithms that use passive learning.

A Simplified Iron Loss Model for Laminated Magnetic Cores <=SEP=> This paper presents a new method for the prediction of iron losses in laminated magnetic cores. The method is based on the evaluation of the magnetization curves and loop shapes by which the iron losses, including hysteresis, excess, and classical eddy-current losses, are calculated from the loop area. The purpose of the method is to omit the numerical solution of the nonlinear diffusion equation (resulting from Maxwell equations) while maintaining reasonable accuracy of the iron losses. Therefore, the proposed method determines, using simple concepts, three magnetic field strength components that are responsible for generating the iron losses. The magnetic field component of the hysteresis loss is calculated by a static hysteresis model. The magnetic field component of the excess loss is calculated using the statistical loss theory. Determining the magnetic field component of the classical eddy-current loss is based upon the assumption that the flux distribution in the steel sheet is uniform, but, here, the eddy-current loss is implicitly enforced to be dependent on the magnetic characteristics of the material. The proposed method is applied to the calculation of iron losses in an electrical steel when the accuracy, stability, and generality of the method are discussed.

15 Planar Antenna Technology for mm-Wave Automotive Radar , Sensing <=SEP=> Planar antennas are common components in sensing applications due to their low cost, low profile and simple integration with systems. They can be used commonly at frequencies as high as 77 GHz and above, for example in automotive sensing. Also, new ultra-wideband communication systems operating around 60 GHz will heavily rely on planar antennas due to their unique properties. One special advantage of planar elements is that they can easily form array structures combining very simple elements, like microstrip patches. Also phased and conformal arrays can be built. Due to all these characteristics, planar antennas are very good candidates for building front ends for mm-wave applications in radar, sensing or communications. For some applications, which are either established today or which will become commercial in a few years, general requirements can be given. In addition to automotive radar, this mainly applies to 60 GHz ultra-broadband wireless indoor communications. Microwave imaging at 94 GHz and above is still much under research, as well as other sensing applications in the millimeter-wave region above 100 GHz. Prominent frequency bands are in the 122 and 140 GHz range1. Typical antenna requirements are multi-beam or scanning capability, high gain up to 30..36 dBi and moderate to low sidelobes. In monopulse radar systems, the sidelobes may be included in the angle determination scheme, then the sidelobe level requirements are rather relaxed. Loss is generally an important issue. Comparing a planar approach to a commercially available dielectric lens, the planar antenna exhibits significantly higher losses, especially if the beamforming device is included. The antenna efficiency of 77 GHz planar array columns is roughly 50%. These losses are caused by dielectric losses and conductor losses. To attain reliable predictions of the losses in full-wave simulations, care has to be taken in the

Cyber-Physical Systems: A New Frontier <=SEP=> The report of the President's Council of Advisors on Science and Technology (PCAST) has placed CPS on the top of the priority list for federal research investment [6]. This article first reviews some of the challenges and promises of CPS, followed by an articulation of some specific challenges and promises that are more closely related to the sensor networks, ubiquitous and trustworthy computing conference.

Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents <=SEP=> Evolution strategies (ES) are a family of black-box optimization algorithms able to train deep neural networks roughly as well as Q-learning and policy gradient methods on challenging deep reinforcement learning (RL) problems, but are much faster (e.g. hours vs. days) because they parallelize better. However, many RL problems require directed exploration because they have reward functions that are sparse or deceptive (i.e. contain local optima), and it is unknown how to encourage such exploration with ES. Here we show that algorithms that have been invented to promote directed exploration in small-scale evolved neural networks via populations of exploring agents, specifically novelty search (NS) and quality diversity (QD) algorithms, can be hybridized with ES to improve its performance on sparse or deceptive deep RL tasks, while retaining scalability. Our experiments confirm that the resultant new algorithms, NS-ES and two QD algorithms, NSR-ES and NSRA-ES, avoid local optima encountered by ES to achieve higher performance on Atari and simulated robots learning to walk around a deceptive trap. This paper thus introduces a family of fast, scalable algorithms for reinforcement learning that are capable of directed exploration. It also adds this new family of exploration algorithms to the RL toolbox and raises the interesting possibility that analogous algorithms with multiple simultaneous paths of exploration might also combine well with existing RL algorithms outside ES.

GlobeSen: an open interconnection framework based on named sensory date for IoT <=SEP=> How to efficiently interconnect ubiquitous wireless sensor networks (WSNs) and the Internet becomes an important challenge of Internet of Things. In this paper, we explore a route of information-centric networking (ICN) to solve this challenge and propose an open interconnection framework named GlobeSen. To overcome the problem that traditional ICN solution (such as NDN) is not suitable for resource-constrained WSNs, we present a new implementation of NDN, NDNs, for WSNs. Specifically, by extracting the spatio-temporal information and data type information of interest, we construct a globally unique name structure, and exploit the spatio-temporal relation operation as the matching method. Based on the new naming strategy and matching method, we further design packet forwarding and routing schemes. Moreover, we also develop a gateway system, NDNt, for protocol translating and an application, SenBrowser, to provide a user-friendly interface for generating interests and illustrating the returned sensory data. We implement a proof-of-concept prototype based on TelosB sensor nodes and an ARM development board, and conduct a series of experiments to evaluate the performance of GlobeSen.