Reviewerly/exHarmony huggingface dataset:
Talk2Nav: Long-Range Vision-and-Language Navigation with Dual Attention and Spatial Memory

Abstract The role of robots in society keeps expanding, bringing with it the necessity of interacting and communicating with humans. In order to keep such interaction intuitive, we provide automatic wayfinding based on verbal navigational instructions. Our first contribution is the creation of a large-scale dataset with verbal navigation instructions. To this end, we have developed an interactive visual navigation environment based on Google Street View; we further design an annotation method to highlight mined anchor landmarks and local directions between them in order to help annotators formulate typical, human references to those. The annotation task was crowdsourced on the AMT platform, to construct a new Talk2Nav dataset with 10, 714 routes. Our second contribution is a new learning method. Inspired by spatial cognition research on the mental conceptualization of navigational instructions, we introduce a soft dual attention mechanism defined over the segmented language instructions to jointly extract two partial instructions—one for matching the next upcoming visual landmark and the other for matching the local directions to the next landmark. On the similar lines, we also introduce spatial memory scheme to encode the local directional transitions. Our work takes advantage of the advance in two lines of research: mental formalization of verbal navigational instructions and training neural network agents for automatic way finding. Extensive experiments show that our method significantly outperforms previous navigation methods. For demo video, dataset and code, please refer to our project page .
Project Achilles: A Prototype Tool for Static Method-Level Vulnerability Detection of Java Source Code Using a Recurrent Neural Network

Software has become an essential component of modern life, but when software vulnerabilities threaten the security of users, new ways of analyzing for software security must be explored. Using the National Institute of Standards and Technology's Juliet Java Suite, containing thousands of examples of defective Java methods for a variety of vulnerabilities, a prototype tool was developed implementing an array of Long-Short Term Memory Recurrent Neural Networks to detect vulnerabilities within source code. The tool employs various data preparation methods to be independent of coding style and to automate the process of extracting methods, labeling data, and partitioning the dataset. The result is a prototype command-line utility that generates an n-dimensional vulnerability prediction vector. The experimental evaluation using 44,495 test cases indicates that the tool can achieve an accuracy higher than 90% for 24 out of 29 different types of CWE vulnerabilities.
Automatic Classification Error Detection and Correction for Robust Human Activity Recognition

One of the main objectives of Ambient Assisted Living (AAL) systems is to proactively provide intelligent services to improve the quality of people's lives in terms of autonomy, safety, and well-being. Designing AAL systems that can autonomously monitor human's activities and provide assistance services poses several challenges of which Human Activity Recognition (HAR) which is critically important to adapt the assistance services to the user. In this letter, a robust multi-label HAR framework is proposed. The proposed framework is composed of two main modules: (i) activity classification module and (ii) classification error detection and correction module. In the first module, machine-learning models are used to predict human activities. Since these models may produce predictions with errors, there is a requirement to detect and correct these errors. The classification error detection and correction module is based on two acyclic directed graphical models and operates in two phases: (i) classification error detection and (ii) classification error correction. The proposed framework is evaluated on the Opportunity dataset, a benchmark and a unique dataset for multi-label human daily living activity recognition. The obtained results demonstrate the ability of the proposed framework to improve the performances of HAR.
A Multi-Period Product Recommender System in Online Food Market based on Recurrent Neural Networks

A recommender system supports customers to find information, products, or services (such as music, books, movies, web sites, and digital contents), so it could help customers to make rapid routine decisions and save their time and money. However, most existing recommender systems do not recommend items that are already purchased by the target customer, so are not suitable for considering customers’ repetitive purchase behavior or purchasing order. In this research, we suggest a multi-period product recommender system, which can learn customers’ purchasing order and customers’ repetitive purchase pattern. For such a purpose we applied the Recurrent Neural Network (RNN), which is one of the artificial neural network structures specialized in time series data analysis, instead of collaborative filtering techniques. Recommendation periods are segmented as various time-steps, and the proposed RNN-based recommender system can recommend items by multiple periods in a time sequence. Several experiments with real online food market data show that the proposed system shows higher performance in accuracy and diversity in a multi-period perspective than the collaborative filtering-based system. From the experimental results, we conclude that the proposed system is suitable for multi-period product recommendation, which results in robust performance considering well customers’ purchasing orders and customers’ repetitive purchase patterns. Moreover, in terms of sustainability, we expect that our study contributes to the reduction of food wastes by inducing planned consumption, and the reduction of shopping time and effort.
Blockchain and the Industrial Internet of Things

Purpose The purpose of this paper is to investigate the suitability of Blockchain technology for applications in the Industrial Internet of Things (IIOT). It provides a taxonomy of system requirements for such applications and maps these requirements against the Blockchain’s technological idiosyncrasies. Design/methodology/approach A requirement taxonomy is built in an iterative process based on a descriptive literature review. In total, 223 studies have been screened leading to a relevant sample of 48 publications that were analyzed in detail regarding posed system requirements. Subsequently, Blockchain’s capabilities are discussed for each requirement dimension. Findings The paper presents a taxonomy of six requirement dimensions. In the mapping process, areas of greater fit (e.g., reliability, nonrepudiation and adaptability) were identified. However, there are also several constraints (e.g., scalability, confidentiality and performance) that limit the use of Blockchain. Research limitations/implications Due to the limited amount of studies and the vibrant development of Blockchain technology, the results may benefit from practical evidence. Researchers are encouraged to validate the results in qualitative practitioner interviews. Focusing on literature-backed public Blockchain, idiosyncrasies of private implementations and specific distributed ledger technologies may be discussed in future studies. Practical implications The paper includes use cases for Blockchain in manufacturing and IIOT applications. Potential caveats for practitioners are presented. Originality/value This paper addresses the need to understand to which degree Blockchain is a suitable technology in manufacturing, especially in context of the IIOT. It contributes a requirement taxonomy which serves as the foundation for a systematic fit assessment.
Task-Motion Planning with Reinforcement Learning for Adaptable Mobile Service Robots

Task-motion planning (TMP) addresses the problem of efficiently generating executable and low-cost task plans in a discrete space such that the (initially unknown) action costs are determined by motion plans in a corresponding continuous space. A task-motion plan for a mobile service robot that behaves in a highly dynamic domain can be sensitive to domain uncertainty and changes, leading to suboptimal behaviors or execution failures. In this paper, we propose a novel framework, TMP-RL, which is an integration of TMP and reinforcement learning (RL), to solve the problem of robust TMP in dynamic and uncertain domains. The robot first generates a low-cost, feasible task-motion plan by iteratively planning in the discrete space and updating relevant action costs evaluated by the motion planner in continuous space. During execution, the robot learns via model-free RL to further improve its task-motion plans. RL enables adaptability to the current domain, but can be costly with regards to experience; using TMP, which does not rely on experience, can jump-start the learning process before executing in the real world. TMP-RL is evaluated in a mobile service robot domain where the robot navigates in an office area, showing significantly improved adaptability to unseen domain dynamics over TMP and task planning (TP)-RL methods.
Scalable and Robust Outlier Detector using Hierarchical Clustering and Long Short-Term Memory (LSTM) Neural Network for the Internet of Things

The emerging centralized entities, like cloud, edge, or Software-Defined Network (SDN), make automated decisions for the Internet of Things (IoT) applications based on the measured data from several sensors. However, the malicious injection of the anomalies or outliers in measured sensor data may disrupt the automated decision making capabilities of the applications running at the centralized location. Therefore, the detection of such outliers is an essential problem for IoT that needs to be researched out. This paper presents a scalable outlier detector that uses Hierarchical clustering in conjunction with Long Short-Term Memory (LSTM) neural network. Hierarchical clustering provides scalability to the outlier detector by finding correlated sensors. The LSTM neural network is coupled with the robust statistics, M-estimator, to accurately detect outliers in time-series data. The simulation results on different data-sets show that the proposed method has an accuracy of more than 90% for different attack strength. Also, the model parameter can be tuned according to the application requirement so that the outlier detector can be tailored to either precision or recall sensitive.
GRIP: Generative Robust Inference and Perception for Semantic Robot Manipulation in Adversarial Environments

Recent advancements have led to a proliferation of machine learning systems used to assist humans in a wide range of tasks. However, we are still far from accurate, reliable, and resource-efficient operations of these systems. For robot perception, convolutional neural networks (CNNs) for object detection and pose estimation are recently coming into widespread use. However, neural networks are known to suffer from overfitting during the training process and are less robust under unforeseen conditions (which makes them especially vulnerable to adversarial scenarios). In this work, we propose Generative Robust Inference and Perception (GRIP) as a two-stage object detection and pose estimation system that aims to combine the relative strengths of discriminative CNNs and generative inference methods to achieve robust estimation. Our results show that a second stage of sample-based generative inference is able to recover from false object detections by CNNs, and produce robust estimations in adversarial conditions. We demonstrate the efficacy of GRIP robustness through comparison with state-of-the-art learning-based pose estimators and pick-and-place manipulation in dark and cluttered environments.
Learning Credible Deep Neural Networks with Rationale Regularization

Recent explainability related studies have shown that state-of-the-art DNNs do not always adopt correct evidences to make decisions. It not only hampers their generalization but also makes them less likely to be trusted by end-users. In pursuit of developing more credible DNNs, in this paper we propose CREX, which encourages DNN models to focus more on evidences that actually matter for the task at hand, and to avoid overfitting to data-dependent bias and artifacts. Specifically, CREX regularizes the training process of DNNs with rationales, i.e., a subset of features highlighted by domain experts as justifications for predictions, to enforce DNNs to generate local explanations that conform with expert rationales. Even when rationales are not available, CREX still could be useful by requiring the generated explanations to be sparse. Experimental results on two text classification datasets demonstrate the increased credibility of DNNs trained with CREX. Comprehensive analysis further shows that while CREX does not always improve prediction accuracy on the held-out test set, it significantly increases DNN accuracy on new and previously unseen data beyond test set, highlighting the advantage of the increased credibility.
Robust Loop Closure Detection based on Bag of SuperPoints and Graph Verification

Loop closure detection (LCD) is a crucial technique for robots, which can correct accumulated localization errors after long time explorations. In this paper, we propose a robust LCD algorithm based on Bag of SuperPoints and graph verification. The system first extracts interest points and feature descriptors using the SuperPoint neural network. Then a visual vocabulary is trained in an incremental and self-supervised manner considering the relations between consecutive training images. Finally, a topological graph is constructed using matched feature points to verify candidate loop closures obtained by a Bag-of-Words (BoW) framework. Comparative experiments with state-of-the-art LCD algorithms on several typical datasets have been carried out. The results demonstrate that our proposed graph verification method can significantly improve the accuracy of image matching and the overall LCD approach outperforms existing methods.
A Power-Efficient CNN Accelerator With Similar Feature Skipping for Face Recognition in Mobile Devices

A low power face recognition (FR) convolutional neural network (CNN) processor is proposed with high power efficiency to achieve always-on FR in mobile devices. Three key features enable a power-efficient FR CNN. First, tile-based clustering (THC) is proposed for reducing the computation overhead of hierarchical clustering. It generates an average of 37.2% duplicated input features in the entire network. Second, a low latency tile-based hierarchical clustering core is proposed. It supports an approximated clustering method that removes distance updates and increases pipeline utilization by up to 98.6%. Finally, a similar feature skipping binary convolution core with separated accumulation cores is proposed to increase power efficiency through skipping redundant MAC operation between duplicated input features and weights. It can skip 38.9% of CNN operations and reduce 35.8% of the total computation considering both CNN and THC. The proposed features enable the proposed processor to operate FR CNN with 17.3 TOPS/W power efficiency, which is 1.3× higher compared to the previous state-of-the-art FR CNN processor. Implemented in 65nm CMOS technology, the 6mm2 FR CNN processor shows 0.5mW power consumption at 1 frames-per-second always-on face recognition.
Blockchain Applied to Vehicular Odometers

Fraud in automobile odometers is a billion dollar problem worldwide. The technical requirements behind any solution for this problem go through the perennial, secure, and immutable registration of vehicular data. Such requirements immediately raise the question of how blockchain technology could be used for this purpose. In this article, the requirements, design choices, and construction details of a blockchain architecture for vehicular odometers are presented and discussed. The architecture advances the state of the art by adopting a more efficient consensus algorithm than the traditional ones, such as proof of work. Results from a prototype proof of concept of the proposed architecture validate the solution, which is able to validate a new block in the blockchain every two seconds. Applicability in real scenarios is one step ahead and very promising.
Vision-Based Potential Pedestrian Risk Analysis on Unsignalized Crosswalk Using Data Mining Techniques

Though the technological advancement of smart city infrastructure has significantly improved urban pedestrians’ health and safety, there remains a large number of road traffic accident victims, making it a pressing current transportation concern. In particular, unsignalized crosswalks present a major threat to pedestrians, but we lack dense behavioral data to understand the risks they face. In this study, we propose a new model for potential pedestrian risky event (PPRE) analysis, using video footage gathered by road security cameras already installed at such crossings. Our system automatically detects vehicles and pedestrians, calculates trajectories, and extracts frame-level behavioral features. We use k-means clustering and decision tree algorithms to classify these events into six clusters, then visualize and interpret these clusters to show how they may or may not contribute to pedestrian risk at these crosswalks. We confirmed the feasibility of the model by applying it to video footage from unsignalized crosswalks in Osan city, South Korea.
A Novel Lightweight Solo Software Development Methodology With Optimum Security Practices

The diffusion of software into all areas of life and all forms of business, increases the demand for high-quality and secure software products. Software development methodologies are designed to improve the quality of software by incorporating practices that promote quality in the developed software. Software security is an important facet of software quality, particularly in this era, where most software is deployed for use over the Internet. Most research on developing high-quality and secure software is normally focused on teams at the expense of individual developers. In trying to fill this gap, in this paper we propose an agile secure-software development methodology. We design a methodology that promotes quality and security in the software products of solo developers. We integrate quality practices with lightweight security practices to produce agile secure software development practices. We draw quality practices from a solo software development framework designed in our previous study, while security practices are drawn from existing lightweight methodologies. We adapt Keramati and Mirian-Hosseinabadi's algorithm to integrate the two sets of practices, taking care to maintain an optimum degree of agility in the target methodology. We evaluate the utility of the resultant methodology through a case study. Results from the case study show that our proposed methodology can be used to build quality and secure software products without compromising the agility of the methodology.
Understanding user acceptance of blockchain-based smart locker

Purpose The purpose of this study is to explore the critical factors that affect users’ acceptance and usage intention toward blockchain-based smart lockers. Design/methodology/approach The research is designed in two phases. In the first phase, the key stakeholders are interviewed. Participants include managers from technology providers and logistics companies. In the second phase, a questionnaire survey is used to validate the proposed model. Findings Based on the final results, this study makes the following three suggestions. First, perceived usefulness and perceived ease of use are the critical factors. In other words, it is important to emphasize the function and convenience of a new service when introducing it to a potential user. Second, safety is not the major concern when using a blockchain-based smart locker. This means that users will trust the service provider for providing a secure service. Users do not worry about the security problem. Finally, the network externality of smart locker is also insignificant. Originality/value This study has three major contributions. First, this study identifies the critical factors that will affect user acceptance of blockchain-based smart lockers. Next, this study combines the opinions from service providers and users to understand the gap between different stakeholders. Finally, this study can enrich our understanding on the applications of blockchain from a managerial perspective and not only from a technical perspective, as in most previous studies.
Ranking-based instance selection for pattern classification

In instance-based learning algorithms, the need to store a large number of examples as the training set results in several drawbacks related to large memory requirements, oversensitivity to noise, and slow execution speed. Instance selection techniques can improve the performance of these algorithms by selecting the best instances from the original data set, removing, for example, redundant information and noisy points. The relationship between an instance and the other patterns in the training set plays an important role and can impact its misclassification by learning algorithms. Such a relationship can be represented as a value that measures how difficult such instance is regarding classification purposes. Based on that, we introduce a novel instance selection algorithm called Ranking-based Instance Selection (RIS) that attributes a score per instance that depends on its relationship with all other instances in the training set. In this sense, instances with higher scores form safe regions (neighborhood of samples with relatively homogeneous class labels) in the feature space, and instances with lower scores form an indecision region (borderline samples of different classes). This information is further used in a selection process to remove instances from both safe and indecision regions that are considered irrelevant to represent their clusters in the feature space. In contrast to previous algorithms, the proposal combines a raking procedure with a selection process aiming to find a promising tradeoff between accuracy and reduction rate. Experiments are conducted on twenty-four real-world classification problems and show the effectiveness of the RIS algorithm when compared against other instance selection algorithms in the literature.
Autonomous navigation for UAVs managing motion and sensing uncertainty

We present a motion planner for the autonomous navigation of UAVs that manages motion and sensing uncertainty at planning time. By doing so, optimal paths in terms of probability of collision, traversal time and uncertainty are obtained. Moreover, our approach takes into account the real dimensions of the UAV in order to reliably estimate the probability of collision from the predicted uncertainty. The motion planner relies on a graduated fidelity state lattice and a novel multi-resolution heuristic which adapt to the obstacles in the map. This allows managing the uncertainty at planning time and yet obtaining solutions fast enough to control the UAV in real time. Experimental results show the reliability and the efficiency of our approach in different real environments and with different motion models. Finally, we also report planning results for the reconstruction of 3D scenarios, showing that with our approach the UAV can obtain a precise 3D model autonomously.