Reviewerly/exHarmony huggingface dataset:
Informed Information Theoretic Model Predictive Control

The problem of minimizing cost in nonlinear control systems with uncertainties or disturbances remains a major challenge. Model predictive control (MPC), and in particular sampling-based MPC has recently shown great success in complex domains such as aggressive driving with highly nonlinear dynamics. Sampling-based methods rely on a prior distribution to generate samples in the first place. Obviously, the choice of this distribution highly influences efficiency of the controller. Existing approaches such as sampling around the control trajectory of the previous time step perform suboptimally, especially in multi-modal or highly dynamic settings. In this work, we therefore propose to learn models that generate samples in low-cost areas of the state-space, conditioned on the environment and on contextual information of the task to solve. By using generative models as an informed sampling distribution, our approach exploits guidance from the learned models and at the same time maintains robustness properties of the MPC methods. We use Conditional Variational Autoencoders (CVAE) to learn distributions that imitate samples from a training dataset containing optimized controls. An extensive evaluation in the autonomous navigation domain suggests that replacing previous sampling schemes with our learned models considerably improves performance in terms of path quality and planning efficiency.
Learning a Controller Fusion Network by Online Trajectory Filtering for Vision-Based UAV Racing

Autonomous UAV racing has recently emerged as an interesting research problem. The dream is to beat humans in this new fast-paced sport. A common approach is to learn an end-to-end policy that directly predicts controls from raw images by imitating an expert. However, such a policy is limited by the expert it imitates and scaling to other environments and vehicle dynamics is difficult. One approach to overcome the drawbacks of an end-to-end policy is to train a network only on the perception task and handle control with a PID or MPC controller. However, a single controller must be extensively tuned and cannot usually cover the whole state space. In this paper, we propose learning an optimized controller using a DNN that fuses multiple controllers. The network learns a robust controller with online trajectory filtering, which suppresses noisy trajectories and imperfections of individual controllers. The result is a network that is able to learn a good fusion of filtered trajectories from different controllers leading to significant improvements in overall performance. We compare our trained network to controllers it has learned from, end-to-end baselines and human pilots in a realistic simulation; our network beats all baselines in extensive experiments and approaches the performance of a professional human pilot.
Muscleblazer: Force-Feedback Suit for Immersive Experience

The increasing use of virtual reality (VR) and augmented reality (AR) systems has opened the possibility of providing immersive experiences to the general population around the world. However, most of the existing systems do not provide highly effective force-feedback experiences to the user. To provide such augmented systems in combination with force-feedback, novel ideas must be introduced that can be easily integrated with the existing VR and AR technologies. This work proposes a first-person VR game integrated with a soft exoskeleton that enhances the quality of interaction between the subject and the virtual environment (VE) through an additional force-feedback element. The effect of introducing the force-feedback element on the user is analyzed by using biosensors and questionnaire feedback. The biosensors are used to measure the level of anxiety induced in the subject during interaction with the virtual environment. Conditions in which this interaction occurs with and without force-feedback are compared. The questionnaire analyzes the perceived change in emotions of users as a result of the introduction of the force-feedback element. This game can be played by most individuals, regardless of age and physical fitness.
LCEval: Learned Composite Metric for Caption Evaluation

Automatic evaluation metrics hold a fundamental importance in the development and fine-grained analysis of captioning systems. While current evaluation metrics tend to achieve an acceptable correlation with human judgements at the system level, they fail to do so at the caption level. In this work, we propose a neural network-based learned metric to improve the caption-level caption evaluation. To get a deeper insight into the parameters which impact a learned metric’s performance, this paper investigates the relationship between different linguistic features and the caption-level correlation of the learned metrics. We also compare metrics trained with different training examples to measure the variations in their evaluation. Moreover, we perform a robustness analysis, which highlights the sensitivity of learned and handcrafted metrics to various sentence perturbations. Our empirical analysis shows that our proposed metric not only outperforms the existing metrics in terms of caption-level correlation but it also shows a strong system-level correlation against human assessments.
Mobile Data Collection and Analysis with Local Differential Privacy

Local Differential Privacy (LDP), where each user perturbs her data locally before sending to an untrusted party, is a new and promising privacy-preserving model for mobile data collection and analysis. LDP has been deployed in many real products recently by several major software and Internet companies, including Google, Apple and Microsoft. This seminar talk first introduces the rationale of LDP model behind these deployed systems to collect and analyze usage data privately, then surveys the current research landscape in LDP, and finally identifies several open problems and research directions in this community.
Composition of Local Potential Functions with Reflection

This paper suggests reflections can be practically useful if they are included in planning for collision capable robot platforms. By modifying a proven strategy for navigation with reflections we maintain global convergence results and reach the goal in less time. An algorithm for identifying reflection surfaces for a given cell decomposition is reported. Baseline and reflected scenarios are compared for two different cell decompositions. Omnipuck, a reflection capable omnidirectional robot meant to store and release impact energy, is used to obtain experimental results and draw conclusions for future work.
Proposing a Blockchain-Based Open Data Platform and Its Decentralized Oracle

Bitcoin has been attracted attention to issue digital currency and also to manage transactions in a decentralized and trustless way without assuming any reliable third-party organization. To enhance the potential of Bitcoin, blockchain technologies have been studied to find more applications to smart contract, identification, and certificate which is valid even after the original issuer disappears. On the other hand, mobile crowdsensing is known as a method to collectively gather and share local data with the help of many and unspecified participants having their portable devices with sensors like smartphones to extract useful information from them. Expectation-Maximization (EM) algorithm is considered to estimate true information from task results from many independent workers and to evaluate workers' scores. By the combination of blockchain and mobile crowdsensing technologies, a open data platform to save true information into the blockchain and issue currency to workers based on their reliability scores. In this research, we explain how to create this open data platform to generate Big and Long-time Open Data (BaLOD) to certify events that occurred in reality and everyone can validate the events.
Determining the Walsh spectra of Taniguchi's and related APN-functions

We introduce a method based on Bezout's theorem on intersection points of two projective plane curves, for determining the nonlinearity of some classes of quadratic functions on F22m. Among those are the functions of Taniguchi (2019) [20], Carlet (2011) [7], and Zhou and Pott (2013) [22], all of which are APN under certain conditions. This approach helps to understand why the majority of the functions in those classes have solely bent and semibent components, which in the case of APN functions is called the classical spectrum. More precisely, we show that all Taniguchi functions have the classical spectrum independent from being APN. We determine the nonlinearity of all functions belonging to Carlet's class and to the class of Zhou and Pott, which also confirms with comparatively simple proofs earlier results on the Walsh spectrum of APN-functions in these classes. Using the Hasse-Weil bound, we show that some simple sufficient conditions for the APN-ness of the Zhou-Pott functions, which are given in the original paper, are also necessary.
Robust depth estimation for multi-occlusion in light-field images

Occlusion is one of the most important issues in light-field depth estimation. In this paper, we propose a light-field multi-occlusion model with the analysis of light transmission. By the model, occlusions in different views are discussed separately. An adaptive algorithm of anti-occlusion in the central view is proposed to obtain more precise consistency regions (unoccluded views) in the angular domain and a subpatch approach of anti-occlusion in other views is presented to optimize the initial depth maps, where depth boundaries are better preserved. Then we propose a curvature confidence analysis approach to make depth evaluation more accurate and it is designed in an energy model to regularize the depth maps. Experimental results demonstrate that the proposed algorithm achieves better subjective and objective quality in depth maps compared with state-of-the-art algorithms.
Decomposing the rationale of code commits: the software developer’s perspective

Communicating the rationale behind decisions is essential for the success of software engineering projects. In particular, understanding the rationale of code commits is an important and often difficult task. We posit that part of such difficulty lies in rationale often being treated as a single piece of information. In this paper, we set to discover the breakdown of components in which developers decompose the rationale of code commits in the context of software maintenance, and to understand their experience with it and with its individual components. For this goal, we apply a mixed-methods approach, interviewing 20 software developers to ask them how they decompose rationale, and surveying an additional 24 developers to understand their experiences needing, finding, and recording those components. We found that developers decompose the rationale of code commits into 15 components, each of which is differently needed, found, and recorded. These components are: goal, need, benefits, constraints, alternatives, selected alternative, dependencies, committer, time, location, modifications, explanation of modifications, validation, maturity stage, and side effects. Our findings provide multiple implications. Educators can now disseminate the multiple dimensions and importance of the rationale of code commits. For practitioners, our decomposition of rationale defines a "common vocabulary" to use when discussing rationale of code commits, which we expect to strengthen the quality of their rationale sharing and documentation process. For researchers, our findings enable techniques for automatically assessing, improving, and generating rationale of code commits to specifically target the components that developers need.
Target-Specific Siamese Attention Network for Real-Time Object Tracking

Deep similarity trackers are able to track above real-time speed. However, their accuracy is considerably lower than deep classification based trackers since they avoid valuable online cues. To feed the target-specific information for real-time object tracking, we propose a novel Siamese attention network. Different types of attention mechanisms are used to capture different contexts of target information and then learned knowledge is used to feed target cues at different representation levels of similarity tracking. In addition, an online learning mechanism is employed to utilise the available target-specific data. The proposed tracker reduces the impact of noise in the target template and improves the accuracy of similarity tracking by feeding target cues into the similarity search. Extensive evaluation performed on OTB-2013/50/100 and VOT2018 benchmark datasets demonstrate the proposed tracker outperforms state-of-the-art approaches while maintaining real-time tracking speed.
Latency Measurement in Head-Mounted Virtual Environments

In this paper, we discuss a generalizable method to measure end-to-end latency. This is the length of time that elapses between when a real-world movement occurs and when the pixels within a head-mounted display are updated to reflect this movement. The method described here utilizes components commonly available at electronics and hobby shops. We demonstrate this measurement method using an HTC Vive and discuss the influence of its low-persistence display on latency measurement.
Open Vocabulary Arabic Diacritics Restoration

Diacritics restoration is a necessary component in order to develop Arabic text to speech systems. When diacritics are present, the phonetic transcription algorithm can be implemented based on a few rules. Restoring Arabic diacritics based on language model scoring is the dominant approach. A fixed vocabulary is usually used to build the language model used for scoring. Since Arabic is a morphologically rich language, the number of the Out-of-vocabulary (OOV) words is large and the diacritization algorithm fails to restore diacritics for these words. In this letter, we present a novel approach to support open vocabulary diacritics restoration based on the Byte Pair Encoding (BPE) method. The BPE method segments the words into variable length sub-word units and allows open vocabulary from fixed sub-word units dictionary. On the Tashkeela diacritization task, this open vocabulary approach outperforms the word and character based methods commonly used in the literature.
Menus on the Desk? System Control in DeskVR

In this work, we evaluate the impact of passive haptic feedback on touch-based menus, given the constraints and possibilities of a seated, desk-based scenario in VR. Therefore, we compare a menu that once is placed on the surface of a desk and once mid-air on a surface in front of the user. The study design is completed by two conditions without passive haptic feedback. In the conducted user study (n=33), we found effects of passive haptics (present vs-non-present) and menu alignment (desk vs. mid-air) on the task performance and subjective look & feel. However, the race between the conditions was close. An overall winner was the mid-air menu with passive haptic feedback, which however raises hardware requirements.
High-Speed Obstacle-Avoidance with Agile Fixed-Wing Aircraft

Agile fixed-wing aircraft aim to bridge the gap between rotor-craft and conventional fixed-wing aircraft, with the capability of maneuverable and even hovering flight like a rotor-craft, and of efficient long distance flight like a conventional fixed-wing aircraft. Avoiding obstacles in unknown environments is a challenging task with these platforms, as they have complicated dynamics and a limited payload, and they fly at high speeds. In this work, we present an obstacle-avoidance strategy that avoids collisions while steering the aircraft to the goal. The strategy does not rely on a prior map of the environment, or the ability to build a map in real-time, and can be run in real-time on-board the aircraft. We utilize a library of optimal trajectories, both conventional and aerobatic maneuvers, that are solved off-line. A sequence of these trajectories is pieced together to form a collision-free motion plan within the field of view of the depth camera that steers the aircraft towards the goal region. We validate the approach in a high-fidelity simulation environment. The aircraft flies autonomously through a forest-like map to a goal region, using conventional maneuvers such as banked and helical turns, as well as aerobatic maneuvers such as an aggressive turnaround.
Image Correspondence With CUR Decomposition-Based Graph Completion and Matching

Establishing correspondence between pictorial descriptions of two images is an important task and can be treated as graph matching problem. However, the process of extracting a favourable graph structure from raw images for matching is influenced by cluttered backgrounds and deformations, which may result in the abundance of noisy graph structures. This paper addresses the problem of point set correspondence and presents a robust graph matching method which recovers the correspondence matches among the graph nodes in a CUR based factorization framework. The graph representation in terms of CUR, inherently preserves the actual nodes connection in sparse manner, this particularly renders the complex space-time realization of affinities among graph nodes. The reformulation of graph matching in terms of small CUR factorization matrices, allows to compute and relax the partially observed graphs, without observing the whole large-scale graph matrix. In particular, we propose two variants of this approach, first, approximating the matching matrix from small CUR observed graph structure, and second, completing the graph structure with higher order CUR form to find correspondence. The CUR based matching algorithms are realized by computing set of compatibility coefficients from pairwise matching graphs and further conducting the probability relaxation procedure to find the matching confidences among nodes. Experiments and analysis on synthetic and natural images dataset prove the effectiveness of proposed methods against state-of-the-art methods. We also explore CUR matching for non-rigid moving object in a video sequence to demonstrate the potential application of graph matching to video analysis.
Integrity Risk-Based Model Predictive Control for Mobile Robots

This paper presents a Model Predictive Controller (MPC) that uses navigation integrity risk as a constraint. Navigation integrity risk accounts for the presence of faults in localization sensors and algorithms, an increasingly important consideration as the number of robots operating in life and mission-critical situations is expected to increase dramatically in near future (e.g. a potential influx of self-driving cars). Specifically, the work uses a local nearest neighbor integrity risk evaluation methodology that accounts for data association faults as a constraint in order to guarantee localization safety over a receding horizon. Moreover, state and control-input constraints have also been enforced in this work. The proposed MPC design is tested using real-world mapped environments, showing that a robot is capable of maintaining a predefined minimum level of localization safety while operating in an urban environment.
Energy Optimal Control Allocation in a Redundantly Actuated Omnidirectional UAV

This paper presents a novel actuation model and control allocation strategy for a redundantly-actuated multirotor unmanned aerial vehicle (UAV), referred to as the omnicopter. With an unconventional configuration, the omnicopter's eight propellers are able to produce all the six components of net force/torque, with two degrees of actuation redundancy. This enables the vehicle to execute motion trajectories unattainable with conventional underactuated multi-rotors. A new inverse actuator model is proposed that accounts for the significant interactions between propeller airflows by relating their output thrust forces to their input motor commands. Actuation redundancy is resolved by solving a convex constrained optimization problem. Its solution yields the most power efficient set of propellers thrusts that would produce a required net force/torque, while respecting the propeller thrust limits. When the required force/torque is infeasible due to the thrust limits, the solution would minimize the norm of the error between the desired and actual net force/torque vectors. Experimental results demonstrate the effectiveness of the proposed model and control allocation strategy.
All models are wrong, some are useful, but are they reproducible? Commentary on Lee et al. (2019)

Lee et al. (2019) make several practical recommendations for replicable and useful cognitive modeling. They also point out that the ultimate test of the usefulness of a cognitive model is its ability to solve practical problems. Solution-oriented modeling requires engaging practitioners who understand the relevantly applied domain but may lack extensive modeling expertise. In this commentary, we argue that for cognitive modeling to reach practitioners, there is a pressing need to move beyond providing the bare minimum information required for reproducibility and instead aim for an improved standard of transparency and reproducibility in cognitive modeling research. We discuss several mechanisms by which reproducible research can foster engagement with applied practitioners. Notably, reproducible materials provide a starting point for practitioners to experiment with cognitive models and evaluate whether they are suitable for their domain of expertise. This is essential because solving complex problems requires exploring a range of modeling approaches, and there may not be time to implement each possible approach from the ground up. Several specific recommendations for best practice are provided, including the application of containerization technologies. We also note the broader benefits of adopting gold standard reproducible practices within the field.
On Using Gated Recurrent Units for Nonlinear System Identification

During recent years Deep Learning (DL) methods facilitated impressive progress on various fields of research: Deep Convolutional Neural Networks (CNN) enabled object classification with to this day unmatched precision, while state of the art results in speech recognition and natural language processing (NLP) were achieved via gated units such as the LSTM. Although recurrent neural network architectures are long established in the field of system identification as a realization of an internal dynamics approach [1] [2], little research has yet been dedicated towards gated units. The purpose of this paper is to evaluate the architectures of recurrent gated units from the viewpoint of system identification and test their performance on a nonlinear system identification task.
A Perspective on Test Methodologies for Supervised Machine Learning Accelerators

Neural Network (NN) accelerators are essential in many emerging applications e.g., autonomous systems in making mission-critical decisions, health-care solutions to assist with diagnoses, etc. Any soft or hard failure during operation can potentially have catastrophic consequences in many of these applications. For instance, inaccurate classification during object recognition and tracking in autonomous vehicles can lead to crashes and subsequent injuries to the passengers. Therefore, testing Neural Network accelerators to ensure reliability and integrity of the underlying hardware is a crucial task to ensure the functionality, especially the ones that are used in missioncritical applications. Conventional functional, stuck-at and delay tests are not sufficient to characterize the ML systems since they face new test and validation challenges. This paper is aimed to provide a perspective on new test requirements and design for test techniques to cover ML features and detect various type of faults in NN accelerator. We discuss First-In-First-Out (FIFO) and Scratchpad based neural network hardware accelerators and propose test methods to detect the faults as well as fault location in different modules of the accelerator including MAC unit, Activation function module, and Processing Element (PE) registers.