OpenRaiser/Intern-Atlas huggingface dataset:
Using Features at Multiple Temporal and Spatial Resolutions to Predict Human Behavior in Real Time
Venue: arXiv cs.LG 2022
Year: 2022

When performing complex tasks, humans naturally reason at multiple temporal and spatial resolutions simultaneously. We contend that for an artificially intelligent agent to effectively model human teammates, i.e., demonstrate computational theory of mind (ToM), it should do the same. In this paper, we present an approach for integrating high and low-resolution spatial and temporal information to predict human behavior in real time and evaluate it on data collected from human subjects performing simulated urban search and rescue (USAR) missions in a Minecraft-based environment. Our model composes neural networks for high and low-resolution feature extraction with a neural network for behavior prediction, with all three networks trained simultaneously. The high-resolution extractor encodes dynamically changing goals robustly by taking as input the Manhattan distance difference between the humans' Minecraft avatars and candidate goals in the environment for the latest few actions, computed from a high-resolution gridworld representation. In contrast, the low-resolution extractor encodes participants' historical behavior using a historical state matrix computed from a low-resolution graph representation. Through supervised learning, our model acquires a robust prior for human behavior prediction, and can effectively deal with long-term observations. Our experimental results demonstrate that our method significantly improves prediction accuracy compared to approaches that only use high-resolution information.
Deep Reinforcement Learning with Vector Quantized Encoding
Venue: arXiv cs.LG 2022
Year: 2022

Human decision-making often involves combining similar states into categories and reasoning at the level of the categories rather than the actual states. Guided by this intuition, we propose a novel method for clustering state features in deep reinforcement learning (RL) methods to improve their interpretability. Specifically, we propose a plug-and-play framework termed \emph{vector quantized reinforcement learning} (VQ-RL) that extends classic RL pipelines with an auxiliary classification task based on vector quantized (VQ) encoding and aligns with policy training. The VQ encoding method categorizes features with similar semantics into clusters and results in tighter clusters with better separation compared to classic deep RL methods, thus enabling neural models to learn similarities and differences between states better. Furthermore, we introduce two regularization methods to help increase the separation between clusters and avoid the risks associated with VQ training. In simulations, we demonstrate that VQ-RL improves interpretability and investigate its impact on robustness and generalization of deep RL.
Lessons from Digital India for the Right to Internet Access
Venue: arXiv cs.NI 2022
Year: 2022

With only 65% of Indian houses having access to the Internet, digital India faces a significant Internet divide across gender and city types. Rendering essential services inaccessible to almost a third of the population necessitates not only provisioning a fundamental right to Internet access but taking specific constructive steps to assure its simple, affordable and safe accessibility. Establishing such a right would also pave way for other ancillary rights required for data privacy, protection from Internet's possible harms and the requirement to be treated fairly. We first discuss two arguments on the universal right to Internet access; from Merten Reglitz, a senior lecturer on Global Ethics and from Vincent Cerf, one of the founding creators of the Internet who has had a profound influence on the field. We specifically argue why Internet access should be treated as a fundamental right. We discuss the learnings from India, contextualizing them with the global debate and argue for establishing Internet access as a fundamental right in India and elsewhere in the form of government legislation to eliminate Internet divide.
Human Autonomy as a Design Principle for Socially Assistive Robots
Venue: arXiv cs.RO 2022
Year: 2022

High levels of robot autonomy are a common goal, but there is a significant risk that the greater the autonomy of the robot the lesser the autonomy of the human working with the robot. For vulnerable populations like older adults who already have a diminished level of autonomy, this is an even greater concern. We propose that human autonomy needs to be at the center of the design for socially assistive robots. Towards this goal, we define autonomy and then provide architectural requirements for social robots to support the user's autonomy. As an example of a design effort, we describe some of the features of our Assist architecture.
Adaptive Nonlinear MPC for Trajectory Tracking of An Overactuated Tiltrotor Hexacopter
Venue: arXiv cs.RO 2022
Year: 2022

Omnidirectional micro aerial vehicles (OMAVs) are more capable of doing environmentally interactive tasks due to their ability to exert full wrenches while maintaining stable poses. However, OMAVs often incorporate additional actuators and complex mechanical structures to achieve omnidirectionality. Obtaining precise mathematical models is difficult, and the mismatch between the model and the real physical system is not trivial. The large model-plant mismatch significantly degrades overall system performance if a non-adaptive model predictive controller (MPC) is used. This work presents the $\mathcal{L}_1$-MPC, an adaptive nonlinear model predictive controller for accurate 6-DOF trajectory tracking of an overactuated tiltrotor hexacopter in the presence of model uncertainties and external disturbances. The $\mathcal{L}_1$-MPC adopts a cascaded system architecture in which a nominal MPC is followed and augmented by an $\mathcal{L}_1$ adaptive controller. The proposed method is evaluated against the non-adaptive MPC, the EKF-MPC, and the PID method in both numerical and PX4 software-in-the-loop simulation with Gazebo. The $\mathcal{L}_1$-MPC reduces the tracking error by around 90% when compared to a non-adaptive MPC, and the $\mathcal{L}_1$-MPC has lower tracking errors, higher uncertainty estimation rates, and less tuning requirements over the EKF-MPC. We will make the implementations, including the hardware-verified PX4 firmware and Gazebo plugins, open-source at https://github.com/HITSZ-NRSL/omniHex.
Improving Deep Facial Phenotyping for Ultra-rare Disorder Verification Using Model Ensembles
Venue: arXiv cs.CV 2022
Year: 2022

Rare genetic disorders affect more than 6% of the global population. Reaching a diagnosis is challenging because rare disorders are very diverse. Many disorders have recognizable facial features that are hints for clinicians to diagnose patients. Previous work, such as GestaltMatcher, utilized representation vectors produced by a DCNN similar to AlexNet to match patients in high-dimensional feature space to support "unseen" ultra-rare disorders. However, the architecture and dataset used for transfer learning in GestaltMatcher have become outdated. Moreover, a way to train the model for generating better representation vectors for unseen ultra-rare disorders has not yet been studied. Because of the overall scarcity of patients with ultra-rare disorders, it is infeasible to directly train a model on them. Therefore, we first analyzed the influence of replacing GestaltMatcher DCNN with a state-of-the-art face recognition approach, iResNet with ArcFace. Additionally, we experimented with different face recognition datasets for transfer learning. Furthermore, we proposed test-time augmentation, and model ensembles that mix general face verification models and models specific for verifying disorders to improve the disorder verification accuracy of unseen ultra-rare disorders. Our proposed ensemble model achieves state-of-the-art performance on both seen and unseen disorders.
The Study of Complex Human Locomotion Behaviors: From Crawling to Walking
Venue: arXiv cs.RO 2022
Year: 2022

This paper uses a simple state machine to develop a control algorithm for controlling an infant humanoid in the context of a simple model system. The algorithm is inspired by a baby who starts learning to stand and walk at 7 to 12 months of age: he or she initially learns to crawl and then, once the lower limb muscles are strong enough, can learn to walk by coming to support his or her upper trunk. Ideally, this algorithm-supported locomotion can take the baby to any desired location: a pile of toys, a tasty snack, or the baby's parents or relatives. In this paper we analyze the crawling stage, the simple 2d bipedal model, and the initial walking form from 8 to 18 months of age, and quantitatively evaluate the ideal kinematics model and simulation results for these stages.
MicroISP: Processing 32MP Photos on Mobile Devices with Deep Learning
Venue: arXiv cs.CV 2022
Year: 2022

While neural networks-based photo processing solutions can provide a better image quality compared to the traditional ISP systems, their application to mobile devices is still very limited due to their very high computational complexity. In this paper, we present a novel MicroISP model designed specifically for edge devices, taking into account their computational and memory limitations. The proposed solution is capable of processing up to 32MP photos on recent smartphones using the standard mobile ML libraries and requiring less than 1 second to perform the inference, while for FullHD images it achieves real-time performance. The architecture of the model is flexible, allowing to adjust its complexity to devices of different computational power. To evaluate the performance of the model, we collected a novel Fujifilm UltraISP dataset consisting of thousands of paired photos captured with a normal mobile camera sensor and a professional 102MP medium-format FujiFilm GFX100 camera. The experiments demonstrated that, despite its compact size, the MicroISP model is able to provide comparable or better visual results than the traditional mobile ISP systems, while outperforming the previously proposed efficient deep learning based solutions. Finally, this model is also compatible with the latest mobile AI accelerators, achieving good runtime and low power consumption on smartphone NPUs and APUs. The code, dataset and pre-trained models are available on the project website: https://people.ee.ethz.ch/~ihnatova/microisp.html
Adversarial and Random Transformations for Robust Domain Adaptation and Generalization
Venue: arXiv cs.CV 2022
Year: 2022

Data augmentation has been widely used to improve generalization in training deep neural networks. Recent works show that using worst-case transformations or adversarial augmentation strategies can significantly improve the accuracy and robustness. However, due to the non-differentiable properties of image transformations, searching algorithms such as reinforcement learning or evolution strategy have to be applied, which are not computationally practical for large scale problems. In this work, we show that by simply applying consistency training with random data augmentation, state-of-the-art results on domain adaptation (DA) and generalization (DG) can be obtained. To further improve the accuracy and robustness with adversarial examples, we propose a differentiable adversarial data augmentation method based on spatial transformer networks (STN). The combined adversarial and random transformations based method outperforms the state-of-the-art on multiple DA and DG benchmark datasets. Besides, the proposed method shows desirable robustness to corruption, which is also validated on commonly used datasets.
Near-Linear Sample Complexity for $L_p$ Polynomial Regression
Venue: arXiv cs.DS 2022
Year: 2022

We study $L_p$ polynomial regression. Given query access to a function $f:[-1,1] \rightarrow \mathbb{R}$, the goal is to find a degree $d$ polynomial $\hat{q}$ such that, for a given parameter $\varepsilon > 0$, $$ \|\hat{q}-f\|_p\le (1+\varepsilon) \cdot \min_{q:\text{deg}(q)\le d}\|q-f\|_p. $$ Here $\|\cdot\|_p$ is the $L_p$ norm, $\|g\|_p = (\int_{-1}^1 |g(t)|^p dt)^{1/p}$. We show that querying $f$ at points randomly drawn from the Chebyshev measure on $[-1,1]$ is a near-optimal strategy for polynomial regression in all $L_p$ norms. In particular, to find $\hat q$, it suffices to sample $O(d\, \frac{\text{polylog}\,d}{\text{poly}\,\varepsilon})$ points from $[-1,1]$ with probabilities proportional to this measure. While the optimal sample complexity for polynomial regression was well understood for $L_2$ and $L_\infty$, our result is the first that achieves sample complexity linear in $d$ and error $(1+\varepsilon)$ for other values of $p$ without any assumptions. Our result requires two main technical contributions. The first concerns $p\leq 2$, for which we provide explicit bounds on the $L_p$ Lewis weight function of the infinite linear operator underlying polynomial regression. Using tools from the orthogonal polynomial literature, we show that this function is bounded by the Chebyshev density. Our second key contribution is to take advantage of the structure of polynomials to reduce the $p>2$ case to the $p\leq 2$ case. By doing so, we obtain a better sample complexity than what is possible for general $p$-norm linear regression problems, for which $Ω(d^{p/2})$ samples are required.
BTO-RRT: A rapid, optimal, smooth and point cloud-based path planning algorithm
Venue: arXiv cs.RO 2022
Year: 2022

This paper explores a rapid, optimal smooth path-planning algorithm for robots (e.g., autonomous vehicles) in point cloud environments. Derivative maps such as dense point clouds, mesh maps, Octomaps, etc. are frequently used for path planning purposes. A bi-directional target-oriented point planning algorithm, directly using point clouds to compute the optimized and dynamically feasible trajectories, is presented in this paper. This approach searches for obstacle-free, low computational cost, smooth, and dynamically feasible paths by analyzing a point cloud of the target environment, using a modified bi-directional and RRT-connect-based path planning algorithm, with a k-d tree-based obstacle avoidance strategy and three-step optimization. This presented approach bypasses the common 3D map discretization, directly leveraging point cloud data and it can be separated into two parts: modified RRT-based algorithm core and the three-step optimization. Simulations on 8 2D maps with different configurations and characteristics are presented to show the efficiency and 2D performance of the proposed algorithm. Benchmark comparison and evaluation with other RRT-based algorithms like RRT, B-RRT, and RRT star are also shown in the paper. Finally, the proposed algorithm successfully achieved different levels of mission goals on three 3D point cloud maps with different densities. The whole simulation proves that not only can our algorithm achieves a better performance on 2D maps compared with other algorithms, but also it can handle different tasks(ground vehicles and UAV applications) on different 3D point cloud maps, which shows the high performance and robustness of the proposed algorithm. The algorithm is open-sourced at \url{https://github.com/zhz03/BTO-RRT}
Out-of-Dynamics Imitation Learning from Multimodal Demonstrations
Venue: arXiv cs.RO 2022
Year: 2022

Existing imitation learning works mainly assume that the demonstrator who collects demonstrations shares the same dynamics as the imitator. However, the assumption limits the usage of imitation learning, especially when collecting demonstrations for the imitator is difficult. In this paper, we study out-of-dynamics imitation learning (OOD-IL), which relaxes the assumption to that the demonstrator and the imitator have the same state spaces but could have different action spaces and dynamics. OOD-IL enables imitation learning to utilize demonstrations from a wide range of demonstrators but introduces a new challenge: some demonstrations cannot be achieved by the imitator due to the different dynamics. Prior works try to filter out such demonstrations by feasibility measurements, but ignore the fact that the demonstrations exhibit a multimodal distribution since the different demonstrators may take different policies in different dynamics. We develop a better transferability measurement to tackle this newly-emerged challenge. We firstly design a novel sequence-based contrastive clustering algorithm to cluster demonstrations from the same mode to avoid the mutual interference of demonstrations from different modes, and then learn the transferability of each demonstration with an adversarial-learning based algorithm in each cluster. Experiment results on several MuJoCo environments, a driving environment, and a simulated robot environment show that the proposed transferability measurement more accurately finds and down-weights non-transferable demonstrations and outperforms prior works on the final imitation learning performance. We show the videos of our experiment results on our website.
Evaluating CNN with Oscillatory Activation Function
Venue: arXiv cs.LG 2022
Year: 2022

The reason behind CNNs capability to learn high-dimensional complex features from the images is the non-linearity introduced by the activation function. Several advanced activation functions have been discovered to improve the training process of neural networks, as choosing an activation function is a crucial step in the modeling. Recent research has proposed using an oscillating activation function to solve classification problems inspired by the human brain cortex. This paper explores the performance of one of the CNN architecture ALexNet on MNIST and CIFAR10 datasets using oscillatory activation function (GCU) and some other commonly used activation functions like ReLu, PReLu, and Mish.
Towards Energy Efficient Autonomous Exploration of Mars Lava Tube with a Martian Coaxial Quadrotor
Venue: arXiv cs.RO 2022
Year: 2022

Mapping and exploration of a Martian terrain with an aerial vehicle has become an emerging research direction, since the successful flight demonstration of the Mars helicopter Ingenuity. Although the autonomy and navigation capability of the state of the art Mars helicopter has proven to be efficient in an open environment, the next area of interest for exploration on Mars are caves or ancient lava tube like environments, especially towards the never-ending search of life on other planets. This article presents an autonomous exploration mission based on a modified frontier approach along with a risk aware planning and integrated collision avoidance scheme with a special focus on energy aspects of a custom designed Mars Coaxial Quadrotor (MCQ) in a Martian simulated lava tube. One of the biggest novelties of the article stems from addressing the exploration capability, while rapidly exploring in local areas and intelligently global re-positioning of the MCQ when reaching dead ends in order to to efficiently use the battery based consumed energy, while increasing the volume of the exploration. The proposed three layer cost based global re-position point selection assists in rapidly redirecting the MCQ to previously partially seen areas that could lead to more unexplored part of the lava tube. The Martian fully simulated mission presented in this article takes into consideration the fidelity of physics of Mars condition in terms of thin atmosphere, low surface pressure and low gravity of the planet, while proves the efficiency of the proposed scheme in exploring an area that is particularly challenging due to the subterranean-like environment. The proposed exploration-planning framework is also validated in simulation by comparing it against the graph based exploration planner.
Distributed Data-Driven Predictive Control for Multi-Agent Collaborative Legged Locomotion
Venue: arXiv cs.RO 2022
Year: 2022

The aim of this work is to define a planner that enables robust legged locomotion for complex multi-agent systems consisting of several holonomically constrained quadrupeds. To this end, we employ a methodology based on behavioral systems theory to model the sophisticated and high-dimensional structure induced by the holonomic constraints. The resulting model is then used in tandem with distributed control techniques such that the computational burden is shared across agents while the coupling between agents is preserved. Finally, this distributed model is framed in the context of a predictive controller, resulting in a robustly stable method for trajectory planning. This methodology is tested in simulation with up to five agents and is further experimentally validated on three A1 quadrupedal robots subject to various uncertainties, including payloads, rough terrain, and push disturbances.
Challenges in Close-Proximity Safe and Seamless Operation of Manned and Unmanned Aircraft in Shared Airspace
Venue: arXiv cs.RO 2022
Year: 2022

We propose developing an integrated system to keep autonomous unmanned aircraft safely separated and behave as expected in conjunction with manned traffic. The main goal is to achieve safe manned-unmanned vehicle teaming to improve system performance, have each (robot/human) teammate learn from each other in various aircraft operations, and reduce the manning needs of manned aircraft. The proposed system anticipates and reacts to other aircraft using natural language instructions and can serve as a co-pilot or operate entirely autonomously. We point out the main technical challenges where improvements on current state-of-the-art are needed to enable Visual Flight Rules to fully autonomous aerial operations, bringing insights to these critical areas. Furthermore, we present an interactive demonstration in a prototypical scenario with one AI pilot and one human pilot sharing the same terminal airspace, interacting with each other using language, and landing safely on the same runway. We also show a demonstration of a vision-only aircraft detection system.
Collaborative Application Security Testing for DevSecOps: An Empirical Analysis of Challenges, Best Practices and Tool Support
Venue: arXiv cs.SE 2022
Year: 2022

DevSecOps is a software development paradigm that places a high emphasis on the culture of collaboration between developers (Dev), security (Sec) and operations (Ops) teams to deliver secure software continuously and rapidly. Adopting this paradigm effectively, therefore, requires an understanding of the challenges, best practices and available solutions for collaboration among these functional teams. However, collaborative aspects related to these teams have received very little empirical attention in the DevSecOps literature. Hence, we present a study focusing on a key security activity, Application Security Testing (AST), in which practitioners face difficulties performing collaborative work in a DevSecOps environment. Our study made novel use of 48 systematically selected webinars, technical talks and panel discussions as a data source to qualitatively analyse software practitioner discussions on the most recent trends and emerging solutions in this highly evolving field. We find that the lack of features that facilitate collaboration built into the AST tools themselves is a key tool-related challenge in DevSecOps. In addition, the lack of clarity related to role definitions, shared goals, and ownership also hinders Collaborative AST (CoAST). We also captured a range of best practices for collaboration (e.g., Shift-left security), emerging communication methods (e.g., ChatOps), and new team structures (e.g., hybrid teams) for CoAST. Finally, our study identified several requirements for new tool features and specific gap areas for future research to provide better support for CoAST in DevSecOps.