OpenRaiser/Intern-Atlas huggingface dataset:
Uncertainty-Aware Boosted Ensembling in Multi-Modal Settings
Venue: arXiv cs.LG 2021
Year: 2021

Reliability of machine learning (ML) systems is crucial in safety-critical applications such as healthcare, and uncertainty estimation is a widely researched method to highlight the confidence of ML systems in deployment. Sequential and parallel ensemble techniques have shown improved performance of ML systems in multi-modal settings by leveraging the feature sets together. We propose an uncertainty-aware boosting technique for multi-modal ensembling in order to focus on the data points with higher associated uncertainty estimates, rather than the ones with higher loss values. We evaluate this method on healthcare tasks related to Dementia and Parkinson's disease which involve real-world multi-modal speech and text data, wherein our method shows an improved performance. Additional analysis suggests that introducing uncertainty-awareness into the boosted ensembles decreases the overall entropy of the system, making it more robust to heteroscedasticity in the data, as well as better calibrating each of the modalities along with high quality prediction intervals. We open-source our entire codebase at https://github.com/usarawgi911/Uncertainty-aware-boosting
Constantine: Automatic Side-Channel Resistance Using Efficient Control and Data Flow Linearization
Venue: arXiv cs.CR 2021
Year: 2021

In the era of microarchitectural side channels, vendors scramble to deploy mitigations for transient execution attacks, but leave traditional side-channel attacks against sensitive software (e.g., crypto programs) to be fixed by developers by means of constant-time programming (i.e., absence of secret-dependent code/data patterns). Unfortunately, writing constant-time code by hand is hard, as evidenced by the many flaws discovered in production side channel-resistant code. Prior efforts to automatically transform programs into constant-time equivalents offer limited security or compatibility guarantees, hindering their applicability to real-world software. In this paper, we present Constantine, a compiler-based system to automatically harden programs against microarchitectural side channels. Constantine pursues a radical design point where secret-dependent control and data flows are completely linearized (i.e., all involved code/data accesses are always executed). This strategy provides strong security and compatibility guarantees by construction, but its natural implementation leads to state explosion in real-world programs. To address this challenge, Constantine relies on carefully designed optimizations such as just-in-time loop linearization and aggressive function cloning for fully context-sensitive points-to analysis, which not only address state explosion, but also lead to an efficient and compatible solution. Constantine yields overheads as low as 16% on standard benchmarks and can handle a fully-fledged component from the production wolfSSL library.
Multi-task Learning with Attention for End-to-end Autonomous Driving
Venue: arXiv cs.RO 2021
Year: 2021

Autonomous driving systems need to handle complex scenarios such as lane following, avoiding collisions, taking turns, and responding to traffic signals. In recent years, approaches based on end-to-end behavioral cloning have demonstrated remarkable performance in point-to-point navigational scenarios, using a realistic simulator and standard benchmarks. Offline imitation learning is readily available, as it does not require expensive hand annotation or interaction with the target environment, but it is difficult to obtain a reliable system. In addition, existing methods have not specifically addressed the learning of reaction for traffic lights, which are a rare occurrence in the training datasets. Inspired by the previous work on multi-task learning and attention modeling, we propose a novel multi-task attention-aware network in the conditional imitation learning (CIL) framework. This does not only improve the success rate of standard benchmarks, but also the ability to react to traffic lights, which we show with standard benchmarks.
MVFuseNet: Improving End-to-End Object Detection and Motion Forecasting through Multi-View Fusion of LiDAR Data
Venue: arXiv cs.CV 2021
Year: 2021

In this work, we propose \textit{MVFuseNet}, a novel end-to-end method for joint object detection and motion forecasting from a temporal sequence of LiDAR data. Most existing methods operate in a single view by projecting data in either range view (RV) or bird's eye view (BEV). In contrast, we propose a method that effectively utilizes both RV and BEV for spatio-temporal feature learning as part of a temporal fusion network as well as for multi-scale feature learning in the backbone network. Further, we propose a novel sequential fusion approach that effectively utilizes multiple views in the temporal fusion network. We show the benefits of our multi-view approach for the tasks of detection and motion forecasting on two large-scale self-driving data sets, achieving state-of-the-art results. Furthermore, we show that MVFusenet scales well to large operating ranges while maintaining real-time performance.
Meta-learning for skin cancer detection using Deep Learning Techniques
Venue: arXiv cs.CV 2021
Year: 2021

This study focuses on automatic skin cancer detection using a Meta-learning approach for dermoscopic images. The aim of this study is to explore the benefits of the generalization of the knowledge extracted from non-medical data in the classification performance of medical data and the impact of the distribution shift problem within limited data by using a simple class and distribution balancer algorithm. In this study, a small sample of a combined dataset from 3 different sources was used to fine-tune a ResNet model pre-trained on non-medical data. The results show an increase in performance on detecting melanoma, malignant (skin cancer), and benign moles with the prior knowledge obtained from images of everyday objects from the ImageNet dataset by 20 points. These findings suggest that features from non-medical images can be used towards the classification of skin moles and that the distribution of the data affects the performance of the model.
Viking: Variational Bayesian Variance Tracking
Venue: arXiv cs.LG 2021
Year: 2021

We consider the problem of time series forecasting in an adaptive setting. We focus on the inference of state-space models under unknown and potentially time-varying noise variances. We introduce an augmented model in which the variances are represented as auxiliary gaussian latent variables in a tracking mode. As variances are nonnegative, a transformation is chosen and applied to these latent variables. The inference relies on the online variational Bayesian methodology, which consists in minimizing a Kullback-Leibler divergence at each time step. We observe that the minimum of the Kullback-Leibler divergence is an extension of the Kalman filter taking into account the variance uncertainty. We design a novel algorithm, named Viking, using these optimal recursive updates. For auxiliary latent variables, we use second-order bounds whose optimum admit closed-form solutions. Experiments on synthetic data show that Viking behaves well and is robust to misspecification.
Exploring 2D Data Augmentation for 3D Monocular Object Detection
Venue: arXiv cs.CV 2021
Year: 2021

Data augmentation is a key component of CNN based image recognition tasks like object detection. However, it is relatively less explored for 3D object detection. Many standard 2D object detection data augmentation techniques do not extend to 3D box. Extension of these data augmentations for 3D object detection requires adaptation of the 3D geometry of the input scene and synthesis of new viewpoints. This requires accurate depth information of the scene which may not be always available. In this paper, we evaluate existing 2D data augmentations and propose two novel augmentations for monocular 3D detection without a requirement for novel view synthesis. We evaluate these augmentations on the RTM3D detection model firstly due to the shorter training times . We obtain a consistent improvement by 4% in the 3D AP (@IoU=0.7) for cars, ~1.8% scores 3D AP (@IoU=0.25) for pedestrians & cyclists, over the baseline on KITTI car detection dataset. We also demonstrate a rigorous evaluation of the mAP scores by re-weighting them to take into account the class imbalance in the KITTI validation dataset.
The Road Less Travelled: Trying And Failing To Generate Walking Simulators
Venue: arXiv cs.AI 2021
Year: 2021

Automated game design is a rapidly growing area of research, yet many aspects of game design lie largely unexamined still, as most systems focus on two-dimensional games with clear objectives and goal-oriented gameplay. This paper describes several attempts to build an automated game designer for 3D games more focused on space, atmosphere and experience. We describe our attempts to build these systems, why they failed, and what steps and future work we believe would be useful for future attempts by others.
Multi-point Coordination in Massive MIMO Systems with Sectorized Antennas
Venue: arXiv cs.IT 2021
Year: 2021

Non-cooperative cellular massive MIMO, combined with power control, is known to lead to significant improvements in per-user throughput compared with conventional LTE technology. In this paper, we investigate further refinements to massive MIMO, first, in the form of three-fold sectorization, and second, coordinated multi-point operation (with and without sectorization), in which the three base stations cooperate in the joint service of their users. For these scenarios, we analyze the downlink performance for both maximum-ratio and zero-forcing precoding and derive closed-form lower-bound expressions on the achievable rate of the users. These expressions are then used to formulate power optimization problems with two throughput fairness criteria: i) network-wide max-min fairness, and ii) per-cell max-min fairness. Furthermore, we provide centralized and decentralized power control strategies to optimize the transmit powers in the network. We demonstrate that employing sectorized antenna elements mitigates the detrimental effects of pilot contamination by rejecting a portion of interfering pilots in the spatial domain during channel estimation phase. Simulation results with practical sectorized antennas reveal that sectorization and multi-point coordination combined with sectorization lead to more than 1.7x and 2.6x improvements in the 95%-likely per-user throughput, respectively.
COVID-19 and Big Data: Multi-faceted Analysis for Spatio-temporal Understanding of the Pandemic with Social Media Conversations
Venue: arXiv cs.SI 2021
Year: 2021

COVID-19 has been devastating the world since the end of 2019 and has continued to play a significant role in major national and worldwide events, and consequently, the news. In its wake, it has left no life unaffected. Having earned the world's attention, social media platforms have served as a vehicle for the global conversation about COVID-19. In particular, many people have used these sites in order to express their feelings, experiences, and observations about the pandemic. We provide a multi-faceted analysis of critical properties exhibited by these conversations on social media regarding the novel coronavirus pandemic. We present a framework for analysis, mining, and tracking the critical content and characteristics of social media conversations around the pandemic. Focusing on Twitter and Reddit, we have gathered a large-scale dataset on COVID-19 social media conversations. Our analyses cover tracking potential reports on virus acquisition, symptoms, conversation topics, and language complexity measures through time and by region across the United States. We also present a BERT-based model for recognizing instances of hateful tweets in COVID-19 conversations, which achieves a lower error-rate than the state-of-the-art performance. Our results provide empirical validation for the effectiveness of our proposed framework and further demonstrate that social media data can be efficiently leveraged to provide public health experts with inexpensive but thorough insight over the course of an outbreak.
Self-optimizing loop sifting and majorization for 3D reconstruction
Venue: arXiv cs.CV 2021
Year: 2021

Visual simultaneous localization and mapping (vSLAM) and 3D reconstruction methods have gone through impressive progress. These methods are very promising for autonomous vehicle and consumer robot applications because they can map large-scale environments such as cities and indoor environments without the need for much human effort. However, when it comes to loop detection and optimization, there is still room for improvement. vSLAM systems tend to add the loops very conservatively to reduce the severe influence of the false loops. These conservative checks usually lead to correct loops rejected, thus decrease performance. In this paper, an algorithm that can sift and majorize loop detections is proposed. Our proposed algorithm can compare the usefulness and effectiveness of different loops with the dense map posterior (DMP) metric. The algorithm tests and decides the acceptance of each loop without a single user-defined threshold. Thus it is adaptive to different data conditions. The proposed method is general and agnostic to sensor type (as long as depth or LiDAR reading presents), loop detection, and optimization methods. Neither does it require a specific type of SLAM system. Thus it has great potential to be applied to various application scenarios. Experiments are conducted on public datasets. Results show that the proposed method outperforms state-of-the-art methods.
Efficient LiDAR Odometry for Autonomous Driving
Venue: arXiv cs.RO 2021
Year: 2021

LiDAR odometry plays an important role in self-localization and mapping for autonomous navigation, which is usually treated as a scan registration problem. Although having achieved promising performance on KITTI odometry benchmark, the conventional searching tree-based approach still has the difficulty in dealing with the large scale point cloud efficiently. The recent spherical range image-based method enjoys the merits of fast nearest neighbor search by spherical mapping. However, it is not very effective to deal with the ground points nearly parallel to LiDAR beams. To address these issues, we propose a novel efficient LiDAR odometry approach by taking advantage of both non-ground spherical range image and bird's-eye-view map for ground points. Moreover, a range adaptive method is introduced to robustly estimate the local surface normal. Additionally, a very fast and memory-efficient model update scheme is proposed to fuse the points and their corresponding normals at different time-stamps. We have conducted extensive experiments on KITTI odometry benchmark, whose promising results demonstrate that our proposed approach is effective.
Soft Walks: Real-Time, Two-Ways Interaction between a Character and Loose Grounds
Venue: arXiv cs.GR 2021
Year: 2021

When walking on loose terrains, possibly covered with vegetation, the ground and grass should deform, but the character's gait should also change accordingly. We propose a method for modeling such two-ways interactions in real-time. We first complement a layered character model by a high-level controller, which uses position and angular velocity inputs to improve dynamic oscillations when walking on various slopes. Secondly, at a refined level, the feet are set to locally deform the ground and surrounding vegetation using efficient procedural functions, while the character's response to such deformations is computed through adapted inverse kinematics. While simple to set up, our method is generic enough to adapt to any character morphology. Moreover, its ability to generate in real time, consistent gaits on a variety of loose grounds of arbitrary slope, possibly covered with grass, makes it an interesting solution to enhance films and games.
Time series analysis with dynamic law exploration
Venue: arXiv cs.AI 2021
Year: 2021

In this paper we examine, how the dynamic laws governing the time evolution of a time series can be identified. We give a finite difference equation as well as a differential equation representation for that. We also study, how the required symmetries, like time reversal can be imposed on the laws. We study the compression performance of linear laws on sound data.
Künstliche Intelligenz, quo vadis?
Venue: arXiv cs.AI 2021
Year: 2021

This paper outlines the state of the art in AI. It then describes basic machine learning and knowledge processing techniques. Based on this, some possibilities and limitations of future AI developments are discussed.
PyArmadillo: a streamlined linear algebra library for Python
Venue: arXiv cs.MS 2021
Year: 2021

PyArmadillo is a linear algebra library for the Python language, with the aim of closely mirroring the programming interface of the widely used Armadillo C++ library, which in turn is deliberately similar to Matlab. PyArmadillo hence facilitates algorithm prototyping with Matlab-like syntax directly in Python, and relatively straightforward conversion of PyArmadillo-based Python code into performant Armadillo-based C++ code. The converted code can be used for purposes such as speeding up Python-based programs in conjunction with pybind11, or the integration of algorithms originally prototyped in Python into larger C++ codebases. PyArmadillo provides objects for matrices and cubes, as well as over 200 associated functions for manipulating data stored in the objects. Integer, floating point and complex numbers are supported. Various matrix factorisations are provided through integration with LAPACK, or one of its high performance drop-in replacements such as Intel MKL or OpenBLAS. PyArmadillo is open-source software, distributed under the Apache 2.0 license; it can be obtained at https://pyarma.sourceforge.io or via the Python Package Index in precompiled form.
Trust as Extended Control: Active Inference and User Feedback During Human-Robot Collaboration
Venue: arXiv cs.RO 2021
Year: 2021

To interact seamlessly with robots, users must infer the causes of a robot's behavior and be confident about that inference. Hence, trust is a necessary condition for human-robot collaboration (HRC). Despite its crucial role, it is largely unknown how trust emerges, develops, and supports human interactions with nonhuman artefacts. Here, we review the literature on trust, human-robot interaction, human-robot collaboration, and human interaction at large. Early models of trust suggest that trust entails a trade-off between benevolence and competence, while studies of human-to-human interaction emphasize the role of shared behavior and mutual knowledge in the gradual building of trust. We then introduce a model of trust as an agent's best explanation for reliable sensory exchange with an extended motor plant or partner. This model is based on the cognitive neuroscience of active inference and suggests that, in the context of HRC, trust can be cast in terms of virtual control over an artificial agent. In this setting, interactive feedback becomes a necessary component of the trustor's perception-action cycle. The resulting model has important implications for understanding human-robot interaction and collaboration, as it allows the traditional determinants of human trust to be defined in terms of active inference, information exchange and empowerment. Furthermore, this model suggests that boredom and surprise may be used as markers for under and over-reliance on the system. Finally, we examine the role of shared behavior in the genesis of trust, especially in the context of dyadic collaboration, suggesting important consequences for the acceptability and design of human-robot collaborative systems.
Uncertainty Quantification in Friction Model for Earthquakes using Bayesian inference
Venue: arXiv cs.CE 2021
Year: 2021

This work presents a framework to inversely quantify uncertainty in the model parameters of the friction model using earthquake data via the Bayesian inference. The forward model is the popular rate- and state- friction (RSF) model along with the spring slider damper idealization. The inverse model is to determine the model parameters using the earthquake data as the response of the RSF model. The conventional solution to the inverse problem is the deterministic parameter values, which may not represent the true value, and quantifying uncertainty in the model parameters increases confidence in the estimation. The uncertainty in the model parameters is estimated by the posterior distribution obtained through the Bayesian inversion.
Ackermann's Function in Iterative Form: A Proof Assistant Experiment
Venue: arXiv cs.LO 2021
Year: 2021

Ackermann's function can be expressed using an iterative algorithm, which essentially takes the form of a term rewriting system. Although the termination of this algorithm is far from obvious, its equivalence to the traditional recursive formulation--and therefore its totality--has a simple proof in Isabelle/HOL. This is a small example of formalising mathematics using a proof assistant, with a focus on the treatment of difficult recursions.
Noise-Robust Deep Spiking Neural Networks with Temporal Information
Venue: arXiv cs.NE 2021
Year: 2021

Spiking neural networks (SNNs) have emerged as energy-efficient neural networks with temporal information. SNNs have shown a superior efficiency on neuromorphic devices, but the devices are susceptible to noise, which hinders them from being applied in real-world applications. Several studies have increased noise robustness, but most of them considered neither deep SNNs nor temporal information. In this paper, we investigate the effect of noise on deep SNNs with various neural coding methods and present a noise-robust deep SNN with temporal information. With the proposed methods, we have achieved a deep SNN that is efficient and robust to spike deletion and jitter.
Maneuver-based Anchor Trajectory Hypotheses at Roundabouts
Venue: arXiv cs.CV 2021
Year: 2021

Predicting future behavior of the surrounding vehicles is crucial for self-driving platforms to safely navigate through other traffic. This is critical when making decisions like crossing an unsignalized intersection. We address the problem of vehicle motion prediction in a challenging roundabout environment by learning from human driver data. We extend existing recurrent encoder-decoder models to be advantageously combined with anchor trajectories to predict vehicle behaviors on a roundabout. Drivers' intentions are encoded by a set of maneuvers that correspond to semantic driving concepts. Accordingly, our model employs a set of maneuver-specific anchor trajectories that cover the space of possible outcomes at the roundabout. The proposed model can output a multi-modal distribution over the predicted future trajectories based on the maneuver-specific anchors. We evaluate our model using the public RounD dataset and the experiment results show the effectiveness of the proposed maneuver-based anchor regression in improving prediction accuracy, reducing the average RMSE to 28% less than the best baseline. Our code is available at https://github.com/m-hasan-n/roundabout.