OpenRaiser/Intern-Atlas huggingface dataset:
Preliminary experiments on thermal emissivity adjustment for face images
Venue: arXiv cs.CV 2022
Year: 2022

In this paper we summarize several applications based on thermal imaging. We emphasize the importance of emissivity adjustment for a proper temperature measurement. A new set of face images acquired at different emissivity values with steps of 0.01 is also presented and will be distributed for free for research purposes. Among the utilities, we can mention: a) the possibility to apply corrections once an image is acquired with a wrong emissivity value and it is not possible to acquire a new one; b) privacy protection in thermal images, which can be obtained with a low emissivity factor, which is still suitable for several applications, but hides the identity of a user; c) image processing for improving temperature detection in scenes containing objects of different emissivity.
A Fast Transformer-based General-Purpose Lossless Compressor
Venue: arXiv cs.LG 2022
Year: 2022

Deep-learning-based compressor has received interests recently due to much improved compression ratio. However, modern approaches suffer from long execution time. To ease this problem, this paper targets on cutting down the execution time of deep-learning-based compressors. Building history-dependencies sequentially (e.g., recurrent neural networks) is responsible for long inference latency. Instead, we introduce transformer into deep learning compressors to build history-dependencies in parallel. However, existing transformer is too heavy in computation and incompatible to compression tasks. This paper proposes a fast general-purpose lossless compressor, TRACE, by designing a compression-friendly structure based on a single-layer transformer. We first design a new metric to advise the selection part of compression model structures. Byte-grouping and Shared-ffn schemes are further proposed to fully utilize the capacity of the single-layer transformer. These features allow TRACE to achieve competitive compression ratio and a much faster speed. In addition, we further accelerate the compression procedure by designing a controller to reduce the parameter updating overhead. Experiments show that TRACE achieves an overall $\sim$3x speedup while keeps a comparable compression ratio to the state-of-the-art compressors. The source code for TRACE and links to the datasets are available at https://github.com/mynotwo/A-Fast-Transformer-based-General-Purpose-LosslessCompressor.
SIT: A Bionic and Non-Linear Neuron for Spiking Neural Network
Venue: arXiv cs.CV 2022
Year: 2022

Spiking Neural Networks (SNNs) have piqued researchers' interest because of their capacity to process temporal information and low power consumption. However, current state-of-the-art methods limited their biological plausibility and performance because their neurons are generally built on the simple Leaky-Integrate-and-Fire (LIF) model. Due to the high level of dynamic complexity, modern neuron models have seldom been implemented in SNN practice. In this study, we adopt the Phase Plane Analysis (PPA) technique, a technique often utilized in neurodynamics field, to integrate a recent neuron model, namely, the Izhikevich neuron. Based on the findings in the advancement of neuroscience, the Izhikevich neuron model can be biologically plausible while maintaining comparable computational cost with LIF neurons. By utilizing the adopted PPA, we have accomplished putting neurons built with the modified Izhikevich model into SNN practice, dubbed as the Standardized Izhikevich Tonic (SIT) neuron. For performance, we evaluate the suggested technique for image classification tasks in self-built LIF-and-SIT-consisted SNNs, named Hybrid Neural Network (HNN) on static MNIST, Fashion-MNIST, CIFAR-10 datasets and neuromorphic N-MNIST, CIFAR10-DVS, and DVS128 Gesture datasets. The experimental results indicate that the suggested method achieves comparable accuracy while exhibiting more biologically realistic behaviors on nearly all test datasets, demonstrating the efficiency of this novel strategy in bridging the gap between neurodynamics and SNN practice.
An I/O-Efficient Disk-based Graph System for Scalable Second-Order Random Walk of Large Graphs
Venue: arXiv cs.DB 2022
Year: 2022

Random walk is widely used in many graph analysis tasks, especially the first-order random walk. However, as a simplification of real-world problems, the first-order random walk is poor at modeling higher-order structures in the data. Recently, second-order random walk-based applications (e.g., Node2vec, Second-order PageRank) have become attractive. Due to the complexity of the second-order random walk models and memory limitations, it is not scalable to run second-order random walk-based applications on a single machine. Existing disk-based graph systems are only friendly to the first-order random walk models and suffer from expensive disk I/Os when executing the second-order random walks. This paper introduces an I/O-efficient disk-based graph system for the scalable second-order random walk of large graphs, called GraSorw. First, to eliminate massive light vertex I/Os, we develop a bi-block execution engine that converts random I/Os into sequential I/Os by applying a new triangular bi-block scheduling strategy, the bucket-based walk management, and the skewed walk storage. Second, to improve the I/O utilization, we design a learning-based block loading model to leverage the advantages of the full-load and on-demand load methods. Finally, we conducted extensive experiments on six large real datasets as well as several synthetic datasets. The empirical results demonstrate that the end-to-end time cost of popular tasks in GraSorw is reduced by more than one order of magnitude compared to the existing disk-based graph systems.
Enhancing AES Using Chaos and Logistic Map-Based Key Generation Technique for Securing IoT-Based Smart Home
Venue: arXiv cs.CR 2022
Year: 2022

The Internet of Things (IoT) has brought new ways for humans and machines to communicate with each other over the internet. Though sensor-driven devices have largely eased our everyday lives, most IoT infrastructures have been suffering from security challenges. Since the emergence of IoT, lightweight block ciphers have been a better option for intelligent and sensor-based applications. When public-key infrastructure dominates worldwide, the symmetric key encipherment such as Advanced Encryption Standard (AES) shows immense prospects to sit with the smart home IoT appliances. As investigated, chaos motivated logistic map shows enormous potential to secure IoT aligned real-time data communication. The unpredictability and randomness features of the logistic map in sync with chaos-based scheduling techniques can pave the way to build a particular dynamic key propagation technique for data confidentiality, availability and integrity. After being motivated by the security prospects of AES and chaos cryptography, the paper illustrates a key scheduling technique using a 3-dimensional S-box (substitution-box). The logistic map algorithm has been incorporated to enhance security. The proposed approach has applicability for lightweight IoT devices such as smart home appliances. The work determines how seeming chaos accelerates the desired key-initiation before message transmission. The proposed model is evaluated based on the key generation delay required for the smart-home sensor devices.
Sensor Data Validation and Driving Safety in Autonomous Driving Systems
Venue: arXiv cs.CV 2022
Year: 2022

Autonomous driving technology has drawn a lot of attention due to its fast development and extremely high commercial values. The recent technological leap of autonomous driving can be primarily attributed to the progress in the environment perception. Good environment perception provides accurate high-level environment information which is essential for autonomous vehicles to make safe and precise driving decisions and strategies. Moreover, such progress in accurate environment perception would not be possible without deep learning models and advanced onboard sensors, such as optical sensors (LiDARs and cameras), radars, GPS. However, the advanced sensors and deep learning models are prone to recently invented attack methods. For example, LiDARs and cameras can be compromised by optical attacks, and deep learning models can be attacked by adversarial examples. The attacks on advanced sensors and deep learning models can largely impact the accuracy of the environment perception, posing great threats to the safety and security of autonomous vehicles. In this thesis, we study the detection methods against the attacks on onboard sensors and the linkage between attacked deep learning models and driving safety for autonomous vehicles. To detect the attacks, redundant data sources can be exploited, since information distortions caused by attacks in victim sensor data result in inconsistency with the information from other redundant sources. To study the linkage between attacked deep learning models and driving safety...
Lay-it-out: Interactive Design of Layout-Sensitive Grammars
Venue: arXiv cs.PL 2022
Year: 2022

Layout-sensitive grammars have been adopted in many modern programming languages. However, tool support for this kind of grammars still remains limited and immature. In this paper, we present Lay-it-out, an interactive framework for layout-sensitive grammar design. Beginning with a user-defined ambiguous grammar, our framework refines it by synthesizing layout constraints through user interaction. For ease of interaction, a shortest nonempty ambiguous sentence (if exists) is automatically generated by our bounded ambiguity checker via SMT solving. The soundness and completeness of our SMT encoding are mechanized in the Coq proof assistant. Case studies on real grammars, including a full grammar, demonstrate the practicality and scalability of our approach.
Physics Community Needs, Tools, and Resources for Machine Learning
Venue: arXiv cs.LG 2022
Year: 2022

Machine learning (ML) is becoming an increasingly important component of cutting-edge physics research, but its computational requirements present significant challenges. In this white paper, we discuss the needs of the physics community regarding ML across latency and throughput regimes, the tools and resources that offer the possibility of addressing these needs, and how these can be best utilized and accessed in the coming years.
Interactive Multi-scale Fusion of 2D and 3D Features for Multi-object Tracking
Venue: arXiv cs.CV 2022
Year: 2022

Multiple object tracking (MOT) is a significant task in achieving autonomous driving. Traditional works attempt to complete this task, either based on point clouds (PC) collected by LiDAR, or based on images captured from cameras. However, relying on one single sensor is not robust enough, because it might fail during the tracking process. On the other hand, feature fusion from multiple modalities contributes to the improvement of accuracy. As a result, new techniques based on different sensors integrating features from multiple modalities are being developed. Texture information from RGB cameras and 3D structure information from Lidar have respective advantages under different circumstances. However, it's not easy to achieve effective feature fusion because of completely distinct information modalities. Previous fusion methods usually fuse the top-level features after the backbones extract the features from different modalities. In this paper, we first introduce PointNet++ to obtain multi-scale deep representations of point cloud to make it adaptive to our proposed Interactive Feature Fusion between multi-scale features of images and point clouds. Specifically, through multi-scale interactive query and fusion between pixel-level and point-level features, our method, can obtain more distinguishing features to improve the performance of multiple object tracking. Besides, we explore the effectiveness of pre-training on each single modality and fine-tuning on the fusion-based model. The experimental results demonstrate that our method can achieve good performance on the KITTI benchmark and outperform other approaches without using multi-scale feature fusion. Moreover, the ablation studies indicates the effectiveness of multi-scale feature fusion and pre-training on single modality.
AmsterTime: A Visual Place Recognition Benchmark Dataset for Severe Domain Shift
Venue: arXiv cs.CV 2022
Year: 2022

We introduce AmsterTime: a challenging dataset to benchmark visual place recognition (VPR) in presence of a severe domain shift. AmsterTime offers a collection of 2,500 well-curated images matching the same scene from a street view matched to historical archival image data from Amsterdam city. The image pairs capture the same place with different cameras, viewpoints, and appearances. Unlike existing benchmark datasets, AmsterTime is directly crowdsourced in a GIS navigation platform (Mapillary). We evaluate various baselines, including non-learning, supervised and self-supervised methods, pre-trained on different relevant datasets, for both verification and retrieval tasks. Our result credits the best accuracy to the ResNet-101 model pre-trained on the Landmarks dataset for both verification and retrieval tasks by 84% and 24%, respectively. Additionally, a subset of Amsterdam landmarks is collected for feature evaluation in a classification task. Classification labels are further used to extract the visual explanations using Grad-CAM for inspection of the learned similar visuals in a deep metric learning models.
AlgebraicSystems: Compositional Verification for Autonomous System Design
Venue: arXiv cs.LO 2022
Year: 2022

Autonomous systems require the management of several model views to assure properties such as safety and security among others. A crucial issue in autonomous systems design assurance is the notion of emergent behavior; we cannot use their parts in isolation to examine their overall behavior or performance. Compositional verification attempts to combat emergence by implementing model transformation as structure-preserving maps between model views. AlgebraicDynamics relies on categorical semantics to draw relationships between algebras and model views. We propose AlgebraicSystems, a conglomeration of algebraic methods to assign semantics and categorical primitives to give computational meaning to relationships between models so that the formalisms and resulting tools are interoperable through vertical and horizontal composition.
Learning multiobjective rough terrain traversability
Venue: arXiv cs.RO 2022
Year: 2022

We present a method that uses high-resolution topography data of rough terrain, and ground vehicle simulation, to predict traversability. Traversability is expressed as three independent measures: the ability to traverse the terrain at a target speed, energy consumption, and acceleration. The measures are continuous and reflect different objectives for planning that go beyond binary classification. A deep neural network is trained to predict the traversability measures from the local heightmap and target speed. To produce training data, we use an articulated vehicle with wheeled bogie suspensions and procedurally generated terrains. We evaluate the model on laser-scanned forest terrains, previously unseen by the model. The model predicts traversability with an accuracy of 90%. Predictions rely on features from the high-dimensional terrain data that surpass local roughness and slope relative to the heading. Correlations show that the three traversability measures are complementary to each other. With an inference speed 3000 times faster than the ground truth simulation and trivially parallelizable, the model is well suited for traversability analysis and optimal path planning over large areas.
A general approach for running Python codes in OpenFOAM using an embedded pybind11 Python interpreter
Venue: arXiv cs.CE 2022
Year: 2022

As the overlap between traditional computational mechanics and machine learning grows, there is an increasing demand for straight-forward approaches to interface Python-based procedures with C++-based OpenFOAM. This article introduces one such general methodology, allowing the execution of Python code directly within an OpenFOAM solver without the need for Python code translation. The proposed approach is based on the lightweight library pybind11, where OpenFOAM data is transferred to an embedded Python interpreter for manipulation, and results are returned as needed. Following a review of related approaches, the article describes the approach, with a particular focus on data transfer between Python and OpenFOAM, executing Python scripts and functions, and practical details about the implementation in OpenFOAM. Three complementary test cases are presented to highlight the functionality and demonstrate the effect of different data transfer approaches: a Python-based velocity profile boundary condition; a Python-based solver for prototyping; and a machine learning mechanical constitutive law class for solids4foam which performs field calculations.
Recognition of polar lows in Sentinel-1 SAR images with deep learning
Venue: arXiv cs.CV 2022
Year: 2022

In this paper, we explore the possibility of detecting polar lows in C-band SAR images by means of deep learning. Specifically, we introduce a novel dataset consisting of Sentinel-1 images divided into two classes, representing the presence and absence of a maritime mesocyclone, respectively. The dataset is constructed using the ERA5 dataset as baseline and it consists of 2004 annotated images. To our knowledge, this is the first dataset of its kind to be publicly released. The dataset is used to train a deep learning model to classify the labeled images. Evaluated on an independent test set, the model yields an F-1 score of 0.95, indicating that polar lows can be consistently detected from SAR images. Interpretability techniques applied to the deep learning model reveal that atmospheric fronts and cyclonic eyes are key features in the classification. Moreover, experimental results show that the model is accurate even if: (i) such features are significantly cropped due to the limited swath width of the SAR, (ii) the features are partly covered by sea ice and (iii) land is covering significant parts of the images. By evaluating the model performance on multiple input image resolutions (pixel sizes of 500m, 1km and 2km), it is found that higher resolution yield the best performance. This emphasises the potential of using high resolution sensors like SAR for detecting polar lows, as compared to conventionally used sensors such as scatterometers.
Intelligent Blockage Prediction and Proactive Handover for Seamless Connectivity in Vision-Aided 5G/6G UDNs
Venue: arXiv cs.NI 2022
Year: 2022

The upsurge in wireless devices and real-time service demands force the move to a higher frequency spectrum. Millimetre-wave (mmWave) and terahertz (THz) bands combined with the beamforming technology offer significant performance enhancements for ultra-dense networks (UDNs). Unfortunately, shrinking cell coverage and severe penetration loss experienced at higher spectrum render mobility management a critical issue in UDNs, especially optimizing beam blockages and frequent handover (HO). Mobility management challenges have become prevalent in city centres and urban areas. To address this, we propose a novel mechanism driven by exploiting wireless signals and on-road surveillance systems to intelligently predict possible blockages in advance and perform timely HO. This paper employs computer vision (CV) to determine obstacles and users' location and speed. In addition, this study introduces a new HO event, called block event {BLK}, defined by the presence of a blocking object and a user moving towards the blocked area. Moreover, the multivariate regression technique predicts the remaining time until the user reaches the blocked area, hence determining best HO decision. Compared to typical wireless networks without blockage prediction, simulation results show that our BLK detection and PHO algorithm achieves 40\% improvement in maintaining user connectivity and the required quality of experience (QoE).
AI Gone Astray: Technical Supplement
Venue: arXiv cs.LG 2022
Year: 2022

This study is a technical supplement to "AI gone astray: How subtle shifts in patient data send popular algorithms reeling, undermining patient safety." from STAT News, which investigates the effect of time drift on clinically deployed machine learning models. We use MIMIC-IV, a publicly available dataset, to train models that replicate commercial approaches by Dascena and Epic to predict the onset of sepsis, a deadly and yet treatable condition. We observe some of these models degrade overtime; most notably an RNN built on Epic features degrades from a 0.729 AUC to a 0.525 AUC over a decade, leading us to investigate technical and clinical drift as root causes of this performance drop.
Can in-home laboratories foster learning, self-efficacy, and motivation during the COVID-19 pandemic? -- A case study in two engineering programs
Venue: arXiv cs.CY 2022
Year: 2022

The COVID-19 pandemic has represented a challenge for higher education in terms to provide quality education despite the lockdown periods, the transformation of the in-person classes to virtual classes, and the demotivation and anxiety that are experimented by the students. Because the basis of engineering is the experimentation through hands-on activities and learning by doing, the lockdown periods and the temporary suspension of the in-person classes and laboratories have meant a problem for educators that try to teach and motivate the students despite the situation. In this context, this study presents an educational methodology based on Problem-Based Learning (PBL) and in-home laboratories in engineering. The methodology was carried out in two phases during 2020, in the academic programs of Industrial Engineering and Technology in Electronics with (n=44) students. The in-home laboratories were sent to the students as part of "kits" with the devices needed in each subject. Besides, due to the difficulties in monitoring the learning process, the students made videos and blogs as a strategy to reinforce their learning and evidence the progress in the courses. The outcomes of the methodology show mainly the following points: (1) An improvement of the academic performance and learning of the students in the courses. (2) A positive influence of the usage of in-home laboratories in motivation, self-efficacy, and reduction of anxiety. (3) Positive correlations between the usage of in-home laboratories, the blogs and videos, and the teacher's feedback for learning, motivation, and self-efficacy. Thus, these results evidence that other alternatives that gather the cognitive and affective learning domains can emerge from engineering to deal with the educational problems produced by the crisis periods.