OpenRaiser/Intern-Atlas huggingface dataset:
A Novel Intrinsic Image Decomposition Method to Recover Albedo for Aerial Images in Photogrammetry Processing
Venue: arXiv cs.CV 2022
Year: 2022

Recovering surface albedos from photogrammetric images for realistic rendering and synthetic environments can greatly facilitate its downstream applications in VR/AR/MR and digital twins. The textured 3D models from standard photogrammetric pipelines are suboptimal to these applications because these textures are directly derived from images, which intrinsically embedded the spatially and temporally variant environmental lighting information, such as the sun illumination, direction, causing different looks of the surface, making such models less realistic when used in 3D rendering under synthetic lightings. On the other hand, since albedo images are less variable by environmental lighting, it can, in turn, benefit basic photogrammetric processing. In this paper, we attack the problem of albedo recovery for aerial images for the photogrammetric process and demonstrate the benefit of albedo recovery for photogrammetry data processing through enhanced feature matching and dense matching. To this end, we proposed an image formation model with respect to outdoor aerial imagery under natural illumination conditions; we then, derived the inverse model to estimate the albedo by utilizing the typical photogrammetric products as an initial approximation of the geometry. The estimated albedo images are tested in intrinsic image decomposition, relighting, feature matching, and dense matching/point cloud generation results. Both synthetic and real-world experiments have demonstrated that our method outperforms existing methods and can enhance photogrammetric processing.
A Credible and Robust approach to Ego-Motion Estimation using an Automotive Radar
Venue: arXiv cs.RO 2022
Year: 2022

Consistent motion estimation is fundamental for all mobile autonomous systems. While this sounds like an easy task, often, it is not the case because of changing environmental conditions affecting odometry obtained from vision, Lidar, or the wheels themselves. Unsusceptible to challenging lighting and weather conditions, radar sensors are an obvious alternative. Usually, automotive radars return a sparse point cloud, representing the surroundings. Utilizing this information to motion estimation is challenging due to unstable and phantom measurements, which result in a high rate of outliers. We introduce a credible and robust probabilistic approach to estimate the ego-motion based on these challenging radar measurements; intended to be used within a loosely-coupled sensor fusion framework. Compared to existing solutions, evaluated on the popular nuScenes dataset and others, we show that our proposed algorithm is more credible while not depending on explicit correspondence calculation.
Custom Sine Waves Are Enough for Imitation Learning of Bipedal Gaits with Different Styles
Venue: arXiv cs.RO 2022
Year: 2022

Not until recently, robust bipedal locomotion has been achieved through reinforcement learning. However, existing implementations rely heavily on insights and efforts from human experts, which is costly for the iterative design of robot systems. Also, styles of the learned motion are strictly limited to that of the reference. In this paper, we propose a new way to learn bipedal locomotion from a simple sine wave as the reference for foot heights. With the naive human insight that the two feet should be lifted up alternatively and periodically, we experimentally demonstrate on the Cassie robot that, a simple reward function is able to make the robot learn to walk end-to-end and efficiently without any explicit knowledge of the model. With custom sine waves, the learned gait pattern can also have customized styles. Codes are released at github.com/WooQi57/sin-cassie-rl.
Towards Using Gameplay Videos for Detecting Issues in Video Games
Venue: arXiv cs.SE 2022
Year: 2022

Context. The game industry is increasingly growing in recent years. Every day, millions of people play video games, not only as a hobby, but also for professional competitions (e.g., e-sports or speed-running) or for making business by entertaining others (e.g., streamers). The latter daily produce a large amount of gameplay videos in which they also comment live what they experience. Since no software and, thus, no video game is perfect, streamers may encounter several problems (such as bugs, glitches, or performance issues). However, it is unlikely that they explicitly report such issues to developers. The identified problems may negatively impact the user's gaming experience and, in turn, can harm the reputation of the game and of the producer. Objective. We aim at proposing and empirically evaluating GELID, an approach for automatically extracting relevant information from gameplay videos by (i) identifying video segments in which streamers experienced anomalies; (ii) categorizing them based on their type and context in which appear (e.g., bugs or glitches appearing in a specific level or scene of the game); and (iii) clustering segments that regard the same specific issue. Method. We will build on top of existing approaches able to identify videos that are relevant for a specific video game. These represent the input of GELID that processes them to achieve the defined objectives. We will experiment GELID on several gameplay videos to understand the extent to which each of its steps is effective.
Internet of Robotic Things: Current Technologies and Applications
Venue: arXiv cs.NI 2022
Year: 2022

The Internet of Robotic Things (IoRT) is a new domain that aims to link the IoT environment with robotic systems and technologies. IoRT connects robotic systems, connects them to the cloud, and transfers critical information as well as knowledge exchange to conduct complicated and intricate activities that a human cannot readily perform. The pertinent notion of IoRT has been discussed in this paper, along with the issues that this area faces on a daily basis. Furthermore, technological applications have been examined in order to provide a better understanding of IoRT and its current development phenomenon. The study describes three layers of IoRT infrastructure: network and control, physical, and service and application layer. In the next section, IoRT problems have been presented, with a focus on data processing and the security and safety of IoRT technological systems. In addition to discussing the difficulties, appropriate solutions have been offered and recommended. IoRT is regarded as an essential technology with the ability to bring about a plethora of benefits in smart society upon adoption, contributing to the generation and development of smart cities and industries in the near future.
Online Emotions During the Storming of the U.S. Capitol: Evidence from the Social Media Network Parler
Venue: arXiv cs.SI 2022
Year: 2022

The storming of the U.S. Capitol on January 6, 2021 has led to the killing of 5 people and is widely regarded as an attack on democracy. The storming was largely coordinated through social media networks such as Parler. Yet little is known regarding how users interacted on Parler during the storming of the Capitol. In this work, we examine the emotion dynamics on Parler during the storming with regard to heterogeneity across time and users. For this, we segment the user base into different groups (e.g., Trump supporters and QAnon supporters). We use affective computing (Kratzwald et al. 2018) to infer the emotions in the contents, thereby allowing us to provide a comprehensive assessment of online emotions. Our evaluation is based on a large-scale dataset from Parler, comprising of 717,300 posts from 144,003 users. We find that the user base responded to the storming of the Capitol with an overall negative sentiment. Akin to this, Trump supporters also expressed a negative sentiment and high levels of unbelief. In contrast to that, QAnon supporters did not express a more negative sentiment during the storming. We further provide a cross-platform analysis and compare the emotion dynamics on Parler and Twitter. Our findings point at a comparatively less negative response to the incidents on Parler compared to Twitter accompanied by higher levels of disapproval and outrage. Our contribution to research is three-fold: (1) We identify online emotions that were characteristic of the storming; (2) we assess emotion dynamics across different user groups on Parler; (3) we compare the emotion dynamics on Parler and Twitter. Thereby, our work offers important implications for actively managing online emotions to prevent similar incidents in the future.
HBFL: A Hierarchical Blockchain-based Federated Learning Framework for a Collaborative IoT Intrusion Detection
Venue: arXiv cs.CR 2022
Year: 2022

The continuous strengthening of the security posture of IoT ecosystems is vital due to the increasing number of interconnected devices and the volume of sensitive data shared. The utilisation of Machine Learning (ML) capabilities in the defence against IoT cyber attacks has many potential benefits. However, the currently proposed frameworks do not consider data privacy, secure architectures, and/or scalable deployments of IoT ecosystems. In this paper, we propose a hierarchical blockchain-based federated learning framework to enable secure and privacy-preserved collaborative IoT intrusion detection. We highlight and demonstrate the importance of sharing cyber threat intelligence among inter-organisational IoT networks to improve the model's detection capabilities. The proposed ML-based intrusion detection framework follows a hierarchical federated learning architecture to ensure the privacy of the learning process and organisational data. The transactions (model updates) and processes will run on a secure immutable ledger, and the conformance of executed tasks will be verified by the smart contract. We have tested our solution and demonstrated its feasibility by implementing it and evaluating the intrusion detection performance using a key IoT data set. The outcome is a securely designed ML-based intrusion detection system capable of detecting a wide range of malicious activities while preserving data privacy.
BioRED: A Rich Biomedical Relation Extraction Dataset
Venue: arXiv cs.CL 2022
Year: 2022

Automated relation extraction (RE) from biomedical literature is critical for many downstream text mining applications in both research and real-world settings. However, most existing benchmarking datasets for bio-medical RE only focus on relations of a single type (e.g., protein-protein interactions) at the sentence level, greatly limiting the development of RE systems in biomedicine. In this work, we first review commonly used named entity recognition (NER) and RE datasets. Then we present BioRED, a first-of-its-kind biomedical RE corpus with multiple entity types (e.g., gene/protein, disease, chemical) and relation pairs (e.g., gene-disease; chemical-chemical) at the document level, on a set of 600 PubMed abstracts. Further, we label each relation as describing either a novel finding or previously known background knowledge, enabling automated algorithms to differentiate between novel and background information. We assess the utility of BioRED by benchmarking several existing state-of-the-art methods, including BERT-based models, on the NER and RE tasks. Our results show that while existing approaches can reach high performance on the NER task (F-score of 89.3%), there is much room for improvement for the RE task, especially when extracting novel relations (F-score of 47.7%). Our experiments also demonstrate that such a rich dataset can successfully facilitate the development of more accurate, efficient, and robust RE systems for biomedicine. The BioRED dataset and annotation guideline are freely available at https://ftp.ncbi.nlm.nih.gov/pub/lu/BioRED/.
Classification of Natural Language Processing Techniques for Requirements Engineering
Venue: arXiv cs.CL 2022
Year: 2022

Research in applying natural language processing (NLP) techniques to requirements engineering (RE) tasks spans more than 40 years, from initial efforts carried out in the 1980s to more recent attempts with machine learning (ML) and deep learning (DL) techniques. However, in spite of the progress, our recent survey shows that there is still a lack of systematic understanding and organization of commonly used NLP techniques in RE. We believe one hurdle facing the industry is lack of shared knowledge of NLP techniques and their usage in RE tasks. In this paper, we present our effort to synthesize and organize 57 most frequently used NLP techniques in RE. We classify these NLP techniques in two ways: first, by their NLP tasks in typical pipelines and second, by their linguist analysis levels. We believe these two ways of classification are complementary, contributing to a better understanding of the NLP techniques in RE and such understanding is crucial to the development of better NLP tools for RE.
FikoRE: 5G and Beyond RAN Emulator for Application Level Experimentation and Prototyping
Venue: arXiv cs.NI 2022
Year: 2022

Novel and cutting-edge use cases have arisen since the first deployments of the fifth generation of telecommunication networks (5G). There are plenty of well-though optimally design 5G simulators and emulators which allow telecommunication technologies engineers and researchers to thoroughly study and test the network. However, the 5G ecosystem is not only limited to the network itself: a fast development of 5G-specific use cases can considerably accelerate the development of telecommunication technologies. We present FikoRE, our real-time Radio Access Networks (RAN) emulator carefully designed for application-level experimentation and prototyping. Its modularity and straightforward implementation allow multidisciplinary user to rapidly use or even modify it to test their own applications. In this article, we present FikoRE's architecture accompanied with relevant validation experiments and results.
Improved Object Pose Estimation via Deep Pre-touch Sensing
Venue: arXiv cs.RO 2022
Year: 2022

For certain manipulation tasks, object pose estimation from head-mounted cameras may not be sufficiently accurate. This is at least in part due to our inability to perfectly calibrate the coordinate frames of today's high degree of freedom robot arms that link the head to the end-effectors. We present a novel framework combining pre-touch sensing and deep learning to more accurately estimate pose in an efficient manner. The use of pre-touch sensing allows our method to localize the object directly with respect to the robot's end effector, thereby avoiding error caused by miscalibration of the arms. Instead of requiring the robot to scan the entire object with its pre-touch sensor, we use a deep neural network to detect object regions that contain distinctive geometric features. By focusing pre-touch sensing on these regions, the robot can more efficiently gather the information necessary to adjust its original pose estimate. Our region detection network was trained using a new dataset containing objects of widely varying geometries and has been labeled in a scalable fashion that is free from human bias. This dataset is applicable to any task that involves a pre-touch sensor gathering geometric information, and has been made publicly available. We evaluate our framework by having the robot re-estimate the pose of a number of objects of varying geometries. Compared to two simpler region proposal methods, we find that our deep neural network performs significantly better. In addition, we find that after a sequence of scans, objects can typically be localized to within 0.5 cm of their true position. We also observe that the original pose estimate can often be significantly improved after collecting a single quick scan.
Segmenting across places: The need for fair transfer learning with satellite imagery
Venue: arXiv cs.CV 2022
Year: 2022

The increasing availability of high-resolution satellite imagery has enabled the use of machine learning to support land-cover measurement and inform policy-making. However, labelling satellite images is expensive and is available for only some locations. This prompts the use of transfer learning to adapt models from data-rich locations to others. Given the potential for high-impact applications of satellite imagery across geographies, a systematic assessment of transfer learning implications is warranted. In this work, we consider the task of land-cover segmentation and study the fairness implications of transferring models across locations. We leverage a large satellite image segmentation benchmark with 5987 images from 18 districts (9 urban and 9 rural). Via fairness metrics we quantify disparities in model performance along two axes -- across urban-rural locations and across land-cover classes. Findings show that state-of-the-art models have better overall accuracy in rural areas compared to urban areas, through unsupervised domain adaptation methods transfer learning better to urban versus rural areas and enlarge fairness gaps. In analysis of reasons for these findings, we show that raw satellite images are overall more dissimilar between source and target districts for rural than for urban locations. This work highlights the need to conduct fairness analysis for satellite imagery segmentation models and motivates the development of methods for fair transfer learning in order not to introduce disparities between places, particularly urban and rural locations.
Beyond 3DMM: Learning to Capture High-fidelity 3D Face Shape
Venue: arXiv cs.CV 2022
Year: 2022

3D Morphable Model (3DMM) fitting has widely benefited face analysis due to its strong 3D priori. However, previous reconstructed 3D faces suffer from degraded visual verisimilitude due to the loss of fine-grained geometry, which is attributed to insufficient ground-truth 3D shapes, unreliable training strategies and limited representation power of 3DMM. To alleviate this issue, this paper proposes a complete solution to capture the personalized shape so that the reconstructed shape looks identical to the corresponding person. Specifically, given a 2D image as the input, we virtually render the image in several calibrated views to normalize pose variations while preserving the original image geometry. A many-to-one hourglass network serves as the encode-decoder to fuse multiview features and generate vertex displacements as the fine-grained geometry. Besides, the neural network is trained by directly optimizing the visual effect, where two 3D shapes are compared by measuring the similarity between the multiview images rendered from the shapes. Finally, we propose to generate the ground-truth 3D shapes by registering RGB-D images followed by pose and shape augmentation, providing sufficient data for network training. Experiments on several challenging protocols demonstrate the superior reconstruction accuracy of our proposal on the face shape.
Investigating Deep Learning Benchmarks for Electrocardiography Signal Processing
Venue: arXiv cs.LG 2022
Year: 2022

In recent years, deep learning has witnessed its blossom in the field of Electrocardiography (ECG) processing, outperforming traditional signal processing methods in various tasks, for example, classification, QRS detection, wave delineation. Although many neural architectures have been proposed in the literature, there is a lack of systematic studies and open-source libraries for ECG deep learning. In this paper, we propose a deep learning framework, named \texttt{torch\_ecg}, which gathers a large number of neural networks, both from literature and novel, for various ECG processing tasks. It establishes a convenient and modular way for automatic building and flexible scaling of the networks, as well as a neat and uniform way of organizing the preprocessing procedures and augmentation techniques for preparing the input data for the models. Besides, \texttt{torch\_ecg} provides benchmark studies using the latest databases, illustrating the principles and pipelines for solving ECG processing tasks and reproducing results from the literature. \texttt{torch\_ecg} offers the ECG research community a powerful tool meeting the growing demand for the application of deep learning techniques.
Adaptive Differential Filters for Fast and Communication-Efficient Federated Learning
Venue: arXiv cs.LG 2022
Year: 2022

Federated learning (FL) scenarios inherently generate a large communication overhead by frequently transmitting neural network updates between clients and server. To minimize the communication cost, introducing sparsity in conjunction with differential updates is a commonly used technique. However, sparse model updates can slow down convergence speed or unintentionally skip certain update aspects, e.g., learned features, if error accumulation is not properly addressed. In this work, we propose a new scaling method operating at the granularity of convolutional filters which 1) compensates for highly sparse updates in FL processes, 2) adapts the local models to new data domains by enhancing some features in the filter space while diminishing others and 3) motivates extra sparsity in updates and thus achieves higher compression ratios, i.e., savings in the overall data transfer. Compared to unscaled updates and previous work, experimental results on different computer vision tasks (Pascal VOC, CIFAR10, Chest X-Ray) and neural networks (ResNets, MobileNets, VGGs) in uni-, bidirectional and partial update FL settings show that the proposed method improves the performance of the central server model while converging faster and reducing the total amount of transmitted data by up to 377 times.
Understanding, Detecting, and Separating Out-of-Distribution Samples and Adversarial Samples in Text Classification
Venue: arXiv cs.CL 2022
Year: 2022

In this paper, we study the differences and commonalities between statistically out-of-distribution (OOD) samples and adversarial (Adv) samples, both of which hurting a text classification model's performance. We conduct analyses to compare the two types of anomalies (OOD and Adv samples) with the in-distribution (ID) ones from three aspects: the input features, the hidden representations in each layer of the model, and the output probability distributions of the classifier. We find that OOD samples expose their aberration starting from the first layer, while the abnormalities of Adv samples do not emerge until the deeper layers of the model. We also illustrate that the models' output probabilities for Adv samples tend to be more unconfident. Based on our observations, we propose a simple method to separate ID, OOD, and Adv samples using the hidden representations and output probabilities of the model. On multiple combinations of ID, OOD datasets, and Adv attacks, our proposed method shows exceptional results on distinguishing ID, OOD, and Adv samples.
Finding the Right Place: Sensor Placement for UWB Time Difference of Arrival Localization in Cluttered Indoor Environments
Venue: arXiv cs.RO 2022
Year: 2022

Ultra-wideband (UWB) time difference of arrival (TDOA)-based localization has recently emerged as a promising indoor positioning solution. However, in cluttered environments, both the UWB radio positions and the obstacle-induced non-line-of-sight (NLOS) measurement biases significantly impact the quality of the position estimate. Consequently, the placement of the UWB radios must be carefully designed to provide satisfactory localization accuracy for a region of interest. In this work, we propose a novel algorithm that optimizes the UWB radio positions for a pre-defined region of interest in the presence of obstacles. The mean-squared error (MSE) metric is used to formulate an optimization problem that balances the influence of the geometry of the radio positions and the NLOS effects. We further apply the proposed algorithm to compute a minimal number of UWB radios required for a desired localization accuracy and their corresponding positions. In a real-world cluttered environment, we show that the designed UWB radio placements provide 47% and 76% localization root-mean-squared error (RMSE) reduction in 2D and 3D experiments, respectively, when compared against trivial placements.