OpenRaiser/Intern-Atlas huggingface dataset:
Machine-learned 3D Building Vectorization from Satellite Imagery
Venue: arXiv cs.CV 2021
Year: 2021

We propose a machine learning based approach for automatic 3D building reconstruction and vectorization. Taking a single-channel photogrammetric digital surface model (DSM) and panchromatic (PAN) image as input, we first filter out non-building objects and refine the building shapes of input DSM with a conditional generative adversarial network (cGAN). The refined DSM and the input PAN image are then used through a semantic segmentation network to detect edges and corners of building roofs. Later, a set of vectorization algorithms are proposed to build roof polygons. Finally, the height information from the refined DSM is added to the polygons to obtain a fully vectorized level of detail (LoD)-2 building model. We verify the effectiveness of our method on large-scale satellite images, where we obtain state-of-the-art performance.
A First Look at Scams on YouTube
Venue: arXiv cs.CR 2021
Year: 2021

YouTube has become the second most popular website according to Alexa, and it represents an enticing platform for scammers to attract victims. Because of the computational difficulty of classifying multimedia, identifying scams on YouTube is more difficult than text-based media. As a consequence, the research community to-date has provided little insight into the prevalence, lifetime, and operational patterns of scammers on YouTube. In this short paper, we present a preliminary exploration of scam videos on YouTube. We begin by identifying 74 search queries likely to lead to scam videos based on the authors' experience seeing scams during routine browsing. We then manually review and characterize the results to identify 668 scams in 3,700 videos. In a detailed analysis of our classifications and metadata, we find that these scam videos have a median lifetime of nearly nine months, and many rely on external websites for monetization. We also explore the potential of detecting scams from metadata alone, finding that metadata does not have enough predictive power to distinguish scams from legitimate videos. Our work demonstrates that scams are a real problem for YouTube users, motivating future work on this topic.
BERT Embeddings Can Track Context in Conversational Search
Venue: arXiv cs.IR 2021
Year: 2021

The use of conversational assistants to search for information is becoming increasingly more popular among the general public, pushing the research towards more advanced and sophisticated techniques. In the last few years, in particular, the interest in conversational search is increasing, not only because of the generalization of conversational assistants but also because conversational search is a step forward in allowing a more natural interaction with the system. In this work, the focus is on exploring the context present of the conversation via the historical utterances and respective embeddings with the aim of developing a conversational search system that helps people search for information in a natural way. In particular, this system must be able to understand the context where the question is posed, tracking the current state of the conversation and detecting mentions to previous questions and answers. We achieve this by using a context-tracking component based on neural query-rewriting models. Another crucial aspect of the system is to provide the most relevant answers given the question and the conversational history. To achieve this objective, we used a Transformer-based re-ranking method and expanded this architecture to use the conversational context. The results obtained with the system developed showed the advantages of using the context present in the natural language utterances and in the neural embeddings generated throughout the conversation.
Simultaneous Face Hallucination and Translation for Thermal to Visible Face Verification using Axial-GAN
Venue: arXiv cs.CV 2021
Year: 2021

Existing thermal-to-visible face verification approaches expect the thermal and visible face images to be of similar resolution. This is unlikely in real-world long-range surveillance systems, since humans are distant from the cameras. To address this issue, we introduce the task of thermal-to-visible face verification from low-resolution thermal images. Furthermore, we propose Axial-Generative Adversarial Network (Axial-GAN) to synthesize high-resolution visible images for matching. In the proposed approach we augment the GAN framework with axial-attention layers which leverage the recent advances in transformers for modelling long-range dependencies. We demonstrate the effectiveness of the proposed method by evaluating on two different thermal-visible face datasets. When compared to related state-of-the-art works, our results show significant improvements in both image quality and face verification performance, and are also much more efficient.
Coinbugs: Enumerating Common Blockchain Implementation-Level Vulnerabilities
Venue: arXiv cs.CR 2021
Year: 2021

A good amount of effort has been dedicated to surveying and systematizing Ethereum smart contract security bug classes. There is, however, a gap in literature when it comes to surveying implementation-level security bugs that commonly occur in basic PoW blockchain node implementations, discovered during the first decade of Bitcoin's existence. This paper attempts to fill this void. In particular, if software which participates in a network by validating and generating new blocks is developed from scratch, WCGW - What Could Go Wrong? Ten broad bug type categories are listed and for each category, known examples are linked. Blockchain, as designed by the Satoshi's paper is exciting and introduces several novel bug classes which are interesting to security researchers. The paper is aimed at security testers aiming to start out in blockchain security reviews and blockchain developers as a reference on common pitfalls.
PMIPv6 Integrated with MIH for Flow Mobility Management: a Real Testbed with Simultaneous Multi-Access in Heterogeneous Mobile Networks
Venue: arXiv cs.NI 2021
Year: 2021

The exponential growth of the number of multihomed mobile devices is changing the way how we can connect to the Internet. Our mobile devices are demanding for more network resources, in terms of traffic volume and QoS requirements. Unfortunately, it is very hard to a multihomed device to be simultaneously connected to the network through multiple links. The current work enhances the network access of multihomed devices agnostically to the deployed access technologies. This enhancement is achieved by using simultaneously all of the mobile devices interfaces, and by routing each individual data flow through the most convenient access technology. The proposed solution is only deployed at the network side and it extends Proxy Mobile IPv6 with flow mobility in a completely transparent way to mobile nodes. In fact, it gives particular attention to the handover mechanisms, by improving the detection and attachment of nodes in the network, with the inclusion of the IEEE 802.21 standard in the solution. This provides the necessary implementation and integration details to extend a network topology with femtocell devices. Each femtocell is equipped with various network interfaces supporting a diverse set of access technologies. There is also a decision entity that manages individually each data flow according to its QoS / QoE requisites. The proposed solution has been developed and extensively tested with a real prototype. Evaluation results evidence that the overhead for using the solution is negligible as compared to the offered advantages such as: the support of flow mobility, the fulfil of VoIP functional requisites, the session continuity in spite of flows mobility, its low overhead, its high scalability, and the complete transparency of the proposed solution to the user terminals.
Towards Causal Federated Learning For Enhanced Robustness and Privacy
Venue: arXiv cs.LG 2021
Year: 2021

Federated Learning is an emerging privacy-preserving distributed machine learning approach to building a shared model by performing distributed training locally on participating devices (clients) and aggregating the local models into a global one. As this approach prevents data collection and aggregation, it helps in reducing associated privacy risks to a great extent. However, the data samples across all participating clients are usually not independent and identically distributed (non-iid), and Out of Distribution(OOD) generalization for the learned models can be poor. Besides this challenge, federated learning also remains vulnerable to various attacks on security wherein a few malicious participating entities work towards inserting backdoors, degrading the generated aggregated model as well as inferring the data owned by participating entities. In this paper, we propose an approach for learning invariant (causal) features common to all participating clients in a federated learning setup and analyze empirically how it enhances the Out of Distribution (OOD) accuracy as well as the privacy of the final learned model.
GAN-Based Interactive Reinforcement Learning from Demonstration and Human Evaluative Feedback
Venue: arXiv cs.LG 2021
Year: 2021

Deep reinforcement learning (DRL) has achieved great successes in many simulated tasks. The sample inefficiency problem makes applying traditional DRL methods to real-world robots a great challenge. Generative Adversarial Imitation Learning (GAIL) -- a general model-free imitation learning method, allows robots to directly learn policies from expert trajectories in large environments. However, GAIL shares the limitation of other imitation learning methods that they can seldom surpass the performance of demonstrations. In this paper, to address the limit of GAIL, we propose GAN-Based Interactive Reinforcement Learning (GAIRL) from demonstration and human evaluative feedback by combining the advantages of GAIL and interactive reinforcement learning. We tested our proposed method in six physics-based control tasks, ranging from simple low-dimensional control tasks -- Cart Pole and Mountain Car, to difficult high-dimensional tasks -- Inverted Double Pendulum, Lunar Lander, Hopper and HalfCheetah. Our results suggest that with both optimal and suboptimal demonstrations, a GAIRL agent can always learn a more stable policy with optimal or close to optimal performance, while the performance of the GAIL agent is upper bounded by the performance of demonstrations or even worse than it. In addition, our results indicate the reason that GAIRL is superior over GAIL is the complementary effect of demonstrations and human evaluative feedback.
Perception Entropy: A Metric for Multiple Sensors Configuration Evaluation and Design
Venue: arXiv cs.RO 2021
Year: 2021

Sensor configuration, including the sensor selections and their installation locations, serves a crucial role in autonomous driving. A well-designed sensor configuration significantly improves the performance upper bound of the perception system. However, as leveraging multiple sensors is becoming the mainstream setting, existing methods mainly focusing on single-sensor configuration problems are hardly utilized in practice. To tackle these issues, we propose a novel method based on conditional entropy in Bayesian theory to evaluate the sensor configurations containing both cameras and LiDARs. Correspondingly, an evaluation metric, perception entropy, is introduced to measure the difference between two configurations, which considers both the perception algorithm performance and the selections of the sensors. To the best of our knowledge, this is the first method to tackle the multi-sensor configuration problem for autonomous vehicles. The simulation results, extensive comparisons, and analysis all demonstrate the superior performance of our proposed approach.
Towards an Interpretable Data-driven Trigger System for High-throughput Physics Facilities
Venue: arXiv cs.AI 2021
Year: 2021

Data-intensive science is increasingly reliant on real-time processing capabilities and machine learning workflows, in order to filter and analyze the extreme volumes of data being collected. This is especially true at the energy and intensity frontiers of particle physics where bandwidths of raw data can exceed 100 Tb/s of heterogeneous, high-dimensional data sourced from hundreds of millions of individual sensors. In this paper, we introduce a new data-driven approach for designing and optimizing high-throughput data filtering and trigger systems such as those in use at physics facilities like the Large Hadron Collider (LHC). Concretely, our goal is to design a data-driven filtering system with a minimal run-time cost for determining which data event to keep, while preserving (and potentially improving upon) the distribution of the output as generated by the hand-designed trigger system. We introduce key insights from interpretable predictive modeling and cost-sensitive learning in order to account for non-local inefficiencies in the current paradigm and construct a cost-effective data filtering and trigger model that does not compromise physics coverage.
Jointly Learning Truth-Conditional Denotations and Groundings using Parallel Attention
Venue: arXiv cs.CL 2021
Year: 2021

We present a model that jointly learns the denotations of words together with their groundings using a truth-conditional semantics. Our model builds on the neurosymbolic approach of Mao et al. (2019), learning to ground objects in the CLEVR dataset (Johnson et al., 2017) using a novel parallel attention mechanism. The model achieves state of the art performance on visual question answering, learning to detect and ground objects with question performance as the only training signal. We also show that the model is able to learn flexible non-canonical groundings just by adjusting answers to questions in the training set.
Influenza Surveillance using Search Engine, SNS, On-line Shopping, Q&A Service and Past Flu Patients
Venue: arXiv cs.CY 2021
Year: 2021

Influenza, an infectious disease, causes many deaths worldwide. Predicting influenza victims during epidemics is an important task for clinical, hospital, and community outbreak preparation. On-line user-generated contents (UGC), primarily in the form of social media posts or search query logs, are generally used for prediction for reaction to sudden and unusual outbreaks. However, most studies rely only on the UGC as their resource and do not use various UGCs. Our study aims to solve these questions about Influenza prediction: Which model is the best? What combination of multiple UGCs works well? What is the nature of each UGC? We adapt some models, LASSO Regression, Huber Regression, Support Vector Machine regression with Linear kernel (SVR) and Random Forest, to test the influenza volume prediction in Japan during 2015 - 2018. For that, we use on-line five data resources: (1) past flu patients, (2) SNS (Twitter), (3) search engines (Yahoo! Japan), (4) shopping services (Yahoo! Shopping), and (5) Q&A services (Yahoo! Chiebukuro) as resources of each model. We then validate respective resources contributions using the best model, Huber Regression, with all resources except one resource. Finally, we use Bayesian change point method for ascertaining whether the trend of time series on any resources is reflected in the trend of flu patient count or not. Our experiments show Huber Regression model based on various data resources produces the most accurate results. Then, from the change point analysis, we get the result that search query logs and social media posts for three years represents these resources as a good predictor. Conclusions: We show that Huber Regression based on various data resources is strong for outliers and is suitable for the flu prediction. Additionally, we indicate the characteristics of each resource for the flu prediction.
Optimal Downlink Training Sequence for Massive MIMO Secret-Key Generation
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, the secret-key capacity is maximized by optimizing the downlink training sequence in a time division duplexing (TDD) massive multiple-input-multiple-output (MIMO) scenario. Both single-user and multiple user cases are considered. As opposed to previous works, the optimal training sequence and the related secret-key capacity is characterized in closed-form in the single-user case and the large antenna multiple-user case. Designs taking into account a constraint on the maximal number of pilots are also proposed. In the multiple-user case, both the max-min and the sum capacity criteria are considered, including potential user priorities. In the end, it is shown that massive MIMO boosts the secret-key capacity by leveraging: i) spatial dimensionality gain and ii) array gain. Moreover, in the large antenna case, the multiple-user capacity is obtained with no extra pilot overhead as compared to the single-user case.
Defending Against Adversarial Denial-of-Service Data Poisoning Attacks
Venue: arXiv cs.CR 2021
Year: 2021

Data poisoning is one of the most relevant security threats against machine learning and data-driven technologies. Since many applications rely on untrusted training data, an attacker can easily craft malicious samples and inject them into the training dataset to degrade the performance of machine learning models. As recent work has shown, such Denial-of-Service (DoS) data poisoning attacks are highly effective. To mitigate this threat, we propose a new approach of detecting DoS poisoned instances. In comparison to related work, we deviate from clustering and anomaly detection based approaches, which often suffer from the curse of dimensionality and arbitrary anomaly threshold selection. Rather, our defence is based on extracting information from the training data in such a generalized manner that we can identify poisoned samples based on the information present in the unpoisoned portion of the data. We evaluate our defence against two DoS poisoning attacks and seven datasets, and find that it reliably identifies poisoned instances. In comparison to related work, our defence improves false positive / false negative rates by at least 50%, often more.
Quadratic GCN for Graph Classification
Venue: arXiv cs.LG 2021
Year: 2021

Graph Convolutional Networks (GCNs) have been extensively used to classify vertices in graphs and have been shown to outperform other vertex classification methods. GCNs have been extended to graph classification tasks (GCT). In GCT, graphs with different numbers of edges and vertices belong to different classes, and one attempts to predict the graph class. GCN based GCT have mostly used pooling and attention-based models. The accuracy of existing GCT methods is still limited. We here propose a novel solution combining GCN, methods from knowledge graphs, and a new self-regularized activation function to significantly improve the accuracy of the GCN based GCT. We present quadratic GCN (QGCN) - A GCN formalism with a quadratic layer. Such a layer produces an output with fixed dimensions, independent of the graph vertex number. We applied this method to a wide range of graph classification problems, and show that when using a self regularized activation function, QGCN outperforms the state of the art methods for all graph classification tasks tested with or without external input on each graph. The code for QGCN is available at: https://github.com/Unknown-Data/QGCN .
Deep Evaluation Metric: Learning to Evaluate Simulated Radar Point Clouds for Virtual Testing of Autonomous Driving
Venue: arXiv cs.CV 2021
Year: 2021

The usage of environment sensor models for virtual testing is a promising approach to reduce the testing effort of autonomous driving. However, in order to deduce any statements regarding the performance of an autonomous driving function based on simulation, the sensor model has to be validated to determine the discrepancy between the synthetic and real sensor data. Since a certain degree of divergence can be assumed to exist, the sufficient level of fidelity must be determined, which poses a major challenge. In particular, a method for quantifying the fidelity of a sensor model does not exist and the problem of defining an appropriate metric remains. In this work, we train a neural network to distinguish real and simulated radar sensor data with the purpose of learning the latent features of real radar point clouds. Furthermore, we propose the classifier's confidence score for the `real radar point cloud' class as a metric to determine the degree of fidelity of synthetically generated radar data. The presented approach is evaluated and it can be demonstrated that the proposed deep evaluation metric outperforms conventional metrics in terms of its capability to identify characteristic differences between real and simulated radar data.
MoSES_2PDF: A GIS-Compatible GPU-accelerated High-Performance Simulation Tool for Grain-Fluid Shallow Flows
Venue: arXiv cs.CE 2021
Year: 2021

We introduce a GPU-accelerated simulation tool, named Modeling on Shallow Flows with Efficient Simulation for Two-Phase Debris Flows (MoSES_2PDF), of which the input and output data can be linked to the GIS system for engineering application. MoSES_2PDF is developed based on the CUDA structure so that it can well run with different NVIDIA GPU cards, once the CUDA vers. 9.2 (or higher) is installed. The performance of the MoSES_2PDF is evaluated, and it is found that the present GPU-CUDA implementation can enhance efficiency by up to 230 folds, depending on the PC/workstations, models of GPU card, and the mesh numbers in the computation domain. Two numerical examples are illustrated with two distinct initial inflow conditions, which are included in two modes of MoSES_2PDF, respectively. In the numerical example of a large-scale event, the 2009 Hsiaolin event, the results computed by two distinct NVIDIA GPU cards (RTX-2080-Ti and Tesla-V100) are found to be identical but only tiny deviation is figured out in comparison with the results computed by the conventional single-core CPU-code. It is speculated to be caused by the different structures in the source codes and some float/double operations. In addition to the illustration in the GIS system, the computed results by MoSES\_2PDF can also be shown with animated 3D graphics in the ANSI-Platform, where the user can interact with 3D scenes. The feasibility, features, and facilities of MoSES\_2PDF are demonstrated with respect to the two numerical examples concerning two real events.
Neural Architecture Search of Deep Priors: Towards Continual Learning without Catastrophic Interference
Venue: arXiv cs.LG 2021
Year: 2021

In this paper we analyze the classification performance of neural network structures without parametric inference. Making use of neural architecture search, we empirically demonstrate that it is possible to find random weight architectures, a deep prior, that enables a linear classification to perform on par with fully trained deep counterparts. Through ablation experiments, we exclude the possibility of winning a weight initialization lottery and confirm that suitable deep priors do not require additional inference. In an extension to continual learning, we investigate the possibility of catastrophic interference free incremental learning. Under the assumption of classes originating from the same data distribution, a deep prior found on only a subset of classes is shown to allow discrimination of further classes through training of a simple linear classifier.