OpenRaiser/Intern-Atlas huggingface dataset:
A Tight Analysis of Hutchinson's Diagonal Estimator
Venue: arXiv cs.DS 2022
Year: 2022

Let $\mathbf{A}\in \mathbb{R}^{n\times n}$ be a matrix with diagonal $\text{diag}(\mathbf{A})$ and let $\bar{\mathbf{A}}$ be $\mathbf{A}$ with its diagonal set to all zeros. We show that Hutchinson's estimator run for $m$ iterations returns a diagonal estimate $\tilde{d}\in \mathbb{R}^n$ such that with probability $(1-δ)$, $$\|\tilde{d} - \text{diag}(\mathbf{A})\|_2 \leq c\sqrt{\frac{\log(2/δ)}{m}}\|\bar{\mathbf{A}}\|_F,$$ where $c$ is a fixed constant independent of all other parameters. This results improves on a recent result of [Baston and Nakatsukasa, 2022] by a $\log(n)$ factor, yielding a bound that is independent of the matrix dimension $n$.
Discovery of partial differential equations from highly noisy and sparse data with physics-informed information criterion
Venue: arXiv cs.LG 2022
Year: 2022

Data-driven discovery of PDEs has made tremendous progress recently, and many canonical PDEs have been discovered successfully for proof-of-concept. However, determining the most proper PDE without prior references remains challenging in terms of practical applications. In this work, a physics-informed information criterion (PIC) is proposed to measure the parsimony and precision of the discovered PDE synthetically. The proposed PIC achieves state-of-the-art robustness to highly noisy and sparse data on seven canonical PDEs from different physical scenes, which confirms its ability to handle difficult situations. The PIC is also employed to discover unrevealed macroscale governing equations from microscopic simulation data in an actual physical scene. The results show that the discovered macroscale PDE is precise and parsimonious, and satisfies underlying symmetries, which facilitates understanding and simulation of the physical process. The proposition of PIC enables practical applications of PDE discovery in discovering unrevealed governing equations in broader physical scenes.
A Survey on Visual Map Localization Using LiDARs and Cameras
Venue: arXiv cs.CV 2022
Year: 2022

As the autonomous driving industry is slowly maturing, visual map localization is quickly becoming the standard approach to localize cars as accurately as possible. Owing to the rich data returned by visual sensors such as cameras or LiDARs, researchers are able to build different types of maps with various levels of details, and use them to achieve high levels of vehicle localization accuracy and stability in urban environments. Contrary to the popular SLAM approaches, visual map localization relies on pre-built maps, and is focused solely on improving the localization accuracy by avoiding error accumulation or drift. We define visual map localization as a two-stage process. At the stage of place recognition, the initial position of the vehicle in the map is determined by comparing the visual sensor output with a set of geo-tagged map regions of interest. Subsequently, at the stage of map metric localization, the vehicle is tracked while it moves across the map by continuously aligning the visual sensors' output with the current area of the map that is being traversed. In this paper, we survey, discuss and compare the latest methods for LiDAR based, camera based and cross-modal visual map localization for both stages, in an effort to highlight the strength and weakness of each approach.
LCCDE: A Decision-Based Ensemble Framework for Intrusion Detection in The Internet of Vehicles
Venue: arXiv cs.CR 2022
Year: 2022

Modern vehicles, including autonomous vehicles and connected vehicles, have adopted an increasing variety of functionalities through connections and communications with other vehicles, smart devices, and infrastructures. However, the growing connectivity of the Internet of Vehicles (IoV) also increases the vulnerabilities to network attacks. To protect IoV systems against cyber threats, Intrusion Detection Systems (IDSs) that can identify malicious cyber-attacks have been developed using Machine Learning (ML) approaches. To accurately detect various types of attacks in IoV networks, we propose a novel ensemble IDS framework named Leader Class and Confidence Decision Ensemble (LCCDE). It is constructed by determining the best-performing ML model among three advanced ML algorithms (XGBoost, LightGBM, and CatBoost) for every class or type of attack. The class leader models with their prediction confidence values are then utilized to make accurate decisions regarding the detection of various types of cyber-attacks. Experiments on two public IoV security datasets (Car-Hacking and CICIDS2017 datasets) demonstrate the effectiveness of the proposed LCCDE for intrusion detection on both intra-vehicle and external networks.
Advantages in Using a Stock Spring Selection Tool that Manages the Uncertainty of the Designer Requirements
Venue: arXiv cs.AI 2022
Year: 2022

This paper analyses the advantages of using a stock spring selection tool that manages the uncertainty of designer requirements. Firstly, the manual search and its main drawbacks are described. Then a computer assisted stock spring selection tool is presented which performs all necessary calculations to extract the most suitable spring from within a database. The algorithm analyses data set with interval values using both multi-criteria analysis and fuzzy logic. Two examples, comparing manual and assisted search, are presented. They show not only that the results are significantly better using the assisted search but it helps designers to detail easily and precisely their specifications and thus increase design process flexibility.
Exploring the Effects of Data Augmentation for Drivable Area Segmentation
Venue: arXiv cs.CV 2022
Year: 2022

The real-time segmentation of drivable areas plays a vital role in accomplishing autonomous perception in cars. Recently there have been some rapid strides in the development of image segmentation models using deep learning. However, most of the advancements have been made in model architecture design. In solving any supervised deep learning problem related to segmentation, the success of the model that one builds depends upon the amount and quality of input training data we use for that model. This data should contain well-annotated varied images for better working of the segmentation model. Issues like this pertaining to annotations in a dataset can lead the model to conclude with overwhelming Type I and II errors in testing and validation, causing malicious issues when trying to tackle real world problems. To address this problem and to make our model more accurate, dynamic, and robust, data augmentation comes into usage as it helps in expanding our sample training data and making it better and more diversified overall. Hence, in our study, we focus on investigating the benefits of data augmentation by analyzing pre-existing image datasets and performing augmentations accordingly. Our results show that the performance and robustness of existing state of the art (or SOTA) models can be increased dramatically without any increase in model complexity or inference time. The augmentations decided on and used in this paper were decided only after thorough research of several other augmentation methodologies and strategies and their corresponding effects that are in widespread usage today. All our results are being reported on the widely used Cityscapes Dataset.
DeepGen: Diverse Search Ad Generation and Real-Time Customization
Venue: arXiv cs.CL 2022
Year: 2022

We present DeepGen, a system deployed at web scale for automatically creating sponsored search advertisements (ads) for BingAds customers. We leverage state-of-the-art natural language generation (NLG) models to generate fluent ads from advertiser's web pages in an abstractive fashion and solve practical issues such as factuality and inference speed. In addition, our system creates a customized ad in real-time in response to the user's search query, therefore highlighting different aspects of the same product based on what the user is looking for. To achieve this, our system generates a diverse choice of smaller pieces of the ad ahead of time and, at query time, selects the most relevant ones to be stitched into a complete ad. We improve generation diversity by training a controllable NLG model to generate multiple ads for the same web page highlighting different selling points. Our system design further improves diversity horizontally by first running an ensemble of generation models trained with different objectives and then using a diversity sampling algorithm to pick a diverse subset of generation results for online selection. Experimental results show the effectiveness of our proposed system design. Our system is currently deployed in production, serving ${\sim}4\%$ of global ads served in Bing.
AFE-CNN: 3D Skeleton-based Action Recognition with Action Feature Enhancement
Venue: arXiv cs.CV 2022
Year: 2022

Existing 3D skeleton-based action recognition approaches reach impressive performance by encoding handcrafted action features to image format and decoding by CNNs. However, such methods are limited in two ways: a) the handcrafted action features are difficult to handle challenging actions, and b) they generally require complex CNN models to improve action recognition accuracy, which usually occur heavy computational burden. To overcome these limitations, we introduce a novel AFE-CNN, which devotes to enhance the features of 3D skeleton-based actions to adapt to challenging actions. We propose feature enhance modules from key joint, bone vector, key frame and temporal perspectives, thus the AFE-CNN is more robust to camera views and body sizes variation, and significantly improve the recognition accuracy on challenging actions. Moreover, our AFE-CNN adopts a light-weight CNN model to decode images with action feature enhanced, which ensures a much lower computational burden than the state-of-the-art methods. We evaluate the AFE-CNN on three benchmark skeleton-based action datasets: NTU RGB+D, NTU RGB+D 120, and UTKinect-Action3D, with extensive experimental results demonstrate our outstanding performance of AFE-CNN.
LFGCF: Light Folksonomy Graph Collaborative Filtering for Tag-Aware Recommendation
Venue: arXiv cs.IR 2022
Year: 2022

Tag-aware recommendation is a task of predicting a personalized list of items for a user by their tagging behaviors. It is crucial for many applications with tagging capabilities like last.fm or movielens. Recently, many efforts have been devoted to improving Tag-aware recommendation systems (TRS) with Graph Convolutional Networks (GCN), which has become new state-of-the-art for the general recommendation. However, some solutions are directly inherited from GCN without justifications, which is difficult to alleviate the sparsity, ambiguity, and redundancy issues introduced by tags, thus adding to difficulties of training and degrading recommendation performance. In this work, we aim to simplify the design of GCN to make it more concise for TRS. We propose a novel tag-aware recommendation model named Light Folksonomy Graph Collaborative Filtering (LFGCF), which only includes the essential GCN components. Specifically, LFGCF first constructs Folksonomy Graphs from the records of user assigning tags and item getting tagged. Then we leverage the simple design of aggregation to learn the high-order representations on Folksonomy Graphs and use the weighted sum of the embeddings learned at several layers for information updating. We share tags embeddings to bridge the information gap between users and items. Besides, a regularization function named TransRT is proposed to better depict user preferences and item features. Extensive hyperparameters experiments and ablation studies on three real-world datasets show that LFGCF uses fewer parameters and significantly outperforms most baselines for the tag-aware top-N recommendations.
Threddy: An Interactive System for Personalized Thread-based Exploration and Organization of Scientific Literature
Venue: arXiv cs.HC 2022
Year: 2022

Reviewing the literature to understand relevant threads of past work is a critical part of research and vehicle for learning. However, as the scientific literature grows the challenges for users to find and make sense of the many different threads of research grow as well. Previous work has helped scholars to find and group papers with citation information or textual similarity using standalone tools or overview visualizations. Instead, in this work we explore a tool integrated into users' reading process that helps them with leveraging authors' existing summarization of threads, typically in introduction or related work sections, in order to situate their own work's contributions. To explore this we developed a prototype that supports efficient extraction and organization of threads along with supporting evidence as scientists read research articles. The system then recommends further relevant articles based on user-created threads. We evaluate the system in a lab study and find that it helps scientists to follow and curate research threads without breaking out of their flow of reading, collect relevant papers and clips, and discover interesting new articles to further grow threads.
Oversquashing in GNNs through the lens of information contraction and graph expansion
Venue: arXiv cs.LG 2022
Year: 2022

The quality of signal propagation in message-passing graph neural networks (GNNs) strongly influences their expressivity as has been observed in recent works. In particular, for prediction tasks relying on long-range interactions, recursive aggregation of node features can lead to an undesired phenomenon called "oversquashing". We present a framework for analyzing oversquashing based on information contraction. Our analysis is guided by a model of reliable computation due to von Neumann that lends a new insight into oversquashing as signal quenching in noisy computation graphs. Building on this, we propose a graph rewiring algorithm aimed at alleviating oversquashing. Our algorithm employs a random local edge flip primitive motivated by an expander graph construction. We compare the spectral expansion properties of our algorithm with that of an existing curvature-based non-local rewiring strategy. Synthetic experiments show that while our algorithm in general has a slower rate of expansion, it is overall computationally cheaper, preserves the node degrees exactly and never disconnects the graph.
HaloAE: An HaloNet based Local Transformer Auto-Encoder for Anomaly Detection and Localization
Venue: arXiv cs.CV 2022
Year: 2022

Unsupervised anomaly detection and localization is a crucial task as it is impossible to collect and label all possible anomalies. Many studies have emphasized the importance of integrating local and global information to achieve accurate segmentation of anomalies. To this end, there has been a growing interest in Transformer, which allows modeling long-range content interactions. However, global interactions through self attention are generally too expensive for most image scales. In this study, we introduce HaloAE, the first auto-encoder based on a local 2D version of Transformer with HaloNet. With HaloAE, we have created a hybrid model that combines convolution and local 2D block-wise self-attention layers and jointly performs anomaly detection and segmentation through a single model. We achieved competitive results on the MVTec dataset, suggesting that vision models incorporating Transformer could benefit from a local computation of the self-attention operation, and pave the way for other applications.
On Rate-Splitting With Non-unique Decoding In Multi-cell Massive MIMO Systems
Venue: arXiv cs.IT 2022
Year: 2022

We consider the downlink of a multi-cell massive MIMO system suffering from asymptotic rate saturation due to pilot contamination. As opposed to treating pilot contamination interference as noise (TIN), we study the performance of decoding the pilot contamination interference. We model pilot-sharing users as an interference channel (IC) and study the performance of schemes that decode this interference partially based on rate-splitting (RS), and compare the performance to schemes that decode the interference in its entirety based on simultaneous unique decoding (SD) or non-unique decoding (SND). For RS, we non-uniquely decode each layer of the pilot contamination interference and use one common power splitting coefficient per IC. Additionally, we establish an achievable region for this RS scheme. Solving a maximum symmetric rate allocation problem based on linear programming (LP), we show that for zero-forcing (ZF) with spatially correlated/uncorrelated channels and with a practical number of BS antennas, RS achieves significantly higher spectral efficiencies than TIN, SD and SND. Furthermore, we numerically examine the impact of increasing the correlation of the channel across antennas, the number of users as well as the degree of shadow fading. In all cases, we show that RS maintains significant gain over TIN, SD and SND.
Cyber Pirates Ahoy! An Analysis of Cybersecurity Challenges in the Shipping Industry
Venue: arXiv cs.CR 2022
Year: 2022

Maritime shipping has become a trillion-dollar industry that now impacts the economy of virtually every country around the world. It is therefore no surprise that countries and companies have spent billions of dollars to modernize shipping vessels and ports with various technologies. However, the implementation of these technologies has also caught the attention of cybercriminals. For example, a cyberattack on one shipping company resulted in nearly $300 millions in financial losses. Hence, this paper describes cybersecurity vulnerabilities present in the international shipping business. The contribution of this paper is the identification and dissection of cyber vulnerabilities specific to the shipping industry, along with how and why these potential vulnerabilities exist.
Fast Online and Relational Tracking
Venue: arXiv cs.CV 2022
Year: 2022

To overcome challenges in multiple object tracking task, recent algorithms use interaction cues alongside motion and appearance features. These algorithms use graph neural networks or transformers to extract interaction features that lead to high computation costs. In this paper, a novel interaction cue based on geometric features is presented aiming to detect occlusion and re-identify lost targets with low computational cost. Moreover, in most algorithms, camera motion is considered negligible, which is a strong assumption that is not always true and leads to ID Switch or mismatching of targets. In this paper, a method for measuring camera motion and removing its effect is presented that efficiently reduces the camera motion effect on tracking. The proposed algorithm is evaluated on MOT17 and MOT20 datasets and it achieves the state-of-the-art performance of MOT17 and comparable results on MOT20. The code is also publicly available.
Maximum Correntropy Value Decomposition for Multi-agent Deep Reinforcemen Learning
Venue: arXiv cs.MA 2022
Year: 2022

We explore value decomposition solutions for multi-agent deep reinforcement learning in the popular paradigm of centralized training with decentralized execution(CTDE). As the recognized best solution to CTDE, Weighted QMIX is cutting-edge on StarCraft Multi-agent Challenge (SMAC), with a weighting scheme implemented on QMIX to place more emphasis on the optimal joint actions. However, the fixed weight requires manual tuning according to the application scenarios, which painfully prevents Weighted QMIX from being used in broader engineering applications. In this paper, we first demonstrate the flaw of Weighted QMIX using an ordinary One-Step Matrix Game (OMG), that no matter how the weight is chosen, Weighted QMIX struggles to deal with non-monotonic value decomposition problems with a large variance of reward distributions. Then we characterize the problem of value decomposition as an Underfitting One-edged Robust Regression problem and make the first attempt to give a solution to the value decomposition problem from the perspective of information-theoretical learning. We introduce the Maximum Correntropy Criterion (MCC) as a cost function to dynamically adapt the weight to eliminate the effects of minimum in reward distributions. We simplify the implementation and propose a new algorithm called MCVD. A preliminary experiment conducted on OMG shows that MCVD could deal with non-monotonic value decomposition problems with a large tolerance of kernel bandwidth selection. Further experiments are carried out on Cooperative-Navigation and multiple SMAC scenarios, where MCVD exhibits unprecedented ease of implementation, broad applicability, and stability.
SINR Analysis of an IRS Assisted MU-MISO System
Venue: arXiv cs.IT 2022
Year: 2022

In this work, we characterize the outage probability (OP) of an intelligent reflecting surface (IRS) assisted multi-user multiple-input-single-output (MU-MISO) communication system. Using a two-step approximation method, we approximate the signal-to-interference-plus-noise ratio (SINR) for any downlink user by a Log-Normal random variable. The impact of various system parameters is studied using the closed-form expression of OP. It is concluded that the position of IRS has a critical role, but an appropriate increase in the number of IRS elements would help to compensate for the loss in performance if the position of IRS is suboptimal.
Projective Geometry, Duality and Plucker Coordinates for Geometric Computations with Determinants on GPUs
Venue: arXiv cs.GR 2022
Year: 2022

Many algorithms used are based on geometrical computation. There are several criteria in selecting appropriate algorithm from already known. Recently, the fastest algorithms have been preferred. Nowadays, algorithms with a high stability are preferred. Also technology and computer architecture, like GPU etc., plays a significant role for large data processing. However, some algorithms are ill-conditioned due to numerical representation used; result of the floating point representation. In this paper, relations between projective representation, duality and Plucker coordinates will be explored with demonstration on simple geometric examples. The presented approach is convenient especially for application on GPUs or vector-vector computational architectures
A Parallel Technique for Multi-objective Bayesian Global Optimization: Using a Batch Selection of Probability of Improvement
Venue: arXiv cs.AI 2022
Year: 2022

Bayesian global optimization (BGO) is an efficient surrogate-assisted technique for problems involving expensive evaluations. A parallel technique can be used to parallelly evaluate the true-expensive objective functions in one iteration to boost the execution time. An effective and straightforward approach is to design an acquisition function that can evaluate the performance of a bath of multiple solutions, instead of a single point/solution, in one iteration. This paper proposes five alternatives of \emph{Probability of Improvement} (PoI) with multiple points in a batch (q-PoI) for multi-objective Bayesian global optimization (MOBGO), taking the covariance among multiple points into account. Both exact computational formulas and the Monte Carlo approximation algorithms for all proposed q-PoIs are provided. Based on the distribution of the multiple points relevant to the Pareto-front, the position-dependent behavior of the five q-PoIs is investigated. Moreover, the five q-PoIs are compared with the other nine state-of-the-art and recently proposed batch MOBGO algorithms on twenty bio-objective benchmarks. The empirical experiments on different variety of benchmarks are conducted to demonstrate the effectiveness of two greedy q-PoIs ($\kpoi_{\mbox{best}}$ and $\kpoi_{\mbox{all}}$) on low-dimensional problems and the effectiveness of two explorative q-PoIs ($\kpoi_{\mbox{one}}$ and $\kpoi_{\mbox{worst}}$) on high-dimensional problems with difficult-to-approximate Pareto front boundaries.