OpenRaiser/Intern-Atlas huggingface dataset:
DOLPHINS: Dataset for Collaborative Perception enabled Harmonious and Interconnected Self-driving
Venue: arXiv cs.CV 2022
Year: 2022

Vehicle-to-Everything (V2X) network has enabled collaborative perception in autonomous driving, which is a promising solution to the fundamental defect of stand-alone intelligence including blind zones and long-range perception. However, the lack of datasets has severely blocked the development of collaborative perception algorithms. In this work, we release DOLPHINS: Dataset for cOllaborative Perception enabled Harmonious and INterconnected Self-driving, as a new simulated large-scale various-scenario multi-view multi-modality autonomous driving dataset, which provides a ground-breaking benchmark platform for interconnected autonomous driving. DOLPHINS outperforms current datasets in six dimensions: temporally-aligned images and point clouds from both vehicles and Road Side Units (RSUs) enabling both Vehicle-to-Vehicle (V2V) and Vehicle-to-Infrastructure (V2I) based collaborative perception; 6 typical scenarios with dynamic weather conditions make the most various interconnected autonomous driving dataset; meticulously selected viewpoints providing full coverage of the key areas and every object; 42376 frames and 292549 objects, as well as the corresponding 3D annotations, geo-positions, and calibrations, compose the largest dataset for collaborative perception; Full-HD images and 64-line LiDARs construct high-resolution data with sufficient details; well-organized APIs and open-source codes ensure the extensibility of DOLPHINS. We also construct a benchmark of 2D detection, 3D detection, and multi-view collaborative perception tasks on DOLPHINS. The experiment results show that the raw-level fusion scheme through V2X communication can help to improve the precision as well as to reduce the necessity of expensive LiDAR equipment on vehicles when RSUs exist, which may accelerate the popularity of interconnected self-driving vehicles. DOLPHINS is now available on https://dolphins-dataset.net/.
Multimodal Open-Vocabulary Video Classification via Pre-Trained Vision and Language Models
Venue: arXiv cs.CV 2022
Year: 2022

Utilizing vision and language models (VLMs) pre-trained on large-scale image-text pairs is becoming a promising paradigm for open-vocabulary visual recognition. In this work, we extend this paradigm by leveraging motion and audio that naturally exist in video. We present \textbf{MOV}, a simple yet effective method for \textbf{M}ultimodal \textbf{O}pen-\textbf{V}ocabulary video classification. In MOV, we directly use the vision encoder from pre-trained VLMs with minimal modifications to encode video, optical flow and audio spectrogram. We design a cross-modal fusion mechanism to aggregate complimentary multimodal information. Experiments on Kinetics-700 and VGGSound show that introducing flow or audio modality brings large performance gains over the pre-trained VLM and existing methods. Specifically, MOV greatly improves the accuracy on base classes, while generalizes better on novel classes. MOV achieves state-of-the-art results on UCF and HMDB zero-shot video classification benchmarks, significantly outperforming both traditional zero-shot methods and recent methods based on VLMs. Code and models will be released.
Support Vector Machines with the Hard-Margin Loss: Optimal Training via Combinatorial Benders' Cuts
Venue: arXiv cs.LG 2022
Year: 2022

The classical hinge-loss support vector machines (SVMs) model is sensitive to outlier observations due to the unboundedness of its loss function. To circumvent this issue, recent studies have focused on non-convex loss functions, such as the hard-margin loss, which associates a constant penalty to any misclassified or within-margin sample. Applying this loss function yields much-needed robustness for critical applications but it also leads to an NP-hard model that makes training difficult, since current exact optimization algorithms show limited scalability, whereas heuristics are not able to find high-quality solutions consistently. Against this background, we propose new integer programming strategies that significantly improve our ability to train the hard-margin SVM model to global optimality. We introduce an iterative sampling and decomposition approach, in which smaller subproblems are used to separate combinatorial Benders' cuts. Those cuts, used within a branch-and-cut algorithm, permit to converge much more quickly towards a global optimum. Through extensive numerical analyses on classical benchmark data sets, our solution algorithm solves, for the first time, 117 new data sets to optimality and achieves a reduction of 50% in the average optimality gap for the hardest datasets of the benchmark.
Maximizing Fair Content Spread via Edge Suggestion in Social Networks
Venue: arXiv cs.SI 2022
Year: 2022

Content spread inequity is a potential unfairness issue in online social networks, disparately impacting minority groups. In this paper, we view friendship suggestion, a common feature in social network platforms, as an opportunity to achieve an equitable spread of content. In particular, we propose to suggest a subset of potential edges (currently not existing in the network but likely to be accepted) that maximizes content spread while achieving fairness. Instead of re-engineering the existing systems, our proposal builds a fairness wrapper on top of the existing friendship suggestion components. We prove the problem is NP-hard and inapproximable in polynomial time unless P = NP. Therefore, allowing relaxation of the fairness constraint, we propose an algorithm based on LP-relaxation and randomized rounding with fixed approximation ratios on fairness and content spread. We provide multiple optimizations, further improving the performance of our algorithm in practice. Besides, we propose a scalable algorithm that dynamically adds subsets of nodes, chosen via iterative sampling, and solves smaller problems corresponding to these nodes. Besides theoretical analysis, we conduct comprehensive experiments on real and synthetic data sets. Across different settings, our algorithms found solutions with nearzero unfairness while significantly increasing the content spread. Our scalable algorithm could process a graph with half a million nodes on a single machine, reducing the unfairness to around 0.0004 while lifting content spread by 43%.
Approximating Highly Inapproximable Problems on Graphs of Bounded Twin-Width
Venue: arXiv cs.DS 2022
Year: 2022

For any $\varepsilon > 0$, we give a polynomial-time $n^\varepsilon$-approximation algorithm for Max Independent Set in graphs of bounded twin-width given with an $O(1)$-sequence. This result is derived from the following time-approximation trade-off: We establish an $O(1)^{2^q-1}$-approximation algorithm running in time $\exp(O_q(n^{2^{-q}}))$, for every integer $q \geqslant 0$. Guided by the same framework, we obtain similar approximation algorithms for Min Coloring and Max Induced Matching. In general graphs, all these problems are known to be highly inapproximable: for any $\varepsilon > 0$, a polynomial-time $n^{1-\varepsilon}$-approximation for any of them would imply that P$=$NP [Hastad, FOCS '96; Zuckerman, ToC '07; Chalermsook et al., SODA '13]. We generalize the algorithms for Max Independent Set and Max Induced Matching to the independent (induced) packing of any fixed connected graph $H$. In contrast, we show that such approximation guarantees on graphs of bounded twin-width given with an $O(1)$-sequence are very unlikely for Min Independent Dominating Set, and somewhat unlikely for Longest Path and Longest Induced Path. Regarding the existence of better approximation algorithms, there is a (very) light evidence that the obtained approximation factor of $n^\varepsilon$ for Max Independent Set may be best possible. This is the first in-depth study of the approximability of problems in graphs of bounded twin-width. Prior to this paper, essentially the only such result was a~polynomial-time $O(1)$-approximation algorithm for Min Dominating Set [Bonnet et al., ICALP '21].
More Data Can Lead Us Astray: Active Data Acquisition in the Presence of Label Bias
Venue: arXiv cs.LG 2022
Year: 2022

An increased awareness concerning risks of algorithmic bias has driven a surge of efforts around bias mitigation strategies. A vast majority of the proposed approaches fall under one of two categories: (1) imposing algorithmic fairness constraints on predictive models, and (2) collecting additional training samples. Most recently and at the intersection of these two categories, methods that propose active learning under fairness constraints have been developed. However, proposed bias mitigation strategies typically overlook the bias presented in the observed labels. In this work, we study fairness considerations of active data collection strategies in the presence of label bias. We first present an overview of different types of label bias in the context of supervised learning systems. We then empirically show that, when overlooking label bias, collecting more data can aggravate bias, and imposing fairness constraints that rely on the observed labels in the data collection process may not address the problem. Our results illustrate the unintended consequences of deploying a model that attempts to mitigate a single type of bias while neglecting others, emphasizing the importance of explicitly differentiating between the types of bias that fairness-aware algorithms aim to address, and highlighting the risks of neglecting label bias during data collection.
Optimizing Data Collection in Deep Reinforcement Learning
Venue: arXiv cs.LG 2022
Year: 2022

Reinforcement learning (RL) workloads take a notoriously long time to train due to the large number of samples collected at run-time from simulators. Unfortunately, cluster scale-up approaches remain expensive, and commonly used CPU implementations of simulators induce high overhead when switching back and forth between GPU computations. We explore two optimizations that increase RL data collection efficiency by increasing GPU utilization: (1) GPU vectorization: parallelizing simulation on the GPU for increased hardware parallelism, and (2) simulator kernel fusion: fusing multiple simulation steps to run in a single GPU kernel launch to reduce global memory bandwidth requirements. We find that GPU vectorization can achieve up to $1024\times$ speedup over commonly used CPU simulators. We profile the performance of different implementations and show that for a simple simulator, ML compiler implementations (XLA) of GPU vectorization outperform a DNN framework (PyTorch) by $13.4\times$ by reducing CPU overhead from repeated Python to DL backend API calls. We show that simulator kernel fusion speedups with a simple simulator are $11.3\times$ and increase by up to $1024\times$ as simulator complexity increases in terms of memory bandwidth requirements. We show that the speedups from simulator kernel fusion are orthogonal and combinable with GPU vectorization, leading to a multiplicative speedup.
A Review of the Operational Use of UAS in Public Safety Emergency Incidents
Venue: arXiv cs.HC 2022
Year: 2022

The domain of public safety in the form of search \& rescue, wildland firefighting, structure firefighting, and law enforcement operations have drawn great interest in the field of aerospace engineering, human-robot teaming, autonomous systems, and robotics. However, a divergence exists in the assumptions made in research and how state-of-the-art technologies may realistically transition into an operational capacity. To aid in the alignment between researchers, technologists, and end users, we aim to provide perspective on how small Uncrewed Aerial Systems (sUAS) have been applied in 114 real world incidents as part of a technical rescue team from 2016 to 2021. We highlight the main applications, integration, tasks, and challenges of employing UAS within five primary use cases including searches, evidence collection, SWAT, wildland firefighting, and structure firefighting. Within these use cases, key incidents are featured that provide perspective on the evolving and dynamic nature of UAS tasking during an operation. Finally, we highlight key technical directions for improving the utilization and efficiency of employing aerial technology in all emergency types.
Anomalous behaviour in loss-gradient based interpretability methods
Venue: arXiv cs.LG 2022
Year: 2022

Loss-gradients are used to interpret the decision making process of deep learning models. In this work, we evaluate loss-gradient based attribution methods by occluding parts of the input and comparing the performance of the occluded input to the original input. We observe that the occluded input has better performance than the original across the test dataset under certain conditions. Similar behaviour is observed in sound and image recognition tasks. We explore different loss-gradient attribution methods, occlusion levels and replacement values to explain the phenomenon of performance improvement under occlusion.
Rate-Splitting Multiple Access for Short-Packet Uplink Communications: A Finite Blocklength Error Probability Analysis
Venue: arXiv cs.IT 2022
Year: 2022

In this letter, we investigate Rate-Splitting Multiple Access (RSMA) for an uplink communication system with finite blocklength. Considering a two-user Single-Input Single-Output (SISO) Multiple Access Channel (MAC), we study the impact of Signal-to-Noise Ratio (SNR), blocklength, power allocation and target rate on the error probability performance of RSMA where one user message is split. We demonstrate that RSMA can improve the error probability performance significantly compared to Non-Orthogonal Multiple Access (NOMA) and RSMA can have a larger rate region than NOMA.
QuaDUE-CCM: Interpretable Distributional Reinforcement Learning using Uncertain Contraction Metrics for Precise Quadrotor Trajectory Tracking
Venue: arXiv cs.RO 2022
Year: 2022

Accuracy and stability are common requirements for Quadrotor trajectory tracking systems. Designing an accurate and stable tracking controller remains challenging, particularly in unknown and dynamic environments with complex aerodynamic disturbances. We propose a Quantile-approximation-based Distributional-reinforced Uncertainty Estimator (QuaDUE) to accurately identify the effects of aerodynamic disturbances, i.e., the uncertainties between the true and estimated Control Contraction Metrics (CCMs). Taking inspiration from contraction theory and integrating the QuaDUE for uncertainties, our novel CCM-based trajectory tracking framework tracks any feasible reference trajectory precisely whilst guaranteeing exponential convergence. More importantly, the convergence and training acceleration of the distributional RL are guaranteed and analyzed, respectively, from theoretical perspectives. We also demonstrate our system under unknown and diverse aerodynamic forces. Under large aerodynamic forces (>2m/s^2), compared with the classic data-driven approach, our QuaDUE-CCM achieves at least a 56.6% improvement in tracking error. Compared with QuaDRED-MPC, a distributional RL-based approach, QuaDUE-CCM achieves at least a 3 times improvement in contraction rate.
BCRLSP: An Offline Reinforcement Learning Framework for Sequential Targeted Promotion
Venue: arXiv cs.LG 2022
Year: 2022

We utilize an offline reinforcement learning (RL) model for sequential targeted promotion in the presence of budget constraints in a real-world business environment. In our application, the mobile app aims to boost customer retention by sending cash bonuses to customers and control the costs of such cash bonuses during each time period. To achieve the multi-task goal, we propose the Budget Constrained Reinforcement Learning for Sequential Promotion (BCRLSP) framework to determine the value of cash bonuses to be sent to users. We first find out the target policy and the associated Q-values that maximizes the user retention rate using an RL model. A linear programming (LP) model is then added to satisfy the constraints of promotion costs. We solve the LP problem by maximizing the Q-values of actions learned from the RL model given the budget constraints. During deployment, we combine the offline RL model with the LP model to generate a robust policy under the budget constraints. Using both online and offline experiments, we demonstrate the efficacy of our approach by showing that BCRLSP achieves a higher long-term customer retention rate and a lower cost than various baselines. Taking advantage of the near real-time cost control method, the proposed framework can easily adapt to data with a noisy behavioral policy and/or meet flexible budget constraints.
Hulls of special typed linear codes and constructions of new EAQECCs
Venue: arXiv cs.IT 2022
Year: 2022

In this paper, we study Euclidean and Hermitian hulls of generalized Reed-Solomon codes and twisted generalized Reed-Solomon codes, as well as the Hermitian hulls of Roth-Lempel typed codes. We present explicit constructions of MDS and AMDS linear codes for which their hull dimensions are well determined. As an application, we provide several classes of entanglement-assisted quantum error correcting codes with new parameters.
Sotto Voce: Federated Speech Recognition with Differential Privacy Guarantees
Venue: arXiv cs.CR 2022
Year: 2022

Speech data is expensive to collect, and incredibly sensitive to its sources. It is often the case that organizations independently collect small datasets for their own use, but often these are not performant for the demands of machine learning. Organizations could pool these datasets together and jointly build a strong ASR system; sharing data in the clear, however, comes with tremendous risk, in terms of intellectual property loss as well as loss of privacy of the individuals who exist in the dataset. In this paper, we offer a potential solution for learning an ML model across multiple organizations where we can provide mathematical guarantees limiting privacy loss. We use a Federated Learning approach built on a strong foundation of Differential Privacy techniques. We apply these to a senone classification prototype and demonstrate that the model improves with the addition of private data while still respecting privacy.
Structural Prior Guided Generative Adversarial Transformers for Low-Light Image Enhancement
Venue: arXiv cs.CV 2022
Year: 2022

We propose an effective Structural Prior guided Generative Adversarial Transformer (SPGAT) to solve low-light image enhancement. Our SPGAT mainly contains a generator with two discriminators and a structural prior estimator (SPE). The generator is based on a U-shaped Transformer which is used to explore non-local information for better clear image restoration. The SPE is used to explore useful structures from images to guide the generator for better structural detail estimation. To generate more realistic images, we develop a new structural prior guided adversarial learning method by building the skip connections between the generator and discriminators so that the discriminators can better discriminate between real and fake features. Finally, we propose a parallel windows-based Swin Transformer block to aggregate different level hierarchical features for high-quality image restoration. Experimental results demonstrate that the proposed SPGAT performs favorably against recent state-of-the-art methods on both synthetic and real-world datasets.
Robust AI Driving Strategy for Autonomous Vehicles
Venue: arXiv cs.RO 2022
Year: 2022

There has been significant progress in sensing, perception, and localization for automated driving, However, due to the wide spectrum of traffic/road structure scenarios and the long tail distribution of human driver behavior, it has remained an open challenge for an intelligent vehicle to always know how to make and execute the best decision on road given available sensing / perception / localization information. In this chapter, we talk about how artificial intelligence and more specifically, reinforcement learning, can take advantage of operational knowledge and safety reflex to make strategical and tactical decisions. We discuss some challenging problems related to the robustness of reinforcement learning solutions and their implications to the practical design of driving strategies for autonomous vehicles. We focus on automated driving on highway and the integration of reinforcement learning, vehicle motion control, and control barrier function, leading to a robust AI driving strategy that can learn and adapt safely.
Utility Driven Job Selection Problem on Road Networks
Venue: arXiv cs.DB 2022
Year: 2022

In this paper, we study the problem of \textsc{Utility Driven Job Selection} on Road Networks for which the inputs are: a road network with the vertices as the set of Point-Of-Interests (Henceforth mentioned as POI) and the edges are road segments joining the POIs, a set of jobs with their originating POI, starting time, duration, and the utility. A worker can earn the utility associated with the job if (s)he performs this. As the jobs are originating at different POIs, the worker has to move from one POI to the other one to take up the job. Some budget is available for this purpose. Any two jobs can be taken up by the worker only if the finishing time of the first job plus traveling time from the POI of the first job to the second one should be less than or equal to the starting time of the second job. We call this constraint as the temporal constraint. The goal of this problem is to choose a subset of the jobs to maximize the earned utility such that the budget and temporal constraints should not be violated. We present two solution approaches with detailed analysis. First one of them works based on finding the locally optimal job at the end of every job and we call this approach as the \emph{Best First Search Approach}. The other approach is based on the Nearest Neighbor Search on road networks. We perform a set of experiments with real\mbox{-}world trajectory datasets to demonstrate the efficiency and effectiveness of the proposed solution approaches. We observe that the proposed approaches lead to more utility compared to baseline methods.
Envy\mbox{-}free Trip Planning in Group Trip Planning Query Problem
Venue: arXiv cs.DB 2022
Year: 2022

In recent times, Group Trip Planning Query (henceforth referred to as GTP Query) is one of the well\mbox{-}studied problems in Spatial Databases. The inputs to the problem are a road network where the vertices represent the Point-of-Interests (mentioned as POIs henceforth) and they are grouped into different categories, edges represent the road segments, and edge weight represents the distance and a group of users along with their source and destination location. This problem asks to return one POI from every category such that the aggregated distance traveled by the group is minimized. As the objective is to minimize the aggregated distance, the existing solution methodologies do not consider the individual distances traveled by the group members. To address this issue, we introduce and study the \textsc{Envy Free Group Trip Planning Query} Problem. Along with the inputs of the GTP Query Problem, in this variant, we also have a threshold distance $D$ such that aggregated distance traveled by the group is minimized and for any member pairs the difference between their individual distance traveled is less than equal to $D$. However, it may so happen that a given $D$ value no such set POIs are found. To tackle this issue, we introduce the surrogate problem \textsc{Envy Free Group Trip Planning Query with Minimum Additional Distance} Problem which asks what is the minimum distance to be added with $D$ to obtain at least one solution. For these problems, we design efficient solution approaches and experiment with real-world datasets. From the experiments, we observe that the proposed solution approaches lead to less aggregated distance compared to baseline methods with reasonable computational overhead.