OpenRaiser/Intern-Atlas huggingface dataset:
Vision-Centric BEV Perception: A Survey
Venue: arXiv cs.CV 2023
Year: 2023

In recent years, vision-centric Bird's Eye View (BEV) perception has garnered significant interest from both industry and academia due to its inherent advantages, such as providing an intuitive representation of the world and being conducive to data fusion. The rapid advancements in deep learning have led to the proposal of numerous methods for addressing vision-centric BEV perception challenges. However, there has been no recent survey encompassing this novel and burgeoning research field. To catalyze future research, this paper presents a comprehensive survey of the latest developments in vision-centric BEV perception and its extensions. It compiles and organizes up-to-date knowledge, offering a systematic review and summary of prevalent algorithms. Additionally, the paper provides in-depth analyses and comparative results on various BEV perception tasks, facilitating the evaluation of future works and sparking new research directions. Furthermore, the paper discusses and shares valuable empirical implementation details to aid in the advancement of related algorithms.
The Role of Morphological Variation in Evolutionary Robotics: Maximizing Performance and Robustness
Venue: arXiv cs.NE 2023
Year: 2023

Exposing an Evolutionary Algorithm that is used to evolve robot controllers to variable conditions is necessary to obtain solutions which are robust and can cross the reality gap. However, we do not yet have methods for analyzing and understanding the impact of the varying morphological conditions which impact the evolutionary process, and therefore for choosing suitable variation ranges. By morphological conditions, we refer to the starting state of the robot, and to variations in its sensor readings during operation due to noise. In this article, we introduce a method that permits us to measure the impact of these morphological variations and we analyze the relation between the amplitude of variations, the modality with which they are introduced, and the performance and robustness of evolving agents. Our results demonstrate that (i) the evolutionary algorithm can tolerate morphological variations which have a very high impact, (ii) variations affecting the actions of the agent are tolerated much better than variations affecting the initial state of the agent or of the environment, and (iii) improving the accuracy of the fitness measure through multiple evaluations is not always useful. Moreover, our results show that morphological variations permit generating solutions which perform better both in varying and non-varying conditions.
Analyzing Data-Centric Properties for Graph Contrastive Learning
Venue: arXiv cs.LG 2023
Year: 2023

Recent analyses of self-supervised learning (SSL) find the following data-centric properties to be critical for learning good representations: invariance to task-irrelevant semantics, separability of classes in some latent space, and recoverability of labels from augmented samples. However, given their discrete, non-Euclidean nature, graph datasets and graph SSL methods are unlikely to satisfy these properties. This raises the question: how do graph SSL methods, such as contrastive learning (CL), work well? To systematically probe this question, we perform a generalization analysis for CL when using generic graph augmentations (GGAs), with a focus on data-centric properties. Our analysis yields formal insights into the limitations of GGAs and the necessity of task-relevant augmentations. As we empirically show, GGAs do not induce task-relevant invariances on common benchmark datasets, leading to only marginal gains over naive, untrained baselines. Our theory motivates a synthetic data generation process that enables control over task-relevant information and boasts pre-defined optimal augmentations. This flexible benchmark helps us identify yet unrecognized limitations in advanced augmentation techniques (e.g., automated methods). Overall, our work rigorously contextualizes, both empirically and theoretically, the effects of data-centric properties on augmentation strategies and learning paradigms for graph SSL.
Nonstationary Continuum-Armed Bandit Strategies for Automated Trading in a Simulated Financial Market
Venue: arXiv cs.MA 2023
Year: 2023

We approach the problem of designing an automated trading strategy that can consistently profit by adapting to changing market conditions. This challenge can be framed as a Nonstationary Continuum-Armed Bandit (NCAB) problem. To solve the NCAB problem, we propose PRBO, a novel trading algorithm that uses Bayesian optimization and a ``bandit-over-bandit'' framework to dynamically adjust strategy parameters in response to market conditions. We use Bristol Stock Exchange (BSE) to simulate financial markets containing heterogeneous populations of automated trading agents and compare PRBO with PRSH, a reference trading strategy that adapts strategy parameters through stochastic hill-climbing. Results show that PRBO generates significantly more profit than PRSH, despite having fewer hyperparameters to tune. The code for PRBO and performing experiments is available online open-source (https://github.com/HarmoniaLeo/PRZI-Bayesian-Optimisation).
Drive Right: Shaping Public's Trust, Understanding, and Preference Towards Autonomous Vehicles Using a Virtual Reality Driving Simulator
Venue: arXiv cs.HC 2023
Year: 2023

Autonomous vehicles are increasingly introduced into our lives. Yet, people's misunderstanding and mistrust have become the major obstacles to the use of these technologies. In response to this problem, proper work must be done to increase public's understanding and awareness and help drivers rationally evaluate the system. The method proposed in this paper is a virtual reality driving simulator which serves as a low-cost platform for autonomous vehicle demonstration and education. To test the validity of the platform, we recruited 36 participants and conducted a test training drive using three different scenarios. The results show that our simulator successfully increased participants' understanding while favorably changing their attitude towards the autonomous system. The methodology and findings presented in this paper can be further explored by driving schools, auto manufacturers, and policy makers, to improve training for autonomous vehicles.
Tailoring to the Tails: Risk Measures for Fine-Grained Tail Sensitivity
Venue: arXiv cs.LG 2023
Year: 2023

Expected risk minimization (ERM) is at the core of many machine learning systems. This means that the risk inherent in a loss distribution is summarized using a single number - its average. In this paper, we propose a general approach to construct risk measures which exhibit a desired tail sensitivity and may replace the expectation operator in ERM. Our method relies on the specification of a reference distribution with a desired tail behaviour, which is in a one-to-one correspondence to a coherent upper probability. Any risk measure, which is compatible with this upper probability, displays a tail sensitivity which is finely tuned to the reference distribution. As a concrete example, we focus on divergence risk measures based on f-divergence ambiguity sets, which are a widespread tool used to foster distributional robustness of machine learning systems. For instance, we show how ambiguity sets based on the Kullback-Leibler divergence are intricately tied to the class of subexponential random variables. We elaborate the connection of divergence risk measures and rearrangement invariant Banach norms.
Towards Antisymmetric Neural Ansatz Separation
Venue: arXiv cs.LG 2023
Year: 2023

We study separations between two fundamental models (or \emph{Ansätze}) of antisymmetric functions, that is, functions $f$ of the form $f(x_{σ(1)}, \ldots, x_{σ(N)}) = \text{sign}(σ)f(x_1, \ldots, x_N)$, where $σ$ is any permutation. These arise in the context of quantum chemistry, and are the basic modeling tool for wavefunctions of Fermionic systems. Specifically, we consider two popular antisymmetric Ansätze: the Slater representation, which leverages the alternating structure of determinants, and the Jastrow ansatz, which augments Slater determinants with a product by an arbitrary symmetric function. We construct an antisymmetric function in $N$ dimensions that can be efficiently expressed in Jastrow form, yet provably cannot be approximated by Slater determinants unless there are exponentially (in $N^2$) many terms. This represents the first explicit quantitative separation between these two Ansätze.
Agents Incorporating Identity and Dynamic Teams in Social Dilemmas
Venue: arXiv cs.MA 2023
Year: 2023

We present our preliminary work on a multi-agent system involving the complex human phenomena of identity and dynamic teams. We outline our ongoing experimentation into understanding how these factors can eliminate some of the naive assumptions of current multi-agent approaches. These include a lack of complex heterogeneity between agents and unchanging team structures. We outline the human social psychological basis for identity, one's sense of self, and dynamic teams, the changing nature of human teams. We describe our application of these factors to a multi-agent system and our expectations for how they might improve the system's applicability to more complex problems, with specific relevance to ad hoc teamwork. We expect that the inclusion of more complex human processes, like identity and dynamic teams, will help with the eventual goal of having effective human-agent teams.
Variational Autoencoders for Anomaly Detection in Respiratory Sounds
Venue: arXiv cs.SD 2023
Year: 2023

This paper proposes a weakly-supervised machine learning-based approach aiming at a tool to alert patients about possible respiratory diseases. Various types of pathologies may affect the respiratory system, potentially leading to severe diseases and, in certain cases, death. In general, effective prevention practices are considered as major actors towards the improvement of the patient's health condition. The proposed method strives to realize an easily accessible tool for the automatic diagnosis of respiratory diseases. Specifically, the method leverages Variational Autoencoder architectures permitting the usage of training pipelines of limited complexity and relatively small-sized datasets. Importantly, it offers an accuracy of 57 %, which is in line with the existing strongly-supervised approaches.
TPM: Transition Probability Matrix -- Graph Structural Feature based Embedding
Venue: arXiv cs.LG 2023
Year: 2023

In this work, Transition Probability Matrix (TPM) is proposed as a new method for extracting the features of nodes in the graph. The proposed method uses random walks to capture the connectivity structure of a node's close neighborhood. The information obtained from random walks is converted to anonymous walks to extract the topological features of nodes. In the embedding process of nodes, anonymous walks are used since they capture the topological similarities of connectivities better than random walks. Therefore the obtained embedding vectors have richer information about the underlying connectivity structure. The method is applied to node classification and link prediction tasks. The performance of the proposed algorithm is superior to the state-of-the-art algorithms in the recent literature. Moreover, the extracted information about the connectivity structure of similar networks is used to link prediction and node classification tasks for a completely new graph.
Type-Theoretic Approaches to Ordinals
Venue: arXiv cs.LO 2023
Year: 2023

In a constructive setting, no concrete formulation of ordinal numbers can simultaneously have all the properties one might be interested in; for example, being able to calculate limits of sequences is constructively incompatible with deciding extensional equality. Using homotopy type theory as the foundational setting, we develop an abstract framework for ordinal theory and establish a collection of desirable properties and constructions. We then study and compare three concrete implementations of ordinals in homotopy type theory: first, a notation system based on Cantor normal forms (binary trees); second, a refined version of Brouwer trees (infinitely-branching trees); and third, extensional well-founded orders. Each of our three formulations has the central properties expected of ordinals, such as being equipped with an extensional and well-founded ordering as well as allowing basic arithmetic operations, but they differ with respect to what they make possible in addition. For example, for finite collections of ordinals, Cantor normal forms have decidable properties, but suprema of infinite collections cannot be computed. In contrast, extensional well-founded orders work well with infinite collections, but almost all properties are undecidable. Brouwer trees take the sweet spot in the middle by combining a restricted form of decidability with the ability to work with infinite increasing sequences. Our three approaches are connected by canonical order-preserving functions from the "more decidable" to the "less decidable" notions. We have formalised the results on Cantor normal forms and Brouwer trees in cubical Agda, while extensional well-founded orders have been studied and formalised thoroughly by Escardo and his collaborators. Finally, we compare the computational efficiency of our implementations with the results reported by Berger.
Achievable Refined Asymptotics for Successive Refinement Using Gaussian Codebooks
Venue: arXiv cs.IT 2023
Year: 2023

We study the mismatched successive refinement problem where one uses Gaussian codebooks to compress an arbitrary memoryless source with successive minimum Euclidean distance encoding under the quadratic distortion measure. Specifically, we derive achievable refined asymptotics under both the joint excess-distortion probability (JEP) and the separate excess-distortion probabilities (SEP) criteria. For both second-order and moderate deviations asymptotics, we consider two types of codebooks: the spherical codebook where each codeword is drawn independently and uniformly from the surface of a sphere and the i.i.d. Gaussian codebook where each component of each codeword is drawn independently from a Gaussian distribution. We establish the achievable second-order rate-region under JEP and we show that under SEP any memoryless source satisfying mild moment conditions is strongly successively refinable. When specialized to a Gaussian memoryless source (GMS), our results provide an alternative achievability proof with specific code design. We show that under JEP and SEP, the same moderate deviations constant is achievable. For large deviations asymptotics, we only consider the i.i.d. Gaussian codebook since the i.i.d. Gaussian codebook has better performance than the spherical codebook in this regime for the one layer mismatched rate-distortion problem (Zhou, Tan, Motani, TIT, 2019). We derive achievable exponents of both JEP and SEP and specialize our results to a GMS, which appears to be a novel result of independent interest.
Hybrid-ARQ Based Relaying Strategies for Enhancing Reliability in Delay-Bounded Networks
Venue: arXiv cs.IT 2023
Year: 2023

Inspired by several delay-bounded mission-critical applications, this paper investigates chase-combining-based hybrid automatic repeat request (CC-HARQ) protocols to achieve high reliability in delay-constrained applications. A salient feature of our approach is to use the end-to-end delay constraint for computing the total number of ARQs permitted in the network, and then optimally distributing them across the nodes so as to minimize packet-drop-probability (PDP), which is the end-to-end reliability metric of interest. Since the chase-combining strategy combines the received packets across multiple attempts, we observe that the PDP of the network depends on the coherence-time of the intermediate wireless channels. As a result, we address the question of computing optimal allocation of ARQs for CC-HARQ strategies under both slow-fading and fast-fading scenarios. For both the channel conditions, we derive closed-form expressions for the PDP, and then formulate several optimization problems for minimizing the PDP for a given delay-bound. Using extensive theoretical results on the local minima of the optimization problems, we synthesize low-complexity algorithms to obtain near-optimal ARQ distributions. Besides using extensive simulation results to validate our findings, a detailed end-to-end delay analysis is also presented to show that the proposed CC-HARQ strategies outperform already known Type-1 ARQ based strategies in several scenarios.
300 GHz Dual-Band Channel Measurement, Analysis and Modeling in an L-shaped Hallway
Venue: arXiv cs.IT 2023
Year: 2023

The Terahertz (THz) band (0.1-10 THz) has been envisioned as one of the promising spectrum bands for sixth-generation (6G) and beyond communications. In this paper, a dual-band angular-resolvable wideband channel measurement in an indoor L-shaped hallway is presented and THz channel characteristics at 306-321 GHz and 356-371 GHz are analyzed. It is found that conventional close-in and alpha-beta path loss models cannot take good care of large-scale fading in the non-line-of-sight (NLoS) case, for which a modified alpha-beta path loss model for the NLoS case is proposed and verified in the NLoS case for both indoor and outdoor L-shaped scenarios. To describe both large-scale and small-scale fading, a ray-tracing (RT)-statistical hybrid channel model is proposed in the THz hallway scenario. Specifically in the hybrid model, the deterministic part in hybrid channel modeling uses RT modeling of dominant multi-path components (MPCs), i.e., LoS and multi-bounce reflected paths in the near-NLoS region, while dominant MPCs at far-NLoS positions can be deduced based on the developed statistical evolving model. The evolving model describes the continuous change of arrival angle, power and delay of dominant MPCs in the NLoS region. On the other hand, non-dominant MPCs are generated statistically. The proposed hybrid approach reduces the computational cost and solves the inaccuracy or even missing of dominant MPCs through RT at far-NLoS positions.
Learning to Learn to Predict Performance Regressions in Production at Meta
Venue: arXiv cs.SE 2023
Year: 2023

Catching and attributing code change-induced performance regressions in production is hard; predicting them beforehand, even harder. A primer on automatically learning to predict performance regressions in software, this article gives an account of the experiences we gained when researching and deploying an ML-based regression prediction pipeline at Meta. In this paper, we report on a comparative study with four ML models of increasing complexity, from (1) code-opaque, over (2) Bag of Words, (3) off-the-shelve Transformer-based, to (4) a bespoke Transformer-based model, coined SuperPerforator. Our investigation shows the inherent difficulty of the performance prediction problem, which is characterized by a large imbalance of benign onto regressing changes. Our results also call into question the general applicability of Transformer-based architectures for performance prediction: an off-the-shelve CodeBERT-based approach had surprisingly poor performance; our highly customized SuperPerforator architecture initially achieved prediction performance that was just on par with simpler Bag of Words models, and only outperformed them for down-stream use cases. This ability of SuperPerforator to transfer to an application with few learning examples afforded an opportunity to deploy it in practice at Meta: it can act as a pre-filter to sort out changes that are unlikely to introduce a regression, truncating the space of changes to search a regression in by up to 43%, a 45x improvement over a random baseline. To gain further insight into SuperPerforator, we explored it via a series of experiments computing counterfactual explanations. These highlight which parts of a code change the model deems important, thereby validating the learned black-box model.
Stronger Privacy Amplification by Shuffling for Rényi and Approximate Differential Privacy
Venue: arXiv cs.CR 2023
Year: 2023

The shuffle model of differential privacy has gained significant interest as an intermediate trust model between the standard local and central models [EFMRTT19; CSUZZ19]. A key result in this model is that randomly shuffling locally randomized data amplifies differential privacy guarantees. Such amplification implies substantially stronger privacy guarantees for systems in which data is contributed anonymously [BEMMRLRKTS17]. In this work, we improve the state of the art privacy amplification by shuffling results both theoretically and numerically. Our first contribution is the first asymptotically optimal analysis of the Rényi differential privacy parameters for the shuffled outputs of LDP randomizers. Our second contribution is a new analysis of privacy amplification by shuffling. This analysis improves on the techniques of [FMT20] and leads to tighter numerical bounds in all parameter settings.
An NLP-Assisted Bayesian Time Series Analysis for Prevalence of Twitter Cyberbullying During the COVID-19 Pandemic
Venue: arXiv cs.SI 2023
Year: 2023

COVID-19 has brought about many changes in social dynamics. Stay-at-home orders and disruptions in school teaching can influence bullying behavior in-person and online, both of which leading to negative outcomes in victims. To study cyberbullying specifically, 1 million tweets containing keywords associated with abuse were collected from the beginning of 2019 to the end of 2021 with the Twitter API search endpoint. A natural language processing model pre-trained on a Twitter corpus generated probabilities for the tweets being offensive and hateful. To overcome limitations of sampling, data was also collected using the count endpoint. The fraction of tweets from a given daily sample marked as abusive is multiplied to the number reported by the count endpoint. Once these adjusted counts are assembled, a Bayesian autoregressive Poisson model allows one to study the mean trend and lag functions of the data and how they vary over time. The results reveal strong weekly and yearly seasonality in hateful speech but with slight differences across years that may be attributed to COVID-19.
High-Order Mesh Morphing for Boundary and Interface Fitting to Implicit Geometries
Venue: arXiv cs.CG 2023
Year: 2023

We propose a method that morphs high-orger meshes such that their boundaries and interfaces coincide/align with implicitly defined geometries. Our focus is particularly on the case when the target surface is prescribed as the zero isocontour of a smooth discrete function. Common examples of this scenario include using level set functions to represent material interfaces in multimaterial configurations, and evolving geometries in shape and topology optimization. The proposed method formulates the mesh optimization problem as a variational minimization of the sum of a chosen mesh-quality metric using the Target-Matrix Optimization Paradigm (TMOP) and a penalty term that weakly forces the selected faces of the mesh to align with the target surface. The distinct features of the method are use of a source mesh to represent the level set function with sufficient accuracy, and adaptive strategies for setting the penalization weight and selecting the faces of the mesh to be fit to the target isocontour of the level set field. We demonstrate that the proposed method is robust for generating boundary- and interface-fitted meshes for curvilinear domains using different element types in 2D and 3D.