Miracle-dz/UniFAR-FTUs huggingface dataset:
BOTT: Box Only Transformer Tracker for 3D Object Tracking

Tracking 3D objects is an important task in autonomous driving. Classical Kalman Filtering based methods are still the most popular solutions. However, these methods require handcrafted designs in motion modeling and can not benefit from the growing data amounts. In this paper, Box Only Transformer Tracker (BOTT) is proposed to learn to link 3D boxes of the same object from the different frames, by taking all the 3D boxes in a time window as input. Specifically, transformer self-attention is applied to exchange information between all the boxes to learn global-informative box embeddings. The similarity between these learned embeddings can be used to link the boxes of the same object. BOTT can be used for both online and offline tracking modes seamlessly. Its simplicity enables us to significantly reduce engineering efforts required by traditional Kalman Filtering based methods. Experiments show BOTT achieves competitive performance on two largest 3D MOT benchmarks: 69.9 and 66.7 AMOTA on nuScenes validation and test splits, respectively, 56.45 and 59.57 MOTA L2 on Waymo Open Dataset validation and test splits, respectively. This work suggests that tracking 3D objects by learning features directly from 3D boxes using transformers is a simple yet effective way.
Topographic complexity and biotic resilience to climate change

Topographically complex alpine terrains create a mosaic of diverse microclimates over short distances. This study investigated the extent of small-scale variation in temperature and soil moisture using dense arrays of temperature loggers and moisture measurements, and how this microclimatic variation influenced vascular-plant species richness and intra-specific trait variation across five pairs of 40x40 m landscape plots of contrasting complexity in alpine tundra at Finse, Norway. Spatial variation in mean temperature within landscape plots was in the range of 2-4 °C, similar to what is expected across large altitudinal or latitudinal distances, suggesting an important buffering capacity of such landscapes in the event of climate warming. Rough landscapes contained more species than flatter ones, while patterns of within-species phenotypic variation were less clear and differed between species. These results suggest that local reshuffling and short-distance migration will be important biotic responses to climate change in this system, with assumed associated changes in biotic interactions and ecosystem function. The study also highlights the importance of mountains as target areas for biodiversity conservation.
AI-Assisted Investigation of on-Chain Parameters: Risky Cryptocurrencies and Price Factors

Cryptocurrencies have become a popular and widely researched topic of interest in recent years for investors and scholars. In order to make informed investment decisions, it is essential to comprehend the factors that impact cryptocurrency prices and to identify risky cryptocurrencies. This paper focuses on analyzing historical data and using artificial intelligence algorithms on on-chain parameters to identify the factors affecting a cryptocurrency's price and to find risky cryptocurrencies. We conducted an analysis of historical cryptocurrencies' on-chain data and measured the correlation between the price and other parameters. In addition, we used clustering and classification in order to get a better understanding of a cryptocurrency and classify it as risky or not. The analysis revealed that a significant proportion of cryptocurrencies (39%) disappeared from the market, while only a small fraction (10%) survived for more than 1000 days. Our analysis revealed a significant negative correlation between cryptocurrency price and maximum and total supply, as well as a weak positive correlation between price and 24-hour trading volume. Moreover, we clustered cryptocurrencies into five distinct groups using their on-chain parameters, which provides investors with a more comprehensive understanding of a cryptocurrency when compared to those clustered with it. Finally, by implementing multiple classifiers to predict whether a cryptocurrency is risky or not, we obtained the best f1-score of 76% using K-Nearest Neighbor.
Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis

We present a method that simultaneously addresses the tasks of dynamic scene novel-view synthesis and six degree-of-freedom (6-DOF) tracking of all dense scene elements. We follow an analysis-by-synthesis framework, inspired by recent work that models scenes as a collection of 3D Gaussians which are optimized to reconstruct input images via differentiable rendering. To model dynamic scenes, we allow Gaussians to move and rotate over time while enforcing that they have persistent color, opacity, and size. By regularizing Gaussians' motion and rotation with local-rigidity constraints, we show that our Dynamic 3D Gaussians correctly model the same area of physical space over time, including the rotation of that space. Dense 6-DOF tracking and dynamic reconstruction emerges naturally from persistent dynamic view synthesis, without requiring any correspondence or flow as input. We demonstrate a large number of downstream applications enabled by our representation, including first-person view synthesis, dynamic compositional scene synthesis, and 4D video editing.
The Inflation Attention Threshold and Inflation Surges

At the outbreak of the recent inflation surge, the public's attention to inflation was low but increased rapidly once inflation started to rise. I develop a framework where this behavior is optimal: agents pay little attention to inflation when inflation is low and stable, but they increase their attention once inflation exceeds a certain threshold. Using survey inflation expectations, I estimate the attention threshold to be at an inflation rate of about 4\%, with attention in the high-attention regime being twice as high as in the low-attention regime. Embedding this into a general equilibrium monetary model, I find that the inflation attention threshold gives rise to a dynamic non-linearity in the Phillips Curve, rendering inflation more sensitive to fluctuations in the output gap during periods of high attention. When calibrated to match the empirical findings, the model generates inflation and inflation expectation dynamics consistent with the recent inflation surge in the US. The attention threshold induces a state dependency: cost-push shocks become more inflationary in times of loose monetary policy. These state-dependent effects are absent in the model with constant attention or under rational expectations. Following simple Taylor rules triggers frequent and prolonged episodes of heightened attention, thereby increasing the volatility of inflation, and - due to the asymmetry of the attention threshold - also the average level of inflation, which leads to substantial welfare losses.
Sparse reconstruction of ordinary differential equations with inference

Sparse regression has emerged as a popular technique for learning dynamical systems from temporal data, beginning with the SINDy (Sparse Identification of Nonlinear Dynamics) framework proposed by arXiv:1509.03580. Quantifying the uncertainty inherent in differential equations learned from data remains an open problem, thus we propose leveraging recent advances in statistical inference for sparse regression to address this issue. Focusing on systems of ordinary differential equations (ODEs), SINDy assumes that each equation is a parsimonious linear combination of a few candidate functions, such as polynomials, and uses methods such as sequentially-thresholded least squares or the Lasso to identify a small subset of these functions that govern the system's dynamics. We instead employ bias-corrected versions of the Lasso and ridge regression estimators, as well as an empirical Bayes variable selection technique known as SEMMS, to estimate each ODE as a linear combination of terms that are statistically significant. We demonstrate through simulations that this approach allows us to recover the functional terms that correctly describe the dynamics more often than existing methods that do not account for uncertainty.
META-SELD: Meta-Learning for Fast Adaptation to the new environment in Sound Event Localization and Detection

For learning-based sound event localization and detection (SELD) methods, different acoustic environments in the training and test sets may result in large performance differences in the validation and evaluation stages. Different environments, such as different sizes of rooms, different reverberation times, and different background noise, may be reasons for a learning-based system to fail. On the other hand, acquiring annotated spatial sound event samples, which include onset and offset time stamps, class types of sound events, and direction-of-arrival (DOA) of sound sources is very expensive. In addition, deploying a SELD system in a new environment often poses challenges due to time-consuming training and fine-tuning processes. To address these issues, we propose Meta-SELD, which applies meta-learning methods to achieve fast adaptation to new environments. More specifically, based on Model Agnostic Meta-Learning (MAML), the proposed Meta-SELD aims to find good meta-initialized parameters to adapt to new environments with only a small number of samples and parameter updating iterations. We can then quickly adapt the meta-trained SELD model to unseen environments. Our experiments compare fine-tuning methods from pre-trained SELD models with our Meta-SELD on the Sony-TAU Realistic Spatial Soundscapes 2023 (STARSSS23) dataset. The evaluation results demonstrate the effectiveness of Meta-SELD when adapting to new environments.
RLIPv2: Fast Scaling of Relational Language-Image Pre-training

Relational Language-Image Pre-training (RLIP) aims to align vision representations with relational texts, thereby advancing the capability of relational reasoning in computer vision tasks. However, hindered by the slow convergence of RLIPv11 architecture and the limited availability of existing scene graph data, scaling RLIPv1 is challenging. In this paper, we propose RLIPv2, a fast converging model that enables the scaling of relational pre-training to large-scale pseudo-labelled scene graph data. To enable fast scaling, RLIPv2 introduces Asymmetric Language-Image Fusion (ALIF), a mechanism that facilitates earlier and deeper gated cross-modal fusion with sparsified language encoding layers. ALIF leads to comparable or better performance than RLIPv1 in a fraction of the time for pre-training and fine-tuning. To obtain scene graph data at scale, we extend object detection datasets with free-form relation labels by introducing a captioner (e.g., BLIP) and a designed Relation Tagger. The Relation Tagger assigns BLIP-generated relation texts to region pairs, thus enabling larger-scale relational pre-training. Through extensive experiments conducted on Human-Object Interaction Detection and Scene Graph Generation, RLIPv2 shows state-of-the-art performance on three benchmarks under fully-finetuning, few-shot and zero-shot settings. Notably, the largest RLIPv2 achieves 23.29mAP on HICO-DET without any fine-tuning, yields 32.22mAP with just 1% data and yields 45.09mAP with 100% data. Code and models are publicly available at https://github.com/JacobYuan7/RLIPv2.
Towards Personalized Federated Learning via Heterogeneous Model Reassembly

This paper focuses on addressing the practical yet challenging problem of model heterogeneity in federated learning, where clients possess models with different network structures. To track this problem, we propose a novel framework called pFedHR, which leverages heterogeneous model reassembly to achieve personalized federated learning. In particular, we approach the problem of heterogeneous model personalization as a model-matching optimization task on the server side. Moreover, pFedHR automatically and dynamically generates informative and diverse personalized candidates with minimal human intervention. Furthermore, our proposed heterogeneous model reassembly technique mitigates the adverse impact introduced by using public data with different distributions from the client data to a certain extent. Experimental results demonstrate that pFedHR outperforms baselines on three datasets under both IID and Non-IID settings. Additionally, pFedHR effectively reduces the adverse impact of using different public data and dynamically generates diverse personalized models in an automated manner.
Learning the meanings of function words from grounded language using a visual question answering model

Interpreting a seemingly-simple function word like"or","behind", or"more"can require logical, numerical, and relational reasoning. How are such words learned by children? Prior acquisition theories have often relied on positing a foundation of innate knowledge. Yet recent neural-network based visual question answering models apparently can learn to use function words as part of answering questions about complex visual scenes. In this paper, we study what these models learn about function words, in the hope of better understanding how the meanings of these words can be learnt by both models and children. We show that recurrent models trained on visually grounded language learn gradient semantics for function words requiring spacial and numerical reasoning. Furthermore, we find that these models can learn the meanings of logical connectives"and"and"or"without any prior knowledge of logical reasoning, as well as early evidence that they are sensitive to alternative expressions when interpreting language. Finally, we show that word learning difficulty is dependent on frequency in models' input. Our findings offer proof-of-concept evidence that it is possible to learn the nuanced interpretations of function words in visually grounded context by using non-symbolic general statistical learning algorithms, without any prior knowledge of linguistic meaning.
Addressing Socio-Cultural Animation as Community Based Social Work with Street Children in Maputo, Mozambique

It is rare to find a book written in English focused on the study of socio-cultural animation that is not the product of a translation from French, Spanish or Portuguese. Socio-cultural animation is a methodology for social and educational intervention that was originated in the middle of the 50’s of the last century in France and soon spread to the French-speaking countries and Spain, and through the latter, to Hispanic and Lusitanian countries. It can be said that there is not in the academic tradition or practice of the Anglo-Saxon countries anything like socio-cultural animation. Currently, a Google search of the terms “socio-cultural animation” generates very little results. Community Development would be the closest Anglo-Saxon equivalent. The author himself, when expresses his initial expectations regarding his research, notes that. the findings of the study may promote reflection on issues related to community development and social work practices (p. 33).
VALERIE22 - A photorealistic, richly metadata annotated dataset of urban environments

The VALERIE tool pipeline is a synthetic data generator [14] developed with the goal to contribute to the understanding of domain-specific factors that influence perception performance of DNNs (deep neural networks). This work was carried out under the German research project KI Absicherung in order to develop a methodology for the validation of DNNs in the context of pedestrian detection in urban environments for automated driving. The VALERIE22 dataset was generated with the VALERIE procedural tools pipeline providing a photorealistic sensor simulation rendered from automatically synthesized scenes. The dataset provides a uniquely rich set of metadata, allowing extraction of specific scene and semantic features (like pixel-accurate occlusion rates, positions in the scene and distance + angle to the camera). This enables a multitude of possible tests on the data and we hope to stimulate research on understanding performance of DNNs. Based on cross-domain semantic segmentation experiments, i.e. training on synthetic data and evaluation on target real world data, a comparison with several other publicly available datasets is provided, demonstrating that VALERIE22 is one of best performing synthetic datasets currently available in the open domain. 1
Linearity of Relation Decoding in Transformer Language Models

Much of the knowledge encoded in transformer language models (LMs) may be expressed in terms of relations: relations between words and their synonyms, entities and their attributes, etc. We show that, for a subset of relations, this computation is well-approximated by a single linear transformation on the subject representation. Linear relation representations may be obtained by constructing a first-order approximation to the LM from a single prompt, and they exist for a variety of factual, commonsense, and linguistic relations. However, we also identify many cases in which LM predictions capture relational knowledge accurately, but this knowledge is not linearly encoded in their representations. Our results thus reveal a simple, interpretable, but heterogeneously deployed knowledge representation strategy in transformer LMs.
Evaluation of Kato-Katz and multiplex quantitative polymerase chain reaction performance for clinical helminth infections in Thailand using a latent class analysis

Using an appropriate diagnostic tool is essential to soil-transmitted helminth control and elimination efforts. Kato-Katz (KK) is the most commonly used diagnostic, but recently other tools, such as real-time quantitative polymerase chain reaction (multiplex qPCR), are starting to be employed more. Here, we evaluated the performance of these two diagnostic tools for five helminth species in Thailand. In the absence of a gold standard, diagnostic performance can be evaluated using latent class analysis. Our results suggest that in moderate to high prevalence settings above 2% multiplex qPCR could be more sensitive than KK, this was particularly apparent for Opisthorchis viverrini in the northeastern provinces. However, for low prevalence, both diagnostics suffered from low sensitivity. Specificity of both diagnostics was estimated to be high (above 70%) across all settings. For some specific helminth infection such as O. viverrini, multiplex qPCR is still a preferable choice of diagnostic test. KK performed equally well in detecting Ascaris lumbricoides and Taenia solium when the prevalence is moderate to high (above 2%). Neither test performed well when the prevalence of infection is low (below 2%), and certainly in the case for hookworm and Trichuris trichiura. Combination of two or more diagnostic tests can improve the performance although the cost would be high. Development of new methods for helminth surveillance at the pre-elimination phase is therefore very important. This article is part of the theme issue ‘Challenges and opportunities in the fight against neglected tropical diseases: a decade from the London Declaration on NTDs’.
Co-lexicographically Ordering Automata and Regular Languages -- Part II

In the present work, we tackle the regular language indexing problem by first studying the hierarchy of $p$-sortable languages: regular languages accepted by automata of width $p$. We show that the hierarchy is strict and does not collapse, and provide (exponential in $p$) upper and lower bounds relating the minimum widths of equivalent NFAs and DFAs. Our bounds indicate the importance of being able to index NFAs, as they enable indexing regular languages with much faster and smaller indexes. Our second contribution solves precisely this problem, optimally: we devise a polynomial-time algorithm that indexes any NFA with the optimal value $p$ for its width, without explicitly computing $p$ (NP-hard to find). In particular, this implies that we can index in polynomial time the well-studied case $p=1$ (Wheeler NFAs). More in general, in polynomial time we can build an index breaking the worst-case conditional lower bound of $\Omega(|P| m)$, whenever the input NFA's width is $p \in o(\sqrt{m})$.
An Investigation on Hardware-Aware Vision Transformer Scaling

Vision Transformer (ViT) has demonstrated promising performance in various computer vision tasks, and recently attracted a lot of research attention. Many recent works have focused on proposing new architectures to improve ViT and deploying it into real-world applications. However, little effort has been made to analyze and understand ViT’s architecture design space and its implication of hardware-cost on different devices. In this work, by simply scaling ViT’s depth, width, input size, and other basic configurations, we show that a scaled vanilla ViT model without bells and whistles can achieve comparable or superior accuracy-efficiency trade-off than most of the latest ViT variants. Specifically, compared to DeiT-Tiny, our scaled model achieves a \(\uparrow 1.9\% \) higher ImageNet top-1 accuracy under the same FLOPs and a \(\uparrow 3.7\% \) better ImageNet top-1 accuracy under the same latency on an NVIDIA Edge GPU TX2. Motivated by this, we further investigate the extracted scaling strategies from the following two aspects: (1) “can these scaling strategies be transferred across different real hardware devices? ”; and (2) “can these scaling strategies be transferred to different ViT variants and tasks? ”. For (1), our exploration, based on various devices with different resource budgets, indicates that the transferability effectiveness depends on the underlying device together with its corresponding deployment tool; for (2), we validate the effective transferability of the aforementioned scaling strategies obtained from a vanilla ViT model on top of an image classification task to the PiT model, a strong ViT variant targeting efficiency, as well as object detection and video classification tasks. In particular, when transferred to PiT, our scaling strategies lead to a boosted ImageNet top-1 accuracy of from \(74.6\% \) to \(76.7\% \) ( \(\uparrow 2.1\% \) ) under the same 0.7G FLOPs; and when transferred to the COCO object detection task, the average precision is boosted by \(\uparrow 0.7\% \) under a similar throughput on a V100 GPU.