Miracle-dz/UniFAR-FTUs huggingface dataset:
TCTrack: Temporal Contexts for Aerial Tracking

Temporal contexts among consecutive frames are far from being fully utilized in existing visual trackers. In this work, we present TCTrack11https://github.com/vision4robotics/TCTrack, a comprehensive framework to fully exploit temporal contexts for aerial tracking. The temporal contexts are incorporated at two levels: the extraction of features and the refinement of similarity maps. Specifically, for feature extraction, an online temporally adaptive convolution is proposed to enhance the spatial features using temporal information, which is achieved by dynamically calibrating the convolution weights according to the previous frames. For similarity map refinement, we propose an adaptive temporal transformer, which first effectively encodes temporal knowledge in a memory-efficient way, before the temporal knowledge is decoded for accurate adjustment of the similarity map. TCTrack is effective and efficient: evaluation on four aerial tracking benchmarks shows its impressive performance; real-world UAV tests show its high speed of over 27 FPS on NVIDIA Jetson AGX Xavier.
Accelerated SGD for Non-Strongly-Convex Least Squares

We consider stochastic approximation for the least squares regression problem in the non-strongly convex setting. We present the first practical algorithm that achieves the optimal prediction error rates in terms of dependence on the noise of the problem, as O(d/t) while accelerating the forgetting of the initial conditions to O(d/t). Our new algorithm is based on a simple modification of the accelerated gradient descent. We provide convergence results for both the averaged and the last iterate of the algorithm. In order to describe the tightness of these new bounds, we present a matching lower bound in the noiseless setting and thus show the optimality of our algorithm.
Translating the internal climate variability from climate variables to hydropower production

Quantifying uncertainties in estimating future hydropower production directly or indirectly affects India’s energy security, planning, and management. The chaotic and nonlinear nature of atmospheric processes results in considerable Internal Climate Variability (ICV) for future projections of climate variables. Multiple initial condition ensembles and multi-model ensembles of earth system models are often used to analyze the role of ICV and model uncertainty in precipitation and temperature. However, there are limited studies focusing on quantifying the role of internal variability on impact variables, including hydropower production. In this study, we analyze the role of internal variability and model uncertainty on three prominent hydropower plants of India using multiple initial condition ensembles of the EC-Earth3 and multiple models from CMIP6. We estimate the streamflow projections for all ensembles using the Variable Infiltration Capacity (VIC) hydrological model for four time periods, historical (1985-2014), near-term (2015-2044), mid-term (2045-2074) and far-term (2075-2100). We estimate maximum hydropower production generated using monthly release and hydraulic head available at the reservoir. We also analyzed the role of bias correction in hydropower production. The results show that ICV plays a significant role in estimating streamflow and hydropower estimation for monsoon and throughout the year, respectively. Model uncertainty contributes more to total uncertainty than ICV in estimating the streamflow and potential hydropower. However, ICV is increasing towards the far-term. We also show that bias correction does not preserve the internal variability in estimating the streamflow. Although there is an increase in uncertainty for estimated streamflow, mean hydropower shows the decrease towards the far-term for February to May, more prominent for MICE than MME. The results suggest a need to incorporate uncertainty due to internal variability for addressing power security in changing climate scenarios.
Duality via convolution of W-algebras

Feigin-Frenkel duality is the isomorphism between the principal W-algebras of a simple Lie algebra g and its Langlands dual Lie algebra g. A generalization of this duality to a larger family of W-algebras called hook-type was recently conjectured by Gaiotto and Rapčák and proved by the first two authors. It says that the affine cosets of two different hook-type W-algebras are isomorphic. A natural question is whether the duality between affine cosets can be enhanced to a duality between the full W-algebras. There is a convolution operation that maps a hook-type W-algebra W to a certain relative semi-infinite cohomology of W tensored with a suitable kernel VOA. The first two authors conjectured previously that this cohomology is isomorphic to the Feigin-Frenkel dual hook-type W-algebra. Our main result is a proof of this conjecture.
Recent Advances in Vision Transformer: A Survey and Outlook of Recent Work

Vision Transformers (ViTs) are becoming more popular and dominating technique for various vision tasks, compare to Convolutional Neural Networks (CNNs). As a demanding technique in computer vision, ViTs have been successfully solved various vision problems while focusing on long-range relationships. In this paper, we begin by introducing the fundamental concepts and background of the self-attention mechanism. Next, we provide a comprehensive overview of recent top-performing ViT methods describing in terms of strength and weakness, computational cost as well as training and testing dataset. We thoroughly compare the performance of various ViT algorithms and most representative CNN methods on popular benchmark datasets. Finally, we explore some limitations with insightful observations and provide further research direction. The project page along with the collections of papers are available at https://github.com/khawar512/ViT-Survey
LILE: Look In-Depth before Looking Elsewhere - A Dual Attention Network using Transformers for Cross-Modal Information Retrieval in Histopathology Archives

The volume of available data has grown dramatically in recent years in many applications. Furthermore, the age of networks that used multiple modalities separately has practically ended. Therefore, enabling bidirectional cross-modality data retrieval capable of processing has become a requirement for many domains and disciplines of research. This is especially true in the medical field, as data comes in a multitude of types, including various types of images and reports as well as molecular data. Most contemporary works apply cross attention to highlight the essential elements of an image or text in relation to the other modalities and try to match them together. However, regardless of their importance in their own modality, these approaches usually consider features of each modality equally. In this study, self-attention as an additional loss term will be proposed to enrich the internal representation provided into the cross attention module. This work suggests a novel architecture with a new loss term to help represent images and texts in the joint latent space. Experiment results on two benchmark datasets, i.e. MS-COCO and ARCH, show the effectiveness of the proposed method.
Neural Window Fully-connected CRFs for Monocular Depth Estimation

Estimating the accurate depth from a single image is challenging since it is inherently ambiguous and ill-posed. While recent works design increasingly complicated and powerful networks to directly regress the depth map, we take the path of CRFs optimization. Due to the expensive computation, CRFs are usually performed between neighborhoods rather than the whole graph. To leverage the potential of fully-connected CRFs, we split the input into windows and perform the FC-CRFs optimization within each window, which reduces the computation complexity and makes FC-CRFs feasible. To better capture the relationships between nodes in the graph, we exploit the multi-head attention mechanism to compute a multi-head potential function, which is fed to the networks to output an optimized depth map. Then we build a bottom-up-top-down structure, where this neural window FC-CRFs module serves as the decoder, and a vision transformer serves as the encoder. The experiments demonstrate that our method significantly improves the performance across all metrics on both the KITTI and NYUv2 datasets, compared to previous methods. Furthermore, the proposed method can be directly applied to panorama images and outperforms all previous panorama methods on the MatterPort3D dataset.11Project page: https://weihaosky.github. io/newcrfs
Liouvillian skin effect in an exactly solvable model

The interplay between dissipation, topology and sensitivity to boundary conditions has recently attracted tremendous amounts of attention at the level of effective non-Hermitian descriptions. Here we exactly solve a quantum mechanical Lindblad master equation describing a dissipative topological Su-Schrieffer-Heeger (SSH) chain of fermions for both open boundary condition (OBC) and periodic boundary condition (PBC). We find that the extreme sensitivity on the boundary conditions associated with the non-Hermitian skin effect is directly reflected in the rapidities governing the time evolution of the density matrix giving rise to a Liouvillian skin effect. This leads to several intriguing phenomena including boundary sensitive damping behavior, steady state currents in finite periodic systems, and diverging relaxation times in the limit of large systems. We illuminate how the role of topology in these systems differs in the effective non-Hermitian Hamiltonian limit and the full master equation framework.
Few-Shot Partial Multi-View Learning

It is often the case that data are with multiple views in real-world applications. Fully exploring the information of each view is significant for making data more representative. However, due to various limitations and failures in data collection and pre-processing, it is inevitable for real data to suffer from view missing and data scarcity. The coexistence of these two issues makes it more challenging to achieve the pattern classification task. Currently, to our best knowledge, few appropriate methods can well-handle these two issues simultaneously. Aiming to draw more attention from the community to this challenge, we propose a new task in this paper, called few-shot partial multi-view learning, which focuses on overcoming the negative impact of the view-missing issue in the low-data regime. The challenges of this task are twofold: (i) it is difficult to overcome the impact of data scarcity under the interference of missing views; (ii) the limited number of data exacerbates information scarcity, thus making it harder to address the view-missing issue in turn. To address these challenges, we propose a new unified Gaussian dense-anchoring method. The unified dense anchors are learned for the limited partial multi-view data, thereby anchoring them into a unified dense representation space where the influence of data scarcity and view missing can be alleviated. We conduct extensive experiments to evaluate our method. The results on Cub-googlenet-doc2vec, Handwritten, Caltech102, Scene15, Animal, ORL, tieredImagenet, and Birds-200-2011 datasets validate its effectiveness. The codes will be released at https://github.com/zhouyuan888888/UGDA.
Intensity Image-Based LiDAR Fiducial Marker System

The fiducial marker system for LiDAR is crucial for the robotic application but it is still rare to date. In this letter, an Intensity Image-based LiDAR Fiducial Marker (IILFM) system is developed. This system only requires an unstructured point cloud with intensity as the input and it has no restriction on marker placement and shape. A marker detection method that locates the predefined 3D fiducials in the point cloud through the intensity image is introduced. Then, an approach that utilizes the detected 3D fiducials to estimate the LiDAR 6-DOF pose that describes the transmission from the world coordinate system to the LiDAR coordinate system is developed. Moreover, all these processes run in real-time (approx 40 Hz on Livox Mid-40 and approx 143 Hz on VLP-16). Qualitative and quantitative experiments are conducted to demonstrate that the proposed system has similar convenience and accuracy as the conventional visual fiducial marker system.
Queueing Systems with Preferred Service Delivery Times and Multiple Customer Classes

Motivated by the operational problems in click and collect systems, such as curbside pickup programs, we study a joint admission control and capacity allocation problem. We consider a system where arriving customers have preferred service delivery times and gauge the service quality based on the service provider’s ability to complete the service as close as possible to the preferred time. Customers can be of different priority classes, and their priority may increase as they wait longer in the queue. The service provider can reject customers upon their arrival if the system is overloaded or outsource the service (alternatively work overtime) when the capacity is not enough. The service provider’s goal is to find the minimum-cost admission and capacity allocation policy to dynamically decide when to serve and whom to serve. We model this problem as a Markov Decision Process. Our structural results partially characterize a set of suboptimal solutions, and we develop solution methods using these results. We also develop a problem-specific approximation method that is based on state aggregation to overcome the computational challenges. We present extensive computational results and discuss the impact of problem parameters on the optimal policy.
Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations

Spurious correlations pose a major challenge for robust machine learning. Models trained with empirical risk minimization (ERM) may learn to rely on correlations between class labels and spurious attributes, leading to poor performance on data groups without these correlations. This is particularly challenging to address when spurious attribute labels are unavailable. To improve worst-group performance on spuriously correlated data without training attribute labels, we propose Correct-N-Contrast (CNC), a contrastive approach to directly learn representations robust to spurious correlations. As ERM models can be good spurious attribute predictors, CNC works by (1) using a trained ERM model's outputs to identify samples with the same class but dissimilar spurious features, and (2) training a robust model with contrastive learning to learn similar representations for same-class samples. To support CNC, we introduce new connections between worst-group error and a representation alignment loss that CNC aims to minimize. We empirically observe that worst-group error closely tracks with alignment loss, and prove that the alignment loss over a class helps upper-bound the class's worst-group vs. average error gap. On popular benchmarks, CNC reduces alignment loss drastically, and achieves state-of-the-art worst-group accuracy by 3.6% average absolute lift. CNC is also competitive with oracle methods that require group labels.
What Makes Transfer Learning Work for Medical Images: Feature Reuse & Other Factors

Transfer learning is a standard technique to transfer knowledge from one domain to another. For applications in medical imaging, transfer from ImageNet has become the de-facto approach, despite differences in the tasks and im-age characteristics between the domains. However, it is un-clear what factors determine whether - and to what extent- transfer learning to the medical domain is useful. The long- standing assumption that features from the source domain get reused has recently been called into question. Through a series of experiments on several medical image bench-mark datasets, we explore the relationship between transfer learning, data size, the capacity and inductive bias of the model, as well as the distance between the source and tar-get domain. Our findings suggest that transfer learning is beneficial in most cases, and we characterize the important role feature reuse plays in its success.
CAFE Learning to Condense Dataset by Aligning Features

Dataset condensation aims at reducing the network training effort through condensing a cumbersome training set into a compact synthetic one. State-of-the-art approaches largely rely on learning the synthetic data by matching the gradients between the real and synthetic data batches. Despite the intuitive motivation and promising re-sults, such gradient-based methods, by nature, easily over-ﬁt to a biased set of samples that produce dominant gradients, and thus lack a global supervision of data distribution. In this paper, we propose a novel scheme to Condense dataset by Aligning FEatures (CAFE), which explicitly attempts to preserve the real-feature distribution as well as the discriminant power of the resulting synthetic set, lending itself to strong generalization capability to various architectures. At the heart of our approach is an effective strategy to align features from the real and synthetic data across various scales, while accounting for the classiﬁcation of real samples. Our scheme is further backed up by a novel dynamic bi-level optimization, which adaptively adjusts parameter updates to prevent over-/under-ﬁtting. We validate the proposed CAFE across various datasets, and demonstrate that it generally outperforms the state of the art: on the SVHN dataset, for example, the performance gain is up to 11%. Extensive experiments and analysis verify the effectiveness and necessity of proposed designs.
Nemo: Guiding and Contextualizing Weak Supervision for Interactive Data Programming

Weak Supervision (WS) techniques allow users to efficiently create large training datasets by programmatically labeling data with heuristic sources of supervision. While the success of WS relies heavily on the provided labeling heuristics, the process of how these heuristics are created in practice has remained under-explored. In this work, we formalize the development process of labeling heuristics as an interactive procedure, built around the existing workflow where users draw ideas from a selected set of development data for designing the heuristic sources. With the formalism, shown in Figure 1, we study two core problems of (1) how to strategically select the development data to guide users in efficiently creating informative heuristics, and (2) how to exploit the information within the development process to contextualize and better learn from the resultant heuristics. Building upon two novel methodologies that effectively tackle the respective problems considered, we present Nemo, an end-to-end interactive system that improves the overall productivity of WS learning pipeline by an average 20% (and up to 47% in one task) compared to the prevailing WS approach.
Modularity of the ABCD Random Graph Model with Community Structure

The Artificial Benchmark for Community Detection (ABCD) graph is a random graph model with community structure and power-law distribution for both degrees and community sizes. The model generates graphs with similar properties as the well-known LFR one, and its main parameter $\xi$ can be tuned to mimic its counterpart in the LFR model, the mixing parameter $\mu$. In this paper, we investigate various theoretical asymptotic properties of the ABCD model. In particular, we analyze the modularity function, arguably, the most important graph property of networks in the context of community detection. Indeed, the modularity function is often used to measure the presence of community structure in networks. It is also used as a quality function in many community detection algorithms, including the widely used Louvain algorithm.
Tracking Object-State Representations During Real-Time Language Comprehension by Native and Non-native Speakers of English

The present “visual world” eye-tracking study examined the time-course of how native and non-native speakers keep track of implied object-state representations during real-time language processing. Fifty-two native speakers of English and 46 non-native speakers with advanced English proficiency joined this study. They heard short stories describing a target object (e.g., an onion) either having undergone a substantial change-of-state (e.g., chop the onion) or a minimal change-of-state (e.g., weigh the onion) while their eye movements toward competing object-states (e.g., a chopped onion vs. an intact onion) and two unrelated distractors were tracked. We found that both groups successfully directed their visual attention toward the end-state of the target object that was implied in the linguistic context. However, neither group showed anticipatory eye movements toward the implied object-state when hearing the critical verb (e.g., “weigh/chop”). Only native English speakers but not non-native speakers showed a bias in visual attention during the determiner (“the”) before the noun (e.g., “onion”). Our results suggested that although native and non-native speakers of English largely overlapped in their time-courses of keeping track of object-state representations during real-time language comprehension, non-native speakers showed a short delay in updating the implied object-state representations.
Virtual Reality as a Moderator of Psychedelic-Assisted Psychotherapy

Psychotherapy with the use of psychedelic substances, including psilocybin, lysergic acid diethylamide (LSD), ketamine, and 3,4-methylenedioxymethamphetamine (MDMA), has demonstrated promise in treatment of post-traumatic stress disorder (PTSD), anxiety, addiction, and treatment-resistant depression. Psychedelic-assisted psychotherapy (PP) represents a unique psychopharmacological model that leverages the profound effects of the psychedelic experience. That experience is characterized by strong dependency on two key factors: participant mindset and the therapeutic environment. As such, therapeutic models that utilize psychedelics reflect the need for careful design that promotes an open, flexible, trusting mindset and a supportive setting. To meet this need, the PP model is increasingly supplemented by auxiliary methods, including meditation, relaxation, visualization or spiritual practices. We suggest virtual reality (VR) as a full-spectrum tool able to capitalize on and catalyze the innately therapeutic aspects of the psychedelic experience, such as detachment from familiar reality, alteration of self-experience, augmentation of sensory perception and induction of mystical-type experiences. This is facilitated by VR’s evidenced capacity to: aid relaxation and reduce anxiety; buffer from external stimuli; promote a mindful presence; train the mind to achieve altered states of consciousness (ASC); evoke mystical states; enhance therapeutic alliance and encourage self-efficacy. While these unique VR features appear promising, VR’s potential role in PP remains speculative due to lack of empirical evidence on the combined use of VR and PP. Given the increased commercial interest in this synergy there is an urgent need to evaluate this approach. We suggest specific VR models and their role within PP protocols to inspire future direction in scientific research, and provide a list of potential disadvantages, side effects and limitations that need to be carefully considered. These include sensory overstimulation, cyber-sickness, triggering memories of past traumatic events as well as distracting from the inner experience or strongly influencing its contents. A balanced, evidence-based approach may provide continuity across all phases of treatment, support transition into and out of an ASC, deepen acute ASC experiences including mystical states and enrich the psychotherapeutic process of integration. We conclude that the potential application of VR in modulating psychedelic-assisted psychotherapy demands further exploration and an evidence-based approach to both design and implementation.