Miracle-dz/UniFAR-FTUs huggingface dataset:
Significance of Anatomical Constraints in Virtual Try-On

The system of Virtual Try-ON (VTON) allows a user to try a product virtually. In general, a VTON system takes a clothing source and a person's image to predict the try-on output of the person in the given clothing. Although existing methods perform well for simple poses, in case of bent or crossed arms posture or when there is a significant difference between the alignment of the source clothing and the pose of the target person, these methods fail by generating inaccurate clothing deformations. In the VTON methods that employ Thin Plate Spline (TPS) based clothing transformations, this mainly occurs for two reasons - (1) the second-order smoothness constraint of TPS that restricts the bending of the object plane. (2) Overlaps among different clothing parts (e.g., sleeves and torso) can not be modeled by a single TPS transformation, as it assumes the clothing as a single planar object; therefore, disregards the independence of movement of different clothing parts. To this end, we make two major contributions. Concerning the bending limitations of TPS, we propose a human AnaTomy-Aware Geometric (ATAG) transformation. Regarding the overlap issue, we propose a part-based warping approach that divides the clothing into independently warpable parts to warp them separately and later combine them. Extensive analysis shows the efficacy of this approach.
Real-and-Present: Investigating the Use of Life-Size 2D Video Avatars in HMD-Based AR Teleconferencing

Augmented Reality (AR) teleconferencing allows separately located users to interact with each other in 3D through agents in their own physical environments. Existing methods leveraging volumetric capturing and reconstruction can provide a high-fidelity experience but are often too complex and expensive for everyday usage. Other solutions target mobile and effortless-to-setup teleconferencing on AR Head Mounted Displays (HMD). They directly transplant the conventional video conferencing onto an AR-HMD platform or use avatars to represent remote participants. However, they can only support either a high fidelity or a high level of co-presence. Moreover, the limited Field of View (FoV) of HMDs could further influence users' immersive experience. To achieve a balance between fidelity and co-presence, we explore using life-size 2D video-based avatars (video avatars for short) in AR teleconferencing. Specifically, with the potential effect of FoV on users' perception of proximity, we first conduct a pilot study to explore the local-user-centered optimal placement of video avatars in small-group AR conversations. With the placement results, we then implement a proof-of-concept prototype of video-avatar-based teleconferencing. We conduct user evaluations with the prototype to verify its effectiveness in balancing fidelity and co-presence. Following the indication in the pilot study, we further quantitatively explore the effect of FoV size on the video avatar's optimal placement through a user study involving more FoV conditions in a VR-simulated environment. We regress placement models to serve as references for computationally determining video avatar placements in such teleconferencing applications on various existing AR HMDs and future ones with bigger FoVs.
Self-Contrast: Better Reflection Through Inconsistent Solving Perspectives

The reflection capacity of Large Language Model (LLM) has garnered extensive attention. A post-hoc prompting strategy, e.g., reflexion and self-refine, refines LLM's response based on self-evaluated or external feedback. However, recent research indicates without external feedback, LLM's intrinsic reflection is unstable. Our investigation unveils that the key bottleneck is the quality of the self-evaluated feedback. We find LLMs often exhibit overconfidence or high randomness when self-evaluate, offering stubborn or inconsistent feedback, which causes poor reflection. To remedy this, we advocate Self-Contrast: It adaptively explores diverse solving perspectives tailored to the request, contrasts the differences, and summarizes these discrepancies into a checklist which could be used to re-examine and eliminate discrepancies. Our method endows LLM with diverse perspectives to alleviate stubborn biases. Moreover, their discrepancies indicate potential errors or inherent uncertainties that LLM often overlooks. Reflecting upon these can catalyze more accurate and stable reflection. Experiments conducted on a series of reasoning and translation tasks with different LLMs serve to underscore the effectiveness and generality of our strategy.
Learning to Prompt with Text Only Supervision for Vision-Language Models

Foundational vision-language models such as CLIP are becoming a new paradigm in vision, due to their excellent generalization abilities. However, adapting these models for downstream tasks while maintaining their generalization remains a challenge. In literature, one branch of methods adapts CLIP by learning prompts using visual information. While effective, most of these works require labeled data which is not practical, and often struggle to generalize towards new datasets due to over-fitting on the source data. An alternative approach resorts to training-free methods by generating class descriptions from large language models (LLMs) and perform prompt ensembling. However, these methods often generate class specific prompts that cannot be transferred to other classes, which incur higher costs by generating LLM descriptions for each class separately. In this work, we propose to combine the strengths of these both streams of methods by learning prompts using only text data derived from LLMs. As supervised training of prompts is not trivial due to absence of images, we develop a training approach that allows prompts to extract rich contextual knowledge from LLM data. Moreover, with LLM contextual data mapped within the learned prompts, it enables zero-shot transfer of prompts to new classes and datasets potentially cutting the LLM prompt engineering cost. To the best of our knowledge, this is the first work that learns generalized prompts using text only data. We perform extensive evaluations on 4 benchmarks where our method improves over prior ensembling works while being competitive to those utilizing labeled images. Our code and pre-trained models are available at https://github.com/muzairkhattak/ProText.
Efficient Cloud-edge Collaborative Inference for Object Re-identification

Current object re-identification (ReID) system follows the centralized processing paradigm, i.e., all computations are conducted in the cloud server and edge devices are only used to capture and send images. As the number of videos experiences a rapid escalation, this paradigm has become impractical due to the finite computational resources. In such a scenario, the ReID system should be converted to fit in the cloud-edge collaborative processing paradigm, which is crucial to boost the scalability and practicality of ReID systems. However, current relevant work lacks research on this issue, making it challenging for ReID methods to be adapted effectively. Therefore, we pioneer a cloud-edge collaborative inference framework for ReID systems and particularly propose a distribution-aware correlation modeling network (DaCM) to make the desired image return to the cloud server as soon as possible via learning to model the spatial-temporal correlations among instances. DaCM embeds the spatial-temporal correlations implicitly included in the timestamps into a graph structure, and it can be applied in the cloud to regulate the size of the upload window and on the edge device to adjust the sequence of images, respectively. Traditional ReID methods can be combined with DaCM seamlessly, enabling their application within our proposed edge-cloud collaborative framework. Extensive experiments demonstrate that our method obviously reduces transmission overhead and significantly improves performance. We will release our code and model.
Fast & Fair: Efficient Second-Order Robust Optimization for Fairness in Machine Learning

This project explores adversarial training techniques to develop fairer Deep Neural Networks (DNNs) to mitigate the inherent bias they are known to exhibit. DNNs are susceptible to inheriting bias with respect to sensitive attributes such as race and gender, which can lead to life-altering outcomes (e.g., demographic bias in facial recognition software used to arrest a suspect). We propose a robust optimization problem, which we demonstrate can improve fairness in several datasets, both synthetic and real-world, using an affine linear model. Leveraging second order information, we are able to find a solution to our optimization problem more efficiently than a purely first order method.
Multi-Source Domain Adaptation with Transformer-based Feature Generation for Subject-Independent EEG-based Emotion Recognition

Although deep learning-based algorithms have demonstrated excellent performance in automated emotion recognition via electroencephalogram (EEG) signals, variations across brain signal patterns of individuals can diminish the model's effectiveness when applied across different subjects. While transfer learning techniques have exhibited promising outcomes, they still encounter challenges related to inadequate feature representations and may overlook the fact that source subjects themselves can possess distinct characteristics. In this work, we propose a multi-source domain adaptation approach with a transformer-based feature generator (MSDA-TF) designed to leverage information from multiple sources. The proposed feature generator retains convolutional layers to capture shallow spatial, temporal, and spectral EEG data representations, while self-attention mechanisms extract global dependencies within these features. During the adaptation process, we group the source subjects based on correlation values and aim to align the moments of the target subject with each source as well as within the sources. MSDA-TF is validated on the SEED dataset and is shown to yield promising results.
Sending-or-not-sending quantum key distribution with phase postselection

Quantum key distribution (QKD) could help to share secure key between two distant peers. In recent years, twin-field (TF) QKD has been widely investigated because of its long transmission distance. One of the popular variants of TF QKD is sending-or-not-sending (SNS) QKD, which has been experimentally verified to realize 1000-km level fibre key distribution. In this article, the authors introduce phase postselection into the SNS protocol. With this modification, the probability of selecting"sending"can be substantially improved. The numerical simulation shows that the transmission distance can be improved both with and without the actively odd-parity pairing method. With discrete phase randomization, the variant can have both a larger key rate and a longer distance.
TR-DETR: Task-Reciprocal Transformer for Joint Moment Retrieval and Highlight Detection

Video moment retrieval (MR) and highlight detection (HD) based on natural language queries are two highly related tasks, which aim to obtain relevant moments within videos and highlight scores of each video clip. Recently, several methods have been devoted to building DETR-based networks to solve both MR and HD jointly. These methods simply add two separate task heads after multi-modal feature extraction and feature interaction, achieving good performance. Nevertheless, these approaches underutilize the reciprocal relationship between two tasks. In this paper, we propose a task-reciprocal transformer based on DETR (TR-DETR) that focuses on exploring the inherent reciprocity between MR and HD. Specifically, a local-global multi-modal alignment module is first built to align features from diverse modalities into a shared latent space. Subsequently, a visual feature refinement is designed to eliminate query-irrelevant information from visual features for modal interaction. Finally, a task cooperation module is constructed to refine the retrieval pipeline and the highlight score prediction process by utilizing the reciprocity between MR and HD. Comprehensive experiments on QVHighlights, Charades-STA and TVSum datasets demonstrate that TR-DETR outperforms existing state-of-the-art methods. Codes are available at https://github.com/mingyao1120/TR-DETR.
Universal Approximation Theorem for Vector- and Hypercomplex-Valued Neural Networks

The universal approximation theorem states that a neural network with one hidden layer can approximate continuous functions on compact sets with any desired precision. This theorem supports using neural networks for various applications, including regression and classification tasks. Furthermore, it is valid for real-valued neural networks and some hypercomplex-valued neural networks such as complex-, quaternion-, tessarine-, and Clifford-valued neural networks. However, hypercomplex-valued neural networks are a type of vector-valued neural network defined on an algebra with additional algebraic or geometric properties. This paper extends the universal approximation theorem for a wide range of vector-valued neural networks, including hypercomplex-valued models as particular instances. Precisely, we introduce the concept of non-degenerate algebra and state the universal approximation theorem for neural networks defined on such algebras.
asimulation: Domain formation and impact on observables in resolved cosmological simulations of the (a)symmetron

The symmetron is a dark energy and dark matter candidate that forms topological defects in the late-time universe and holds promise to resolve some of the cosmological tensions. We perform high resolution simulations of the dynamical and non-linear (a)symmetron using the recently developed relativistic N-body code asevolution. By extensively testing the temporal and spatial convergence of domain decompositioning and domain wall stability, we find criteria and physical intuition for the convergence. We apply the resolution criteria to run five high resolution, $1280^3$ grids and 500 Mpc/h boxsize, simulations of the (a)symmetron and consider the behaviour of the scalar field and the domain walls in each scenario. We find the effect on the matter power spectra, the halo mass function and observables computed over the past lightcone of an observer such as the integrated Sachs-Wolfe and non-linear Rees-Sciama effect (ISW-RS) and the lensing, compared to LCDM. We show local oscillations of the fifth force strength and the formation of planar structures in the density field. The dynamics of the field is visualised in animations with high resolution in time. The simulation code is made publicly available.
Dynamics of point-vortex systems near thermal equilibrium: relaxation or not?

This article is devoted to the long-time dynamics of point-vortex systems near thermal equilibrium and to the possible emergence of collisional relaxation. More precisely, we consider a tagged particle coupled to a large number of background particles that are initially at equilibrium, and we analyze its resulting slow dynamics. On the one hand, in the spirit of the Lenard-Balescu relaxation for plasmas, we establish in a generic setting the outset of the slow thermalization of the tagged particle. On the other hand, we show that a completely different phenomenology is also possible in some degenerate regime: the slow dynamics of the tagged particle then remains conservative and the thermalization no longer holds in a strict sense. We provide the first detailed description of this degenerate regime and of its mixing properties. Note that it is particularly delicate to handle due to statistical closure problems, which manifest themselves as a lack of self-adjointness of the effective Hamiltonian.
Enhancing RAW-to-sRGB with Decoupled Style Structure in Fourier Domain

RAW to sRGB mapping, which aims to convert RAW images from smartphones into RGB form equivalent to that of Digital Single-Lens Reflex (DSLR) cameras, has become an important area of research. However, current methods often ignore the difference between cell phone RAW images and DSLR camera RGB images, a difference that goes beyond the color matrix and extends to spatial structure due to resolution variations. Recent methods directly rebuild color mapping and spatial structure via shared deep representation, limiting optimal performance. Inspired by Image Signal Processing (ISP) pipeline, which distinguishes image restoration and enhancement, we present a novel Neural ISP framework, named FourierISP. This approach breaks the image down into style and structure within the frequency domain, allowing for independent optimization. FourierISP is comprised of three subnetworks: Phase Enhance Subnet for structural refinement, Amplitude Refine Subnet for color learning, and Color Adaptation Subnet for blending them in a smooth manner. This approach sharpens both color and structure, and extensive evaluations across varied datasets confirm that our approach realizes state-of-the-art results. Code will be available at https://github.com/alexhe101/FourierISP.
How Do Pedestrians' Perception Change toward Autonomous Vehicles during Unmarked Midblock Multilane Crossings: Role of AV Operation and Signal Indication

One of the primary impediments hindering the widespread acceptance of autonomous vehicles (AVs) among pedestrians is their limited comprehension of AVs. This study employs virtual reality (VR) to provide pedestrians with an immersive environment for engaging with and comprehending AVs during unmarked midblock multilane crossings. Diverse AV driving behaviors were modeled to exhibit negotiation behavior with a yellow signal indication or non-yielding behavior with a blue signal indication. This paper aims to investigate the impact of various factors, such as AV behavior and signaling, pedestrian past behavior, etc., on pedestrians' perception change of AVs. Before and after the VR experiment, participants completed surveys assessing their perception of AVs, focusing on two main aspects:"Attitude"and"System Effectiveness. "The Wilcoxon signed-rank test results demonstrated that both pedestrians' overall attitude score toward AVs and trust in the effectiveness of AV systems significantly increased following the VR experiment. Notably, individuals who exhibited a greater trust in the yellow signals were more inclined to display a higher attitude score toward AVs and to augment their trust in the effectiveness of AV systems. This indicates that the design of the yellow signal instills pedestrians with greater confidence in their interactions with AVs. Further, pedestrians who exhibit more aggressive crossing behavior are less likely to change their perception towards AVs as compared to those pedestrians with more positive crossing behaviors. It is concluded that integrating this paper's devised AV behavior and signaling within an immersive VR setting facilitated pedestrian engagement with AVs, thereby changing their perception of AVs.
Uncertainty-Aware Deep Attention Recurrent Neural Network for Heterogeneous Time Series Imputation

Missingness is ubiquitous in multivariate time series and poses an obstacle to reliable downstream analysis. Although recurrent network imputation achieved the SOTA, existing models do not scale to deep architectures that can potentially alleviate issues arising in complex data. Moreover, imputation carries the risk of biased estimations of the ground truth. Yet, confidence in the imputed values is always unmeasured or computed post hoc from model output. We propose DEep Attention Recurrent Imputation (DEARI), which jointly estimates missing values and their associated uncertainty in heterogeneous multivariate time series. By jointly representing feature-wise correlations and temporal dynamics, we adopt a self attention mechanism, along with an effective residual component, to achieve a deep recurrent neural network with good imputation performance and stable convergence. We also leverage self-supervised metric learning to boost performance by optimizing sample similarity. Finally, we transform DEARI into a Bayesian neural network through a novel Bayesian marginalization strategy to produce stochastic DEARI, which outperforms its deterministic equivalent. Experiments show that DEARI surpasses the SOTA in diverse imputation tasks using real-world datasets, namely air quality control, healthcare and traffic.
AUPIMO: Redefining Visual Anomaly Detection Benchmarks with High Speed and Low Tolerance

Recent advances in visual anomaly detection research have seen AUROC and AUPRO scores on public benchmark datasets such as MVTec and VisA converge towards perfect recall, giving the impression that these benchmarks are near-solved. However, high AUROC and AUPRO scores do not always reflect qualitative performance, which limits the validity of these metrics in real-world applications. We argue that the artificial ceiling imposed by the lack of an adequate evaluation metric restrains progression of the field, and it is crucial that we revisit the evaluation metrics used to rate our algorithms. In response, we introduce Per-IMage Overlap (PIMO), a novel metric that addresses the shortcomings of AUROC and AUPRO. PIMO retains the recall-based nature of the existing metrics but introduces two distinctions: the assignment of curves (and respective area under the curve) is per-image, and its X-axis relies solely on normal images. Measuring recall per image simplifies instance score indexing and is more robust to noisy annotations. As we show, it also accelerates computation and enables the usage of statistical tests to compare models. By imposing low tolerance for false positives on normal images, PIMO provides an enhanced model validation procedure and highlights performance variations across datasets. Our experiments demonstrate that PIMO offers practical advantages and nuanced performance insights that redefine anomaly detection benchmarks -- notably challenging the perception that MVTec AD and VisA datasets have been solved by contemporary models. Available on GitHub: https://github.com/jpcbertoldo/aupimo.
Deconstructing Photospheric Spectral Lines in Solar and Stellar Flares

During solar flares, spectral lines formed in the photosphere have been shown to exhibit changes to their profiles despite the challenges of energy transfer to these depths. Recent work has shown that deep-forming spectral lines are subject to significant contributions from regions above the photosphere throughout the flaring period, resulting in a composite emergent intensity profile from multiple layers of the atmosphere. We employ radiative–hydrodynamic and radiative transfer calculations to simulate the response of the solar/stellar atmosphere to electron beam heating and synthesize spectral lines of Fe i to investigate the line-of-sight velocity fields information available from Doppler shifts of the emergent intensity profile. By utilizing the contribution function to deconstruct the line profile shape into its constituent sources, we show that variations in the line profiles are primarily caused by changes in the chromosphere. Up-flows in this region were found to create blueshifts or false redshifts in the line core dependent on the relative contribution of the chromosphere compared to the photosphere. In extreme solar and stellar flare scenarios featuring explosive chromospheric condensations, redshifted transient components can dominate the temporal evolution of the profile shape, requiring a tertiary component consideration to fully characterize. We conclude that deep-forming lines require a multicomponent understanding and treatment, with different regions of the spectral line being useful for probing individual regions of the atmosphere’s velocity flows.