Miracle-dz/UniFAR-FTUs huggingface dataset:
Self-supervised and Weakly Supervised Contrastive Learning for Frame-wise Action Representations

Previous work on action representation learning focused on global representations for short video clips. In contrast, many practical applications, such as video alignment, strongly demand learning the intensive representation of long videos. In this paper, we introduce a new framework of contrastive action representation learning (CARL) to learn frame-wise action representation in a self-supervised or weakly-supervised manner, especially for long videos. Specifically, we introduce a simple but effective video encoder that considers both spatial and temporal context by combining convolution and transformer. Inspired by the recent massive progress in self-supervised learning, we propose a new sequence contrast loss (SCL) applied to two related views obtained by expanding a series of spatio-temporal data in two versions. One is the self-supervised version that optimizes embedding space by minimizing KL-divergence between sequence similarity of two augmented views and prior Gaussian distribution of timestamp distance. The other is the weakly-supervised version that builds more sample pairs among videos using video-level labels by dynamic time wrapping (DTW). Experiments on FineGym, PennAction, and Pouring datasets show that our method outperforms previous state-of-the-art by a large margin for downstream fine-grained action classification and even faster inference. Surprisingly, although without training on paired videos like in previous works, our self-supervised version also shows outstanding performance in video alignment and fine-grained frame retrieval tasks.
Iterative Next Boundary Detection for Instance Segmentation of Tree Rings in Microscopy Images of Shrub Cross Sections

We address the problem of detecting tree rings in microscopy images of shrub cross sections. This can be regarded as a special case of the instance segmentation task with several unique challenges such as the concentric circular ring shape of the objects and high precision requirements that result in inadequate performance of existing methods. We propose a new iterative method which we term Iterative Next Boundary Detection (INBD). It intuitively models the natural growth direction, starting from the center of the shrub cross section and detecting the next ring boundary in each iteration step. In our experiments, INBD shows superior performance to generic instance segmentation methods and is the only one with a built-in notion of chronological order. Our dataset and source code are available at http://github.com/alexander-g/INBD.
Weakly-Supervised Gaze Estimation from Synthetic Views

3D gaze estimation is most often tackled as learning a direct mapping between input images and the gaze vector or its spherical coordinates. Recently, it has been shown that pose estimation of the face, body and hands beneﬁts from revising the learning target from few pose parameters to dense 3D coordinates. In this work, we leverage this observation and propose to tackle 3D gaze estimation as regression of 3D eye meshes. We overcome the absence of compatible ground truth by ﬁtting a rigid 3D eyeball template on existing gaze datasets and propose to improve generalization by making use of widely available in-the-wild face images. To this end, we propose an automatic pipeline to retrieve robust gaze pseudo-labels from arbitrary face images and design a multi-view supervision framework to balance their effect during training. In our experiments, our method achieves improvement of 30% compared to state-of-the-art in cross-dataset gaze estimation, when no ground truth data are available for training, and 7% when they are. We make our project publicly available at https://github.com/Vagver/dense3Deyes
Front-door Adjustment via Style Transfer for Out-of-distribution Generalisation

Out-of-distribution (OOD) generalisation aims to build a model that can well generalise its learnt knowledge from source domains to an unseen target domain. However, current image classiﬁcation models often perform poorly in the OOD setting due to statistically spurious correlations learning from model training. From causality-based perspective, we formulate the data generation process in OOD image classiﬁcation using a causal graph. On this graph, we show that prediction P ( Y | X ) of a label Y given an image X in statistical learning is formed by both causal effect P ( Y | do ( X )) and spurious effects caused by confounding features (e.g., background). Since the spurious features are domain-variant, the prediction P ( Y | X ) becomes unstable on unseen domains. In this paper, we propose to mitigate the spurious effect of confounders using front-door adjustment. In our method, the mediator variable is hypothesized as semantic features that are essential to determine a label for an image. Inspired by capability of style transfer in image generation, we interpret the combination of the mediator variable with different generated images in the front-door formula and propose novel algorithms to estimate it. Extensive experimental results on widely used benchmark datasets verify the effectiveness of our method.
How to Compare Fuzzers

Fuzzing is a key method to discover vulnerabilities in programs. Despite considerable progress in this area in the past years, measuring and comparing the effectiveness of fuzzers is still an open research question. In software testing, the gold standard for evaluating test quality is mutation analysis , assessing the ability of a test to detect synthetic bugs; if a set of tests fails to detect such mutations, it will also fail to detect real bugs. Mutation analysis subsumes various coverage measures and provides a large and diverse set of faults that can be arbitrarily hard to trigger and detect, thus preventing the problems of saturation and overﬁtting. Unfortunately, the cost of traditional mutation analysis is exorbitant for fuzzing, as mutations need independent evaluation. In this paper, we apply modern mutation analysis techniques that pool multiple mutations; allowing us, for the ﬁrst time, to evaluate and compare fuzzers with mutation analysis. We introduce an evaluation bench for fuzzers and apply it to a number of popular fuzzers and subjects. In a comprehensive evaluation, we show how it allows us to assess fuzzer performance and measure the impact of improved techniques. While we ﬁnd that today’s fuzzers can detect only a small percentage of mutations, this should be seen as a challenge for future research—notably in improving (1) detecting failures beyond generic crashes; and (2) triggering mutations (and thus faults).
Characterizing Interplanetary Coronal Mass Ejection-related Forbush Decreases at Mercury Using MESSENGER Observations: Identification of a One- or Two-step Structure

The large-scale magnetic structure of interplanetary coronal mass ejections (ICMEs) has been shown to cause decreases in the galactic cosmic ray (GCR) flux measured in situ by spacecraft, known as Forbush decreases (Fds). We use measurements of the GCR count rate obtained by NASA’s MErcury Surface, Space ENvironment, GEochemistry, and Ranging (MESSENGER) spacecraft during its orbital phase around Mercury to identify such Fds related to the passage of ICMEs and characterize their structure. Of the 42 ICMEs with corresponding high-quality GCR data, 79% are associated with a Fd. Thus, a total of 33 ICME-related Fds were identified, 24 of which (73%) have a two-step structure. We use a superposed epoch analysis to build an average Fd profile at MESSENGER and find that despite the variability of individual events, a two-step structure is produced and is directly linked with the magnetic boundaries of the ICME. By using results from previous studies at Earth and Mars, we also address whether two-step Fds are more commonly observed closer to the Sun; we found that, although likely, this is not conclusive when comparing to the wide range of results of previous studies conducted at Earth. Finally, we find that the percentage decrease in GCR flux of the Fd is greater at MESSENGER on average than at Earth and Mars, decreasing with increasing heliocentric distance. The relationship between the percentage decrease and maximum hourly decrease is also in agreement with previous studies, and follows trends relating to the expansion of ICMEs as they propagate through the heliosphere.
PØDA: Prompt-driven Zero-shot Domain Adaptation

Domain adaptation has been vastly investigated in computer vision but still requires access to target images at train time, which might be intractable in some uncommon conditions. In this paper, we propose the task of ‘Prompt-driven Zero-shot Domain Adaptation’, where we adapt a model trained on a source domain using only a general description in natural language of the target domain, i.e., a prompt. First, we leverage a pretrained contrastive vision-language model (CLIP) to optimize affine transformations of source features, steering them towards the target text embedding while preserving their content and semantics. To achieve this, we propose Prompt-driven Instance Normalization (PIN). Second, we show that these prompt-driven augmentations can be used to perform zero-shot domain adaptation for semantic segmentation. Experiments demonstrate that our method significantly outperforms CLIP-based style transfer baselines on several datasets for the downstream task at hand, even surpassing one-shot unsupervised domain adaptation. A similar boost is observed on object detection and image classification. The code is available at https://github.com/astra-vision/PODA.
Analyses of braced excavation considering parameter uncertainties using a finite element code

This study presents a simplified procedure to quantify the effect of parameter uncertainty with consideration of soil spatial variability in analysis of braced excavations. A well-documented case history is adopted in conjunction with PLAXIS, a proprietary finite element code. The simplified approach based on first-order approximation is easy to use, requires less computational work and provides a practical tool to estimate the variation of the wall and ground responses caused by the uncertainty in input parameters. Moreover, to facilitate the numerical analysis, this study also documents a successful implementation of user-defined soil models, including the hyperbolic and the modified pseudoplasticity soil models, into PLAXIS. The results show that PLAXIS, with these user-defined soil models, can capture the laboratory stress–strain relationships and predict, accurately, the wall and ground responses in a braced excavation.
Ref-NPR: Reference-Based Non-Photorealistic Radiance Fields for Controllable Scene Stylization

Current 3D scene stylization methods transfer textures and colors as styles using arbitrary style references, lacking meaningful semantic correspondences. We introduce Reference-Based Non-Photorealistic Radiance Fields (Ref-NP R) to address this limitation. This controllable method stylizes a 3D scene using radiance fields with a single stylized 2D view as a reference. We propose a ray registration process based on the stylized reference view to obtain pseudo-ray supervision in novel views. Then we exploit semantic correspondences in content images to fill occluded regions with perceptually similar styles, resulting in non-photorealistic and continuous novel view sequences. Our experimental results demonstrate that Ref-NPR out-performs existing scene and video stylization methods regarding visual quality and semantic correspondence. The code and data are publicly available on the project page at https://ref-npr.github. io.
Semantic-aware Message Broadcasting for Efficient Unsupervised Domain Adaptation

Vision transformer has demonstrated great potential in abundant vision tasks. However, it also inevitably suffers from poor generalization capability when the distribution shift occurs in testing (i.e., out-of-distribution data). To mitigate this issue, we propose a novel method, Semantic-aware Message Broadcasting (SAMB), which enables more informative and flexible feature alignment for unsupervised domain adaptation (UDA). Particularly, we study the attention module in the vision transformer and notice that the alignment space using one global class token lacks enough flexibility, where it interacts information with all image tokens in the same manner but ignores the rich semantics of different regions. In this paper, we aim to improve the richness of the alignment features by enabling semantic-aware adaptive message broadcasting. Particularly, we introduce a group of learned group tokens as nodes to aggregate the global information from all image tokens, but encourage different group tokens to adaptively focus on the message broadcasting to different semantic regions. In this way, our message broadcasting encourages the group tokens to learn more informative and diverse information for effective domain alignment. Moreover, we systematically study the effects of adversarial-based feature alignment (ADA) and pseudo-label based self-training (PST) on UDA. We find that one simple two-stage training strategy with the cooperation of ADA and PST can further improve the adaptation capability of the vision transformer. Extensive experiments on DomainNet, OfficeHome, and VisDA-2017 demonstrate the effectiveness of our methods for UDA.
Absence of Ginzburg-Landau mechanism for vestigial order in the normal phase above a two-component superconductor

A two-component superconductor may hypothetically support a vestigial order phase above its superconducting transition temperature, with rotational or time-reversal symmetry spontaneously broken while remain non-superconducting. This has been suggested as an explanation to the observed normal state nematicity of the nematic superconductor $M_x$Bi$_2$Se$_3$. We examine the condition for this vestigial order to occur within Ginzburg-Landau theory with order parameter fluctuations, on both the nematic and chiral sides of the theory. Contrary to prior theoretical results, we rule out a large portion of parameter space for possible vestigial order. We argue that very extreme anisotropy is one prerequisite for a stable vestigial phase, which is likely not met in real materials.
Editing a Woman's Voice

Prior work shows that men and women speak with different levels of confidence, though it's often assumed that these differences are innate or are learned in early childhood. Using academic publishing as a setting, we find that language differences across male and female authors are initially negligible: in first drafts of academic manuscripts, men and women write with similar levels of uncertainty. However, when we trace those early drafts to their published versions, a substantial gender gap in linguistic uncertainty arises. That is, women increase their use of cautionary language through the publication process more than men. We show this increase in the linguistic gender gap varies substantially based on editor assignment. Specifically, our author-to-editor matched dataset allows us to estimate editor-specific fixed effects, capturing how specific editors impact the change in linguistic uncertainty for female authors relative to male authors (the editor's author-gender gap). Editors' author-gender gaps vary widely, and correlate with observable editor characteristics such as societal norms in their country-of-origin, their work history, and the year that they obtained their PhD. Overall, our study suggests that a woman's"voice"is partially shaped by external forces, and it highlights the critical role of editors in shaping how female academics communicate.
Acoustic source‐level measurements for a variety of merchant ships

This report presents a set of 50 source spectra obtained from merchant ships of opportunity near Genova, Italy. The source spectra were calculated from radiated‐noise spectra measured on a towed array together with a transmission‐loss spectrum computed from a parabolic equation model. The aggregate source spectra are characterized in terms of a mean source spectrum and three source‐level histograms computed for different frequency bands. It is shown that the mean spectrum is comparable in level and shape to a spectrum computed from a well‐known empirical model and that the source‐level histograms are approximately Gaussian with standard deviations of 5, 5.5, and 6.8 dB, respectively. Finally, a sub‐sample of 36 spectra is drawn from the aggregate and separated into three identifiable ship classes, namely, passenger/ferries, cargo ships, and tankers. It is seen that the source spectra for the three different ship classes have comparable means and standard deviations. From this result it is conjectured that both the mean spectrum and the source‐level histograms are not sensitive to ship class and can therefore be taken as representative of shipping in other regions with different ship class percentages.
Self-Supervised Audio-Visual Speech Representations Learning by Multimodal Self-Distillation

In this work, we present a novel method, named AV2vec, for learning audio-visual speech representations by multimodal self-distillation. AV2vec has a student and a teacher module, in which the student performs a masked latent feature regression task using the multimodal target features generated online by the teacher. The parameters of the teacher model are a momentum update of the student. Since our target features are generated online, AV2vec needs no iteration step like AV-HuBERT and the total training time cost is reduced to less than one-fifth. We further propose AV2vec-MLM in this study, which augments AV2vec with a masked language model (MLM)-style loss using multitask learning. Our experimental results show that AV2vec achieved comparable performance to the AV-HuBERT baseline. When combined with an MLM-style loss, AV2vec-MLM outperformed baselines and achieved the best performance on the downstream tasks.
Integration of a systolic array based hardware accelerator into a DNN operator auto-tuning framework

The deployment of neural networks on heterogeneous SoCs coupled with custom accelerators is a challenging task because of the lack of end-to-end software tools provided for these systems. Moreover, the already available low level schedules and mapping strategies provided by the accelerator developers for typical tensor operations are not necessarily the best possible ones for each particular use case. This is why frameworks which automatically test the performance of the generated code on a specific hardware configuration are of special interest. In this work, the integration between the code generation framework TVM and the systolic array-based accelerator Gemmini is presented. A generic schedule to offload the GEneral Matrix Multiply (GEMM) tensor operation onto Gemmini is detailed, and its suitability is tested by executing the AutoTVM tuning process on it. Our generated code achieves a peak throughput of 46 giga-operations per second (GOPs) under a 100 MHz clock on a Xilinx ZCU102 FPGA, outperforming previous work. Furthermore, the code generated by this integration was able to surpass the default hand-tuned schedules provided by the Gemmini developers in real-world workloads.
Ancestry-specific analyses of genome-wide data confirm the settlement sequence of Polynesia

: By demonstrating the role that historical population replacements and waves of admixture have played around the world, the prior work of Reich and colleagues has provided a paradigm for understanding human history [Reich et al. 2009; Reich et al. 2012; Patterson et al. 2012]. Although we show in Ioannidis et al. [2021] that the peopling of Polynesia was a range expansion, and not, as suggested by Huang et al. [2022], yet another example of waves of admixture and large-scale gene flow between populations, we believe that our result in this recently settled oceanic expanse is the exception that proves the rule. In
Spatial evolution of supersonic streamwise vortices

In this paper, the spatial developments of high Mach number streamwise vortices have been investigated using direct numerical simulations for Mach numbers from 2.5 to 9.0. The particular streamwise vortices studied here are those with uniform stagnation temperatures and those with constant entropy. For vortices with uniform stagnation temperatures, the calculated spatial evolutions show that for small swirl parameters, a bending wave mode is excited and spiral structures of low-wavenumber modes only develop downstream. In contrast, for swirl parameters larger than a moderate amount (approximately 0.5), relaminarization occurs downstream. This study also found that the relation between the circulation and the freestream Mach number contributes to a necessary condition as to whether perturbations can grow in supersonic flows. For isentropic streamwise vortices, instability properties at high Mach numbers parallel those in low-speed flows because the evolution can give rise to high-wavenumbers within the cor.
Retrievability of information in quantum and realistic hidden variable theories

We propose a generalisation of the Leggett-Garg conditions for macrorealistic behaviour. Our proposal relies on relaxing the postulate of non-invasive measurability with that of retrievability of information. This leads to a strictly broader class of hidden variable theories than those having a macrorealistic description. Crucially, whereas quantum mechanical tests of macrorealism require one to optimise over all possible state updates, for retrievability of information it suffices to use the basic L\"uders state update, which is present in every quantum measurement. We show that in qubit systems the optimal retrieving protocols further relate to the fundamental precision limit of quantum theory given by Busch-Lahti-Werner error-disturbance uncertainty relations. We implement an optimal protocol using a photonic setting, and report an experimental violation of the proposed generalisation of macrorealism.
Gravity from thermodynamics: Optimal transport and negative effective dimensions

We prove an equivalence between the classical equations of motion governing vacuum gravity compactifications (and more general warped-product spacetimes) and a concavity property of entropy under time evolution. This is obtained by linking the theory of optimal transport to the Raychaudhuri equation in the internal space, where the warp factor introduces effective notions of curvature and (negative) internal dimension. When the Reduced Energy Condition is satisfied, concavity can be characterized in terms of the cosmological constant \LambdaΛ; as a consequence, the masses of the spin-two Kaluza-Klein fields obey bounds in terms of \LambdaΛ alone. We show that some Cheeger bounds on the KK spectrum hold even without assuming synthetic Ricci lower bounds, in the large class of infinitesimally Hilbertian metric measure spaces, which includes D-brane and O-plane singularities. As an application, we show how some approximate string theory solutions in the literature achieve scale separation, and we construct a new explicit parametrically scale-separated AdS solution of M-theory supported by Casimir energy.