Miracle-dz/UniFAR-FTUs huggingface dataset:
Video Diffusion Models

Generating temporally coherent high fidelity video is an important milestone in generative modeling research. We make progress towards this milestone by proposing a diffusion model for video generation that shows very promising initial results. Our model is a natural extension of the standard image diffusion architecture, and it enables jointly training from image and video data, which we find to reduce the variance of minibatch gradients and speed up optimization. To generate long and higher resolution videos we introduce a new conditional sampling technique for spatial and temporal video extension that performs better than previously proposed methods. We present the first results on a large text-conditioned video generation task, as well as state-of-the-art results on established benchmarks for video prediction and unconditional video generation. Supplementary material is available at https://video-diffusion.github. io/
DaViT: Dual Attention Vision Transformers

In this work, we introduce Dual Attention Vision Transformers (DaViT), a simple yet effective vision transformer architecture that is able to capture global context while maintaining computational efficiency. We propose approaching the problem from an orthogonal angle: exploiting self-attention mechanisms with both"spatial tokens"and"channel tokens". With spatial tokens, the spatial dimension defines the token scope, and the channel dimension defines the token feature dimension. With channel tokens, we have the inverse: the channel dimension defines the token scope, and the spatial dimension defines the token feature dimension. We further group tokens along the sequence direction for both spatial and channel tokens to maintain the linear complexity of the entire model. We show that these two self-attentions complement each other: (i) since each channel token contains an abstract representation of the entire image, the channel attention naturally captures global interactions and representations by taking all spatial positions into account when computing attention scores between channels; (ii) the spatial attention refines the local representations by performing fine-grained interactions across spatial locations, which in turn helps the global information modeling in channel attention. Extensive experiments show our DaViT achieves state-of-the-art performance on four different tasks with efficient computations. Without extra data, DaViT-Tiny, DaViT-Small, and DaViT-Base achieve 82.8%, 84.2%, and 84.6% top-1 accuracy on ImageNet-1K with 28.3M, 49.7M, and 87.9M parameters, respectively. When we further scale up DaViT with 1.5B weakly supervised image and text pairs, DaViT-Gaint reaches 90.4% top-1 accuracy on ImageNet-1K. Code is available at https://github.com/dingmyu/davit.
Zero-Shot Category-Level Object Pose Estimation

Object pose estimation is an important component of most vision pipelines for embodied agents, as well as in 3D vision more generally. In this paper we tackle the problem of estimating the pose of novel object categories in a zero-shot manner. This extends much of the existing literature by removing the need for pose-labelled datasets or category-specific CAD models for training or inference. Specifically, we make the following contributions. First, we formalise the zero-shot, category-level pose estimation problem and frame it in a way that is most applicable to real-world embodied agents. Secondly, we propose a novel method based on semantic correspondences from a self-supervised vision transformer to solve the pose estimation problem. We further re-purpose the recent CO3D dataset to present a controlled and realistic test setting. Finally, we demonstrate that all baselines for our proposed task perform poorly, and show that our method provides a six-fold improvement in average rotation accuracy at 30 degrees. Our code is available at https://github.com/applied-ai-lab/zero-shot-pose.
SOMOS: The Samsung Open MOS Dataset for the Evaluation of Neural Text-to-Speech Synthesis

In this work, we present the SOMOS dataset, the ﬁrst large-scale mean opinion scores (MOS) dataset consisting of solely neural text-to-speech (TTS) samples. It can be employed to train automatic MOS prediction systems focused on the assessment of modern synthesizers, and can stimulate advancements in acoustic model evaluation. It consists of 20K synthetic utterances of the LJ Speech voice, a public domain speech dataset which is a common benchmark for building neural acoustic models and vocoders. Utterances are generated from 200 TTS systems including vanilla neural acoustic models as well as models which allow prosodic variations. An LPCNet vocoder is used for all systems, so that the samples’ variation depends only on the acoustic models. The synthesized utterances provide bal-anced and adequate domain and length coverage. We collect MOS naturalness evaluations on 3 English Amazon Mechanical Turk locales and share practices leading to reliable crowdsourced annotations for this task. Baseline results of state-of-the-art MOS prediction models on the SOMOS dataset are presented, while we show the challenges that such models face when assigned to evaluate synthetic utterances.
A sub-Riemannian Gauss-Bonnet theorem for surfaces in contact manifolds

We obtain a sub-Riemannian version of the classical Gauss-Bonnet theorem. We consider subsurfaces of a three dimensional contact sub-Riemannian manifolds, and using a family of taming Riemannian metric, we obtain a pure sub-Riemannian result in the limit. In particular, we are able to recover topological information of the surface from the geometry around the characteristic set, i.e., the points where the tangent space to the surface and contact structure coincide. We both give a version for surfaces without boundary and surfaces with boundary.
Mapping the Multilingual Margins: Intersectional Biases of Sentiment Analysis Systems in English, Spanish, and Arabic

As natural language processing systems become more widespread, it is necessary to address fairness issues in their implementation and deployment to ensure that their negative impacts on society are understood and minimized. However, there is limited work that studies fairness using a multilingual and intersectional framework or on downstream tasks. In this paper, we introduce four multilingual Equity Evaluation Corpora, supplementary test sets designed to measure social biases, and a novel statistical framework for studying unisectional and intersectional social biases in natural language processing. We use these tools to measure gender, racial, ethnic, and intersectional social biases across five models trained on emotion regression tasks in English, Spanish, and Arabic. We find that many systems demonstrate statistically significant unisectional and intersectional social biases. We make our code and datasets available for download.
Bayesian vector autoregressive analysis of macroeconomic and transport influences on urban traffic accidents

The macro inﬂuencing factors analysis of urban traﬃc safety is important to guide the direc- tion of urban development to reduce the frequency of traﬃc accidents. In this study, a Bayesian vector autoregressive(BVAR) model was developed to exploring the impact of six macro-level economic and transport factors, including population, GDP, private vehicle ownership, bus ownership, subway rail mileage and road average speed on traﬃc accidents with the small sample size transport annual report data in Beijing. The results show that the BVAR model was suitable for time series analysis of traﬃc accidents in small sample situations. In macroeconomic factors, GDP growth was considered to reduce the number of traﬃc accidents in the long term, while population growth had a positive eﬀect on traﬃc accidents in the short term. With the respect to macro-transport factors, road average speed and private vehicle ownership was perceived to increase traﬃc accidents in long duration, whereas bus ownership and subway rail mileage had long-term negative eﬀects, with the greatest positive eﬀect for road average speed and the greatest negative eﬀect for subway rail mileage. This study suggests that government departments can reduce the number of traﬃc accidents by increasing investment in public transportation in-frastructures, limiting private vehicles and road speed.
Betelgeuse constraints on coupling between axionlike particles and electrons

Axion-like particles (ALPs) can be produced by thermal processes in a stellar interior, escape from the star and, if suﬃciently light, be converted into photons in the external Galactic magnetic ﬁeld. Such a process could produce a detectable hard X-ray excess in the direction of the star. In this scenario, a promising class of targets is the red supergiants, massive stars which are experiencing the late part of their evolution. We report on a search for ALP-induced X-ray emission from Betelgeuse, produced via the combined processes of Bremsstrahlung, Compton and Primakoﬀ. Using a 50 ks observation of Betelgeuse by the NuSTAR satellite telescope, we set 95% C.L. upper limits on the ALP-electron ( g ae ) and ALP-photon ( g aγ ) couplings. For masses m a ≤ (3 . 5 − 5 . 5) × 10 − 11 eV, we ﬁnd g aγ × g ae < (0 . 4 − 2 . 8) × 10 − 24 GeV − 1 (depending on the stellar model and assuming a value of the regular Galactic magnetic ﬁeld in the direction transverse to Betelgeuse of B T =1.4 µ G). This corresponds to g ae < (0 . 4 − 2 . 8) × 10 − 12 for g aγ > 1 . 0 × 10 − 12 GeV − 1 . This analysis supercedes by over an order of magnitude the limit on g ae × g aγ placed by the CAST solar axion experiment and is among the strongest constraints on these couplings.
Graphene oxide 2D films integrated with nanowires and ring resonators for enhanced nonlinear optics

We report enhanced nonlinear optics in nanowires, waveguides, and ring resonators by introducing layered two-dimensional (2D) graphene oxide (GO) films through experimental demonstration. The GO films are integrated on silicon-on-insulator nanowires (SOI), high index doped silica glass, and silicon nitride (SiN) waveguides and microring resonators (MRRs), to demonstrate an improved optical nonlinearity including Kerr nonlinearity and four-wave mixing (FWM). By using a large-area, transfer-free, layer-by-layer GO coating method with photolithography and lift-off processes, we integrate GO films on these complementary metal-oxide-semiconductor (CMOS)-compatible devices. For SOI nanowires, significant spectral broadening of optical pulses in GO-coated SOI nanowires induced by self-phase modulation (SPM) is observed, achieving a high spectral broadening factor of 4.34 for a device with a patterned film including 10 layers of GO. A significant enhancement in the nonlinear figure of merit (FOM) for silicon nanowires by a factor of 20 is also achieved, resulting in a FOM > 5. For Hydex and SiN waveguides, enhanced FWM in the GO-coated waveguides is achieved, where conversion efficiency (CE) enhancements of up to 6.9 dB and 9.1 dB relative to the uncoated waveguides. For MRRs, an increase of up to ~10.3 dB in the FWM CE is achieved due to the resonant enhancement effect. These results reveal the strong potential of GO films to improve the nonlinear optics of nanowires, waveguides, and ring resonators.
Output-sensitive ERM-based techniques for data-driven algorithm design

Data-driven algorithm design is a promising, learning-based approach for beyond worst-case analysis of algorithms with tunable parameters. An important open problem is the design of computationally efficient data-driven algorithms for combinatorial algorithm families with multiple parameters. As one fixes the problem instance and varies the parameters, the"dual"loss function typically has a piecewise-decomposable structure, i.e. is well-behaved except at certain sharp transition boundaries. In this work we initiate the study of techniques to develop efficient ERM learning algorithms for data-driven algorithm design by enumerating the pieces of the sum dual loss functions for a collection of problem instances. The running time of our approach scales with the actual number of pieces that appear as opposed to worst case upper bounds on the number of pieces. Our approach involves two novel ingredients -- an output-sensitive algorithm for enumerating polytopes induced by a set of hyperplanes using tools from computational geometry, and an execution graph which compactly represents all the states the algorithm could attain for all possible parameter values. We illustrate our techniques by giving algorithms for pricing problems, linkage-based clustering and dynamic-programming based sequence alignment.
Event Transformer. A sparse-aware solution for efficient event data processing

Event cameras are sensors of great interest for many applications that run in low-resource and challenging environments. They log sparse illumination changes with high temporal resolution and high dynamic range, while they present minimal power consumption. However, top-performing methods often ignore specific event-data properties, leading to the development of generic but computationally expensive algorithms. Efforts toward efficient solutions usually do not achieve top-accuracy results for complex tasks. This work proposes a novel framework, Event Transformer (EvT)1, that effectively takes advantage of event-data properties to be highly efficient and accurate. We introduce a new patch-based event representation and a compact transformer-like architecture to process it. EvT is evaluated on different event-based benchmarks for action and gesture recognition. Evaluation results show better or comparable accuracy to the state-of-the-art while requiring significantly less computation resources, which makes EvT able to work with minimal latency both on GPU and CPU.
Birth of the Galactic Disk Revealed by the H3 Survey

We use chemistry ([ α /Fe] and [Fe/H]), main sequence turnoff ages, and kinematics determined from H3 Survey spectroscopy and Gaia astrometry to identify the birth of the Galactic disk. We separate in-situ and accreted stars on the basis of angular momenta and eccentricities. The sequence of high − α in-situ stars persists down to at least [Fe/H] ≈ − 2 . 5 and shows unexpected non-monotonic behavior: with increasing metallicity the population ﬁrst declines in [ α /Fe], then increases over the range − 1 . 3 (cid:46) [Fe/H] (cid:46) − 0 . 7, and then declines again at higher metallicities. The number of stars in the in-situ population rapidly increases above [Fe/H] ≈ − 1. The average kinematics of these stars are hot and independent of metallicity at [Fe/H] (cid:46) − 1 and then become increasingly cold and disk-like at higher metallicities. The ages of the in-situ, high − α stars are uniformly very old ( ≈ 13 Gyr) at [Fe/H] (cid:46) − 1 . 3, and span a wider range (8 − 12 Gyr) at higher metallicities. Interpreting the chemistry with a simple chemical evolution model suggests that the non-monotonic behavior is due to a signiﬁcant increase in star formation efﬁciency, which began ≈ 13 Gyr ago. These results support a picture in which the ﬁrst ≈ 1 Gyr of the Galaxy was characterized by a “simmering phase” in which the star formation efﬁciency was low and the kinematics had substantial disorder with some net rotation. The disk then underwent a dramatic transformation to a “boiling phase”, in which the star formation efﬁciency increased substantially, the kinematics became disk-like, and the number of stars formed increased tenfold. We interpret this transformation as the birth of the Galactic disk at z ≈ 4. The physical origin of this transformation is unclear and does not seem to be reproduced in current galaxy formation models.
Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformer

Videos are created to express emotion, exchange information, and share experiences. Video synthesis has intrigued researchers for a long time. Despite the rapid progress driven by advances in visual synthesis, most existing studies focus on improving the frames' quality and the transitions between them, while little progress has been made in generating longer videos. In this paper, we present a method that builds on 3D-VQGAN and transformers to generate videos with thousands of frames. Our evaluation shows that our model trained on 16-frame video clips from standard benchmarks such as UCF-101, Sky Time-lapse, and Taichi-HD datasets can generate diverse, coherent, and high-quality long videos. We also showcase conditional extensions of our approach for generating meaningful long videos by incorporating temporal information with text and audio. Videos and code can be found at https://songweige.github. io/projects/tats/index.html.
BERTuit: Understanding Spanish language in Twitter through a native transformer

The appearance of complex attention-based language models such as BERT, Roberta or GPT-3 has allowed to address highly complex tasks in a plethora of scenarios. However, when applied to speciﬁc domains, these models encounter considerable difﬁculties. This is the case of Social Networks such as Twitter, an ever-changing stream of information written with informal and complex language, where each message requires careful evaluation to be understood even by humans given the important role that context plays. Addressing tasks in this domain through Natural Language Processing involves severe challenges. When powerful state-of-the-art multilingual language models are applied to this scenario, language speciﬁc nuances use to get lost in translation. To face these challenges we present BERTuit , the larger transformer proposed so far for Spanish language, pre-trained on a massive dataset of 230M Spanish tweets using RoBERTa optimization. Our motivation is to provide a powerful resource to better understand Spanish Twitter and to be used on applications focused on this social network, with special emphasis on solutions devoted to tackle the spreading of misinformation in this platform. BERTuit is evaluated on several tasks and compared against M-BERT, XLM-RoBERTa and XLM-T, very competitive multilingual transformers. The utility of our approach is shown with applications, in this case: a zero-shot methodology to visualize groups of hoaxes and proﬁling authors spreading disinformation. Misinformation spreads wildly on platforms such as Twitter in languages other than English, meaning performance of transformers may suffer when transferred outside English speaking communities.
Pretraining Text Encoders with Adversarial Mixture of Training Signal Generators

We present a new framework AMOS that pretrains text encoders with an Adversarial learning curriculum via a Mixture Of Signals from multiple auxiliary generators. Following ELECTRA-style pretraining, the main encoder is trained as a discriminator to detect replaced tokens generated by auxiliary masked language models (MLMs). Different from ELECTRA which trains one MLM as the generator, we jointly train multiple MLMs of different sizes to provide training signals at various levels of difficulty. To push the discriminator to learn better with challenging replaced tokens, we learn mixture weights over the auxiliary MLMs' outputs to maximize the discriminator loss by backpropagating the gradient from the discriminator via Gumbel-Softmax. For better pretraining efficiency, we propose a way to assemble multiple MLMs into one unified auxiliary model. AMOS outperforms ELECTRA and recent state-of-the-art pretrained models by about 1 point on the GLUE benchmark for BERT base-sized models.
Enhancing Semantic Code Search with Multimodal Contrastive Learning and Soft Data Augmentation

Code search aims to retrieve the most semantically relevant code snippet for a given natural language query. Recently, large-scale code pre-trained models such as CodeBERT and GraphCodeBERT learn generic representations of source code and have achieved substantial improvement on code search task. However, the high-quality sequence-level representations of code snippets have not been sufficiently explored. In this paper, we propose a new approach with multimodal contrastive learning and soft data augmentation for code search. Multimodal contrastive learning is used to pull together the representations of code-query pairs and push apart the unpaired code snippets and queries. Moreover, data augmentation is critical in contrastive learning for learning high-quality representations. However, only semantic-preserving augmentations for source code are considered in existing work. In this work, we propose to do soft data augmentation by dynamically masking and replacing some tokens in code sequences to generate code snippets that are similar but not necessarily semantic-preserving as positive samples for paired queries. We conduct extensive experiments to evaluate the effectiveness of our approach on a large-scale dataset with six programming languages. The experimental results show that our approach significantly outperforms the state-of-the-art methods. We also adapt our techniques to several pre-trained models such as RoBERTa and CodeBERT, and significantly boost their performance on the code search task.
The Eel Pulsar Wind Nebula: A PeVatron-candidate Origin for HAWC J1826−128 and HESS J1826−130

HAWC J1826−128 is one of the brightest Galactic TeV γ-ray sources detected by the High Altitude Water Cherenkov (HAWC) observatory, with photon energies extending up to nearly ∼100 TeV. This HAWC source spatially coincides with the H.E. S.S. TeV source HESS J1826−130 and the “Eel” pulsar wind nebula (PWN), which is associated with the GeV pulsar PSR J1826−1256. In the X-ray band, Chandra and XMM-Newton revealed that the Eel PWN is composed of both a compact nebula (∼15″) and diffuse X-ray emission (∼6′ × 2′) extending away from the pulsar. Our NuSTAR observation detected hard X-ray emission from the compact PWN up to ∼20 keV and evidence of the synchrotron burn-off effect. In addition to the spatial coincidence between HESS J1826−130 and the diffuse X-ray PWN, our multiwavelength spectral energy distribution (SED) analysis using X-ray and γ-ray data establishes a leptonic origin of the TeV emission associated with the Eel PWN. Furthermore, our evolutionary PWN SED model suggests (1) a low PWN B-field of ∼1 μG, (2) a significantly younger pulsar age (t ∼ 5.7 kyr) than the characteristic age (τ = 14.4 kyr), and (3) a maximum electron energy of Emax=2 PeV. The low B-field, as well as the putative supersonic motion of the pulsar, may account for the asymmetric morphology of the diffuse X-ray emission. Our results suggest that the Eel PWN may be a leptonic PeVatron particle accelerator powered by the ∼6 kyr old pulsar PSR J1826−1256 with a spin-down power of 3.6 × 1036 erg s−1.