Miracle-dz/UniFAR-FTUs huggingface dataset:
Hierarchical DCC-HEAVY Model for High-Dimensional Covariance Matrices

We introduce a new HD DCC-HEAVY class of hierarchical-type factor models for conditional covariance matrices of high-dimensional returns, employing the corresponding realized measures built from higher-frequency data. The modelling approach features sophisticated asymmetric dynamics in covariances coupled with straightforward estimation and forecasting schemes, independent of the cross-sectional dimension of the assets under consideration. Empirical analyses suggest the HD DCC-HEAVY models have a better in-sample fit, and deliver statistically and economically significant out-of-sample gains relative to the standard benchmarks and existing hierarchical factor models. The results are robust under different market conditions.
CLIP-VG: Self-paced Curriculum Adapting of CLIP via Exploiting Pseudo-Language Labels for Visual Grounding

Visual Grounding (VG) refers to locating a region described by expressions in a speciﬁc image, which is a critical topic in vision-language ﬁelds. To alleviate the dependence on labeled data, existing unsupervised methods try to locate regions using task-unrelated pseudo-labels. However, a large proportion of pseudo-labels are noisy and diversity scarcity in language taxonomy. Inspired by the advances in V-L pretraining, we consider utilizing the VLP models to realize unsupervised transfer learning in down-stream grounding task. Thus, we propose CLIP-VG, a novel method that can conduct self-paced curriculum adapting of CLIP via exploiting pseudo-language labels to solve VG problem. By elaborating an efﬁcient model structure, we ﬁrst propose a single-source and multi-source curriculum adapting method for unsupervised VG to progressively sample more reliable cross-modal pseudo-labels to obtain the optimal model, thus achieving implicit knowledge exploiting and denoising. Our method outperforms the existing state-of-the-art unsupervised VG method Pseudo-Q in both single-source and multi-source scenarios with a large margin, i.e., 6
GSB: Group Superposition Binarization for Vision Transformer with Limited Training Samples

Vision Transformer (ViT) has performed remarkably in various computer vision tasks. Nonetheless, affected by the massive amount of parameters, ViT usually suffers from serious overfitting problems with a relatively limited number of training samples. In addition, ViT generally demands heavy computing resources, which limit its deployment on resource-constrained devices. As a type of model-compression method, model binarization is potentially a good choice to solve the above problems. Compared with the full-precision one, the model with the binarization method replaces complex tensor multiplication with simple bit-wise binary operations and represents full-precision model parameters and activations with only 1-bit ones, which potentially solves the problem of model size and computational complexity, respectively. In this paper, we investigate a binarized ViT model. Empirically, we observe that the existing binarization technology designed for Convolutional Neural Networks (CNN) cannot migrate well to a ViT's binarization task. We also find that the decline of the accuracy of the binary ViT model is mainly due to the information loss of the Attention module and the Value vector. Therefore, we propose a novel model binarization technique, called Group Superposition Binarization (GSB), to deal with these issues. Furthermore, in order to further improve the performance of the binarization model, we have investigated the gradient calculation procedure in the binarization process and derived more proper gradient calculation equations for GSB to reduce the influence of gradient mismatch. Then, the knowledge distillation technique is introduced to alleviate the performance degradation caused by model binarization. Analytically, model binarization can limit the parameter's search space during parameter updates while training a model. Therefore, the binarization process can actually play an implicit regularization role and help solve the problem of overfitting in the case of insufficient training data. Experiments on three datasets with limited numbers of training samples demonstrate that the proposed GSB model achieves state-of-the-art performance among the binary quantization schemes and exceeds its full-precision counterpart on some indicators. Code and models are available at: https://github.com/IMRL/GSB-Vision-Transformer.
Model Predictive Control with Reach-avoid Analysis

In this paper we investigate the optimal controller synthesis problem, so that the system under the controller can reach a specified target set while satisfying given constraints. Existing model predictive control (MPC) methods learn from a set of discrete states visited by previous (sub-)optimized trajectories and thus result in computationally expensive mixed-integer nonlinear optimization. In this paper a novel MPC method is proposed based on reach-avoid analysis to solve the controller synthesis problem iteratively. The reach-avoid analysis is concerned with computing a reach-avoid set which is a set of initial states such that the system can reach the target set successfully. It not only provides terminal constraints, which ensure feasibility of MPC, but also expands discrete states in existing methods into a continuous set (i.e., reach-avoid sets) and thus leads to nonlinear optimization which is more computationally tractable online due to the absence of integer variables. Finally, we evaluate the proposed method and make comparisons with state-of-the-art ones based on several examples.
Generative Adversarial Networks for Spatio-Spectral Compression of Hyperspectral Images

The development of deep learning-based models for the compression of hyperspectral images (HSIs) has recently attracted great attention in remote sensing due to the sharp growing of hyperspectral data archives. Most of the existing models achieve either spectral or spatial compression, and do not jointly consider the spatio-spectral redundancies present in HSIs. To address this problem, in this paper we focus our attention on the High Fidelity Compression (HiFiC) model (which is proven to be highly effective for spatial compression problems) and adapt it to perform spatio-spectral compression of HSIs. In detail, we introduce two new models: i) HiFiC using Squeeze and Excitation (SE) blocks (denoted as HiFiC$_{SE}$); and ii) HiFiC with 3D convolutions (denoted as HiFiC$_{3D}$) in the framework of compression of HSIs. We analyze the effectiveness of HiFiC$_{SE}$ and HiFiC$_{3D}$ in compressing the spatio-spectral redundancies with channel attention and inter-dependency analysis. Experimental results show the efficacy of the proposed models in performing spatio-spectral compression, while reconstructing images at reduced bitrates with higher reconstruction quality. The code of the proposed models is publicly available at https://git.tu-berlin.de/rsim/HSI-SSC .
An Ising Model of Bursts and Fizzles in Intergroup Violence

During intergroup confrontations, agitating stimuli such as opponents' threats and provocations can trigger collective violence. We examine video recordings of street fights between groups of young men. Collective violence sometimes breaks out in a burst and at other times in a fizzle wherein only few group members participate. These two patterns are accurately predicted by an Ising model with asymmetric spins.
Trillion Dollar Words: A New Financial Dataset, Task & Market Analysis

Monetary policy pronouncements by Federal Open Market Committee (FOMC) are a major driver of financial market returns. We construct the largest tokenized and annotated dataset of FOMC speeches, meeting minutes, and press conference transcripts in order to understand how monetary policy influences financial markets. In this study, we develop a novel task of hawkish-dovish classification and benchmark various pre-trained language models on the proposed dataset. Using the best-performing model (RoBERTa-large), we construct a measure of monetary policy stance for the FOMC document release days. To evaluate the constructed measure, we study its impact on the treasury market, stock market, and macroeconomic indicators. Our dataset, models, and code are publicly available on Huggingface and GitHub under CC BY-NC 4.0 license.
Frequency-aware Dimension Selection for Static Word Embedding by Mixed Product Distance

Static word embedding is still useful, particularly for context-unavailable tasks, because in the case of no context available, pre-trained language models often perform worse than static word embeddings. Although dimension is a key factor determining the quality of static word embeddings, automatic dimension selection is rarely discussed. In this paper, we investigate the impact of word frequency on the dimension selection, and empirically find that word frequency is so vital that it needs to be taken into account during dimension selection. Based on such an empirical finding, this paper proposes a dimension selection method that uses a metric (Mixed Product Distance, MPD) to select a proper dimension for word embedding algorithms without training any word embedding. Through applying a post-processing function to oracle matrices, the MPD-based method can de-emphasize the impact of word frequency. Experiments on both context-unavailable and context-available tasks demonstrate the better efficiency-performance trade-off of our MPD-based dimension selection method over baselines.
Learning Better Contrastive View from Radiologist's Gaze

Recent self-supervised contrastive learning methods greatly benefit from the Siamese structure that aims to minimizing distances between positive pairs. These methods usually apply random data augmentation to input images, expecting the augmented views of the same images to be similar and positively paired. However, random augmentation may overlook image semantic information and degrade the quality of augmented views in contrastive learning. This issue becomes more challenging in medical images since the abnormalities related to diseases can be tiny, and are easy to be corrupted (e.g., being cropped out) in the current scheme of random augmentation. In this work, we first demonstrate that, for widely-used X-ray images, the conventional augmentation prevalent in contrastive pre-training can affect the performance of the downstream diagnosis or classification tasks. Then, we propose a novel augmentation method, i.e., FocusContrast, to learn from radiologists' gaze in diagnosis and generate contrastive views for medical images with guidance from radiologists' visual attention. Specifically, we track the gaze movement of radiologists and model their visual attention when reading to diagnose X-ray images. The learned model can predict visual attention of the radiologists given a new input image, and further guide the attention-aware augmentation that hardly neglects the disease-related abnormalities. As a plug-and-play and framework-agnostic module, FocusContrast consistently improves state-of-the-art contrastive learning methods of SimCLR, MoCo, and BYOL by 4.0~7.0% in classification accuracy on a knee X-ray dataset.
Stability of liquid film coating a horizontal cylinder: interplay of capillary and gravity forces

We study the drainage of a viscous liquid film coating outside a horizontal cylinder. We first study the evolution of the axially invariant draining flow, initiated at rest with uniform film thickness $\delta$. Non-linear simulations indicate that for each $\delta$, there is a threshold in the Bond number ($Bo$), which compares the gravitational effects with surface tension, above which the draining liquid bulk ruptures. This critical $Bo$ is found to scale inversely with $\delta$, defining the existence of a quasi-stationary pendant curtain sustained below the cylinder by surface tension. The interface of the pendant curtain is unconditionally linearly unstable and is prone to Rayleigh-Plateau-like, capillarity-driven, and Rayleigh-Taylor, gravity-driven, instabilities. The linear stability of the quasi-static state along with an energy analysis of the unstable mode illustrates that while the Rayleigh-Taylor instability is always present, capillary effects dominate the instability at small $Bo$, which promotes the formation of pearls enveloping the cylinder. In contrast, at large $Bo$, capillarity acts in a stabilising way and the instability is purely gravity-driven, forming underside modulations. We present the asymptotic energy repartition representing the different physical mechanisms at play in the instability of the saturated curtains for a wide range of $\{Bo,\delta\}$. The results of the linear analysis agree with the pre-existing experiments of~\citeauthor{Bruyn1997} and non-linear simulations of~\citeauthor{Weidner1997} on thin film and extend the results for thick films. Additionally, based on the volume made available for droplet growth by the development of the most linearly amplified wavelength, we build a tentative regime diagram predicting the final patterns emerging from the pendant curtain: an array of pearls or pendant drops or three-dimensional droplet pinch-off.
Efficient Semiparametric Estimation of Average Treatment Effects Under Covariate Adaptive Randomization

Experiments that use covariate adaptive randomization (CAR) are commonplace in applied economics and other fields. In such experiments, the experimenter first stratifies the sample according to observed baseline covariates and then assigns treatment randomly within these strata so as to achieve balance according to pre-specified stratum-specific target assignment proportions. In this paper, we compute the semiparametric efficiency bound for estimating the average treatment effect (ATE) in such experiments with binary treatments allowing for the class of CAR procedures considered in Bugni, Canay, and Shaikh (2018, 2019). This is a broad class of procedures and is motivated by those used in practice. The stratum-specific target proportions play the role of the propensity score conditional on all baseline covariates (and not just the strata) in these experiments. Thus, the efficiency bound is a special case of the bound in Hahn (1998), but conditional on all baseline covariates. Additionally, this efficiency bound is shown to be achievable under the same conditions as those used to derive the bound by using a cross-fitted Nadaraya-Watson kernel estimator to form nonparametric regression adjustments.
Understanding and Bridging the Modality Gap for Speech Translation

How to achieve better end-to-end speech translation (ST) by leveraging (text) machine translation (MT) data? Among various existing techniques, multi-task learning is one of the effective ways to share knowledge between ST and MT in which additional MT data can help to learn source-to-target mapping. However, due to the differences between speech and text, there is always a gap between ST and MT. In this paper, we first aim to understand this modality gap from the target-side representation differences, and link the modality gap to another well-known problem in neural machine translation: exposure bias. We find that the modality gap is relatively small during training except for some difficult cases, but keeps increasing during inference due to the cascading effect. To address these problems, we propose the Cross-modal Regularization with Scheduled Sampling (Cress) method. Specifically, we regularize the output predictions of ST and MT, whose target-side contexts are derived by sampling between ground truth words and self-generated words with a varying probability. Furthermore, we introduce token-level adaptive training which assigns different training weights to target tokens to handle difficult cases with large modality gaps. Experiments and analysis show that our approach effectively bridges the modality gap, and achieves significant improvements over a strong baseline in all eight directions of the MuST-C dataset.
A Language Model of Java Methods with Train/Test Deduplication

This tool demonstration presents a research toolkit for a language model of Java source code. The target audience includes researchers studying problems at the granularity level of subroutines, statements, or variables in Java. In contrast to many existing language models, we prioritize features for researchers including an open and easily-searchable training set, a held out test set with different levels of deduplication from the training set, infrastructure for deduplicating new examples, and an implementation platform suitable for execution on equipment accessible to a relatively modest budget. Our model is a GPT2-like architecture with 350m parameters. Our training set includes 52m Java methods (9b tokens) and 13m StackOverflow threads (10.5b tokens). To improve accessibility of research to more members of the community, we limit local resource requirements to GPUs with 16GB video memory. We provide a test set of held out Java methods that include descriptive comments, including the entire Java projects for those methods. We also provide deduplication tools using precomputed hash tables at various similarity thresholds to help researchers ensure that their own test examples are not in the training set. We make all our tools and data open source and available via Huggingface and Github.
Federated TD Learning Over Finite-Rate Erasure Channels: Linear Speedup Under Markovian Sampling

Federated learning (FL) has recently gained much attention due to its effectiveness in speeding up supervised learning tasks under communication and privacy constraints. However, whether similar speedups can be established for reinforcement learning remains much less understood theoretically. Towards this direction, we study a federated policy evaluation problem where agents communicate via a central aggregator to expedite the evaluation of a common policy. To capture typical communication constraints in FL, we consider finite capacity up-link channels that can drop packets based on a Bernoulli erasure model. Given this setting, we propose and analyze QFedTD - a quantized federated temporal difference learning algorithm with linear function approximation. Our main technical contribution is to provide a finite-sample analysis of QFedTD that (i) highlights the effect of quantization and erasures on the convergence rate; and (ii) establishes a linear speedup w.r. t. the number of agents under Markovian sampling. Notably, while different quantization mechanisms and packet drop models have been extensively studied in the FL, distributed optimization, and networked control systems literature, our work is the first to provide a non-asymptotic analysis of their effects in multi-agent and federated reinforcement learning.
The two Higgs doublet type-II seesaw model: Naturalness and $\bar{B}\to X_s\gamma$ versus heavy Higgs masses

We extend the work \cite{Ouazghour:2018mld} to a more general and detailed analysis through studying the naturalness problem and $B$ physics constraints within the context of two higgs doublets model augmented with a complex scalar triplet field ($2HDMcT$). We first derive the modified Vetman conditions and show that naturalness problem might be evaded at the electroweak scale. Then, we evaluate the branching ratio of $\bar{B}\to X_s\gamma$ radiative decay at the Next to leading order. Besides unitarity, boundedness from below constraints and the present collider data from Higgs signal rate measurements, the parameter space of $2HDMcT$ is then revisited to see how the modified Veltman conditions and $\bar{B}\to X_s\gamma$ experimental bounds induce significant delimitations. Our analysis shows that the naturalness affects drastically the masses of heavy Higgs $h_3$, $A_2$, $H_2^\pm$ and $H^{\pm\pm}$. More specifically, we considered three benchmak scenarios corresponding to $\mu_2>20, 40, 70$ GeV. For $\mu_2>40$, we find stringent upper limits on $m_{h_2}$, $m_{A_1}$ and $m_{H_1^\pm}$ at $880$, $920$ and $918$ GeV respectively. If, in addition, the measurements of $\bar{B}\to X_s\gamma$ decay rate are also considered, the lower masses limits of nonstandard Higgs bosons ($h_2$, $h_3$, $A_1$, $A_2$, $H_1^\pm$, $H_2^\pm$, $H^{\pm\pm}$) are pushed up to higher values located between $490$ and $750$ GeV. Besides, we also demonstrate that the $\bar{B}\to X_s\gamma$ experimental limits can only be accommodated within $2HDMcT$ if two conditions are fullfilled : $\alpha_1$ parameter positive and the Higgs mass hierarchy $m_{h_{3}}>m_{H_2^\pm}>m_{H^{\pm\pm}}$.
Current and lasting effects of affect labeling on late positive potential (LPP) amplitudes elicited by negative events

Labeling the emotional aspect of self‐unrelated stimuli (i.e., affect labeling) is a crucial strategy for implicit emotion regulation. However, it is uncertain whether affect labeling influences event‐related potential (ERP) responses (e.g., the late positive potential, LPP) to negative stimuli in comparison with control conditions in which attention is shifted to the emotional content of the stimuli (e.g., affect matching). Additionally, it is unknown whether affect labeling has a lasting effect on the processing of negative stimuli.
Impact of accommodative insufficiency and accommodative/vergence therapy on ciliary muscle thickness in the eye

Recent evidence suggests that the ciliary muscle apical fibres are most responsive to accommodative load; however, the structure of the ciliary muscle in individuals with accommodative insufficiency is unknown. This study examined ciliary muscle structure in individuals with accommodative insufficiency (AI). We also determined the response of the ciliary muscle to accommodative/vergence therapy and increasing accommodative demands to investigate the muscle's responsiveness to workload.
Can Extant Vaping Prevention Message Experiments Tell Us Something About What Works? A Response to O’Keefe

ABSTRACT In his commentary, O’Keefe raises some interesting questions about the meaning of effect sizes in meta-analyses of experiments, focusing on our recent meta-analysis on vaping prevention messages. In this commentary, we respond to O’Keefe’s comments and make several points. First, it is not uncommon to include experiments with different control conditions in a meta-analysis. Second, the set of studies in our meta-analysis were relatively homogenous, all being experiments testing messages to discourage vaping among adolescents and young adults. Third, the control conditions in each of the studies in the meta-analysis were appropriate for each given study, and our results show homogenous effects on most outcomes. Fourth, our meta-analysis finds meaningful effects that are timely and will be useful to researchers and practitioners alike. As this literature continues to grow, so too will knowledge about the effects of vaping prevention messages and moderators of those effects.