Miracle-dz/UniFAR-FTUs huggingface dataset:
Fantastic Gains and Where to Find Them: On the Existence and Prospect of General Knowledge Transfer between Any Pretrained Model

Training deep networks requires various design decisions regarding for instance their architecture, data augmentation, or optimization. In this work, we find these training variations to result in networks learning unique feature sets from the data. Using public model libraries comprising thousands of models trained on canonical datasets like ImageNet, we observe that for arbitrary pairings of pretrained models, one model extracts significant data context unavailable in the other -- independent of overall performance. Given any arbitrary pairing of pretrained models and no external rankings (such as separate test sets, e.g. due to data privacy), we investigate if it is possible to transfer such"complementary"knowledge from one model to another without performance degradation -- a task made particularly difficult as additional knowledge can be contained in stronger, equiperformant or weaker models. Yet facilitating robust transfer in scenarios agnostic to pretrained model pairings would unlock auxiliary gains and knowledge fusion from any model repository without restrictions on model and problem specifics - including from weaker, lower-performance models. This work therefore provides an initial, in-depth exploration on the viability of such general-purpose knowledge transfer. Across large-scale experiments, we first reveal the shortcomings of standard knowledge distillation techniques, and then propose a much more general extension through data partitioning for successful transfer between nearly all pretrained models, which we show can also be done unsupervised. Finally, we assess both the scalability and impact of fundamental model properties on successful model-agnostic knowledge transfer.
Decay constants of $c \bar b$ mesons involving the ten heavy flavor-changing currents at N$^3$LO QCD

Within the nonrelativistic QCD (NRQCD) framework, we complete the three-loop calculations of the NRQCD renormalization constants and the matching coefficients, for the heavy flavor-changing temporal vector, spatial-spatial tensor, spatial-temporal axial-tensor currents, which are coupled to the $P$-wave $c\bar b$ mesons. We further study the ten decay constants for the $c\bar b$ mesons ($B_c,B_c^*,B_{c0}^*,B_{c1}$) coupled with the ten heavy flavor-changing currents involving (pseudo-)scalar, (axial-)vector and (axial-)tensor up to the next-to-next-to-next-to-leading order (N$^3$LO) of $\alpha_s$. We obtain the six ratios of decay constants by approximating them to the corresponding ratios of matching coefficients. We find the N$^3$LO QCD corrections to the six ratios of decay constants have good convergence and weak scale-dependence. We finally predict the hierarchical relationship among the ten decay constants for the $c\bar b$ mesons coupled with the ten currents.
C-Disentanglement: Discovering Causally-Independent Generative Factors under an Inductive Bias of Confounder

Representation learning assumes that real-world data is generated by a few semantically meaningful generative factors (i.e., sources of variation) and aims to discover them in the latent space. These factors are expected to be causally disentangled, meaning that distinct factors are encoded into separate latent variables, and changes in one factor will not affect the values of the others. Compared to statistical independence, causal disentanglement allows more controllable data generation, improved robustness, and better generalization. However, most existing work assumes unconfoundedness in the discovery process, that there are no common causes to the generative factors and thus obtain only statistical independence. In this paper, we recognize the importance of modeling confounders in discovering causal generative factors. Unfortunately, such factors are not identifiable without proper inductive bias. We fill the gap by introducing a framework entitled Confounded-Disentanglement (C-Disentanglement), the first framework that explicitly introduces the inductive bias of confounder via labels from domain expertise. In addition, we accordingly propose an approach to sufficiently identify the causally disentangled factors under any inductive bias of the confounder. We conduct extensive experiments on both synthetic and real-world datasets. Our method demonstrates competitive results compared to various SOTA baselines in obtaining causally disentangled features and downstream tasks under domain shifts.
Pitch-angle Anisotropy Imprinted by Relativistic Magnetic Reconnection

Radiation emitted by nonthermal particles accelerated during relativistic magnetic reconnection is critical for understanding the nonthermal emission in a variety of astrophysical systems, including blazar jets, black hole coronae, pulsars, and magnetars. By means of fully kinetic particle-in-cell simulations, we demonstrate that reconnection-driven particle acceleration imprints an energy-dependent pitch-angle anisotropy and gives rise to broken power laws in both the particle energy spectrum and the pitch-angle anisotropy. The particle distributions depend on the relative strength of the non-reconnecting (guide field) versus the reconnecting component of the magnetic field (B g /B 0) and the lepton magnetization (σ 0). Below the break Lorentz factor γ 0 (injection), the particle energy spectrum is ultra-hard (p < < 1), while above γ 0, the spectral index p > is highly sensitive to B g /B 0. Particles’ velocities align with the magnetic field, reaching minimum pitch angle α at a Lorentz factor γminα controlled by B g /B 0 and σ 0. The energy-dependent pitch-angle anisotropy, evaluated through the mean of sin2α of particles at a given energy, exhibits power-law ranges with negative (m <) and positive (m >) slopes below and above γminα , becoming steeper as B g /B 0 increases. The generation of anisotropic pitch-angle distributions has important astrophysical implications. We address their effects on regulating synchrotron luminosity, spectral energy distribution, polarization, particle cooling, the synchrotron burn-off limit, emission beaming, and temperature anisotropy.
Graphical Object-Centric Actor-Critic

There have recently been significant advances in the problem of unsupervised object-centric representation learning and its application to downstream tasks. The latest works support the argument that employing disentangled object representations in image-based object-centric reinforcement learning tasks facilitates policy learning. We propose a novel object-centric reinforcement learning algorithm combining actor-critic and model-based approaches to utilize these representations effectively. In our approach, we use a transformer encoder to extract object representations and graph neural networks to approximate the dynamics of an environment. The proposed method fills a research gap in developing efficient object-centric world models for reinforcement learning settings that can be used for environments with discrete or continuous action spaces. Our algorithm performs better in a visually complex 3D robotic environment and a 2D environment with compositional structure than the state-of-the-art model-free actor-critic algorithm built upon transformer architecture and the state-of-the-art monolithic model-based algorithm.
Measurement of the Multi-Neutron $\bar{\nu}_{\mu}$ Charged Current Differential Cross Section at Low Available Energy on Hydrocarbon

Neutron production in antineutrino interactions can lead to bias in energy reconstruction in neutrino oscillation experiments, but these interactions have rarely been studied. MINERvA previously studied neutron production at an average antineutrino energy of ~3 GeV in 2016 and found deficiencies in leading models. In this paper, the MINERvA 6 GeV average antineutrino energy data set is shown to have similar disagreements. A measurement of the cross section for an antineutrino to produce two or more neutrons and have low visible energy is presented as an experiment-independent way to explore neutron production modeling. This cross section disagrees with several leading models' predictions. Neutron modeling techniques from nuclear physics are used to quantify neutron detection uncertainties on this result.
Orthonormal representations, vector chromatic number, and extension complexity

We construct a bipartite generalization of Alon and Szegedy's nearly orthogonal vectors, thereby obtaining strong bounds for several extremal problems involving the Lov\'asz theta function, vector chromatic number, minimum semidefinite rank, nonnegative rank, and extension complexity of polytopes. In particular, we derive some general lower bounds for the vector chromatic number which may be of independent interest.
Quenched pair breaking by interlayer correlations as a key to superconductivity in La$_3$Ni$_2$O$_7$

The recent discovery of superconductivity in La$_3$Ni$_2$O$_7$ with $T_\mathrm{c} \simeq 80~\mathrm{K}$ under high pressure opens up a new route to high-$T_\mathrm{c}$ superconductivity. This material realizes a bilayer square lattice model featuring a strong interlayer hybridization unlike many unconventional superconductors. A key question in this regard concerns how electronic correlations driven by the interlayer hybridization affect the low-energy electronic structure and the concomitant superconductivity. Here, we demonstrate using a cluster dynamical mean-field theory that the interlayer electronic correlations (IECs) induce a Lifshitz transition resulting in a change of Fermi surface topology. By solving an appropriate gap equation, we further show that the dominant pairing instability (intraorbital $s$-wave/interorbital $d_{x^2-y^2}$-wave, or $s \pm$-wave in a bonding-antibonding basis) is enhanced by the IECs. The underlying mechanism is the quenching of a strong ferromagnetic channel, resulting from the Lifshitz transition driven by the IECs. Based on this picture, we provide a possible reason of why superconductivity emerges only under high pressure.
HyperFields: Towards Zero-Shot Generation of NeRFs from Text

We introduce HyperFields, a method for generating text-conditioned Neural Radiance Fields (NeRFs) with a single forward pass and (optionally) some fine-tuning. Key to our approach are: (i) a dynamic hypernetwork, which learns a smooth mapping from text token embeddings to the space of NeRFs; (ii) NeRF distillation training, which distills scenes encoded in individual NeRFs into one dynamic hypernetwork. These techniques enable a single network to fit over a hundred unique scenes. We further demonstrate that HyperFields learns a more general map between text and NeRFs, and consequently is capable of predicting novel in-distribution and out-of-distribution scenes -- either zero-shot or with a few finetuning steps. Finetuning HyperFields benefits from accelerated convergence thanks to the learned general map, and is capable of synthesizing novel scenes 5 to 10 times faster than existing neural optimization-based methods. Our ablation experiments show that both the dynamic architecture and NeRF distillation are critical to the expressivity of HyperFields.
Dual-Class Stocks: Can They Serve as Effective Predictors?

Kardemir Karabuk Iron Steel Industry Trade&Co. Inc., ranked as the 24th largest industrial company in Turkey, offers three distinct stocks listed on the Borsa Istanbul: KRDMA, KRDMB, and KRDMD. These stocks, sharing the sole difference in voting power, have exhibited significant price divergence over an extended period. This paper conducts an in-depth analysis of the divergence patterns observed in these three stock prices from January 2001 to July 2023. Additionally, it introduces an innovative training set selection rule tailored for LSTM models, incorporating a rolling training set, and demonstrates its significant predictive superiority over the conventional use of LSTM models with large training sets. Despite their strong correlation, the study found no compelling evidence supporting the efficiency of dual-class stocks as predictors of each other's performance.
Multitask Online Learning: Listen to the Neighborhood Buzz

We study multitask online learning in a setting where agents can only exchange information with their neighbors on an arbitrary communication network. We introduce $\texttt{MT-CO}_2\texttt{OL}$, a decentralized algorithm for this setting whose regret depends on the interplay between the task similarities and the network structure. Our analysis shows that the regret of $\texttt{MT-CO}_2\texttt{OL}$ is never worse (up to constants) than the bound obtained when agents do not share information. On the other hand, our bounds significantly improve when neighboring agents operate on similar tasks. In addition, we prove that our algorithm can be made differentially private with a negligible impact on the regret when the losses are linear. Finally, we provide experimental support for our theory.
Likelihood-based Out-of-Distribution Detection with Denoising Diffusion Probabilistic Models

Out-of-Distribution detection between dataset pairs has been extensively explored with generative models. We show that likelihood-based Out-of-Distribution detection can be extended to diffusion models by leveraging the fact that they, like other likelihood-based generative models, are dramatically affected by the input sample complexity. Currently, all Out-of-Distribution detection methods with Diffusion Models are reconstruction-based. We propose a new likelihood ratio for Out-of-Distribution detection with Deep Denoising Diffusion Models, which we call the Complexity Corrected Likelihood Ratio. Our likelihood ratio is constructed using Evidence Lower-Bound evaluations from an individual model at various noising levels. We present results that are comparable to state-of-the-art Out-of-Distribution detection methods with generative models.
LONG-TERM CHANGES lN THE VERTICAL DISTRIBUTION OF PHYTOPLANKTON BlOMASS AND PRIMARY PRODUCTION lN LAKE GENEV A: A RESPONSE TO THE OLIGOTROPHICA TION

ln Lake Geneva, after several years of eutrophication dissolved inorganic phosphorus concentrations was reduced from 76 Jlg.r' to around 30 Jlg,r', This decrease lead to changes in the phytoplankton composition, but no decrease in the phytoplankton crop has been observed. An analysis of the changes in the vertical distribution of chlorophyll a and primary production reveals a long-term tendancy towards the deepening of the maximum and a more homogeneous distribution of the primary production within the photic layer. This evolution is coherent with the one expected for a lake in course of oligotrophication. Even though the transparency controls the maximum depth of phytoplankton growth, it seems that the observed changes in the vertical distribution of phytoplankton are more likely to be induced by the evolution in the phosphorus vertical gradient. We conclude that the general decrease in phosphorus concentrations induced a change in the vertical distribution of phytoplankton, and altered the summer phytoplankton community toward the selection of species adapted to low-light intensities. RlASSUNTO
Affective Video Content Analysis: Decade Review and New Perspectives

Video content is rich in semantics and has the ability to evoke various emotions in viewers. In recent years, with the rapid development of affective computing and the explosive growth of visual data, affective video content analysis (AVCA) as an essential branch of affective computing has become a widely researched topic. In this study, we comprehensively review the development of AVCA over the past decade, particularly focusing on the most advanced methods adopted to address the three major challenges of video feature extraction, expression subjectivity, and multimodal feature fusion. We first introduce the widely used emotion representation models in AVCA and describe commonly used datasets. We summarize and compare representative methods in the following aspects: (1) unimodal AVCA models, including facial expression recognition and posture emotion recognition; (2) multimodal AVCA models, including feature fusion, decision fusion, and attention-based multimodal models; (3) model performance evaluation standards. Finally, we discuss future challenges and promising research directions, such as emotion recognition and public opinion analysis, human-computer interaction, and emotional intelligence.
Dynamic Factor Models: a Genealogy

Dynamic factor models have been developed out of the need of analyzing and forecasting time series in increasingly high dimensions. While mathematical statisticians faced with inference problems in high-dimensional observation spaces were focusing on the so-called spiked-model-asymptotics, econometricians adopted an entirely and considerably more effective asymptotic approach, rooted in the factor models originally considered in psychometrics. The so-called dynamic factor model methods, in two decades, has grown into a wide and successful body of techniques that are widely used in central banks, financial institutions, economic and statistical institutes. The objective of this chapter is not an extensive survey of the topic but a sketch of its historical growth, with emphasis on the various assumptions and interpretations, and a family tree of its main variants.
Controlled Decoding from Language Models

KL-regularized reinforcement learning (RL) is a popular alignment framework to control the language model responses towards high reward outcomes. We propose a modular solver for this RL objective, called controlled decoding (CD), which exerts control through a separate prefix scorer module. At training time, the prefix scorer learns a value function for the reward, and it is used at inference time to control the generation from a frozen base model, provably sampling from a solution to the RL objective. We empirically demonstrate that CD is effective as a control mechanism on popular benchmarks. We also show that a single prefix scorer can learn multiple rewards and different reward combinations can be configurable at inference time, effectively solving a multi-objective RL problem with no additional training. We show that the benefits of applying CD transfer to an unseen base model with no further tuning. Finally, we show that CD can be applied in a blockwise decoding fashion at inference-time, essentially bridging the gap between the popular best-of-$n$ strategy and token-level control through reinforcement learning. This makes CD a promising approach for alignment of language models.
SPA: A Graph Spectral Alignment Perspective for Domain Adaptation

Unsupervised domain adaptation (UDA) is a pivotal form in machine learning to extend the in-domain model to the distinctive target domains where the data distributions differ. Most prior works focus on capturing the inter-domain transferability but largely overlook rich intra-domain structures, which empirically results in even worse discriminability. In this work, we introduce a novel graph SPectral Alignment (SPA) framework to tackle the tradeoff. The core of our method is briefly condensed as follows: (i)-by casting the DA problem to graph primitives, SPA composes a coarse graph alignment mechanism with a novel spectral regularizer towards aligning the domain graphs in eigenspaces; (ii)-we further develop a fine-grained message propagation module -- upon a novel neighbor-aware self-training mechanism -- in order for enhanced discriminability in the target domain. On standardized benchmarks, the extensive experiments of SPA demonstrate that its performance has surpassed the existing cutting-edge DA methods. Coupled with dense model analysis, we conclude that our approach indeed possesses superior efficacy, robustness, discriminability, and transferability. Code and data are available at: https://github.com/CrownX/SPA.
BOOST: Harnessing Black-Box Control to Boost Commonsense in LMs' Generation

Large language models (LLMs) such as GPT-3 have demonstrated a strong capability to generate coherent and contextually relevant text. However, amidst their successes, a crucial issue persists: their generated outputs still lack commonsense at times. Moreover, fine-tuning the entire LLM towards more commonsensical outputs is computationally expensive if not infeasible. In this paper, we present a computation-efficient framework that steers a frozen Pre-Trained Language Model (PTLM) towards more commonsensical generation (i.e., producing a plausible output that incorporates a list of concepts in a meaningful way). Specifically, we first construct a reference-free evaluator that assigns a sentence with a commonsensical score by grounding the sentence to a dynamic commonsense knowledge base from four different relational aspects. We then use the scorer as the oracle for commonsense knowledge, and extend the controllable generation method called NADO to train an auxiliary head that guides a fixed PTLM to better satisfy the oracle. We test our framework on a series of GPT-2-, Flan-T5-, and Alpaca-based language models (LMs) on two constrained concept-to-sentence benchmarks. Human evaluation results demonstrate that our method consistently leads to the most commonsensical outputs.
IndustReal: A Dataset for Procedure Step Recognition Handling Execution Errors in Egocentric Videos in an Industrial-Like Setting

Although action recognition for procedural tasks has received notable attention, it has a fundamental flaw in that no measure of success for actions is provided. This limits the applicability of such systems especially within the industrial domain, since the outcome of procedural actions is often significantly more important than the mere execution. To address this limitation, we define the novel task of procedure step recognition (PSR), focusing on recognizing the correct completion and order of procedural steps. Alongside the new task, we also present the multi-modal IndustReal dataset. Unlike currently available datasets, IndustReal contains procedural errors (such as omissions) as well as execution errors. A significant part of these errors are exclusively present in the validation and test sets, making IndustReal suitable to evaluate robustness of algorithms to new, unseen mistakes. Additionally, to encourage reproducibility and allow for scalable approaches trained on synthetic data, the 3D models of all parts are publicly available. Annotations and benchmark performance are provided for action recognition and assembly state detection, as well as the new PSR task. IndustReal, along with the code and model weights, is available at: https://github.com/TimSchoonbeek/IndustReal .
MaxEnt Loss: Constrained Maximum Entropy for Calibration under Out-of-Distribution Shift

We present a new loss function that addresses the out-of-distribution (OOD) network calibration problem. While many objective functions have been proposed to effectively calibrate models in-distribution, our findings show that they do not always fare well OOD. Based on the Principle of Maximum Entropy, we incorporate helpful statistical constraints observed during training, delivering better model calibration without sacrificing accuracy. We provide theoretical analysis and show empirically that our method works well in practice, achieving state-of-the-art calibration on both synthetic and real-world benchmarks. Our code is available at https://github.com/dexterdley/MaxEnt-Loss.
The effect of altitude and development stage on the synthetic activity of some polyphenols in Teucrium polium L. in Tessala mountains (Western Algeria).

Corresponding Author: Nadjia Fertout-Mouri, Laboratory of Plant Biodiversity: conservation and enhancement, faculty of Natural and Life Sciences, Djillali Liabes University, Sidi Bel Abbes, 22000, Algeria. Tel : +213 772 43 57 93; E-mail: nadjiafertout@yahoo.fr The effect of altitude and development stage on the synthetic activity of some polyphenols in Teucrium polium L. in Tessala mountains (Western Algeria).