Miracle-dz/UniFAR-FTUs huggingface dataset:
On the Formation of Continental Polar Air

Abstract This research investigates the transformation of maritime polar air into continental polar air in the Arctic during winter. The evolution of the vertical profiles of temperature and humidity is simulated using a one-dimensional model. The following physical processes are examined 1) surface enthalpy flux; 2) infrared radiative cooling due to emission by CO2, water vapor, water droplets, and ice crystals; 3) gravitational setting of the condensed water, 4) turbulent mixing, and 5) subsidence. The modeled formation of continental polar air is dominated by the radiative cooling due to emission by ice crystals and water droplets. The model reproduces the formation of low-level clouds that are frequently observed in these cold air masses, and also the phenomenon of “cloudless” ice crystal precipitation. The model requires two weeks for the formation of fully-developed continental polar air, although after only four days of cooling the air has acquired most of the air mass properties. The rate of cooli.
Nonlinear generalizations of principal component learning algorithms

In this paper, we introduce and study nonlinear generalizations of several neural algorithms that learn the principal eigenvectors of the data covariance matrix. We first consider the robust versions that optimize a nonquadratic criterion under orthonormality constraints. As an important byproduct, Sanger's GHA and Oja's SGA algorithms for learning principal components are derived from a natural optimization problem. We also introduce a fully nonlinear generalization that has signal separation capabilities not possessed by standard principal component analysis learning algorithms.
When is every Hausdorff continuous interval-valued function bounded by continuous functions?

Abstract In this paper we characterize the topological spaces X for which ℍ ft (X) = ℍ cm (X). Here ℍ ft (X) is the set of all finite Hausdorff continuous interval-valued functions on X and ℍ cm (X) is the subset of ℍ ft (X) formed by those functions f ε ℍ ft (X) for which there exist continuous real-valued functions on X, ϕ and Φ such that ϕ ≤ f ≤ Φ. These sets of interval-valued functions are important in functional analysis and in partial differential equations (PDEs) since ℍ ft (X) is Dedekind order complete and the Dedekind order completion of the set C(X) of all continuous real-valued functions on a completely regular topological space X is ℍ cm (X).
When Deep Learning Meets Polyhedral Theory: A Survey

In the past decade, deep learning became the prevalent methodology for predictive modeling thanks to the remarkable accuracy of deep neural networks in tasks such as computer vision and natural language processing. Meanwhile, the structure of neural networks converged back to simpler representations based on piecewise constant and piecewise linear functions such as the Rectified Linear Unit (ReLU), which became the most commonly used type of activation function in neural networks. That made certain types of network structure $\unicode{x2014}$such as the typical fully-connected feedforward neural network$\unicode{x2014}$ amenable to analysis through polyhedral theory and to the application of methodologies such as Linear Programming (LP) and Mixed-Integer Linear Programming (MILP) for a variety of purposes. In this paper, we survey the main topics emerging from this fast-paced area of work, which bring a fresh perspective to understanding neural networks in more detail as well as to applying linear optimization techniques to train, verify, and reduce the size of such networks.
A preferential interpretation of MultiLayer Perceptrons in a conditional logic with typicality

In this paper we investigate the relationships between a multipreferential semantics for defeasible reasoning in knowledge representation and a multilayer neural network model. Weighted knowledge bases for a simple description logic with typicality are considered under a (many-valued) ``concept-wise"multipreference semantics. The semantics is used to provide a preferential interpretation of MultiLayer Perceptrons (MLPs). A model checking and an entailment based approach are exploited in the verification of conditional properties of MLPs.
On the sub-additivity of stochastic matching

We consider a stochastic matching model with a general compatibility graph, as introduced in \cite{MaiMoy16}. We prove that most common matching policies (including FCFM, priorities and random) satisfy a particular sub-additive property, which we exploit to show in many cases, the coupling-from-the-past to the steady state, using a backwards scheme {\em \`a la} Loynes. We then use these results to explicitly construct perfect bi-infinite matchings, and to build a perfect simulation algorithm in the case where the buffer of the system is finite.
NLNDE at SemEval-2023 Task 12: Adaptive Pretraining and Source Language Selection for Low-Resource Multilingual Sentiment Analysis

This paper describes our system developed for the SemEval-2023 Task 12 “Sentiment Analysis for Low-resource African Languages using Twitter Dataset”. Sentiment analysis is one of the most widely studied applications in natural language processing. However, most prior work still focuses on a small number of high-resource languages. Building reliable sentiment analysis systems for low-resource languages remains challenging, due to the limited training data in this task. In this work, we propose to leverage language-adaptive and task-adaptive pretraining on African texts and study transfer learning with source language selection on top of an African language-centric pretrained language model.Our key findings are: (1) Adapting the pretrained model to the target language and task using a small yet relevant corpus improves performance remarkably by more than 10 F1 score points. (2) Selecting source languages with positive transfer gains during training can avoid harmful interference from dissimilar languages, leading to better results in multilingual and cross-lingual settings. In the shared task, our system wins 8 out of 15 tracks and, in particular, performs best in the multilingual evaluation.
Enhancing Adversarial Contrastive Learning via Adversarial Invariant Regularization

Adversarial contrastive learning (ACL) is a technique that enhances standard contrastive learning (SCL) by incorporating adversarial data to learn a robust representation that can withstand adversarial attacks and common corruptions without requiring costly annotations. To improve transferability, the existing work introduced the standard invariant regularization (SIR) to impose style-independence property to SCL, which can exempt the impact of nuisance style factors in the standard representation. However, it is unclear how the style-independence property benefits ACL-learned robust representations. In this paper, we leverage the technique of causal reasoning to interpret the ACL and propose adversarial invariant regularization (AIR) to enforce independence from style factors. We regulate the ACL using both SIR and AIR to output the robust representation. Theoretically, we show that AIR implicitly encourages the representational distance between different views of natural data and their adversarial variants to be independent of style factors. Empirically, our experimental results show that invariant regularization significantly improves the performance of state-of-the-art ACL methods in terms of both standard generalization and robustness on downstream tasks. To the best of our knowledge, we are the first to apply causal reasoning to interpret ACL and develop AIR for enhancing ACL-learned robust representations. Our source code is at https://github.com/GodXuxilie/Enhancing_ACL_via_AIR.
Entropy production in the nonreciprocal Cahn-Hilliard model.

We study the nonreciprocal Cahn-Hilliard model with thermal noise as a prototypical example of a generic class of non-Hermitian stochastic field theories, analyzed in two companion papers [Suchanek, Kroy, and Loos, Phys. Rev. Lett. 131, 258302 (2023)10.1103/PhysRevLett.131. 258302; Phys. Rev. E 108, 064123 (2023)10.1103/PhysRevE.108. 064123]. Due to the nonreciprocal coupling between two field components, the model is inherently out of equilibrium and can be regarded as an active field theory. Beyond the conventional homogeneous and static-demixed phases, it exhibits a traveling-wave phase, which can be entered via either an oscillatory instability or a critical exceptional point. By means of a Fourier decomposition of the entropy production rate, we quantify the associated scale-resolved time-reversal symmetry breaking, in all phases and across the transitions, in the low-noise regime. Our perturbative calculation reveals its dependence on the strength of the nonreciprocal coupling. Surging entropy production near the static-dynamic transitions can be attributed to entropy-generating fluctuations in the longest wavelength Fourier mode and heralds the emerging traveling wave. Its translational dynamics can be mapped on the dissipative ballistic motion of an active (quasi)particle.
Spatio-temporal characterization of non-linear forcing in turbulence

The quadratic convection term in the incompressible Navier-Stokes equations is considered as a non-linear forcing to the linear operator, and it is studied in the Fourier domain through the analysis of interactions between triadically compatible wavenumber-frequency triplets. Interaction coefficients are proposed to quantify the contribution to the forcing by each pair of triplets and are computed using data from direct numerical simulations of a turbulent channel at $Re_{\tau} \approx 550$. The coefficients show the importance of interactions involving streamwise large scales. The regions of non-linear interactions permitted under the quasi-linear (QL) and generalized quasi-linear (GQL) assumptions are shown to be significant contributors to the forcing and increasing the number of GQL-large scales is shown to monotonically increase the total forcing captured, providing a possible reason for the success of QL and GQL simulations. The tools presented are expected to be useful for improving modeling of the nonlinearity, especially in QL, GQL, and resolvent analyses, and understanding the amplitude modulation mechanism relating large-scale fluctuations to the modulation of near-wall structures.
An Iterative Algorithm for Rescaled Hyperbolic Functions Regression

Large language models (LLMs) have numerous real-life applications across various domains, such as natural language translation, sentiment analysis, language modeling, chatbots and conversational agents, creative writing, text classification, summarization, and generation. LLMs have shown great promise in improving the accuracy and efficiency of these tasks, and have the potential to revolutionize the field of natural language processing (NLP) in the years to come. Exponential function based attention unit is a fundamental element in LLMs. Several previous works have studied the convergence of exponential regression and softmax regression. The exponential regression [Li, Song, Zhou 2023] and softmax regression [Deng, Li, Song 2023] can be formulated as follows. Given matrix $A \in \mathbb{R}^{n \times d}$ and vector $b \in \mathbb{R}^n$, the goal of exponential regression is to solve \begin{align*} \min_{x} \| \exp(Ax) - b \|_2 \end{align*} and the goal of softmax regression is to solve \begin{align*} \min_{x} \| \langle \exp(Ax) , {\bf 1}_n \rangle^{-1} \exp(Ax) - b \|_2 . \end{align*} In this work, we define a slightly different formulation than softmax regression. \begin{align*} \min_{x \in \mathbb{R}^d } \| u(x) - \langle u(x) , {\bf 1}_n \rangle \cdot b \|_2 \end{align*} where $u(x) \in \{ \exp(Ax), \cosh(Ax) , \sinh(Ax) \}$. We provide an input sparsity time algorithm for this problem. Our algorithm framework is very general and can be applied to functions like $\cosh()$ and $\sinh()$ as well. Our technique is also general enough to be applied to in-context learning for rescaled softmax regression.
POUF: Prompt-oriented unsupervised fine-tuning for large pre-trained models

Through prompting, large-scale pre-trained models have become more expressive and powerful, gaining significant attention in recent years. Though these big models have zero-shot capabilities, in general, labeled data are still required to adapt them to downstream tasks. To overcome this critical limitation, we propose an unsupervised fine-tuning framework to directly fine-tune the model or prompt on the unlabeled target data. We demonstrate how to apply our method to both language-augmented vision and masked-language models by aligning the discrete distributions extracted from the prompts and target data. To verify our approach's applicability, we conduct extensive experiments on image classification, sentiment analysis, and natural language inference tasks. Across 13 image-related tasks and 15 language-related ones, the proposed approach achieves consistent improvements over the baselines.
Scaling Description of Dynamical Heterogeneity and Avalanches of Relaxation in Glass-Forming Liquids

We provide a theoretical description of dynamical heterogeneities in glass-forming liquids, based on the premise that relaxation occurs via local rearrangements coupled by elasticity. In our framework, the growth of the dynamical correlation length $\xi$ and of the correlation volume $\chi_4$ are controlled by a zero-temperature fixed point. We connect this critical behavior to the properties of the distribution of local energy barriers at zero temperature. Our description makes a direct connection between dynamical heterogeneities and avalanche-type relaxation associated to dynamic facilitation, allowing us to relate the size distribution of heterogeneities to their time evolution. Within an avalanche, a local region relaxes multiple times, the more the larger is the avalanche. This property, related to the nature of the zero-temperature fixed point, directly leads to decoupling of particle diffusion and relaxation time (the so-called Stokes-Einstein violation). Our most salient predictions are tested and confirmed by numerical simulations of scalar and tensorial thermal elasto-plastic models.
Adaptive procedures for directional false discovery rate control

In multiple hypothesis testing, it is well known that adaptive procedures can enhance power via incorporating information about the number of true nulls present. Under independence, we establish that two adaptive false discovery rate (FDR) methods, upon augmenting sign declarations, also offer directional false discovery rate (FDR$_\text{dir}$) control in the strong sense. Such FDR$_\text{dir}$ controlling properties are appealing because adaptive procedures have the greatest potential to reap substantial gain in power when the underlying parameter configurations contain little to no true nulls, which are precisely settings where the FDR$_\text{dir}$ is an arguably more meaningful error rate to be controlled than the FDR.
An Integral Associated With the Modified Saigo Operators Involving Two I-Functions and Generalized Polynomials

In this paper, we obtained two fractional integrals involving the product of two I-functions (10), general class of polynomials and Gauss hypergeometric function. By making use of these integrals, we have obtained two theorems based on modified Saigo operators of fractional integration. Keywords: fractional integral, Saigo operators, Riemann-Liouville operator, Weyl operator, Erd�� lyi- Kober operator, I-function, gauss hypergeometric function and general class of multivariable polynomial. , , ,    n h n g n f z n n n z d d       
What Do Self-Supervised Vision Transformers Learn?

We present a comparative study on how and why contrastive learning (CL) and masked image modeling (MIM) differ in their representations and in their performance of downstream tasks. In particular, we demonstrate that self-supervised Vision Transformers (ViTs) have the following properties: (1) CL trains self-attentions to capture longer-range global patterns than MIM, such as the shape of an object, especially in the later layers of the ViT architecture. This CL property helps ViTs linearly separate images in their representation spaces. However, it also makes the self-attentions collapse into homogeneity for all query tokens and heads. Such homogeneity of self-attention reduces the diversity of representations, worsening scalability and dense prediction performance. (2) CL utilizes the low-frequency signals of the representations, but MIM utilizes high-frequencies. Since low- and high-frequency information respectively represent shapes and textures, CL is more shape-oriented and MIM more texture-oriented. (3) CL plays a crucial role in the later layers, while MIM mainly focuses on the early layers. Upon these analyses, we find that CL and MIM can complement each other and observe that even the simplest harmonization can help leverage the advantages of both methods. The code is available at https://github.com/naver-ai/cl-vs-mim.
Adversarial Representation Learning for Robust Privacy Preservation in Audio

Sound event detection systems are widely used in various applications such as surveillance and environmental monitoring where data is automatically collected, processed, and sent to a cloud for sound recognition. However, this process may inadvertently reveal sensitive information about users or their surroundings, hence raising privacy concerns. In this study, we propose a novel adversarial training method for learning representations of audio recordings that effectively prevents the detection of speech activity from the latent features of the recordings. The proposed method trains a model to generate invariant latent representations of speech-containing audio recordings that cannot be distinguished from non-speech recordings by a speech classifier. The novelty of our work is in the optimization algorithm, where the speech classifier's weights are regularly replaced with the weights of classifiers trained in a supervised manner. This increases the discrimination power of the speech classifier constantly during the adversarial training, motivating the model to generate latent representations in which speech is not distinguishable, even using new speech classifiers trained outside the adversarial training loop. The proposed method is evaluated against a baseline approach with no privacy measures and a prior adversarial training method, demonstrating a significant reduction in privacy violations compared to the baseline approach. Additionally, we show that the prior adversarial method is practically ineffective for this purpose.
Learning Locally Editable Virtual Humans

In this paper, we propose a novel hybrid representation and end-to-end trainable network architecture to model fully editable and customizable neural avatars. At the core of our work lies a representation that combines the modeling power of neural fields with the ease of use and inherent 3D consistency of skinned meshes. To this end, we construct a trainable feature codebook to store local geometry and texture features on the vertices of a deformable body model, thus exploiting its consistent topology under articulation. This representation is then employed in a generative auto-decoder architecture that admits fitting to unseen scans and sampling of realistic avatars with varied appearances and geometries. Furthermore, our representation allows local editing by swapping local features between 3D assets. To verify our method for avatar creation and editing, we contribute a new highquality dataset, dubbed CustomHumans, for training and evaluation. Our experiments quantitatively and qualitatively show that our method generates diverse detailed avatars and achieves better model fitting performance compared to state-of-the-art methods. Our code and dataset are available at https://ait.ethz. ch/customhumans.
Towards Improving Operation Economics: A Bilevel MIP-Based Closed-Loop Predict-and-Optimize Framework for Prescribing Unit Commitment

Generally, system operators conduct the economic operation of power systems in an open-loop predict-then-optimize process: the renewable energy source (RES) availability and system reserve requirements are first predicted; given the predictions, system operators solve optimization models such as unit commitment (UC) to determine the economical operation plans accordingly. However, such an open-loop process could essentially compromise the operation economics because its predictors myopically seek to improve the immediate statistical prediction errors instead of the ultimate operation cost. To this end, this paper presents a closed-loop predict-and-optimize framework, offering a prescriptive UC to improve the operation economics. First, a bilevel mixed-integer programming model is leveraged to train cost-oriented predictors tailored for optimal system operations: the upper level trains the RES and reserve predictors based on their induced operation cost; the lower level, with given predictions, mimics the system operation process and feeds the induced operation cost back to the upper level. Furthermore, the embeddability of the trained predictors grants a prescriptive UC model, which simultaneously provides RES-reserve predictions and UC decisions with enhanced operation economics. Finally, numerical case studies using real-world data illustrate the potential economic and practical advantages of prescriptive UC over deterministic, robust, and stochastic UC models.
Lower bounds on the energy of the Bose gas

We present an overview of the approach to establish a lower bound to the ground state energy for the dilute, interacting Bose gas in a periodic box. In this paper the size of the box is larger than the Gross-Pitaevski length scale. The presentation includes both the 2 and 3 dimensional cases, and catches the second order correction, i.e. the Lee-Huang-Yang term. The calculation on a box of this length scale is the main step to calculate the energy in the thermodynamic limit. However, the periodic boundary condition simplifies many steps of the argument considerably compared to the localized problem coming from the thermodynamic case.
Magnetar Central Engines in Gamma-Ray Bursts Follow the Universal Relation of Accreting Magnetic Stars

Gamma-ray bursts (GRBs), both long and short, are explosive events whose inner engine is generally expected to be a black hole or a highly magnetic neutron star (magnetar) accreting high-density matter. Recognizing the nature of GRB central engines, and in particular the formation of neutron stars (NSs), is of high astrophysical significance. A possible signature of NSs in GRBs is the presence of a plateau in the early X-ray afterglow. Here we carefully select a subset of long and short GRBs with a clear plateau, and look for an additional NS signature in their prompt emission, namely a transition between the accretion and propeller phases in analogy with accreting, magnetic compact objects in other astrophysical sources. We estimate from the prompt emission the minimum accretion luminosity below which the propeller mechanism sets in, and the NS magnetic field and spin period from the plateau. We demonstrate that these three quantities obey the same universal relation in GRBs as in other accreting compact objects switching from accretion to propeller. This relation provides also an estimate of the radiative efficiency of GRBs, which we find to be several times lower than radiatively efficient accretion in X-ray binaries and in agreement with theoretical expectations. These results provide additional support to the idea that at least some GRBs are powered by magnetars surrounded by an accretion disk.