Miracle-dz/UniFAR-FTUs huggingface dataset:
Hydrologic Prediction over the Conterminous United States Using the National Multi-Model Ensemble

AbstractThe authors analyzed the skill of monthly and seasonal soil moisture (SM) and runoff (RO) forecasts over the United States performed by driving the Variable Infiltration Capacity (VIC) hydrologic model with forcings derived from the National Multi-Model Ensemble hindcasts (NMME_VIC). The grand ensemble mean NMME_VIC forecasts were compared to ensemble streamflow prediction (ESP) forecasts derived from the VIC model forced by resampling of historical observations during the forecast period (ESP_VIC), using the same initial conditions as NMME_VIC. The forecast period is from 1982 to 2010, with the forecast initialized on 1 January, 1 April, 5 July, and 3 October. Overall, forecast skill is seasonally and regionally dependent. The authors found that 1) the skill of the grand ensemble mean NMME_VIC forecasts is comparable with that of the individual model that has the highest skill; 2) for all forecast initiation dates, the initial conditions play a dominant role in forecast skill at 1-month lead, and.
Weak-Form Latent Space Dynamics Identification

Recent work in data-driven modeling has demonstrated that a weak formulation of model equations enhances the noise robustness of a wide range of computational methods. In this paper, we demonstrate the power of the weak form to enhance the LaSDI (Latent Space Dynamics Identification) algorithm, a recently developed data-driven reduced order modeling technique. We introduce a weak form-based version WLaSDI (Weak-form Latent Space Dynamics Identification). WLaSDI first compresses data, then projects onto the test functions and learns the local latent space models. Notably, WLaSDI demonstrates significantly enhanced robustness to noise. With WLaSDI, the local latent space is obtained using weak-form equation learning techniques. Compared to the standard sparse identification of nonlinear dynamics (SINDy) used in LaSDI, the variance reduction of the weak form guarantees a robust and precise latent space recovery, hence allowing for a fast, robust, and accurate simulation. We demonstrate the efficacy of WLaSDI vs. LaSDI on several common benchmark examples including viscid and inviscid Burgers', radial advection, and heat conduction. For instance, in the case of 1D inviscid Burgers' simulations with the addition of up to 100% Gaussian white noise, the relative error remains consistently below 6% for WLaSDI, while it can exceed 10,000% for LaSDI. Similarly, for radial advection simulations, the relative errors stay below 15% for WLaSDI, in stark contrast to the potential errors of up to 10,000% with LaSDI. Moreover, speedups of several orders of magnitude can be obtained with WLaSDI. For example applying WLaSDI to 1D Burgers' yields a 140X speedup compared to the corresponding full order model. Python code to reproduce the results in this work is available at (https://github.com/MathBioCU/PyWSINDy_ODE) and (https://github.com/MathBioCU/PyWLaSDI).
An Embodied Generalist Agent in 3D World

Leveraging massive knowledge and learning schemes from large language models (LLMs), recent machine learning models show notable successes in building generalist agents that exhibit the capability of general-purpose task solving in diverse domains, including natural language processing, computer vision, and robotics. However, a significant challenge remains as these models exhibit limited ability in understanding and interacting with the 3D world. We argue this limitation significantly hinders the current models from performing real-world tasks and further achieving general intelligence. To this end, we introduce an embodied multi-modal and multi-task generalist agent that excels in perceiving, grounding, reasoning, planning, and acting in the 3D world. Our proposed agent, referred to as LEO, is trained with shared LLM-based model architectures, objectives, and weights in two stages: (i) 3D vision-language alignment and (ii) 3D vision-language-action instruction tuning. To facilitate the training, we meticulously curate and generate an extensive dataset comprising object-level and scene-level multi-modal tasks with exceeding scale and complexity, necessitating a deep understanding of and interaction with the 3D world. Through rigorous experiments, we demonstrate LEO's remarkable proficiency across a wide spectrum of tasks, including 3D captioning, question answering, embodied reasoning, embodied navigation, and robotic manipulation. Our ablation results further provide valuable insights for the development of future embodied generalist agents.
Influence Maximization based on Simplicial Contagion Models in Hypergraphs

In recent years, the exploration of node centrality has received significant attention and extensive investigation, primarily fuelled by its applications in diverse domains such as product recommendations, opinion propagation, disease spread, and other scenarios requiring the maximization of node influence. Despite various perspectives emphasizing the indispensability of higher-order networks, research specifically delving into node centrality within the realm of hypergraphs has been relatively constrained. This paper focuses on the problem of influence maximization on the Simplicial Contagion Model (SCM), using the susceptible-infected-recovered (SIR) model as an example. To find practical solutions to this optimization problem, we have developed a theoretical framework based on message passing process and conducted stability analysis of equilibrium solutions for the self-consistent equations. Furthermore, we introduce a metric called collective influence and propose an adaptive algorithm, known as the Collective Influence Adaptive (CIA), to identify influential propagators in the spreading process. Notably, our algorithm distinguishes itself by prioritizing collective influence over individual influence, resulting in demonstrably superior performance, a characteristic substantiated by a comprehensive array of experiments.
Learning principle and mathematical realization of the learning mechanism in the brain

While deep learning has achieved remarkable success, there is no clear explanation about why it works so well. In order to discuss this question quantitatively, we need a mathematical framework that explains what learning is in the first place. After several considerations, we succeeded in constructing a mathematical framework that can provide a unified understanding of all types of learning, including deep learning and learning in the brain. We call it learning principle, and it follows that all learning is equivalent to estimating the probability of input data. We not only derived this principle, but also mentioned its application to actual machine learning models. For example, we found that conventional supervised learning is equivalent to estimating conditional probabilities, and succeeded in making supervised learning more effective and generalized. We also proposed a new method of defining the values of estimated probability using differentiation, and showed that unsupervised learning can be performed on arbitrary dataset without any prior knowledge. Namely, this method is a general-purpose machine learning in the true sense. Moreover, we succeeded in describing the learning mechanism in the brain by considering the time evolution of a fully or partially connected model and applying this new method. The learning principle provides solutions to many unsolved problems in deep learning and cognitive neuroscience.
Improving performance of heart rate time series classification by grouping subjects

Unlike the more commonly analyzed ECG or PPG data for activity classification, heart rate time series data is less detailed, often noisier and can contain missing data points. Using the BigIdeasLab_STEP dataset, which includes heart rate time series annotated with specific tasks performed by individuals, we sought to determine if general classification was achievable. Our analyses showed that the accuracy is sensitive to the choice of window/stride size. Moreover, we found variable classification performances between subjects due to differences in the physical structure of their hearts. Various techniques were used to minimize this variability. First of all, normalization proved to be a crucial step and significantly improved the performance. Secondly, grouping subjects and performing classification inside a group helped to improve performance and decrease inter-subject variability. Finally, we show that including handcrafted features as input to a deep learning (DL) network improves the classification performance further. Together, these findings indicate that heart rate time series can be utilized for classification tasks like predicting activity. However, normalization or grouping techniques need to be chosen carefully to minimize the issue of subject variability.
From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Models

Passively collected behavioral health data from ubiquitous sensors holds significant promise to provide mental health professionals insights from patient's daily lives; however, developing analysis tools to use this data in clinical practice requires addressing challenges of generalization across devices and weak or ambiguous correlations between the measured signals and an individual's mental health. To address these challenges, we take a novel approach that leverages large language models (LLMs) to synthesize clinically useful insights from multi-sensor data. We develop chain of thought prompting methods that use LLMs to generate reasoning about how trends in data such as step count and sleep relate to conditions like depression and anxiety. We first demonstrate binary depression classification with LLMs achieving accuracies of 61.1% which exceed the state of the art. While it is not robust for clinical use, this leads us to our key finding: even more impactful and valued than classification is a new human-AI collaboration approach in which clinician experts interactively query these tools and combine their domain expertise and context about the patient with AI generated reasoning to support clinical decision-making. We find models like GPT-4 correctly reference numerical data 75% of the time, and clinician participants express strong interest in using this approach to interpret self-tracking data.
Text-Guided Texturing by Synchronized Multi-View Diffusion

This paper introduces a novel approach to synthesize texture to dress up a given 3D object, given a text prompt. Based on the pretrained text-to-image (T2I) diffusion model, existing methods usually employ a project-and-inpaint approach, in which a view of the given object is first generated and warped to another view for inpainting. But it tends to generate inconsistent texture due to the asynchronous diffusion of multiple views. We believe such asynchronous diffusion and insufficient information sharing among views are the root causes of the inconsistent artifact. In this paper, we propose a synchronized multi-view diffusion approach that allows the diffusion processes from different views to reach a consensus of the generated content early in the process, and hence ensures the texture consistency. To synchronize the diffusion, we share the denoised content among different views in each denoising step, specifically blending the latent content in the texture domain from views with overlap. Our method demonstrates superior performance in generating consistent, seamless, highly detailed textures, comparing to state-of-the-art methods.
Comment on"Ironing, sweeping, and multidimensional screening''

In their study of price discrimination for a monopolist selling heterogeneous products to consumers having private information about their own multidimensional types, Rochet and Chon\'e (1998) discovered a new form of screening in which consumers with intermediate types are bunched together into isochoice groups of various dimensions incentivized to purchase the same product. They analyzed a particular example involving customer types distributed uniformly over the unit square. For this example, we prove that their proposed solution is not selfconsistent, and we indicate how consistency can be restored.
Non-deterministic linear thresholding systems reveal their deterministic origins

Linear thresholding systems have been used as a model of neural activation and have more recently been proposed as a model of gene activation. Deterministic linear thresholding systems can be turned into non-deterministic systems by the introduction of noise. Under mild conditions on the noise, we show that the deterministic model can be deduced from the probabilities of the non-deterministic model.
Quantum Sensing of Magnetic Fields with Molecular Spins

Spins are prototypical systems with the potential to probe magnetic fields down to the atomic scale limit. Exploiting their quantum nature through appropriate sensing protocols allows to enlarge their applicability to fields not always accessible by classical sensors. Here we first show that quantum sensing protocols for AC magnetic fields can be implemented on molecular spin ensembles embedded into hybrid quantum circuits. We then show that, using only echo detection at microwave frequency and no optical readout, Dynamical Decoupling protocols synchronized with the AC magnetic fields can enhance the sensitivity up to $S = 10^{-10}-10^{-9}T/\sqrt{Hz}$ with a low (4-5) number of applied pulses. These results paves the way for the development of strategies to exploit molecular spins as quantum sensors.
AC Power Flow Informed Parameter Learning for DC Power Flow Network Equivalents

This paper presents an algorithm to optimize the parameters of power systems equivalents to enhance the accuracy of the DC power flow approximation in reduced networks. Based on a zonal division of the network, the algorithm produces a reduced power system equivalent that captures inter-zonal flows with aggregated buses and equivalent transmission lines. The algorithm refines coefficient and bias parameters for the DC power flow model of the reduced network, aiming to minimize discrepancies between inter-zonal flows in DC and AC power flow results. Using optimization methods like Broyden-Fletcher-Goldfarb-Shanno (BFGS), Limited-memory BFGS (L-BFGS), and Truncated Newton Conjugate-Gradient (TNC) in an offline training phase, these parameters boost the accuracy of online DC power flow computations. In contrast to existing network equivalencing methods, the proposed algorithm optimizes accuracy over a specified range of operation as opposed to only considering a single nominal point. Numerical tests demonstrate substantial accuracy improvements over traditional equivalencing and approximation methods.
LM-Cocktail: Resilient Tuning of Language Models via Model Merging

The pre-trained language models are continually fine-tuned to better support downstream applications. However, this operation may result in significant performance degeneration on general tasks beyond the targeted domain. To overcome this problem, we propose LM-Cocktail which enables the fine-tuned model to stay resilient in general perspectives. Our method is conducted in the form of model merging, where the fine-tuned language model is merged with the pre-trained base model or the peer models from other domains through weighted average. Despite simplicity, LM-Cocktail is surprisingly effective: the resulted model is able to achieve a strong empirical performance in the whole scope of general tasks while preserving a superior capacity in its targeted domain. We conduct comprehensive experiments with LLama and BGE model on popular benchmarks, including FLAN, MMLU, MTEB, whose results validate the efficacy of our proposed method. The code and checkpoints are available at https://github.com/FlagOpen/FlagEmbedding/tree/master/LM_Cocktail.
AnimateAnything: Fine-Grained Open Domain Image Animation with Motion Guidance

Image animation is a key task in computer vision which aims to generate dynamic visual content from static image. Recent image animation methods employ neural based rendering technique to generate realistic animations. Despite these advancements, achieving fine-grained and controllable image animation guided by text remains challenging, particularly for open-domain images captured in diverse real environments. In this paper, we introduce an open domain image animation method that leverages the motion prior of video diffusion model. Our approach introduces targeted motion area guidance and motion strength guidance, enabling precise control the movable area and its motion speed. This results in enhanced alignment between the animated visual elements and the prompting text, thereby facilitating a fine-grained and interactive animation generation process for intricate motion sequences. We validate the effectiveness of our method through rigorous experiments on an open-domain dataset, with the results showcasing its superior performance. Project page can be found at https://animationai.github. io/AnimateAnything.
Enhancing Scene Graph Generation with Hierarchical Relationships and Commonsense Knowledge

This work presents an enhanced approach to generating scene graphs by incorporating a relationship hierarchy and commonsense knowledge. Specifically, we propose a Bayesian classification head that exploits an informative hierarchical structure. It jointly predicts the super-category or type of relationship between the two objects, along with the detailed relationship under each super-category. We design a commonsense validation pipeline that uses a large language model to critique the results from the scene graph prediction system and then use that feedback to enhance the model performance. The system requires no external large language model assistance at test time, making it more convenient for practical applications. Experiments on the Visual Genome and the OpenImage V6 datasets demonstrate that harnessing hierarchical relationships enhances the model performance by a large margin. The proposed Bayesian head can also be incorporated as a portable module in existing scene graph generation algorithms to improve their results. In addition, the commonsense validation enables the model to generate an extensive set of reasonable predictions beyond dataset annotations.
Camera-Independent Single Image Depth Estimation from Defocus Blur

Monocular depth estimation is an important step in many downstream tasks in machine vision. We address the topic of estimating monocular depth from defocus blur which can yield more accurate results than the semantic based depth estimation methods. The existing monocular depth from defocus techniques are sensitive to the particular camera that the images are taken from. We show how several camera-related parameters affect the defocus blur using optical physics equations and how they make the defocus blur depend on these parameters. The simple correction procedure we propose can alleviate this problem which does not require any retraining of the original model. We created a synthetic dataset which can be used to test the camera independent performance of depth from defocus blur models. We evaluate our model on both synthetic and real datasets (DDFF12 and NYU depth V2) obtained with different cameras and show that our methods are significantly more robust to the changes of cameras. Code: https://github.com/sleekEagle/defocus_camind. git
Efficient Spatial Dataset Search over Multiple Data Sources

In this paper, we investigate a novel spatial dataset search paradigm over multiple spatial data sources, which enables users to conduct join and union searches seamlessly. Specifically, we define two search problems called Maximum Intersection Query (MIQ) and Maximum Coverage Query with a Connection constraint (MCQC). To address these problems, we propose a unified Multi-source Spatial Dataset Search (MSDS) framework. In MSDS, we design a multi-layer index to accelerate the MIQ and MCQC. In addition, we prove that the MCQC is NP-hard and design two greedy algorithms to solve the problem. To deal with the constant update of spatial datasets in each data source, we design a dynamic index updating strategy and optimize search algorithms to reduce communication costs and improve search efficiency. We evaluate the efficiency of MSDS on five real-world data sources, and the experimental results show that our framework is able to achieve a significant reduction in running time and communication cost.
Jordan correspondence and block distribution of characters

We complete the determination of the $\ell$-block distribution of characters for quasi-simple exceptional groups of Lie type up to some minor ambiguities relating to non-uniqueness of Jordan decomposition. For this, we first determine the $\ell$-block distribution for finite reductive groups whose ambient algebraic group defined in characteristic different from $\ell$ has connected centre. As a consequence we derive a compatibility between $\ell$-blocks, $e$-Harish-Chandra series and Jordan decomposition. Further we apply our results to complete the proof of Robinson's conjecture on defects of characters.
Turbulence modulation by suspended finite-sized particles -- Towards physics-based multiphase subgrid modeling

The presence of a dispersed phase substantially modifies small-scale turbulence. However, there has not been a comprehensive mechanistically-based understanding to predict turbulence modulation. Based on the energy flux balance, we propose a theoretical model to predict the turbulent kinetic energy modulation in isotropic turbulence due to the dispersed phase. The comparison between model predictions and results from particle-resolved simulations and high-fidelity experiments validates the performance of the model over a wide range of turbulence and particle parameters.
Self-Supervised Music Source Separation Using Vector-Quantized Source Category Estimates

Music source separation is focused on extracting distinct sonic elements from composite tracks. Historically, many methods have been grounded in supervised learning, necessitating labeled data, which is occasionally constrained in its diversity. More recent methods have delved into N-shot techniques that utilize one or more audio samples to aid in the separation. However, a challenge with some of these methods is the necessity for an audio query during inference, making them less suited for genres with varied timbres and effects. This paper offers a proof-of-concept for a self-supervised music source separation system that eliminates the need for audio queries at inference time. In the training phase, while it adopts a query-based approach, we introduce a modification by substituting the continuous embedding of query audios with Vector Quantized (VQ) representations. Trained end-to-end with up to N classes as determined by the VQ's codebook size, the model seeks to effectively categorise instrument classes. During inference, the input is partitioned into N sources, with some potentially left unutilized based on the mix's instrument makeup. This methodology suggests an alternative avenue for considering source separation across diverse music genres. We provide examples and additional results online.