Miracle-dz/UniFAR-FTUs huggingface dataset:
A Boundary Element Method for the Analysis of Nonlinear Sloshing in Three-Dimensional Containers.

A new computational method has been developed for the analysis of three-dimensional large-amplitude motion of liquids with free surfaces in moving containers. The problem is formulated mathematically as a nonlinear initial-boundary value problem under the assumption of irrotational flow of an inviscid fluid. The free surface is moved during each time step in a Lagrangian manner, and its new position is calculated using a forward-time Taylor series expansion. One of the key features of the present method exists in the algorithm to compute the values of Lagrangian time derivatives in the Taylor series. To evaluate these derivatives, a boundary element method is used. The use of the Taylor series expansion has enabled us to employ a variable time-stepping method. The size of a time increment is determined at each time step so that the remainders of the truncated Taylor series should be equal to or less than a given small error limit. Such a variable time-stepping technique has made a great contribution to numerically stable computations. As a numerical example, a swirl motion of liquid in a cylindrical container subjected to a forced horizontal oscillation has been analyzed.
Higgs Inflation and the Electroweak Gauge Sector

We introduce a new method that allows for the Higgs to be the inflaton. That is, we let the Higgs be a pseudo-Nambu-Goldstone (pNG) boson of a global coset symmetry $G/H$ that spontaneously breaks at an energy scale $\sim 4\pi f$ and give it a suitable $SU(2) \subset G$ Chern-Simons interaction, with $\beta$ the dimensionless Chern-Simons coupling strength and $f$ an $SU(2)$ decay constant. As a result, slow-roll inflation occurs via $SU(2)$-induced friction down a steep sinusoidal potential. In order to obey electroweak $SU(2)_{\rm L}\times U(1)_Y$ symmetry, the lowest-order Chern-Simons interaction is required to be quadratic in the Higgs with coupling strength $\propto \beta^2/f^2$. Higher-order interaction terms keep the full Lagrangian nearly invariant under the approximate pNG shift symmetry. Employing the simplest symmetry coset $SU(5)/SO(5)$, $N$ $e$-folds of inflation occur when $N \approx 60 \left(g/0.64\right)^2\left[\beta/\left(3\times 10^6\right)\right]^{8/3}\left[f/\left(5\times 10^{11}\ {\rm GeV}\right)\right]^{2/3}$, with $g$ the weak isospin gauge coupling constant. Small values of the decay constant, $f \lesssim 5 \times 10^{11} {\rm GeV}$, which are needed to address the Higgs hierarchy problem, are ruled out by electric dipole measurements and so successfully explaining inflation requires large $\beta$. We discuss possible methods to achieve such large couplings and other alternative Higgs inflation scenarios outside the standard modified-gravity framework.
AutoML Systems For Medical Imaging

The integration of machine learning in medical image analysis can greatly enhance the quality of healthcare provided by physicians. The combination of human expertise and computerized systems can result in improved diagnostic accuracy. An automated machine learning approach simplifies the creation of custom image recognition models by utilizing neural architecture search and transfer learning techniques. Medical imaging techniques are used to non-invasively create images of internal organs and body parts for diagnostic and procedural purposes. This article aims to highlight the potential applications, strategies, and techniques of AutoML in medical imaging through theoretical and empirical evidence.
Capturing (Optimal) Relaxed Plans with Stable and Supported Models of Logic Programs

We establish a novel relation between delete-free planning, an important task for the AI planning community also known as relaxed planning, and logic programming. We show that given a planning problem, all subsets of actions that could be ordered to produce relaxed plans for the problem can be bijectively captured with stable models of a logic program describing the corresponding relaxed planning problem. We also consider the supported model semantics of logic programs, and introduce one causal and one diagnostic encoding of the relaxed planning problem as logic programs, both capturing relaxed plans with their supported models. Our experimental results show that these new encodings can provide major performance gain when computing optimal relaxed plans, with our diagnostic encoding outperforming state-of-the-art approaches to relaxed planning regardless of the given time limit when measured on a wide collection of STRIPS planning benchmarks.
A multi-band AGN-SFG classifier for extragalactic radio surveys using machine learning

Extragalactic radio continuum surveys play an increasingly more important role in galaxy evolution and cosmology studies. While radio galaxies and radio quasars dominate at the bright end, star-forming galaxies (SFGs) and radio-quiet Active Galactic Nuclei (AGNs) are more common at fainter flux densities. Our aim is to develop a machine learning classifier that can efficiently and reliably separate AGNs and SFGs in radio continuum surveys. We perform supervised classification of SFGs vs AGNs using the Light Gradient Boosting Machine (LGBM) on three LOFAR Deep Fields (Lockman Hole, Bootes and ELAIS-N1), which benefit from a wide range of high-quality multi-wavelength data and classification labels derived from extensive spectral energy distribution (SED) analyses. Our trained model has a precision of 0.92(0.01) and a recall of 0.87(0.02) for SFGs. For AGNs, the model has slightly worse performance, with a precision of 0.87(0.02) and recall of 0.78(0.02). These results demonstrate that our trained model can successfully reproduce the classification labels derived from detailed SED analysis. The model performance decreases towards higher redshifts, mainly due to smaller training sample sizes. To make the classifier more adaptable to other radio galaxy surveys, we also investigate how our classifier performs with a poorer multi-wavelength sampling of the SED. In particular, we find that the far-infrared (FIR) and radio bands are of great importance. We also find that higher S/N in some photometric bands leads to a significant boost in the model's performance. In addition to using the 150 MHz radio data, our model can also be used with 1.4 GHz radio data. Converting 1.4 GHz to 150 MHz radio data reduces performance by about 4% in precision and 3% in recall. The final trained model is publicly available at https://github.com/Jesper-Karsten/MBASC
Bayesian Optimisation of Functions on Graphs

The increasing availability of graph-structured data motivates the task of optimising over functions defined on the node set of graphs. Traditional graph search algorithms can be applied in this case, but they may be sample-inefficient and do not make use of information about the function values; on the other hand, Bayesian optimisation is a class of promising black-box solvers with superior sample efficiency, but it has been scarcely been applied to such novel setups. To fill this gap, we propose a novel Bayesian optimisation framework that optimises over functions defined on generic, large-scale and potentially unknown graphs. Through the learning of suitable kernels on graphs, our framework has the advantage of adapting to the behaviour of the target function. The local modelling approach further guarantees the efficiency of our method. Extensive experiments on both synthetic and real-world graphs demonstrate the effectiveness of the proposed optimisation framework.
On control synthesis for uncertain dynamical discrete-time systems through polyhedral techniques

Problems of feedback terminal target control for linear discrete-time systems without and with uncertainties are considered. We continue the development of methods of control synthesis using polyhedral (parallelotope-valued) solvability tubes. The cases without uncertainties, with additive parallelotope-bounded uncertainties, and also with interval uncertainties in coefficients of the system are considered. Also the same systems under state constraints are considered. Nonlinear recurrent relations are presented for polyhedral solvability tubes for each of the mentioned cases. Two types of control strategies, which can be calculated on the base of the mentioned tubes, are proposed. Controls of the second type can be calculated by explicit formulas. Results of computer simulations are presented.
An Image Compensation Method Based on the Illumination Component Correction

For illumination compensation of images,the traditional method based on gray-level mapping is defective since the resultant images are unnatural in grey level transition and short of three-dimensional sense.This paper proposes to separate the image into the illumination and reflection components by image smoothing based on a knowledge that the illumination component usually changes slowly in the image;then a Gamma correction can be made only to the illumination component rather than the gray image as a whole.Experimental results show that the proposed method performs better in image compensation.
Weakly Supervised 3D Object Detection with Multi-Stage Generalization

With the rapid development of large models, the need for data has become increasingly crucial. Especially in 3D object detection, costly manual annotations have hindered further advancements. To reduce the burden of annotation, we study the problem of achieving 3D object detection solely based on 2D annotations. Thanks to advanced 3D reconstruction techniques, it is now feasible to reconstruct the overall static 3D scene. However, extracting precise object-level annotations from the entire scene and generalizing these limited annotations to the entire scene remain challenges. In this paper, we introduce a novel paradigm called BA$^2$-Det, encompassing pseudo label generation and multi-stage generalization. We devise the DoubleClustering algorithm to obtain object clusters from reconstructed scene-level points, and further enhance the model's detection capabilities by developing three stages of generalization: progressing from complete to partial, static to dynamic, and close to distant. Experiments conducted on the large-scale Waymo Open Dataset show that the performance of BA$^2$-Det is on par with the fully-supervised methods using 10% annotations. Additionally, using large raw videos for pretraining,BA$^2$-Det can achieve a 20% relative improvement on the KITTI dataset. The method also has great potential for detecting open-set 3D objects in complex scenes. Project page: https://ba2det.site.
Online Multi-Contact Receding Horizon Planning via Value Function Approximation

Planning multi-contact motions in a receding horizon fashion requires a value function to guide the planning with respect to the future, e.g., building momentum to traverse large obstacles. Traditionally, the value function is approximated by computing trajectories in a prediction horizon (never executed) that foresees the future beyond the execution horizon. However, given the non-convex dynamics of multi-contact motions, this approach is computationally expensive. To enable online Receding Horizon Planning (RHP) of multi-contact motions, we find efficient approximations of the value function. Specifically, we propose a trajectory-based and a learning-based approach. In the former, namely RHP with Multiple Levels of Model Fidelity, we approximate the value function by computing the prediction horizon with a convex relaxed model. In the latter, namely Locally-Guided RHP, we learn an oracle to predict local objectives for locomotion tasks, and we use these local objectives to construct local value functions for guiding a short-horizon RHP. We evaluate both approaches in simulation by planning centroidal trajectories of a humanoid robot walking on moderate slopes, and on large slopes where the robot cannot maintain static balance. Our results show that locally-guided RHP achieves the best computation efficiency (95\%-98.6\% cycles converge online). This computation advantage enables us to demonstrate online receding horizon planning of our real-world humanoid robot Talos walking in dynamic environments that change on-the-fly.
How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources

In this work we explore recent advances in instruction-tuning language models on a range of open instruction-following datasets. Despite recent claims that open models can be on par with state-of-the-art proprietary models, these claims are often accompanied by limited evaluation, making it difficult to compare models across the board and determine the utility of various resources. We provide a large set of instruction-tuned models from 6.7B to 65B parameters in size, trained on 12 instruction datasets ranging from manually curated (e.g., OpenAssistant) to synthetic and distilled (e.g., Alpaca) and systematically evaluate them on their factual knowledge, reasoning, multilinguality, coding, and open-ended instruction following abilities through a collection of automatic, model-based, and human-based metrics. We further introduce T\"ulu, our best performing instruction-tuned model suite finetuned on a combination of high-quality open resources. Our experiments show that different instruction-tuning datasets can uncover or enhance specific skills, while no single dataset (or combination) provides the best performance across all evaluations. Interestingly, we find that model and human preference-based evaluations fail to reflect differences in model capabilities exposed by benchmark-based evaluations, suggesting the need for the type of systemic evaluation performed in this work. Our evaluations show that the best model in any given evaluation reaches on average 87% of ChatGPT performance, and 73% of GPT-4 performance, suggesting that further investment in building better base models and instruction-tuning data is required to close the gap. We release our instruction-tuned models, including a fully finetuned 65B T\"ulu, along with our code, data, and evaluation framework at https://github.com/allenai/open-instruct to facilitate future research.
Achieving Consensus over Compact Submanifolds

We consider the consensus problem in a decentralized network, focusing on a compact submanifold that acts as a nonconvex constraint set. By leveraging the proximal smoothness of the compact submanifold, which encompasses the local singleton property and the local Lipschitz continuity of the projection operator on the manifold, and establishing the connection between the projection operator and general retraction, we show that the Riemannian gradient descent with a unit step size has locally linear convergence if the network has a satisfactory level of connectivity. Moreover, based on the geometry of the compact submanifold, we prove that a convexity-like regularity condition, referred to as the restricted secant inequality, always holds in an explicitly characterized neighborhood around the solution set of the nonconvex consensus problem. By leveraging this restricted secant inequality and imposing a weaker connectivity requirement on the decentralized network, we present a comprehensive analysis of the linear convergence of the Riemannian gradient descent, taking into consideration appropriate initialization and step size. Furthermore, if the network is well connected, we demonstrate that the local Lipschitz continuity endowed by proximal smoothness is a sufficient condition for the restricted secant inequality, thus contributing to the local error bound. We believe that our established results will find more application in the consensus problems over a more general proximally smooth set. Numerical experiments are conducted to validate our theoretical findings.
Automatic retrieval of corresponding US views in longitudinal examinations

Skeletal muscle atrophy is a common occurrence in critically ill patients in the intensive care unit (ICU) who spend long periods in bed. Muscle mass must be recovered through physiotherapy before patient discharge and ultrasound imaging is frequently used to assess the recovery process by measuring the muscle size over time. However, these manual measurements are subject to large variability, particularly since the scans are typically acquired on different days and potentially by different operators. In this paper, we propose a self-supervised contrastive learning approach to automatically retrieve similar ultrasound muscle views at different scan times. Three different models were compared using data from 67 patients acquired in the ICU. Results indicate that our contrastive model outperformed a supervised baseline model in the task of view retrieval with an AUC of 73.52% and when combined with an automatic segmentation model achieved 5.7%+/-0.24% error in cross-sectional area. Furthermore, a user study survey confirmed the efficacy of our model for muscle view retrieval.
JGAT: a joint spatio-temporal graph attention model for brain decoding

The decoding of brain neural networks has been an intriguing topic in neuroscience for a well-rounded understanding of different types of brain disorders and cognitive stimuli. Integrating different types of connectivity, e.g., Functional Connectivity (FC) and Structural Connectivity (SC), from multi-modal imaging techniques can take their complementary information into account and therefore have the potential to get better decoding capability. However, traditional approaches for integrating FC and SC overlook the dynamical variations, which stand a great chance to over-generalize the brain neural network. In this paper, we propose a Joint kernel Graph Attention Network (JGAT), which is a new multi-modal temporal graph attention network framework. It integrates the data from functional Magnetic Resonance Images (fMRI) and Diffusion Weighted Imaging (DWI) while preserving the dynamic information at the same time. We conduct brain-decoding tasks with our JGAT on four independent datasets: three of 7T fMRI datasets from the Human Connectome Project (HCP) and one from animal neural recordings. Furthermore, with Attention Scores (AS) and Frame Scores (FS) computed and learned from the model, we can locate several informative temporal segments and build meaningful dynamical pathways along the temporal domain for the HCP datasets. The URL to the code of JGAT model: https://github.com/BRAINML-GT/JGAT.
An introduction to the analysis of gradients systems

The present notes provide an extended version of a small lecture course given at the Humboldt Universit\"at zu Berlin in the Winter Term 2022/23 (of 36 hours). The material starting in Section 5.4 was added afterwards. The aim of these notes to give an introductory overview on the analytical approaches for gradient-flow equations in Hilbert spaces, Banach spaces, and metric spaces and to show that on the first entry level these theories have a lot in common. The theories and their specific setups are illustrated by suitable examples and counterexamples.
Unified Embedding Based Personalized Retrieval in Etsy Search

Embedding-based neural retrieval is a prevalent approach to address the semantic gap problem which often arises in product search on tail queries. In contrast, popular queries typically lack context and have a broad intent where additional context from users historical interaction can be helpful. In this paper, we share our novel approach to address both: the semantic gap problem followed by an end to end trained model for personalized semantic retrieval. We propose learning a unified embedding model incorporating graph, transformer and term-based embeddings end to end and share our design choices for optimal tradeoff between performance and efficiency. We share our learnings in feature engineering, hard negative sampling strategy, and application of transformer model, including a novel pre-training strategy and other tricks for improving search relevance and deploying such a model at industry scale. Our personalized retrieval model significantly improves the overall search experience, as measured by a 5.58% increase in search purchase rate and a 2.63% increase in site-wide conversion rate, aggregated across multiple A/B tests - on live traffic.
Federated Learning under Covariate Shifts with Generalization Guarantees

This paper addresses intra-client and inter-client covariate shifts in federated learning (FL) with a focus on the overall generalization performance. To handle covariate shifts, we formulate a new global model training paradigm and propose Federated Importance-Weighted Empirical Risk Minimization (FTW-ERM) along with improving density ratio matching methods without requiring perfect knowledge of the supremum over true ratios. We also propose the communication-efficient variant FITW-ERM with the same level of privacy guarantees as those of classical ERM in FL. We theoretically show that FTW-ERM achieves smaller generalization error than classical ERM under certain settings. Experimental results demonstrate the superiority of FTW-ERM over existing FL baselines in challenging imbalanced federated settings in terms of data distribution shifts across clients.
Actively Supervised Clustering for Open Relation Extraction

Current clustering-based Open Relation Extraction (OpenRE) methods usually adopt a two-stage pipeline, which simultaneously learns relation representations and assignments in the first stage, then manually labels relation for each cluster.However, unsupervised objectives struggle to explicitly optimize clusters to align with relational semantics, and the number of clusters K has to be supplied in advance.In this paper, we present a novel setting, named actively supervised clustering for OpenRE. Our insight lies in that clustering learning and relation labeling can be performed simultaneously, which provides the necessary guidance for clustering without a significant increase in human effort. Along with this setting, we propose an active labeling strategy tailored for clustering. Instead of only focusing on improving the clustering of relations that have been discovered, our strategy is encouraged to discover new relations through diversity regularization. This is particularly beneficial for long-tail relations in the real world.Experimental results show that our method is able to discover almost all relational clusters in the data and improve the SOTA methods by 13.8% and 10.6%, on two datasets respectively.
Enabling tabular deep learning when d ≫ n with an auxiliary knowledge graph

Machine learning models exhibit strong performance on datasets with abundant labeled samples. However, for tabular datasets with extremely high $d$-dimensional features but limited $n$ samples (i.e. $d \gg n$), machine learning models struggle to achieve strong performance due to the risk of overfitting. Here, our key insight is that there is often abundant, auxiliary domain information describing input features which can be structured as a heterogeneous knowledge graph (KG). We propose PLATO, a method that achieves strong performance on tabular data with $d \gg n$ by using an auxiliary KG describing input features to regularize a multilayer perceptron (MLP). In PLATO, each input feature corresponds to a node in the auxiliary KG. In the MLP's first layer, each input feature also corresponds to a weight vector. PLATO is based on the inductive bias that two input features corresponding to similar nodes in the auxiliary KG should have similar weight vectors in the MLP's first layer. PLATO captures this inductive bias by inferring the weight vector for each input feature from its corresponding node in the KG via a trainable message-passing function. Across 6 $d \gg n$ datasets, PLATO outperforms 13 state-of-the-art baselines by up to 10.19%.