Miracle-dz/UniFAR-FTUs huggingface dataset:
JADES + JEMS: A Detailed Look at the Buildup of Central Stellar Cores and Suppression of Star Formation in Galaxies at Redshifts 3

We present a spatially resolved study of stellar populations in 6 galaxies with stellar masses $M_*\sim10^{10}M_\odot$ at $z\sim3.7$ using 14-filter JWST/NIRCam imaging from the JADES and JEMS surveys. The 6 galaxies are visually selected to have clumpy substructures with distinct colors over rest-frame $3600-4100$\r{A}, including a bright dominant stellar core that is close to their stellar-light centroids. With 23-filter photometry from HST to JWST, we measure the stellar-population properties of individual structural components via SED fitting using Prospector. We find that the central stellar cores are $\gtrsim2$ times more massive than the Toomre mass, indicating they may not form via in-situ fragmentation. The stellar cores have stellar ages of $0.4-0.7$ Gyr that are similar to the timescale of clump inward migration due to dynamical friction, suggesting that they likely instead formed through the coalescence of giant stellar clumps. While they have not yet quenched, the 6 galaxies are below the star-forming main sequence by $0.2-0.7$ dex. Within each galaxy, we find that the specific star formation rate is lower in the central stellar core, and the stellar-mass surface density of the core is already similar to quenched galaxies of the same masses and redshifts. Meanwhile, the stellar ages of the cores are either comparable to or younger than the extended, smooth parts of the galaxies. Our findings are consistent with model predictions of the gas-rich compaction scenario for the buildup of galaxies' central regions at high redshifts. We are likely witnessing the coeval formation of dense central cores, along with the onset of galaxy-wide quenching at $z>3$.
Federated Learning of Gboard Language Models with Differential Privacy

We train and deploy language models (LMs) with federated learning (FL) and differential privacy (DP) in Google Keyboard (Gboard). The recent DP-Follow the Regularized Leader (DP-FTRL) algorithm is applied to achieve meaningfully formal DP guarantees without requiring uniform sampling of clients. To provide favorable privacy-utility trade-offs, we introduce a new client participation criterion and discuss the implication of its configuration in large scale systems. We show how quantile-based clip estimation can be combined with DP-FTRL to adaptively choose the clip norm during training or reduce the hyperparameter tuning in preparation of training. With the help of pretraining on public data, we trained and deployed more than fifteen Gboard LMs that achieve high utility and $\rho-$zCDP privacy guarantees with $\rho \in (0.3, 2)$, with one model additionally trained with secure aggregation.We summarize our experience and provide concrete suggestions on DP training for practitioners.
Optimal Dynamic Subset Sampling: Theory and Applications

We study the fundamental problem of sampling independent events, called subset sampling. Specifically, consider a set of n distinct events S=x1, …, xn, in which each event xi has an associated probability p(xi). The subset sampling problem aims to sample a subset T ⊆ S, such that every xi is independently included in T with probability p(xi). A naive solution is to flip a coin for each event, which takes O(n) time. However, an ideal solution is a data structure that allows drawing a subset sample in time proportional to the expected output size μ=∑i=1n p(xi), which can be significantly smaller than n in many applications. The subset sampling problem serves as an important building block in many tasks and has been the subject of various research for more than a decade. However, the majority of existing subset sampling methods are designed for a static setting, where the events in set S or their associated probabilities remain unchanged over time. These algorithms incur either large query time or update time in a dynamic setting despite the ubiquitous time-evolving events with varying probabilities in real life. Therefore, it is a pressing need, but still, an open problem, to design efficient dynamic subset sampling algorithms. In this paper, we propose ODSS, the first optimal dynamic subset sampling algorithm. The expected query time and update time of ODSS are both optimal, matching the lower bounds of the subset sampling problem. We present a nontrivial theoretical analysis to demonstrate the superiority of ODSS. We also conduct comprehensive experiments to empirically evaluate the performance of ODSS. Moreover, we apply ODSS to a concrete application: Influence Maximization. We empirically show that our ODSS can improve the complexities of existing Influence Maximization algorithms on large real-world evolving social networks.
Stabilized immersed isogeometric analysis for the Navier-Stokes-Cahn-Hilliard equations, with applications to binary-fluid flow through porous media

Binary-fluid flows can be modeled using the Navier-Stokes-Cahn-Hilliard equations, which represent the boundary between the fluid constituents by a diffuse interface. The diffuse-interface model allows for complex geometries and topological changes of the binary-fluid interface. In this work, we propose an immersed isogeometric analysis framework to solve the Navier-Stokes-Cahn-Hilliard equations on domains with geometrically complex external binary-fluid boundaries. The use of optimal-regularity B-splines results in a computationally efficient higher-order method. The key features of the proposed framework are a generalized Navier-slip boundary condition for the tangential velocity components, Nitsche's method for the convective impermeability boundary condition, and skeleton- and ghost-penalties to guarantee stability. A binary-fluid Taylor-Couette flow is considered for benchmarking. Porous medium simulations demonstrate the ability of the immersed isogeometric analysis framework to model complex binary-fluid flow phenomena such as break-up and coalescence in complex geometries.
Statistically Optimal K-means Clustering via Nonnegative Low-rank Semidefinite Programming

$K$-means clustering is a widely used machine learning method for identifying patterns in large datasets. Semidefinite programming (SDP) relaxations have recently been proposed for solving the $K$-means optimization problem that enjoy strong statistical optimality guarantees, but the prohibitive cost of implementing an SDP solver renders these guarantees inaccessible to practical datasets. By contrast, nonnegative matrix factorization (NMF) is a simple clustering algorithm that is widely used by machine learning practitioners, but without a solid statistical underpinning nor rigorous guarantees. In this paper, we describe an NMF-like algorithm that works by solving a nonnegative low-rank restriction of the SDP relaxed $K$-means formulation using a nonconvex Burer--Monteiro factorization approach. The resulting algorithm is just as simple and scalable as state-of-the-art NMF algorithms, while also enjoying the same strong statistical optimality guarantees as the SDP. In our experiments, we observe that our algorithm achieves substantially smaller mis-clustering errors compared to the existing state-of-the-art.
Chatbots put to the test in math and logic problems: A preliminary comparison and assessment of ChatGPT-3.5, ChatGPT-4, and Google Bard

A comparison between three chatbots which are based on large language models, namely ChatGPT-3.5, ChatGPT-4 and Google Bard is presented, focusing on their ability to give correct answers to mathematics and logic problems. In particular, we check their ability to Understand the problem at hand; Apply appropriate algorithms or methods for its solution; and Generate a coherent response and a correct answer. We use 30 questions that are clear, without any ambiguities, fully described with plain text only, and have a unique, well defined correct answer. The questions are divided into two sets of 15 each. The questions of Set A are 15"Original"problems that cannot be found online, while Set B contains 15"Published"problems that one can find online, usually with their solution. Each question is posed three times to each chatbot. The answers are recorded and discussed, highlighting their strengths and weaknesses. It has been found that for straightforward arithmetic, algebraic expressions, or basic logic puzzles, chatbots may provide accurate solutions, although not in every attempt. However, for more complex mathematical problems or advanced logic tasks, their answers, although written in a usually"convincing"way, may not be reliable. Consistency is also an issue, as many times a chatbot will provide conflicting answers when given the same question more than once. A comparative quantitative evaluation of the three chatbots is made through scoring their final answers based on correctness. It was found that ChatGPT-4 outperforms ChatGPT-3.5 in both sets of questions. Bard comes third in the original questions of Set A, behind the other two chatbots, while it has the best performance (first place) in the published questions of Set B. This is probably because Bard has direct access to the internet, in contrast to ChatGPT chatbots which do not have any communication with the outside world.
Dink-Net: Neural Clustering on Large Graphs

Deep graph clustering, which aims to group the nodes of a graph into disjoint clusters with deep neural networks, has achieved promising progress in recent years. However, the existing methods fail to scale to the large graph with million nodes. To solve this problem, a scalable deep graph clustering method (Dink-Net) is proposed with the idea of dilation and shrink. Firstly, by discriminating nodes, whether being corrupted by augmentations, representations are learned in a self-supervised manner. Meanwhile, the cluster centres are initialized as learnable neural parameters. Subsequently, the clustering distribution is optimized by minimizing the proposed cluster dilation loss and cluster shrink loss in an adversarial manner. By these settings, we unify the two-step clustering, i.e., representation learning and clustering optimization, into an end-to-end framework, guiding the network to learn clustering-friendly features. Besides, Dink-Net scales well to large graphs since the designed loss functions adopt the mini-batch data to optimize the clustering distribution even without performance drops. Both experimental results and theoretical analyses demonstrate the superiority of our method. Compared to the runner-up, Dink-Net achieves 9.62% NMI improvement on the ogbn-papers100M dataset with 111 million nodes and 1.6 billion edges. The source code is released at https://github.com/yueliu1999/Dink-Net. Besides, a collection (papers, codes, and datasets) of deep graph clustering is shared at https://github.com/yueliu1999/Awesome-Deep-Graph-Clustering.
Adaptive Sparsity Level during Training for Efficient Time Series Forecasting with Transformers

Efficient time series forecasting has become critical for real-world applications, particularly with deep neural networks (DNNs). Efficiency in DNNs can be achieved through sparse connectivity and reducing the model size. However, finding the sparsity level automatically during training remains a challenging task due to the heterogeneity in the loss-sparsity tradeoffs across the datasets. In this paper, we propose \enquote{\textbf{P}runing with \textbf{A}daptive \textbf{S}parsity \textbf{L}evel} (\textbf{PALS}), to automatically seek an optimal balance between loss and sparsity, all without the need for a predefined sparsity level. PALS draws inspiration from both sparse training and during-training methods. It introduces the novel"expand"mechanism in training sparse neural networks, allowing the model to dynamically shrink, expand, or remain stable to find a proper sparsity level. In this paper, we focus on achieving efficiency in transformers known for their excellent time series forecasting performance but high computational cost. Nevertheless, PALS can be applied directly to any DNN. In the scope of these arguments, we demonstrate its effectiveness also on the DLinear model. Experimental results on six benchmark datasets and five state-of-the-art transformer variants show that PALS substantially reduces model size while maintaining comparable performance to the dense model. More interestingly, PALS even outperforms the dense model, in 12 and 14 cases out of 30 cases in terms of MSE and MAE loss, respectively, while reducing 65% parameter count and 63% FLOPs on average. Our code will be publicly available upon acceptance of the paper.
Bottleneck Structure in Learned Features: Low-Dimension vs Regularity Tradeoff

Previous work has shown that DNNs with large depth $L$ and $L_{2}$-regularization are biased towards learning low-dimensional representations of the inputs, which can be interpreted as minimizing a notion of rank $R^{(0)}(f)$ of the learned function $f$, conjectured to be the Bottleneck rank. We compute finite depth corrections to this result, revealing a measure $R^{(1)}$ of regularity which bounds the pseudo-determinant of the Jacobian $\left|Jf(x)\right|_{+}$ and is subadditive under composition and addition. This formalizes a balance between learning low-dimensional representations and minimizing complexity/irregularity in the feature maps, allowing the network to learn the `right' inner dimension. Finally, we prove the conjectured bottleneck structure in the learned features as $L\to\infty$: for large depths, almost all hidden representations are approximately $R^{(0)}(f)$-dimensional, and almost all weight matrices $W_{\ell}$ have $R^{(0)}(f)$ singular values close to 1 while the others are $O(L^{-\frac{1}{2}})$. Interestingly, the use of large learning rates is required to guarantee an order $O(L)$ NTK which in turns guarantees infinite depth convergence of the representations of almost all layers.
Training an Ising Machine with Equilibrium Propagation

Ising machines, which are hardware implementations of the Ising model of coupled spins, have been influential in the development of unsupervised learning algorithms at the origins of Artificial Intelligence (AI). However, their application to AI has been limited due to the complexities in matching supervised training methods with Ising machine physics, even though these methods are essential for achieving high accuracy. In this study, we demonstrate a novel approach to train Ising machines in a supervised way through the Equilibrium Propagation algorithm, achieving comparable results to software-based implementations. We employ the quantum annealing procedure of the D-Wave Ising machine to train a fully-connected neural network on the MNIST dataset. Furthermore, we demonstrate that the machine's connectivity supports convolution operations, enabling the training of a compact convolutional network with minimal spins per neuron. Our findings establish Ising machines as a promising trainable hardware platform for AI, with the potential to enhance machine learning applications.
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets

The development of large language models (LLMs) such as ChatGPT has brought a lot of attention recently. However, their evaluation in the benchmark academic datasets remains under-explored due to the difficulty of evaluating the generative outputs produced by this model against the ground truth. In this paper, we aim to present a thorough evaluation of ChatGPT's performance on diverse academic datasets, covering tasks like question-answering, text summarization, code generation, commonsense reasoning, mathematical problem-solving, machine translation, bias detection, and ethical considerations. Specifically, we evaluate ChatGPT across 140 tasks and analyze 255K responses it generates in these datasets. This makes our work the largest evaluation of ChatGPT in NLP benchmarks. In short, our study aims to validate the strengths and weaknesses of ChatGPT in various tasks and provide insights for future research using LLMs. We also report a new emergent ability to follow multi-query instructions that we mostly found in ChatGPT and other instruction-tuned models. Our extensive evaluation shows that even though ChatGPT is capable of performing a wide variety of tasks, and may obtain impressive performance in several benchmark datasets, it is still far from achieving the ability to reliably solve many challenging tasks. By providing a thorough assessment of ChatGPT's performance across diverse NLP tasks, this paper sets the stage for a targeted deployment of ChatGPT-like LLMs in real-world applications.
Host group degeneracy in gravitational lensing time delay determination of H 0

Massive elliptical galaxies, that serve as lenses in gravitational lensing time delay measurements of the Hubble parameter H 0, often reside in a host group. We consider degeneracies in the modeling of the group halo. When the group effect on imaging can be summarized by its flexion (the next order term beyond shear in the tidal expansion), the posterior likelihood map can develop disjoint local minima, associated with an approximate discrete symmetry of a dominant flexion term. Monte-Carlo Markov Chain (MCMC) algorithms that are not designed to explore a rich posterior landscape can miss some of the minima, introducing systematic bias. We study mock data and demonstrate that the bias in H 0 can exceed 10%, and pulls the inference value of H 0 above its truth value, for a reason that can be traced to the structure of a mismodeled flexion term. MCMC algorithms that are designed to cope with a rich posterior landscape can uncover the structure. If the group is X-ray bright enough, X-ray data may also help to resolve the degeneracy, by pinpointing the group's center of mass. Finally, we show that some implementations in the literature used an inaccurate kinematical prior, mis-modeling the group velocity dispersion by as much as 20%.
Majority Voting Approach to Ransomware Detection

Crypto-ransomware remains a significant threat to governments and companies alike, with high-profile cyber security incidents regularly making headlines. Many different detection systems have been proposed as solutions to the ever-changing dynamic landscape of ransomware detection. In the majority of cases, these described systems propose a method based on the result of a single test performed on either the executable code, the process under investigation, its behaviour, or its output. In a small subset of ransomware detection systems, the concept of a scorecard is employed where multiple tests are performed on various aspects of a process under investigation and their results are then analysed using machine learning. The purpose of this paper is to propose a new majority voting approach to ransomware detection by developing a method that uses a cumulative score derived from discrete tests based on calculations using algorithmic rather than heuristic techniques. The paper describes 23 candidate tests, as well as 9 Windows API tests which are validated to determine both their accuracy and viability for use within a ransomware detection system. Using a cumulative score calculation approach to ransomware detection has several benefits, such as the immunity to the occasional inaccuracy of individual tests when making its final classification. The system can also leverage multiple tests that can be both comprehensive and complimentary in an attempt to achieve a broader, deeper, and more robust analysis of the program under investigation. Additionally, the use of multiple collaborative tests also significantly hinders ransomware from masking or modifying its behaviour in an attempt to bypass detection.
The Steenrod algebra is self-injective, and the Steenrod algebra is not self-injective

It is well-known that the Steenrod algebra $A$ is self-injective as a graded ring. We make the observation that simply changing the grading on $A$ can make it cease to be self-injective. We see also that $A$ is not self-injective as an ungraded ring. These observations follow from the failure of certain coproducts of injective $A$-modules to be injective. Hence it is natural to ask: which coproducts of graded-injective modules, over a general graded ring, remain graded-injective? We give a complete solution to that question by proving a graded generalization of Carl Faith's characterization of $\Sigma$-injective modules. Specializing again to the Steenrod algebra, we use our graded generalization of Faith's theorem to prove that the covariant embedding of graded $A_*$-comodules into graded $A$-modules preserves injectivity of bounded-above objects, but does not preserve injectivity in general.
Prediction of the 2023 Turkish Presidential Election Results Using Social Media Data

Social media platforms influence the way political campaigns are run and therefore they have become an increasingly important tool for politicians to directly interact with citizens. Previous elections in various countries have shown that social media data may significantly impact election results. In this study, we aim to predict the vote shares of parties participating in the 2023 elections in Turkey by combining social media data from various platforms together with traditional polling data. Our approach is a volume-based approach that considers the number of social media interactions rather than content. We compare several prediction models across varying time windows. Our results show that for all time windows, the ARIMAX model outperforms the other algorithms.
Stability of a Rotating Boundary Layer

This project investigates the unsteady flow around an impulsively-started rotating cylinder. An instability in the flow causes Taylor vortices to form from the initially laminar boundary layer. Experiments are carried out to compare the size of these vortices with the predictions from quasi-steady theory. We find that quasi-steady theory correctly predicts the form of the relation between the size of the vortices and the Reynolds number, but it incorrectly predicts the precise size of the vortices. We then compare the time at which the instability occurs to the quasi-steady predictions, and for Re > 50 we obtain the same result: the theory predicts the general form of the equation, but not the time at which the vortices become visible.When 20 < Re < 50 the thin boundary layer approximation is no longer valid and results deviate from this model. The centrifugal instability in this flow is analogous to the thermalor buoyancydriven instability from a flat heated plate. The extent of this analogy is investigated and evidence of plume-like behaviour is found. However, further quantitative investigations are needed before a definitive statement can be made about the outward velocity of the plumes.
Calculating the Hawking Temperatures of Kerr-Newman Black Holes in the f(R) Gravity Models with the RVB Method

In this study, we conducted a comparison between the RVB method and the conventional method discussed in previous literature for calculating the Hawking temperature of Kerr-Newman black holes under f(R) gravity\cite{9,10,11}. Our research findings are in agreement with the results presented in the literature\cite{17}, with only a variation in the integration constant. Through the comparison of these two methods, we arrived at consistent conclusions.After conducting a thorough comparison between the RVB method and the temperature Green's function method, we have found that these two approaches are fundamentally identical. This significant finding highlights the high level of consistency between the temperature Green's function method and the RVB method.