Miracle-dz/UniFAR-FTUs huggingface dataset:
A Quantum Approach to Synthetic Minority Oversampling Technique (SMOTE)

The paper proposes the Quantum-SMOTE method, a novel solution that uses quantum computing techniques to solve the prevalent problem of class imbalance in machine learning datasets. Quantum-SMOTE, inspired by the Synthetic Minority Oversampling Technique (SMOTE), generates synthetic data points using quantum processes such as swap tests and quantum rotation. The process varies from the conventional SMOTE algorithm's usage of K-Nearest Neighbors (KNN) and Euclidean distances, enabling synthetic instances to be generated from minority class data points without relying on neighbor proximity. The algorithm asserts greater control over the synthetic data generation process by introducing hyperparameters such as rotation angle, minority percentage, and splitting factor, which allow for customization to specific dataset requirements. The approach is tested on a public dataset of TelecomChurn and evaluated alongside two prominent classification algorithms, Random Forest and Logistic Regression, to determine its impact along with varying proportions of synthetic data.
Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models

Sora is a text-to-video generative AI model, released by OpenAI in February 2024. The model is trained to generate videos of realistic or imaginative scenes from text instructions and show potential in simulating the physical world. Based on public technical reports and reverse engineering, this paper presents a comprehensive review of the model's background, related technologies, applications, remaining challenges, and future directions of text-to-video AI models. We first trace Sora's development and investigate the underlying technologies used to build this"world simulator". Then, we describe in detail the applications and potential impact of Sora in multiple industries ranging from film-making and education to marketing. We discuss the main challenges and limitations that need to be addressed to widely deploy Sora, such as ensuring safe and unbiased video generation. Lastly, we discuss the future development of Sora and video generation models in general, and how advancements in the field could enable new ways of human-AI interaction, boosting productivity and creativity of video generation.
Scaling Supervised Local Learning with Augmented Auxiliary Networks

Deep neural networks are typically trained using global error signals that backpropagate (BP) end-to-end, which is not only biologically implausible but also suffers from the update locking problem and requires huge memory consumption. Local learning, which updates each layer independently with a gradient-isolated auxiliary network, offers a promising alternative to address the above problems. However, existing local learning methods are confronted with a large accuracy gap with the BP counterpart, particularly for large-scale networks. This is due to the weak coupling between local layers and their subsequent network layers, as there is no gradient communication across layers. To tackle this issue, we put forward an augmented local learning method, dubbed AugLocal. AugLocal constructs each hidden layer's auxiliary network by uniformly selecting a small subset of layers from its subsequent network layers to enhance their synergy. We also propose to linearly reduce the depth of auxiliary networks as the hidden layer goes deeper, ensuring sufficient network capacity while reducing the computational cost of auxiliary networks. Our extensive experiments on four image classification datasets (i.e., CIFAR-10, SVHN, STL-10, and ImageNet) demonstrate that AugLocal can effectively scale up to tens of local layers with a comparable accuracy to BP-trained networks while reducing GPU memory usage by around 40%. The proposed AugLocal method, therefore, opens up a myriad of opportunities for training high-performance deep neural networks on resource-constrained platforms.Code is available at https://github.com/ChenxiangMA/AugLocal.
Understanding the Dataset Practitioners Behind Large Language Model Development

As large language models (LLMs) become more advanced and impactful, it is increasingly important to scrutinize the data that they rely upon and produce. What is it to be a dataset practitioner doing this work? We approach this in two parts: first, we define the role of"dataset practitioner"by performing a retrospective analysis on the responsibilities of teams contributing to LLM development at Google. Then, we conduct semi-structured interviews with a cross-section of these practitioners (N=10). We find that data quality is the top priority. To evaluate data quality, practitioners either rely on their own intuition or write custom evaluation logic. There is a lack of consensus across practitioners on what quality is and how to evaluate it. We discuss potential reasons for this phenomenon and opportunities for alignment.
Structure of the Northern Los Angeles Basins Revealed in Teleseismic Receiver Functions from Short‐Term Nodal Seismic Arrays

We use teleseismic receiver functions computed from an ∼35 day nodal dataset recorded along three profiles in the northern basins of Los Angeles, California, to map the depth and shape of the sediment–basement interface and to identify possible deep fault offsets. The results show the Moho discontinuity, the bottom of the basement, and intermediary sedimentary layers. There are also indications of midcrustal offsets along strike of the Red Hill and Raymond faults. The results are compared with receiver functions from nearby permanent broadband stations and the 1993 Los Angeles Region Seismic Experiment (LARSE) profile. The images show that dense deployments of node-type sensors can be used to characterize basin structure in a noisy urban environment. Electronic Supplement: Example of the data recorded by the nodal array in the San Gabriel basin for the Bolivia event.
Automated Floodwater Depth Estimation Using Large Multimodal Model for Rapid Flood Mapping

Information on the depth of floodwater is crucial for rapid mapping of areas affected by floods. However, previous approaches for estimating floodwater depth, including field surveys, remote sensing, and machine learning techniques, can be time-consuming and resource-intensive. This paper presents an automated and fast approach for estimating floodwater depth from on-site flood photos. A pre-trained large multimodal model, GPT-4 Vision, was used specifically for estimating floodwater. The input data were flooding photos that contained referenced objects, such as street signs, cars, people, and buildings. Using the heights of the common objects as references, the model returned the floodwater depth as the output. Results show that the proposed approach can rapidly provide a consistent and reliable estimation of floodwater depth from flood photos. Such rapid estimation is transformative in flood inundation mapping and assessing the severity of the flood in near-real time, which is essential for effective flood response strategies.
Scaling properties of the action in the Riemann-Liouville fractional standard map

The Riemann-Liouville fractional standard map (RL-fSM) is a two-dimensional nonlinear map with memory given in action-angle variables $(I,\theta)$. The RL-fSM is parameterized by $K$ and $\alpha\in(1,2]$ which control the strength of nonlinearity and the fractional order of the Riemann-Liouville derivative, respectively. In this work, we present a scaling study of the average squared action $\left$ of the RL-fSM along strongly chaotic orbits, i.e. for $K\gg1$. We observe two scenarios depending on the initial action $I_0$, $I_0\ll K$ or $I_0\gg K$. However, we can show that $\left/I_0^2$ is a universal function of the scaled discrete time $nK^2/I_0^2$ ($n$ being the $n$th iteration of the RL-fSM). In addition, we note that $\left$ is independent of $\alpha$ for $K\gg1$. Analytical estimations support our numerical results.
Existence for the Supercooled Stefan Problem in General Dimensions

We prove the global-time existence of weak solutions to the supercooled Stefan problem. Our result holds in general space dimensions and with a general class of initial data. In addition, our solution is maximal in the sense of a certain stochastic order, among all comparable weak solutions starting from the same initial data. Our approach is based on a free target optimization problem for Brownian stopping times, where the main idea is to introduce a superharmonic cost function in the optimization problem. We will show that our choice of the cost function causes the target measure to accumulate near the prescribed domain boundary as much as possible. A central ingredient in our proof lies in the usage of dual problem: we prove the dual attainment and use the dual optimal solution to characterize the primal optimal solution. It follows in turn that the underlying particle dynamics yields a solution to the supercooled Stefan problem.
Beyond Self-learned Attention: Mitigating Attention Bias in Transformer-based Models Using Attention Guidance

Transformer-based models have demonstrated considerable potential for source code modeling tasks in software engineering. However, they are limited by their dependence solely on automatic self-attention weight learning mechanisms. Previous studies have shown that these models overemphasize delimiters added by tokenizers (e.g., [CLS], [SEP]), which may lead to overlooking essential information in the original input source code. To address this challenge, we introduce SyntaGuid, a novel approach that utilizes the observation that attention weights tend to be biased towards specific source code syntax tokens and abstract syntax tree (AST) elements in fine-tuned language models when they make correct predictions. SyntaGuid facilitates the guidance of attention-weight learning, leading to improved model performance on various software engineering tasks. We evaluate the effectiveness of SyntaGuid on multiple tasks and demonstrate that it outperforms existing state-of-the-art models in overall performance without requiring additional data. Experimental result shows that SyntaGuid can improve overall performance up to 3.25% and fix up to 28.3% wrong predictions. Our work represents the first attempt to guide the attention of Transformer-based models towards critical source code tokens during fine-tuning, highlighting the potential for enhancing Transformer-based models in software engineering.
A p-version of convolution quadrature in wave propagation

We consider a novel way of discretizing wave scattering problems using the general formalism of convolution quadrature, but instead of reducing the timestep size ($h$-method), we achieve accuracy by increasing the order of the method ($p$-method). We base this method on discontinuous Galerkin timestepping and use the Z-transform. We show that for a certain class of incident waves, the resulting schemes observes(root)-exponential convergence rate with respect to the number of boundary integral operators that need to be applied. Numerical experiments confirm the findings.
Linear bounds on treewidth in terms of excluded planar minors

One of the fundamental results in graph minor theory is that for every planar graph $H$, there is a minimum integer $f(H)$ such that graphs with no minor isomorphic to $H$ have treewidth at most $f(H)$. A lower bound for ${f(H)}$ can be obtained by considering the maximum integer $k$ such that $H$ contains $k$ vertex-disjoint cycles. There exists a graph of treewidth ${\Omega(k\log k)}$ which does not contain $k$ vertex-disjoint cycles, from which it follows that ${f(H) = \Omega(k\log k)}$. In particular, if ${f(H)}$ is linear in ${\lvert{V(H)}\rvert}$ for graphs $H$ from a subclass of planar graphs, it is necessary that $n$-vertex graphs from the class contain at most ${O(n/\log(n))}$ vertex-disjoint cycles. We ask whether this is also a sufficient condition, and demonstrate that this is true for classes of planar graphs with bounded component size. For an $n$-vertex graph $H$ which is a disjoint union of $r$ cycles, we show that ${f(H) \leq 3n/2 + O(r^2 \log r)}$, and improve this to ${f(H) \leq n + O(\sqrt{n})}$ when ${r = 2}$. In particular this bound is linear when ${r=O(\sqrt{n}/\log(n))}$. We present a linear bound for ${f(H)}$ when $H$ is a subdivision of an $r$-edge planar graph for any constant $r$. We also improve the best known bounds for ${f(H)}$ when $H$ is the wheel graph or the ${4 \times 4}$ grid, obtaining a bound of $160$ for the latter.
Isoperimetric Profiles and Regular Embeddings of locally compact groups

In this article we extend the notion of $L^p$-measured subgroups couplings; a quantitative asymmetric version of measure equivalence that was introduced by Delabie, Koivisto, Le Ma\^itre and Tessera for finitely generated groups, to the setting of locally compact compactly generated unimodular groups. As an example of these couplings; using ideas from Bader and Rosendal, we prove a"dynamical criteria"for the existence of regular embeddings between amenable locally compact compactly generated unimodular groups, namely the existence of an $L^\infty$-measured subgroup coupling that is coarsely $m$ to $1$. We then proceed to prove that the existence of an $L^p$-measured subgroup that is coarsely $m$ to $1$ implies the monotonicity of the $L^p$-isoperimetric profile. We conclude then that the $L^p$-isoperimetric profile is monotonous under regular embeddings, as well as coarse embeddings, between amenable unimodular locally compact compactly generated groups.
Batched Nonparametric Contextual Bandits

We study nonparametric contextual bandits under batch constraints, where the expected reward for each action is modeled as a smooth function of covariates, and the policy updates are made at the end of each batch of observations. We establish a minimax regret lower bound for this setting and propose Batched Successive Elimination with Dynamic Binning (BaSEDB) that achieves optimal regret (up to logarithmic factors). In essence, BaSEDB dynamically splits the covariate space into smaller bins, carefully aligning their widths with the batch size. We also show the suboptimality of static binning under batch constraints, highlighting the necessity of dynamic binning. Additionally, our results suggest that a nearly constant number of policy updates can attain optimal regret in the fully online setting.
CLAPSep: Leveraging Contrastive Pre-trained Models for Multi-Modal Query-Conditioned Target Sound Extraction

Universal sound separation (USS) aims to extract arbitrary types of sounds from real-world sound recordings. Language-queried target sound extraction (TSE) is an effective approach to achieving USS. Such systems consist of two components: a query network that converts user queries into conditional embeddings, and a separation network that extracts the target sound based on conditional embeddings. Existing methods mainly suffer from two issues: firstly, they require training a randomly initialized model from scratch, lacking the utilization of pre-trained models, and substantial data and computational resources are needed to ensure model convergence; secondly, existing methods need to jointly train a query network and a separation network, which tends to lead to overfitting. To address these issues, we build the CLAPSep model based on contrastive language-audio pre-trained model (CLAP). We achieve this by using a pre-trained text encoder of CLAP as the query network and introducing pre-trained audio encoder weights of CLAP into the separation network to fully utilize the prior knowledge embedded in the pre-trained model to assist in target sound extraction tasks. Extensive experimental results demonstrate that the proposed method saves training resources while ensuring the model's performance and generalizability. Additionally, we explore the model's ability to comprehensively utilize language/audio multi-modal and positive/negative multi-valent user queries, enhancing system performance while providing diversified application modes.
Outdoor Environment Reconstruction with Deep Learning on Radio Propagation Paths

Conventional methods for outdoor environment reconstruction rely predominantly on vision-based techniques like photogrammetry and LiDAR, facing limitations such as constrained coverage, susceptibility to environmental conditions, and high computational and energy demands. These challenges are particularly pronounced in applications like augmented reality navigation, especially when integrated with wearable devices featuring constrained computational resources and energy budgets. In response, this paper proposes a novel approach harnessing ambient wireless signals for outdoor environment reconstruction. By analyzing radio frequency (RF) data, the paper aims to deduce the environmental characteristics and digitally reconstruct the outdoor surroundings. Investigating the efficacy of selected deep learning (DL) techniques on the synthetic RF dataset WAIR-D, the study endeavors to address the research gap in this domain. Two DL-driven approaches are evaluated (convolutional U-Net and CLIP+ based on vision transformers), with performance assessed using metrics like intersection-over-union (IoU), Hausdorff distance, and Chamfer distance. The results demonstrate promising performance of the RF-based reconstruction method, paving the way towards lightweight and scalable reconstruction solutions.
LocalGCL: Local-aware Contrastive Learning for Graphs

Graph representation learning (GRL) makes considerable progress recently, which encodes graphs with topological structures into low-dimensional embeddings. Meanwhile, the time-consuming and costly process of annotating graph labels manually prompts the growth of self-supervised learning (SSL) techniques. As a dominant approach of SSL, Contrastive learning (CL) learns discriminative representations by differentiating between positive and negative samples. However, when applied to graph data, it overemphasizes global patterns while neglecting local structures. To tackle the above issue, we propose \underline{Local}-aware \underline{G}raph \underline{C}ontrastive \underline{L}earning (\textbf{\methnametrim}), a self-supervised learning framework that supplementarily captures local graph information with masking-based modeling compared with vanilla contrastive learning. Extensive experiments validate the superiority of \methname against state-of-the-art methods, demonstrating its promise as a comprehensive graph representation learner.
Cross-Modal Contextualized Diffusion Models for Text-Guided Visual Generation and Editing

Conditional diffusion models have exhibited superior performance in high-fidelity text-guided visual generation and editing. Nevertheless, prevailing text-guided visual diffusion models primarily focus on incorporating text-visual relationships exclusively into the reverse process, often disregarding their relevance in the forward process. This inconsistency between forward and reverse processes may limit the precise conveyance of textual semantics in visual synthesis results. To address this issue, we propose a novel and general contextualized diffusion model (ContextDiff) by incorporating the cross-modal context encompassing interactions and alignments between text condition and visual sample into forward and reverse processes. We propagate this context to all timesteps in the two processes to adapt their trajectories, thereby facilitating cross-modal conditional modeling. We generalize our contextualized diffusion to both DDPMs and DDIMs with theoretical derivations, and demonstrate the effectiveness of our model in evaluations with two challenging tasks: text-to-image generation, and text-to-video editing. In each task, our ContextDiff achieves new state-of-the-art performance, significantly enhancing the semantic alignment between text condition and generated samples, as evidenced by quantitative and qualitative evaluations. Our code is available at https://github.com/YangLing0818/ContextDiff
Current potential patches

A novel form of the current potential, a mathematical tool for the design of stellarators and stellarator coils, is developed. Specifically, these are current potentials with a finite-element-like basis, called \textit{current potential patches}. Current potential patches leverage the relationship between distributions of magnetic dipoles and current potentials to explore limits of the access properties of stellarator coil sets. An example calculation is shown using the Helically Symmetric Experiment (HSX) equilibrium, demonstrating the method's use in coil design and understanding the limits of the access properties of coil sets. Current potential patches have additional desirable properties such as of promoting sparse current sheet solutions and identifying crucial locations of shaping current placement. A result is found for the HSX equilibrium that shaping currents covering only 25\% of the winding surface is sufficient to produce the equilibrium to a good accuracy.
Transfer Learning Bayesian Optimization to Design Competitor DNA Molecules for Use in Diagnostic Assays

With the rise in engineered biomolecular devices, there is an increased need for tailor-made biological sequences. Often, many similar biological sequences need to be made for a specific application meaning numerous, sometimes prohibitively expensive, lab experiments are necessary for their optimization. This paper presents a transfer learning design of experiments workflow to make this development feasible. By combining a transfer learning surrogate model with Bayesian optimization, we show how the total number of experiments can be reduced by sharing information between optimization tasks. We demonstrate the reduction in the number of experiments using data from the development of DNA competitors for use in an amplification-based diagnostic assay. We use cross-validation to compare the predictive accuracy of different transfer learning models, and then compare the performance of the models for both single objective and penalized optimization tasks.
Impact of Rossby and Kelvin Wave Components on MJO Eastward Propagation

There are contrasting views concerning the impact of Rossby wave component of MJO flow on its eastward propagation. One view (called “drag effect”) argues that because Rossby waves propagate westward, a stronger Rossby wave component slows down the eastward propagation. The other view (called “acceleration effect”) argues that a stronger Rossby wave enhances east–west asymmetry of moist static energy (MSE) tendency and thus favors the eastward propagation. This study aims to resolve this issue through diagnosis of both idealized aquaplanet simulations and 26 models from the MJO Task Force/GEWEX Atmospheric System Studies (MJOTF/GASS). In the aquaplanet experiments, three sets of zonally uniform, equatorially symmetric SST distributions are specified. The MJO phase speed is faster in the presence of a narrower SST meridional profile, in which both the Rossby and the Kelvin wave components are stronger and the east–west asymmetry of MSE tendency is larger. A further analysis of the 26 general circulation models reveals that the MJO propagation skill and phase speed are positively correlated to both the Rossby wave and the Kelvin wave strength in the lower free atmosphere (above 800 hPa). Models that have a stronger Rossby and Kelvin wave component tend to simulate realistic and faster eastward propagation. Therefore, both the aquaplanet and the multimodel simulations support the Rossby wave acceleration effect hypothesis.