Miracle-dz/UniFAR-FTUs huggingface dataset:
GEVO-ML: Optimizing Machine Learning Code with Evolutionary Computation

Parallel accelerators, such as GPUs, are key enablers for large-scale Machine Learning (ML) applications. However, ML model developers often lack detailed knowledge of the underlying system architectures, while system programmers usually do not have a high-level understanding of the ML model that runs on the specific system. To mitigate this gap between two relevant aspects of domain knowledge, this paper proposes GEVO-ML, a tool for automatically discovering optimization opportunities and tuning the performance of ML kernels, where the model and training/prediction processes are uniformly represented in a single intermediate language, the Multiple-Layer Intermediate Representation (MLIR). GEVO-ML uses multi-objective evolutionary search to find edits (mutations) to MLIR code that ultimately runs on GPUs, improving performance on desired criteria while retaining required functionality. We demonstrate GEVO-ML on two different ML workloads for both model training and prediction. GEVO-ML finds significant Pareto improvements for these models, achieving 90.43% performance improvement when model accuracy is relaxed by 2%, from 91.2% to 89.3%. For the training workloads, GEVO-ML finds a 4.88% improvement in model accuracy, from 91% to 96%, without sacrificing training or testing speed. Our analysis of key GEVO-ML mutations reveals diverse code modifications, while might be foreign to human developers, achieving similar effects with how human developers improve model design, for example, by changing learning rates or pruning non-essential layer parameters.
Directional atomic layer etching of MgO-doped lithium niobate using sequential exposures of H$_2$ and SF$_6$ plasma

Lithium niobate (LiNbO$_3$, LN) is a ferroelectric crystal of interest for integrated photonics owing to its large second-order optical nonlinearity and the ability to impart periodic poling via an external electric field. However, on-chip device performance based on thin-film lithium niobate (TFLN) is presently limited by optical loss arising from corrugations between poled regions and sidewall surface roughness. Atomic layer etching (ALE) could potentially smooth these features and thereby increase photonic performance, but no ALE process has been reported for LN. Here, we report a directional ALE process for $x$-cut MgO-doped LN using sequential exposures of H$_2$ and SF$_6$/Ar plasmas. We observe etch rates up to $1.01 \pm 0.05$ nm/cycle with a synergy of $94$%. We also demonstrate ALE can be achieved with SF$_6$/O$_2$ or Cl$_2$/BCl$_3$ plasma exposures in place of the SF$_6$/Ar plasma step with synergies above $90$%. When combined with a wet post-process to remove redeposited compounds, the process yields a 50% decrease in surface roughness. With additional optimization to reduce the quantity of redeposited compounds, these processes could be used to smoothen surfaces of TFLN waveguides etched by physical Ar$^+$ milling, thereby increasing the performance of TFLN nanophotonic devices or enabling new integrated photonic capabilities.
Negative Sampling with Adaptive Denoising Mixup for Knowledge Graph Embedding

Knowledge graph embedding (KGE) aims to map entities and relations of a knowledge graph (KG) into a low-dimensional and dense vector space via contrasting the positive and negative triples. In the training process of KGEs, negative sampling is essential to find high-quality negative triples since KGs only contain positive triples. Most existing negative sampling methods assume that non-existent triples with high scores are high-quality negative triples. However, negative triples sampled by these methods are likely to contain noise. Specifically, they ignore that non-existent triples with high scores might also be true facts due to the incompleteness of KGs, which are usually called false negative triples. To alleviate the above issue, we propose an easily pluggable denoising mixup method called DeMix, which generates high-quality triples by refining sampled negative triples in a self-supervised manner. Given a sampled unlabeled triple, DeMix firstly classifies it into a marginal pseudo-negative triple or a negative triple based on the judgment of the KGE model itself. Secondly, it selects an appropriate mixup partner for the current triple to synthesize a partially positive or a harder negative triple. Experimental results on the knowledge graph completion task show that the proposed DeMix is superior to other negative sampling techniques, ensuring corresponding KGEs a faster convergence and better link prediction results.
Oil formation volume factor modeling: Traditional vs. Stochastically optimized neural networks

Oil formation volume factor (FVF) is considered as relative change in oil volume between reservoir condition and standard surface condition. FVF, always greater than one, is dominated by reservoir temperature, amount of dissolved gas in oil, and specific gravity of oil and dissolved gas. In addition to limitations on reliable sampling, experimental determination of FVF is associated with high costs and time-consumption. Therefore, this study proposes a novel approach based on hybrid genetic algorithm-pattern search (GA-PS) optimized neural network (NN) for fast, accurate, and cheap determination of oil FVF from available measured pressure-volume-temperature (PVT) data. Contrasting to traditional neural network which is in danger of sticking in local minima, GA-PS optimized NN is in charge of escaping from local minima and converging to global minimum. A group of 342 data points were used for model construction and a group of 219 data points were employed for model assessment. Results indicated superiority of GA-PS optimized NN to traditional NN. Oil FVF values, determined by GA-PS optimized NN were in good agreement with reality.
Shuffle Bases and Quasisymmetric Power Sums

The algebra of quasisymmetric functions QSym and the shuffle algebra of compositions Sh are isomorphic as graded Hopf algebras (in characteristic zero), and isomorphisms between them can be specified via shuffle bases of QSym. We use the notion of infinitesimal characters to characterize shuffle bases, and we establish a universal property for Sh in the category of connected graded Hopf algebras equipped with an infinitesimal character, analogous to the universal property of QSym as a combinatorial Hopf algebra described by Aguiar, Bergeron, and Sottile. We then use these results to give general constructions for quasisymmetric power sums, recovering four previous constructions from the literature, and study their properties.
Diversifying the Mixture-of-Experts Representation for Language Models with Orthogonal Optimizer

The Mixture of Experts (MoE) has emerged as a highly successful technique in deep learning, based on the principle of divide-and-conquer to maximize model capacity without significant additional computational cost. Even in the era of large-scale language models (LLMs), MoE continues to play a crucial role, as some researchers have indicated that GPT-4 adopts the MoE structure to ensure diverse inference results. However, MoE is susceptible to performance degeneracy, particularly evident in the issues of imbalance and homogeneous representation among experts. While previous studies have extensively addressed the problem of imbalance, the challenge of homogeneous representation remains unresolved. In this study, we shed light on the homogeneous representation problem, wherein experts in the MoE fail to specialize and lack diversity, leading to frustratingly high similarities in their representations (up to 99% in a well-performed MoE model). This problem restricts the expressive power of the MoE and, we argue, contradicts its original intention. To tackle this issue, we propose a straightforward yet highly effective solution: OMoE, an orthogonal expert optimizer. Additionally, we introduce an alternating training strategy that encourages each expert to update in a direction orthogonal to the subspace spanned by other experts. Our algorithm facilitates MoE training in two key ways: firstly, it explicitly enhances representation diversity, and secondly, it implicitly fosters interaction between experts during orthogonal weights computation. Through extensive experiments, we demonstrate that our proposed optimization algorithm significantly improves the performance of fine-tuning the MoE model on the GLUE benchmark, SuperGLUE benchmark, question-answering task, and name entity recognition tasks.
Structure-Aware Analyses and Algorithms for Interpolative Decompositions

Low-rank approximation is a task of critical importance in modern science, engineering, and statistics. Many low-rank approximation algorithms, such as the randomized singular value decomposition (RSVD), project their input matrix into a subspace approximating the span of its leading singular vectors. Other algorithms compress their input into a small subset of representative rows or columns, leading to a so-called interpolative decomposition. This paper investigates how the accuracy of interpolative decompositions is affected by the structural properties of the input matrix being operated on, including how these properties affect the performance comparison between interpolative decompositions and RSVD. We also introduce a novel method of interpolative decomposition in the form of the randomized Golub-Klema-Stewart (RGKS) algorithm, which combines RSVD with a pivoting strategy for column subset selection. Through numerical experiments, we find that matrix structures including singular subspace geometry and singular spectrum decay play a significant role in determining the performance comparison between these different algorithms. We also prove inequalities which bound the error of a general interpolative decomposition in terms of these matrix structures. Lastly, we develop forms of these bounds specialized to RGKS while considering how randomization affects the approximation error of this algorithm.
The inhomogeneous multispecies PushTASEP: Dynamics and symmetry

We introduce and study a natural multispecies variant of the inhomogeneous PushTASEP with site-dependent rates on the finite ring. We show that the stationary distribution of this process is proportional to the ASEP polynomials at $q = 1$ and $t = 0$. This is done by constructing a multiline process which projects to the multispecies PushTASEP, and identifying its stationary distribution using time-reversal arguments. We also study symmetry properties of the process under interchange of the rates associated to the sites. These results hold not just for events depending on the configuration at a single time in equilibrium, but also for systems out of equilibrium and for events depending on the path of the process over time. Lastly, we give explicit formulas for nearest-neighbour two-point correlations in terms of Schur functions.
SVM based Multiclass Classifier for Gait phase Classification using Shank IMU Sensor

In this study, a gait phase classification method based on SVM multiclass classification is introduced, with a focus on the precise identification of the stance and swing phases, which are further subdivided into seven phases. Data from individual IMU sensors, such as Shank Acceleration X, Y, Z, Shank Gyro X, and Knee Angles, are used as features in this classification model. The suggested technique successfully classifies the various gait phases with a significant accuracy of about 90.3%. Gait phase classification is crucial, especially in the domains of exoskeletons and prosthetics, where accurate identification of gait phases enables seamless integration with assistive equipment, improving mobility, stability, and energy economy. This study extends the study of gait and offers an effective method for correctly identifying gait phases from Shank IMU sensor data, with potential applications in biomechanical research, exoskeletons, rehabilitation, and prosthetics.
Efficient Representation of Lattice Path Matroids

Efficient deterministic algorithms to construct representations of lattice path matroids over finite fields are presented. They are built on known constructions of hierarchical secret sharing schemes, a recent characterization of hierarchical matroid ports, and the existence of isolating weight functions for lattice path matroids whose values are polynomial on the size of the ground set.
An Anytime Algorithm for Good Arm Identification

In good arm identification (GAI), the goal is to identify one arm whose average performance exceeds a given threshold, referred to as good arm, if it exists. Few works have studied GAI in the fixed-budget setting, when the sampling budget is fixed beforehand, or the anytime setting, when a recommendation can be asked at any time. We propose APGAI, an anytime and parameter-free sampling rule for GAI in stochastic bandits. APGAI can be straightforwardly used in fixed-confidence and fixed-budget settings. First, we derive upper bounds on its probability of error at any time. They show that adaptive strategies are more efficient in detecting the absence of good arms than uniform sampling. Second, when APGAI is combined with a stopping rule, we prove upper bounds on the expected sampling complexity, holding at any confidence level. Finally, we show good empirical performance of APGAI on synthetic and real-world data. Our work offers an extensive overview of the GAI problem in all settings.
Impact of tropical teleconnections on the long-range predictability of the atmosphere at midlatitudes: a reduced-order multi-scale model perspective

Teleconnections between the tropical and the extratropical climates are often considered as a potential source of long-term predictability at seasonal to decadal time scales in the extratropics. This claim is taken up in the present work by investigating the predictability of a coupled ocean–atmosphere extratropical model under a one-way forcing generated by a tropical model. Both models display a chaotic dynamics, and the dominant variable of the extratropical model displays a high correlation with the tropical forcing in the reference simulation, inducing a low-frequency variability signal in the extratropics. Numerical experiments emulating the presence of initial condition errors in the tropical model are conducted to clarify their impact on the predictability in the extratropics. It is shown that: (i) the correlation skill of the dominant observable affected by the forcing is considerably degraded at interannual time scales due to the presence of initial condition errors in the tropics, limiting the potential of teleconnections; (ii) averaging of an ensemble of forecasts–with a small number of members–may substantially improve the quality of the forecasts; and (iii) temporal averaging may also improve the quality of the forecasts (at the expense of being able to forecast extreme events), in particular when the forcing affects weakly the observable under interest.
MEMTRACK: A Deep Learning-Based Approach to Microrobot Tracking in Dense and Low-Contrast Environments

Tracking microrobots is challenging, considering their minute size and high speed. As the field progresses towards developing microrobots for biomedical applications and conducting mechanistic studies in physiologically relevant media (e.g., collagen), this challenge is exacerbated by the dense surrounding environments with feature size and shape comparable to microrobots. Herein, we report Motion Enhanced Multi-level Tracker (MEMTrack), a robust pipeline for detecting and tracking microrobots using synthetic motion features, deep learning-based object detection, and a modified Simple Online and Real-time Tracking (SORT) algorithm with interpolation for tracking. Our object detection approach combines different models based on the object's motion pattern. We trained and validated our model using bacterial micro-motors in collagen (tissue phantom) and tested it in collagen and aqueous media. We demonstrate that MEMTrack accurately tracks even the most challenging bacteria missed by skilled human annotators, achieving precision and recall of 77% and 48% in collagen and 94% and 35% in liquid media, respectively. Moreover, we show that MEMTrack can quantify average bacteria speed with no statistically significant difference from the laboriously-produced manual tracking data. MEMTrack represents a significant contribution to microrobot localization and tracking, and opens the potential for vision-based deep learning approaches to microrobot control in dense and low-contrast settings. All source code for training and testing MEMTrack and reproducing the results of the paper have been made publicly available https://github.com/sawhney-medha/MEMTrack.
InfoGCN++: Learning Representation by Predicting the Future for Online Human Skeleton-based Action Recognition

Skeleton-based action recognition has made significant advancements recently, with models like InfoGCN showcasing remarkable accuracy. However, these models exhibit a key limitation: they necessitate complete action observation prior to classification, which constrains their applicability in real-time situations such as surveillance and robotic systems. To overcome this barrier, we introduce InfoGCN++, an innovative extension of InfoGCN, explicitly developed for online skeleton-based action recognition. InfoGCN++ augments the abilities of the original InfoGCN model by allowing real-time categorization of action types, independent of the observation sequence's length. It transcends conventional approaches by learning from current and anticipated future movements, thereby creating a more thorough representation of the entire sequence. Our approach to prediction is managed as an extrapolation issue, grounded on observed actions. To enable this, InfoGCN++ incorporates Neural Ordinary Differential Equations, a concept that lets it effectively model the continuous evolution of hidden states. Following rigorous evaluations on three skeleton-based action recognition benchmarks, InfoGCN++ demonstrates exceptional performance in online action recognition. It consistently equals or exceeds existing techniques, highlighting its significant potential to reshape the landscape of real-time action recognition applications. Consequently, this work represents a major leap forward from InfoGCN, pushing the limits of what's possible in online, skeleton-based action recognition. The code for InfoGCN++ is publicly available at https://github.com/stnoah1/infogcn2 for further exploration and validation.
Direct measurement of the coherent light proportion from a practical laser source

We present a technique to estimate the proportion of coherent emission in the light emitted by a practical laser source without spectral filtering. The technique is based on measuring interferometric photon correlations between the output ports of an asymmetric Mach-Zehnder interferometer. With this, we characterize the fraction of coherent emission in the light emitted by a laser diode when transiting through the lasing threshold.
An Empirical Study of Super-resolution on Low-resolution Micro-expression Recognition

Micro-expression recognition (MER) in low-resolution (LR) scenarios presents an important and complex challenge, particularly for practical applications such as group MER in crowded environments. Despite considerable advancements in super-resolution techniques for enhancing the quality of LR images and videos, few study has focused on investigate super-resolution for improving LR MER. The scarcity of investigation can be attributed to the inherent difficulty in capturing the subtle motions of micro-expressions, even in original-resolution MER samples, which becomes even more challenging in LR samples due to the loss of distinctive features. Furthermore, a lack of systematic benchmarking and thorough analysis of super-resolution-assisted MER methods has been noted. This paper tackles these issues by conducting a series of benchmark experiments that integrate both super-resolution (SR) and MER methods, guided by an in-depth literature survey. Specifically, we employ seven cutting-edge state-of-the-art (SOTA) MER techniques and evaluate their performance on samples generated from 13 SOTA SR techniques, thereby addressing the problem of super-resolution in MER. Through our empirical study, we uncover the primary challenges associated with SR-assisted MER and identify avenues to tackle these challenges by leveraging recent advancements in both SR and MER methodologies. Our analysis provides insights for progressing toward more efficient SR-assisted MER.
Population-based wind farm monitoring based on a spatial autoregressive approach

An important challenge faced by wind farm operators is to reduce operation and maintenance cost. Structural health monitoring provides a means of cost reduction through minimising unnecessary maintenance trips as well as prolonging turbine service life. Population-based structural health monitoring can further reduce the cost of health monitoring systems by implementing one system for multiple structures (i.e.~turbines). At the same time, shared data within a population of structures may improve the predictions of structural behaviour. To monitor turbine performance at a population/farm level, an important initial step is to construct a model that describes the behaviour of all turbines under normal conditions. This paper proposes a population-level model that explicitly captures the spatial and temporal correlations (between turbines) induced by the wake effect. The proposed model is a Gaussian process-based spatial autoregressive model, named here a GP-SPARX model. This approach is developed since (a) it reflects our physical understanding of the wake effect, and (b) it benefits from a stochastic data-based learner. A case study is provided to demonstrate the capability of the GP-SPARX model in capturing spatial and temporal variations as well as its potential applicability in a health monitoring system.
Dynamic Prediction of Full-Ocean Depth SSP by a Hierarchical LSTM: An Experimental Result

Sound speed profile (SSP) distribution is an important parameter for underwater positioning, navigation, and timing (PNT) because it affects the propagation mode of underwater acoustic signals. To accurately predict future sound speed distribution, we propose a hierarchical long short-term memory (H-LSTM) neural network, which explores the distribution pattern of sound velocity in the time dimension. To verify the feasibility and effectiveness, we conducted both simulations and real experiments. The ocean experiment was held in the South China Sea in April 2023. Results show that the predictive error of the proposed method is only 0.1530 m/s, demonstrating the proposed method outperforms the state-of-the-art methods and providing additional evidence of its heightened predictive capability in predicting future sound speed distribution.
TOI-1801 b: A temperate mini-Neptune around a young M0.5 dwarf

We report the discovery, mass, and radius determination of TOI-1801 b, a temperate mini-Neptune around a young M dwarf. TOI-1801 b was observed in TESS sectors 22 and 49, and the alert that this was a TESS planet candidate with a period of 21.3 days went out in April 2020. However, ground-based follow-up observations, including seeing-limited photometry in and outside transit together with precise radial velocity (RV) measurements with CARMENES and HIRES revealed that the true period of the planet is 10.6 days. These observations also allowed us to retrieve a mass of 5.74 $\pm$ 1.46 $M_\oplus$, which together with a radius of 2.08 $\pm$ 0.12 $R_\oplus$, means that TOI-1801 b is most probably composed of water and rock, with an upper limit of 2\% by mass of H$_{2}$ in its atmosphere. The stellar rotation period of 16 days is readily detectable in our RV time series and in the ground-based photometry. We derived a likely age of 600--800 Myr for the parent star TOI-1801, which means that TOI-1801 b is the least massive young mini-Neptune with precise mass and radius determinations. Our results suggest that if TOI-1801 b had a larger atmosphere in the past, it must have been removed by some evolutionary mechanism on timescales shorter than 1 Gyr.