Miracle-dz/UniFAR-FTUs huggingface dataset:
Efficient Fine-Tuning of Compressed Language Models with Learners

Fine-tuning BERT-based models is resource-intensive in memory, computation, and time. While many prior works aim to improve inference efficiency via compression techniques, e.g., pruning, these works do not explicitly address the computational challenges of training to downstream tasks. We introduce Learner modules and priming, novel methods for fine-tuning that exploit the overparameterization of pre-trained language models to gain benefits in convergence speed and resource utilization. Learner modules navigate the double bind of 1) training efficiently by fine-tuning a subset of parameters, and 2) training effectively by ensuring quick convergence and high metric scores. Our results on DistilBERT demonstrate that learners perform on par with or surpass the baselines. Learners train 7x fewer parameters than state-of-the-art methods on GLUE. On CoLA, learners fine-tune 20% faster, and have significantly lower resource utilization.
Finding a Lower Bound for k-Unbounded Hamiltonian Cycles

Methods to determine the existence of Hamiltonian Cycles in graphs have been extensively studied. However, little research has been done following cases when no Hamiltonian Cycle exists. Let a vertex be"unbounded"if it is visited more than once in a path. Furthermore, let a k-Unbounded Hamiltonian Cycle be a path with finite length that visits every vertex, has adjacent start and end vertices, and contains k unbounded vertices. We consider a novel variant of the Hamiltonian Cycle Problem in which the objective is to find an m-Unbounded Hamiltonian Cycle where m is the minimum value of k such that a k-Unbounded Hamiltonian Cycle exists. We first consider the task on well-known non-Hamiltonian graphs. We then provide an exponential-time brute-force algorithm for the determination of an m-Unbounded Hamiltonian Cycle and discuss approaches to solve the variant through transformations to the Hamiltonian Cycle Problem and the Asymmetric Traveling Salesman Problem. Finally, we present a polynomial-time heuristic for the determination of an m-Unbounded Hamiltonian Cycle that is also shown to be an effective heuristic for the original Hamiltonian Cycle Problem.
Deep Reinforcement Learning for Multi-Agent Interaction

The development of autonomous agents which can interact with other agents to accomplish a given task is a core area of research in artificial intelligence and machine learning. Towards this goal, the Autonomous Agents Research Group develops novel machine learning algorithms for autonomous systems control, with a specific focus on deep reinforcement learning and multi-agent reinforcement learning. Research problems include scalable learning of coordinated agent policies and inter-agent communication; reasoning about the behaviours, goals, and composition of other agents from limited observations; and sample-efficient learning based on intrinsic motivation, curriculum learning, causal inference, and representation learning. This article provides a broad overview of the ongoing research portfolio of the group and discusses open problems for future directions.
Machine learning optimization of Majorana hybrid nanowires

As the complexity of quantum systems such as quantum bit arrays increases, efforts to automate expensive tuning are increasingly worthwhile. We investigate machine learning based tuning of gate arrays using the covariance matrix adaptation evolution strategy algorithm for the case study of Majorana wires with strong disorder. We find that the algorithm is able to efficiently improve the topological signatures, learn intrinsic disorder profiles, and completely eliminate disorder effects. For example, with only 20 gates, it is possible to fully recover Majorana zero modes destroyed by disorder by optimizing gate voltages.
Joint Sensing and Communications for Deep Reinforcement Learning-based Beam Management in 6G

User location is a piece of critical information for network management and control. However, location uncertainty is unavoidable in certain settings leading to localization errors. In this paper, we consider the user location uncertainty in the mmWave networks, and investigate joint vision-aided sensing and communications using deep reinforcement learning-based beam management for future 6G networks. In particular, we first extract pixel characteristic-based features from satellite images to improve localization accuracy. Then we propose a UK-medoids based method for user clustering with location uncertainty, and the clustering results are consequently used for the beam management. Finally, we apply the DRL algorithm for intra-beam radio resource allocation. The simulations first show that our proposed vision-aided method can substantially reduce the localization error. The proposed UK-medoids and DRL based scheme (UKM-DRL) is compared with two other schemes: K-means based clustering and DRL based resource allocation (K-DRL) and UK-means based clustering and DRL based resource allocation (UK-DRL). The proposed method has 17.2% higher throughput and 7.7% lower delay than UK-DRL, and more than doubled throughput and 55.8% lower delay than K-DRL.
Entanglement spectrum and entropy in Floquet topological matter

Entanglement is one of the most fundamental features of quantum systems. In this work, we obtain the entanglement spectrum and entropy of Floquet noninteracting fermionic lattice models and build their connections with Floquet topological phases. Topological winding and Chern numbers are introduced to characterize the entanglement spectrum and eigenmodes. Correspondences between the spectrum and topology of entanglement Hamiltonians under periodic boundary conditions and topological edge states under open boundary conditions are further established. The theory is applied to Floquet topological insulators in different symmetry classes and spatial dimensions. Our work thus provides a useful framework for the study of rich entanglement patterns in Floquet topological matter.
Searching for ultralight axions with twisted cavity resonators of anyon rotational symmetry with bulk modes of nonzero helicity

M\"obius-ring resonators stem from a well-studied and fascinating geometrical structure that features a one-sided topology; the M\"obius strip, and have been shown to exhibit fermion rotational symmetry with respect to a ring resonator with no twist (which exhibits boson rotational symmetry) (see PhysRevLett.101.247701). Here, we present a new type of resonator through the formation of twisted hollow structures using equilateral triangular cross-sections, which leads to the realization of a cavity with anyon rotational symmetry. Unlike all previous cavity resonators, the anyon resonator permits the existence of bulk resonant modes that exhibit non-zero electromagnetic helicity in vacuo, with a non-zero overlap of the electric and magnetic mode eigenvectors, $\int \mathbf{E}_p\cdot\mathbf{B}_p~d\tau$, integrated over the cavity volume. In the upconversion limit, we show that these non-zero helical modes couple naturally to ultra-light dark matter axions within the bandwidth of the resonator by adding amplitude-modulated sidebands through the axion-photon chiral anomaly. Thus, we show a sensitive ultra-light dark matter experiment may be realized by implementing such a resonator in an ultra-stable oscillator configuration and searching for signals in the Fourier spectrum of amplitude fluctuations. This removes the typical requirement for an external magnetic field and therefore permits the use of superconducting materials to reduce surface losses and enhance sensitivity to axions.
Statistical Attention Localization (SAL): Methodology and Application to Object Classification

A statistical attention localization (SAL) method is proposed to facilitate the object classification task in this work. SAL consists of three steps: 1) preliminary attention window selection via decision statistics, 2) attention map refinement, and 3) rectangular attention region finalization. SAL computes soft-decision scores of local squared windows and uses them to identify salient regions in Step 1. To accommodate object of various sizes and shapes, SAL refines the preliminary result and obtain an attention map of more flexible shape in Step 2. Finally, SAL yields a rectangular attention region using the refined attention map and bounding box regularization in Step 3. As an application, we adopt E-PixelHop, which is an object classification solution based on successive subspace learning (SSL), as the baseline. We apply SAL so as to obtain a cropped-out and resized attention region as an alternative input. Classification results of the whole image as well as the attention region are ensembled to achieve the highest classification accuracy. Experiments on the CIFAR-10 dataset are given to demonstrate the advantage of the SAL-assisted object classification method.
A new deflation criterion for the QZ algorithm

The QZ algorithm computes the generalized Schur form of a matrix pencil. It is an iterative algorithm and, at some point, it must decide when to deflate, that is when a generalized eigenvalue has converged and to move on to another one. Choosing a deflation criterion that makes this decision is nontrivial. If it is too strict, the algorithm might waste iterations on already converged eigenvalues. If it is not strict enough, the computed eigenvalues might not have full accuracy. Additionally, the criterion should not be computationally expensive to evaluate. There are two commonly used criteria: the elementwise criterion and the normwise criterion. This paper introduces a new deflation criterion based on the size of and the gap between the eigenvalues. We call this new deflation criterion the strict criterion. This new criterion for QZ is analogous to the criterion derived by Ahues and Tisseur for the QR algorithm. Theoretical arguments and numerical experiments suggest that the strict criterion outperforms the normwise and elementwise criteria in terms of accuracy. We also provide an example where the accuracy of the generalized eigenvalues using the elementwise or the normwise criteria is less than two digits whereas the strict criterion leads to generalized eigenvalues which are almost accurate to the working precision. Additionally, this paper evaluates some commonly used criteria for infinite eigenvalues.
Robust Learning of Deep Time Series Anomaly Detection Models with Contaminated Training Data

Time series anomaly detection (TSAD) is an important data mining task with numerous applications in the IoT era. In recent years, a large number of deep neural network-based methods have been proposed, demonstrating significantly better performance than conventional methods on addressing challenging TSAD problems in a variety of areas. Nevertheless, these deep TSAD methods typically rely on a clean training dataset that is not polluted by anomalies to learn the"normal profile"of the underlying dynamics. This requirement is nontrivial since a clean dataset can hardly be provided in practice. Moreover, without the awareness of their robustness, blindly applying deep TSAD methods with potentially contaminated training data can possibly incur significant performance degradation in the detection phase. In this work, to tackle this important challenge, we firstly investigate the robustness of commonly used deep TSAD methods with contaminated training data which provides a guideline for applying these methods when the provided training data are not guaranteed to be anomaly-free. Furthermore, we propose a model-agnostic method which can effectively improve the robustness of learning mainstream deep TSAD models with potentially contaminated data. Experiment results show that our method can consistently prevent or mitigate performance degradation of mainstream deep TSAD models on widely used benchmark datasets.
LSSANet: A Long Short Slice-Aware Network for Pulmonary Nodule Detection

Convolutional neural networks (CNNs) have been demonstrated to be highly effective in the field of pulmonary nodule detection. However, existing CNN based pulmonary nodule detection methods lack the ability to capture long-range dependencies, which is vital for global information extraction. In computer vision tasks, non-local operations have been widely utilized, but the computational cost could be very high for 3D computed tomography (CT) images. To address this issue, we propose a long short slice-aware network (LSSANet) for the detection of pulmonary nodules. In particular, we develop a new non-local mechanism termed long short slice grouping (LSSG), which splits the compact non-local embeddings into a short-distance slice grouped one and a long-distance slice grouped counterpart. This not only reduces the computational burden, but also keeps long-range dependencies among any elements across slices and in the whole feature map. The proposed LSSG is easy-to-use and can be plugged into many pulmonary nodule detection networks. To verify the performance of LSSANet, we compare with several recently proposed and competitive detection approaches based on 2D/3D CNN. Promising evaluation results on the large-scale PN9 dataset demonstrate the effectiveness of our method. Code is at https://github.com/Ruixxxx/LSSANet.
Data-Driven Fast Frequency Control using Inverter-Based Resources

To address the control challenges associated with the increasing share of inverter-connected renewable energy resources, this paper proposes a direct data-driven approach for fast frequency control in the bulk power system. The proposed control scheme partitions the power system into control areas, and leverages local dispatchable inverter-based resources to rapidly mitigate local power imbalances upon events. The controller design is based directly on historical measurement sequences, and does not require identification of a parametric power system model. Theoretical results are provided to support the approach. Simulation studies on a nonlinear three-area test system demonstrate that the controller provides fast and localized frequency control under several types of contingencies.
Resolved Motion Control for 3D Underactuated Bipedal Walking using Linear Inverted Pendulum Dynamics and Neural Adaptation

We present a framework to generate periodic trajectory references for a 3D under-actuated bipedal robot, using a linear inverted pendulum (LIP) based controller with adaptive neural regulation. We use the LIP template model to estimate the robot's center of mass (CoM) position and velocity at the end of the current step, and formulate a discrete controller that determines the next footstep location to achieve a desired walking profile. This controller is equipped on the frontal plane with a Neural-Network-based adaptive term that reduces the model mismatch between the template and physical robot that particularly affects the lateral motion. Then, the foot placement location computed for the LIP model is used to generate task space trajectories (CoM and swing foot trajectories) for the actual robot to realize stable walking. We use a fast, real-time QP-based inverse kinematics algorithm that produces joint references from the task space trajectories, which makes the formulation independent of the knowledge of the robot dynamics. Finally, we implemented and evaluated the proposed approach in simulation and hardware experiments with a Digit robot obtaining stable periodic locomotion for both cases.
Gradient descent provably escapes saddle points in the training of shallow ReLU networks

Dynamical systems theory has recently been applied in optimization to prove that gradient descent algorithms avoid so-called strict saddle points of the loss function. However, in many modern machine learning applications, the required regularity conditions are not satisfied. In particular, this is the case for rectified linear unit (ReLU) networks. In this paper, we prove a variant of the relevant dynamical systems result, a center-stable manifold theorem, in which we relax some of the regularity requirements. Then, we verify that shallow ReLU networks fit into the new framework. Building on a classification of critical points of the square integral loss of shallow ReLU networks measured against an affine target function, we deduce that gradient descent avoids most saddle points. We proceed to prove convergence to global minima if the initialization is sufficiently good, which is expressed by an explicit threshold on the limiting loss.
Fast Hierarchical Deep Unfolding Network for Image Compressed Sensing

By integrating certain optimization solvers with deep neural network, deep unfolding network (DUN) has attracted much attention in recent years for image compressed sensing (CS). However, there still exist several issues in existing DUNs: 1) For each iteration, a simple stacked convolutional network is usually adopted, which apparently limits the expressiveness of these models. 2) Once the training is completed, most hyperparameters of existing DUNs are fixed for any input content, which significantly weakens their adaptability. In this paper, by unfolding the Fast Iterative Shrinkage-Thresholding Algorithm (FISTA), a novel fast hierarchical DUN, dubbed FHDUN, is proposed for image compressed sensing, in which a well-designed hierarchical unfolding architecture is developed to cooperatively explore richer contextual prior information in multi-scale spaces. To further enhance the adaptability, series of hyperparametric generation networks are developed in our framework to dynamically produce the corresponding optimal hyperparameters according to the input content. Furthermore, due to the accelerated policy in FISTA, the newly embedded acceleration module makes the proposed FHDUN save more than 50% of the iterative loops against recent DUNs. Extensive CS experiments manifest that the proposed FHDUN outperforms existing state-of-the-art CS methods, while maintaining fewer iterations.
Historical and contemporary factors shape brooding corals' population structure and 2 genetic diversity 3 Seriatopora hystrix reveals patterns of strong genetic differentiation in the Western Indian Ocean. Heredity

The coral reefs of the Western Indian Ocean (WIO) and the Red Sea offer a unique 18 environment to investigate large scale connectivity of corals along a north-south gradient with variable environmental conditions. However, these highly biodiverse coral reefs remain understudied and few studies have addressed connectivity patterns. In this study, the genetic diversity and connectivity of the brooding scleractinian coral Seriatopora hystrix were assessed on two scales: first, by comparing the Red Sea and the 23 WIO, and second, by comparing sites within the WIO. Ten microsatellites were used to genotype 356 individual colonies from 14 locations. Additionally, a haploweb analysis of an ITS2 marker was used for species delimitation in the genus Seriatopora. Seriatopora hystrix showed limited dispersal capacity and high differentiation both at a large scale between the Red Sea and the WIO, and smaller scale for sites along the coast of 28 East Africa. Red Sea populations appear to belong to a different species than the 29 populations in the WIO. Within the WIO three groups can be distinguished, (I) north 30 Madagascar, (II) south-west Madagascar together with one site in Mozambique (Nacala), 31 and (III) all other sites along the coast of Mozambique, Tanzania and Kenya. These patterns of contemporary dispersal barriers can be explained by the short pelagic larval duration of S. hystrix and by oceanographic conditions, such as eddies in the Mozambique channel causing larval retention in northern Madagascar. This study can guide coral reef management and supports the conservation priority status of the Northern Mozambique Channel region.
Self - Initiative Undergraduate Research

This paper describes an undergraduate research project conducted at a liberal arts institution during the summer of 2019. The undertaking was distinctive in that it was an engineering project conducted at a liberal arts college by undergraduate students enrolled in the college’s engineering program. A multidisciplinary research team was assembled, composed of students majoring in civil, chemical and mechanical engineering. It should be emphasized that this seven-week program was designed to serve primarily as a learning experience for the student researchers. The goals were: (1) to engage undergraduate students in the self-initiative research process; (2) to demonstrate how what they are studying and learning can be put to practical use; and (3) to teach the students the value of their chosen field and how it benefits society. It should be further emphasized that the “nuts and bolts” specifics of the project, though important, were not the primary objective; learning the value of teamwork, prioritization, time management and communication was. detailed
HIF1 (cid:1) and pancreatic (cid:2) -cell development

During early embryogenesis, the pancreas shows a paucity of blood ﬂow, and oxygen tension, the partial pressure of oxygen (pO 2 ), is low. Later, the blood ﬂow increases as (cid:2) -cell differentiation occurs. We have previously reported that pO 2 controls (cid:2) -cell development in rats. Here, we checked that hypoxia inducible factor 1 (cid:1) (HIF1 (cid:1) ) is essential for this control. First, we demonstrated that the effect of pO 2 on (cid:2) -cell differentiation in vitro was independent of epitheliomesenchymal interactions and that neither oxidative nor energetic stress occurred. Second, the effect of pO 2 on pancreas development was shown to be conserved among species, since increasing pO 2 to 21 vs. 3% also induced (cid:2) -cell differentiation in mouse (7-fold, P < 0.001) and human fetal pancreas. Third, the effect of hypoxia was mediated by HIF1 (cid:1) , since the addition of an HIF1 (cid:1) inhibitor at 3% O 2 increased the number of NGN3-expressing progenitors as compared to nontreated controls (9.2-fold, P < 0.001). In contrast, when we stabilized HIF1 (cid:1) by deleting ex vivo the gene encoding pVHL in E13.5 pancreas from Vhl ﬂoxed mice, Ngn 3 expression and (cid:2) -cell development decreased in such Vhl -deleted pancreas compared to controls (2.5 fold, P < 0.05, and 6.6-fold, P < 0.001, respectively). Taken together, these data demonstrate that HIF1 (cid:1) exerts a negative control over (cid:2) -cell differ-entiation. —Heinis,
Kinetics and Metabolism of Biﬁdobacterium adolescentis MB 239 Growing on Glucose, Galactose, Lactose, and Galactooligosaccharides (cid:1)

The kinetics and the metabolism of Biﬁdobacterium adolescentis MB 239 growing on galactooligosaccharides (GOS), lactose, galactose, and glucose were investigated. An unstructured unsegregated model for growth in batch cultures was developed, and kinetic parameters were calculated with a recursive algorithm. The growth rate and cellular yield were highest on galactose, followed by lactose and GOS, and were lowest on glucose. Lactate, acetate, and ethanol yields allowed the calculation of carbon ﬂuxes toward fermentation products. Distributions between two- and three-carbon products were similar on all the carbohydrates (55 and 45%, respectively), but ethanol yields were different on glucose, GOS, lactose, and galactose, in decreasing order of production. Based on the stoichiometry of the fructose-6-phosphate shunt and on the carbon distribution among the products, the ATP yield was calculated. The highest yield was obtained on galactose, while the yields were 5, 8, and 25% lower on lactose, GOS, and glucose, respectively. Therefore, a correspondence among ethanol production, low ATP yields, and low biomass production was established, demonstrating that carbohydrate preferences may result from different distributions