Miracle-dz/UniFAR-FTUs huggingface dataset:
Optimal Pacing of a Cyclist in a Time Trial Based on Individualized Models of Fatigue and Recovery

This article formulates the optimal pacing of a cyclist on hilly terrain time trials as a minimum-time optimal control problem. The maximal power of a cyclist serves as a time-varying constraint and depends on fatigue and recovery which are captured via dynamic models proposed early in the article. Experimental protocols for identifying the individualized parameters of the proposed fatigue and recovery models are detailed and results for six human subjects are shown. In an analytical treatment via necessary conditions of the Pontryagin minimum principle, we show that the cyclist’s optimal power in a time trial is limited to only four modes of all-out, coasting, pedaling at a critical power, or constant speed (bang-singular-bang). To determine when to switch between these modes, we resort to numerical solutions via dynamic programming (DP). One of the subjects is then simulated on four courses, including the Duathlon National Championship in Greenville, SC, USA, in 2019. The DP simulation results show a 24% reduction in travel time over experimental results of the self-paced subject who is a competitive amateur cyclist. This article concludes with a description of a pilot laboratory experiment in which the subject’s trial time was reduced by 3% when the near-optimal pace was communicated to her in real time.
Using Time-Series Privileged Information for Provably Efficient Learning of Prediction Models

We study prediction of future outcomes with supervised models that use privileged information during learning. The privileged information comprises samples of time series observed between the baseline time of prediction and the future outcome; this information is only available at training time which differs from the traditional supervised learning. Our question is when using this privileged data leads to more sample-efficient learning of models that use only baseline data for predictions at test time. We give an algorithm for this setting and prove that when the time series are drawn from a non-stationary Gaussian-linear dynamical system of fixed horizon, learning with privileged information is more efficient than learning without it. On synthetic data, we test the limits of our algorithm and theory, both when our assumptions hold and when they are violated. On three diverse real-world datasets, we show that our approach is generally preferable to classical learning, particularly when data is scarce. Finally, we relate our estimator to a distillation approach both theoretically and empirically.
Labeled sample compression schemes for complexes of oriented matroids

We show that the topes of a complex of oriented matroids (abbreviated COM) of VC-dimension $d$ admit a proper labeled sample compression scheme of size $d$. This considerably extends results of Moran and Warmuth on ample classes, of Ben-David and Litman on affine arrangements of hyperplanes, and of the authors on complexes of uniform oriented matroids, and is a step towards the sample compression conjecture -- one of the oldest open problems in computational learning theory. On the one hand, our approach exploits the rich combinatorial cell structure of COMs via oriented matroid theory. On the other hand, viewing tope graphs of COMs as partial cubes creates a fruitful link to metric graph theory.
Stable Anderson Acceleration for Deep Learning

Anderson acceleration (AA) is an extrapolation technique designed to speed-up fixed-point iterations like those arising from the iterative training of DL models. Training DL models requires large datasets processed in randomly sampled batches that tend to introduce in the fixed-point iteration stochastic oscillations of amplitude roughly inversely proportional to the size of the batch. These oscillations reduce and occasionally eliminate the positive effect of AA. To restore AA's advantage, we combine it with an adaptive moving average procedure that smoothes the oscillations and results in a more regular sequence of gradient descent updates. By monitoring the relative standard deviation between consecutive iterations, we also introduce a criterion to automatically assess whether the moving average is needed. We applied the method to the following DL instantiations: (i) multi-layer perceptrons (MLPs) trained on the open-source graduate admissions dataset for regression, (ii) physics informed neural networks (PINNs) trained on source data to solve 2d and 100d Burgers' partial differential equations (PDEs), and (iii) ResNet50 trained on the open-source ImageNet1k dataset for image classification. Numerical results obtained using up to 1,536 NVIDIA V100 GPUs on the OLCF supercomputer Summit showed the stabilizing effect of the moving average on AA for all the problems above.
Quasi-static magnetoconvection with a tilted magnetic field

A numerical study of convection with stress-free boundary conditions in the presence of an imposed magnetic field that is tilted with respect to the direction of gravity is carried out in the limit of small magnetic Reynolds number. The dynamics are investigated over a range of Rayleigh number Ra and Chandrasekhar numbers up to Q = 2×106, with the tilt angle between the gravity vector and imposed magnetic field vector fixed at 45◦. For a fixed value of Q and increasing Ra, the convection dynamics can be broadly characterized by three primary flow regimes: (1) quasitwo-dimensional convection rolls near the onset of convection; (2) isolated convection columns aligned with the imposed magnetic field; and (3) unconstrained convection reminiscent of nonmagnetic convection. The influence of varying Q and Ra on the various fields is analyzed. Heat and momentum transport, as characterized by the Nusselt and Reynolds numbers, are quantified and compared with the vertical field case. Ohmic dissipation dominates over viscous dissipation in all cases investigated. Various mean fields are investigated and their scaling behavior is analyzed. Provided Ra is sufficiently large, all investigated values of Q exhibit an inverse kinetic energy cascade that yields strong ‘zonal’ flows. Relaxation oscillations, as characterized by a quasi-periodic shift in the predominance of either the zonal or non-zonal component of the mean flow, appear for sufficiently large Ra and Q. 1 ar X iv :2 11 0. 14 72 1v 1 [ ph ys ic s. fl udy n] 2 7 O ct 2 02 1
Overleaf Example

Deep networks have shown remarkable results in the task of object detection. However, their performance suffers critical drops when they are subsequently trained on novel classes without any sample from the base classes originally used to train the model. This phenomenon is known as catastrophic forgetting. Recently, several incremental learning methods are proposed to mitigate catastrophic forgetting for object detection. Despite the effectiveness, these methods require co-occurrence of the unlabeled base classes in the training data of the novel classes. This requirement is impractical in many real-world settings since the base classes do not necessarily co-occur with the novel classes. In view of this limitation, we consider a more practical setting of complete absence of cooccurrence of the base and novel classes for the object detection task. We propose the use of unlabeled in-the-wild data to bridge the non co-occurrence caused by the missing base classes during the training of additional novel classes. To this end, we introduce a blind sampling strategy based on the responses of the baseclass model and pre-trained novel-class model to select a smaller relevant dataset from the large in-the-wild dataset for incremental learning. We then design a dual-teacher distillation framework to transfer the knowledge distilled from the baseand novel-class teacher models to the student model using the sampled inthe-wild data. Experimental results on the PASCAL VOC and MS COCO datasets show that our proposed method significantly outperforms other state-of-the-art class-incremental object detection methods when there is no co-occurrence between the base and novel classes during training. Our source code is available at https://github.com/dongnana777/Bridging-Non-Co-occurrence.
Generalized Shape Metrics on Neural Representations

Understanding the operation of biological and artificial networks remains a difficult and important challenge. To identify general principles, researchers are increasingly interested in surveying large collections of networks that are trained on, or biologically adapted to, similar tasks. A standardized set of analysis tools is now needed to identify how network-level covariates-such as architecture, anatomical brain region, and model organism-impact neural representations (hidden layer activations). Here, we provide a rigorous foundation for these analyses by defining a broad family of metric spaces that quantify representational dissimilarity. Using this framework, we modify existing representational similarity measures based on canonical correlation analysis and centered kernel alignment to satisfy the triangle inequality, formulate a novel metric that respects the inductive biases in convolutional layers, and identify approximate Euclidean embeddings that enable network representations to be incorporated into essentially any off-the-shelf machine learning method. We demonstrate these methods on large-scale datasets from biology (Allen Institute Brain Observatory) and deep learning (NAS-Bench-101). In doing so, we identify relationships between neural representations that are interpretable in terms of anatomical features and model performance.
Incremental Adaptive Dynamic Programming for Approximate Optimal Tracking Control: a Decoupled and Model-Free Approach

This paper proposes a new formulation for the optimal tracking control problem (OTCP) of Euler-Lagrange systems. This formulation extends the incremental adaptive dynamic programming (IADP) technique, a reinforcement learning based method for solving the robust optimal regulation control problem (RORCP), to learn the approximate solution to the OTCP. Departing from available solutions to the OTCP, our developed tracking control scheme settles the curse of complexity problem in value function approximation from a decoupled way, circumvents the learning inefﬁciency regarding varying desired trajectories by avoiding introducing a reference trajectory dynamics into the learning process, and requires neither an accurate nor identiﬁed dynamics through the time delay estimation technique. Speciﬁcally, we ﬁrst convert the intractable OTCP of a high-dimensional uncertain system into multiple manageable sub-RORCPs of low-dimensional incremental error sub-systems. Then, the resulting sub-RORCPs are approximately solved by IADP implemented as a parallel critic learning structure. The proposed tracking control scheme is developed with rigorous theoretical analysis of system stability and weight convergence, and validated numerically on a 6-DoF quadrotor and experimentally on a 3-DoF robot manipulator.
M\"obius group actions in the solvable chimera model

We study actions of Möbius group on two sub-populations in the solvable chimera model proposed by Abrams et al. Dynamics of global variables are given by two coupled Watanabe-Strogatz systems, one for each sub-population. At the first glance, asymptotic dynamics in the model seem to be very simple. For instance, in the stable chimera state distributions of oscillators perform a simple rotations after a certain (sufficiently large) moment. However, a closer look unveils that dynamics are subtler that what can be observed from evolution of densities of oscillators’ phases. In order to gain the full picture, one needs to investigate dynamics on the transformation group that acts on these densities. Such an approach emphasizes impact of the "hidden" variable that is not visible on macroscopic level.
How Important Is Secular Evolution for Black Hole and Neutron Star Mergers in 2+2 and 3+1 Quadruple-star Systems?

Mergers of black holes (BHs) and neutron stars (NSs) result in the emission of gravitational waves that can be detected by LIGO. In this paper, we look at 2+2 and 3+1 quadruple-star systems, which are common among massive stars, the progenitors of BHs and NSs. We carry out a detailed population synthesis of quadruple systems using the Multiple Stellar Evolution code, which seamlessly takes into consideration stellar evolution, binary and tertiary interactions, N-body dynamics, and secular evolution. We find that, although secular evolution plays a role in compact object (BH and NS) mergers, (70–85)% (depending on the model assumptions) of the mergers are solely due to common envelope evolution. Significant eccentricities in the LIGO band (higher than 0.01) are only obtained with zero supernova (SN) kicks and are directly linked to the role of secular evolution. A similar outlier effect is seen in the χ eff distribution, with negative values obtained only with zero SN kicks. When kicks are taken into account, there are no systems that evolve into a quadruple consisting of four compact objects. For our fiducial model, we estimate the merger rates (in units of Gpc−3 yr−1) in 2+2 quadruples (3+1 quadruples) to be 10.8 ± 0.9 (2.9 ± 0.5), 5.7 ± 0.6 (1.4 ± 0.4), and 0.6 ± 0.2 (0.7 ± 0.3) for BH–BH, BH–NS, and NS–NS mergers, respectively. The BH–BH merger rates represent a significant fraction of the current LIGO rates, whereas the other merger rates fall short of LIGO estimates.
Magneto-thermal instability in galaxy clusters II: Three-dimensional simulations

In the intracluster medium (ICM) of galaxies, exchanges of heat across magnetic field lines are strongly suppressed. This anisotropic heat conduction, in the presence of a large-scale temperature gradient, destabilizes the outskirts of galaxy clusters via the magneto-thermal instability (MTI), which could drive observed ICM turbulence. In this paper we continue our investigation of the MTI with 3D simulations using the Boussinesq code SNOOPY. We redress two issues intrinsic to our previous 2D study: an inverse energy cascade and the impossibility of dynamo action. Contrary to 2D simulations, we find inconsequential transport of energy across scales (most energy is dissipated at the same scale as its injection) and that turbulent eddies are vertically elongated at or below the thermal conduction length yet relatively isotropic on larger scales. Similar to 2D, however, the saturated turbulent energy levels and the integral scale follow clear power-laws that depend on the thermal diffusivity, temperature gradient, and buoyancy frequency. We also show that the MTI amplifies magnetic fields, through a fluctuation dynamo, to equipartition strengths provided that the integral scale of MTI turbulence is larger than the viscous dissipation scale. Finally, we show that our scaling laws are consistent with extant observations of ICM turbulence if the thermal conductivity is reduced by a factor of ∼10 from its Spitzer value, and that on global cluster scales the stable stratification significantly reduces the vertical elongation of MTI motions.
Pipeline Parallelism for Inference on Heterogeneous Edge Computing

Deep neural networks with large model sizes achieve state-of-the-art results for tasks in computer vision (CV) and natural language processing (NLP). However, these large-scale models are too compute- or memory-intensive for resource-constrained edge devices. Prior works on parallel and distributed execution primarily focus on training -- rather than inference -- using homogeneous accelerators in data centers. We propose EdgePipe, a distributed framework for edge systems that uses pipeline parallelism to both speed up inference and enable running larger (and more accurate) models that otherwise cannot fit on single edge devices. EdgePipe achieves these results by using an optimal partition strategy that considers heterogeneity in compute, memory, and network bandwidth. Our empirical evaluation demonstrates that EdgePipe achieves $10.59\times$ and $11.88\times$ speedup using 16 edge devices for the ViT-Large and ViT-Huge models, respectively, with no accuracy loss. Similarly, EdgePipe improves ViT-Huge throughput by $3.93\times$ over a 4-node baseline using 16 edge devices, which independently cannot fit the model in memory. Finally, we show up to $4.16\times$ throughput improvement over the state-of-the-art PipeDream when using a heterogeneous set of devices.
On the Importance and Shortcomings of Code Readability Metrics: A Case Study on Reactive Programming

Well structured and readable source code is a pre-requisite for maintainable software and successful collaboration among developers. Static analysis enables the automated extraction of code complexity and readability metrics which can be leveraged to highlight potential improvements in code to both attain software of high quality and reinforce good practices for developers as an educational tool. This assumes reliable readability metrics which are not trivial to obtain since code readability is somewhat subjective. Recent research has resulted in increasingly sophisticated models for predicting readability as perceived by humans primarily with a procedural and object oriented focus, while functional and declarative languages and language extensions advance as they often are said to lead to more concise and readable code. In this paper, we investigate whether the existing complexity and readability metrics reflect that wisdom or whether the notion of readability and its constituents requires overhaul in the light of programming language changes. We therefore compare traditional object oriented and reactive programming in terms of code complexity and readability in a case study. Reactive programming is claimed to increase code quality but few studies have substantiated these claims empirically. We refactored an object oriented open source project into a reactive candidate and compare readability with the original using cyclomatic complexity and two state-of-the-art readability metrics. More elaborate investigations are required, but our findings suggest that both cyclomatic complexity and readability decrease significantly at the same time in the reactive candidate, which seems counter-intuitive. We exemplify and substantiate why readability metrics may require adjustment to better suit popular programming styles other than imperative and object-oriented to better match human expectations.
First Report on Screening of the Profiles of the Essential Oils and Volatiles from the Aerial Parts of Marrubium persicum Using Classical and Advanced Methods Prior to Gas Chromatographic Mass Spectrometric Determination

The Lamiaceae family consists of a broad spectrum of medicinal plants involving Marrubium L. genus. Regarding the diverse pharmaceutical uses of the plants belonging to this genus, they can be considered as proper alternatives for chemical drugs having harmful effects. The present work aims to identify and characterize chemical compositions of the essential oils and volatiles from the aerial partsof Marrubium persicum C. A. Mey. as an herbal plant in Iran using classical hydrodistillation. To establish a comprehensive comparison between the traditional techniques and advanced ones, microwave-based extraction techniques namely MAHD as well as SFME have also been utilized. In another part of this project, the profiles related to the volatile fractions from the aerial parts of Marrubium persicum C. A. Mey.have been assessed and compared with the other categories. The main components in the hydrodistillation (HD) method were α-pinene (21.5%), spathulenol (19.5%), α-thujene (17.4%), while the headspace solid-phase microextraction(HS-SPME) profile mainly consisted of β-caryophyllene (14%), eugenol (11.2%) and methyl eugenol (10.2%). On the other hand, using the SFME approach spathulenol (25.4%), α-pinene (17.4%) and germacrene D (9.5%) were found as the most abundant constituents. Moreover, in the MAHD profile caryophyllene oxide (13.1%), δ-elemene (12.4%), camphene (8.5%) were respectively the predominant natural compounds. According to gas chromatographic-mass spectrometric determinations, a total of 40 compounds were recognized in the corresponding profiles totally covering 94.6-99.7% of the whole chemical compositions. Sesquiterpene hydrocarbons were recognized as the most frequent groups of natural compounds in the profiles of the advanced approaches, whereas in the traditional one monoterpene hydrocarbons were found to be the dominant constituting group.
Mapping conditional distributions for domain adaptation under generalized target shift

We consider the problem of unsupervised domain adaptation (UDA) between a source and a target domain under conditional and label shift a.k. a Generalized Target Shift (GeTarS). Unlike simpler UDA settings, few works have addressed this challenging problem. Recent approaches learn domain-invariant representations, yet they have practical limitations and rely on strong assumptions that may not hold in practice. In this paper, we explore a novel and general approach to align pretrained representations, which circumvents existing drawbacks. Instead of constraining representation invariance, it learns an optimal transport map, implemented as a NN, which maps source representations onto target ones. Our approach is flexible and scalable, it preserves the problem's structure and it has strong theoretical guarantees under mild assumptions. In particular, our solution is unique, matches conditional distributions across domains, recovers target proportions and explicitly controls the target generalization risk. Through an exhaustive comparison on several datasets, we challenge the state-of-the-art in GeTarS.
Improving Super-Resolution Performance Using Meta-Attention Layers

Convolutional Neural Networks (CNNs) have achieved impressive results across many super-resolution (SR) and image restoration tasks. While many such networks can upscalelow-resolution (LR) images using just the raw pixel-level information, the ill-posed nature of SR can make it difficult to accurately super-resolve an image which has undergone multiple different degradations. Additional information (metadata) describing the degradation process (such as the blur kernel applied, compression level, etc.) can guide networks to super-resolve LR images with higher fidelity to the original source. Previous attempts at informing SR networks with degradation parameters have indeed been able to improve performance in a number of scenarios. However, due to the fully-convolutional nature of many SR networks, most of these metadata fusion methods either require a complete architectural change, or necessitate the addition of significant extra complexity. Thus, these approaches are difficult to introduce into arbitrary SR networks without considerable design alterations. In this letter, we introduce meta-attention, a simple mechanism which allows any SR CNN to exploit the information available in relevant degradation parameters. The mechanism functions by translating the metadata into a channel attention vector, which in turn selectively modulates the network's feature maps. Incorporating meta-attention into SR networks is straightforward, as it requires no specific type of architecture to function correctly. Extensive testing has shown that meta-attention can consistently improve the pixel-level accuracy of state-of-the-art (SOTA) networks when provided with relevant degradation metadata. Despite average memory/runtime overheads of less than $\approx$2.6%/0.025 seconds for the datasets and models considered, meta-attention improves the performance for both PSNR and SSIM; for PSNR, the gain onblurred/downsampled (×4) images is of 0.2969 dB (on average) and 0.3320 dB for SOTA general and face SR models, respectively. The coding framework used for this letter is available at: https://github.com/um-dsrg/Super-Resolution-Meta-Attention-Networks.