Miracle-dz/UniFAR-FTUs huggingface dataset:
Towards Generalizable Reinforcement Learning for Trade Execution

Optimized trade execution is to sell (or buy) a given amount of assets in a given time with the lowest possible trading cost. Recently, reinforcement learning (RL) has been applied to optimized trade execution to learn smarter policies from market data. However, we find that many existing RL methods exhibit considerable overfitting which prevents them from real deployment. In this paper, we provide an extensive study on the overfitting problem in optimized trade execution. First, we model the optimized trade execution as offline RL with dynamic context (ORDC), where the context represents market variables that cannot be influenced by the trading policy and are collected in an offline manner. Under this framework, we derive the generalization bound and find that the overfitting issue is caused by large context space and limited context samples in the offline setting. Accordingly, we propose to learn compact representations for context to address the overfitting problem, either by leveraging prior knowledge or in an end-to-end manner. To evaluate our algorithms, we also implement a carefully designed simulator based on historical limit order book (LOB) data to provide a high-fidelity benchmark for different algorithms. Our experiments on the high-fidelity simulator demonstrate that our algorithms can effectively alleviate overfitting and achieve better performance.
Fault‐Tolerant Control Based on Augmented State Estimator and PDF

A new fault‐tolerant control based on augmented state estimator and probability density function (PDF) is proposed for a stochastic distribution system (SDS) with time‐delay and additive fault. First, a system model based on a PDF with the additive fault is constructed by using square‐root rational B‐spline neural networks. Second, an augmented system is obtained by converting the additive fault as an auxiliary state variable. In this framework, a robust augmented state estimator is designed to estimate the original states and the additive fault simultaneously. Then, based on the obtained estimation of fault, a delay‐dependent fault‐tolerant control is designed to compensate the fault. Finally, the numerical simulations show the effectiveness of the proposed method.
PI-VEGAN: Physics Informed Variational Embedding Generative Adversarial Networks for Stochastic Differential Equations

We present a new category of physics-informed neural networks called physics informed variational embedding generative adversarial network (PI-VEGAN), that effectively tackles the forward, inverse, and mixed problems of stochastic differential equations. In these scenarios, the governing equations are known, but only a limited number of sensor measurements of the system parameters are available. We integrate the governing physical laws into PI-VEGAN with automatic differentiation, while introducing a variational encoder for approximating the latent variables of the actual distribution of the measurements. These latent variables are integrated into the generator to facilitate accurate learning of the characteristics of the stochastic partial equations. Our model consists of three components, namely the encoder, generator, and discriminator, each of which is updated alternatively employing the stochastic gradient descent algorithm. We evaluate the effectiveness of PI-VEGAN in addressing forward, inverse, and mixed problems that require the concurrent calculation of system parameters and solutions. Numerical results demonstrate that the proposed method achieves satisfactory stability and accuracy in comparison with the previous physics-informed generative adversarial network (PI-WGAN).
Control- & Task-Aware Optimal Design of Actuation System for Legged Robots Using Binary Integer Linear Programming

Athletic robots demand a whole-body actuation system design that utilizes motors up to the boundaries of their performance. However, creating such robots poses challenges of integrating design principles and reasoning of practical design choices. This paper presents a design framework that guides designers to find optimal design choices to create an actuation system that can rapidly generate torques and velocities required to achieve a given set of tasks, by minimizing inertia and leveraging cooperation between actuators. The framework serves as an interactive tool for designers who are in charge of providing design rules and candidate components such as motors, reduction mechanism, and coupling mechanisms between actuators and joints. A binary integer linear optimization explores design combinations to find optimal components that can achieve a set of tasks. The framework is demonstrated with 200 optimal design studies of a biped with 5-degree-of-freedom (DoF) legs, focusing on the effect of achieving multiple tasks (walking, lifting), constraining the mass budget of all motors in the system and the use of coupling mechanisms. The result provides a comprehensive view of how design choices and rules affect reflected inertia, copper loss of motors, and force capability of optimal actuation systems.
Signature of nodal topology in nonlinear quantum transport across junctions in Weyl and multi-Weyl semimetals

We investigate quantum transport through a rectangular potential barrier in Weyl semimetals (WSMs) and multi-Weyl semimetals (MSMs), within the framework of Landauer-B\"uttiker formalism. Our study uncovers the role of nodal topology imprinted in the electric current and the shot noise. We find that, in contrast to the finite odd-order conductance and noise power, the even-order contributions vanish at the nodes. Additionally, depending on the topological charge ($J$), the linear conductance ($G_1$) scales with the Fermi energy ($E_F$) as $G_1^{E_F>U}\propto E_F^{2/J}$. We demonstrate that the $E_F$-dependence of the second-order conductance and shot noise power could quite remarkably distinguish an MSM from a WSM depending on the band topology, and may induce several smoking gun experiments in nanostructures made out of WSMs and MSMs. Analyzing shot noise and Fano factor, we show that the transport across the rectangular barrier follows the sub-Poissonian statistics. Interestingly, we obtain universal values of Fano factor at the nodes unique to their topological charges. The universality for a fixed $J$, however, indicates that only a fixed number of open channels participate in the transport through evanescent waves at the nodes. The proposed results can serve as a potential diagnostic tool to identify different topological systems in experiments.
Towards Better Fairness-Utility Trade-off: A Comprehensive Measurement-Based Reinforcement Learning Framework

Machine learning is widely used to make decisions with societal impact such as bank loan approving, criminal sentencing, and resume filtering. How to ensure its fairness while maintaining utility is a challenging but crucial issue. Fairness is a complex and context-dependent concept with over 70 different measurement metrics. Since existing regulations are often vague in terms of which metric to use and different organizations may prefer different fairness metrics, it is important to have means of improving fairness comprehensively. Existing mitigation techniques often target at one specific fairness metric and have limitations in improving multiple notions of fairness simultaneously. In this work, we propose CFU (Comprehensive Fairness-Utility), a reinforcement learning-based framework, to efficiently improve the fairness-utility trade-off in machine learning classifiers. A comprehensive measurement that can simultaneously consider multiple fairness notions as well as utility is established, and new metrics are proposed based on an in-depth analysis of the relationship between different fairness metrics. The reward function of CFU is constructed with comprehensive measurement and new metrics. We conduct extensive experiments to evaluate CFU on 6 tasks, 3 machine learning models, and 15 fairness-utility measurements. The results demonstrate that CFU can improve the classifier on multiple fairness metrics without sacrificing its utility. It outperforms all state-of-the-art techniques and has witnessed a 37.5% improvement on average.
Well-posedness for a non-isothermal flow of two viscous incompressible fluids

This work is concerned with a non-isothermal diffuse-interface model which describes the motion of a mixture of two viscous incompressible fluids. The model consists of modified Navier-Stokes equations coupled with a phase-field equation given by a convective Allen-Cahn equation, and energy transport equation for the temperature. This model admits a dissipative energy inequality. It is investigated the well-posedness of the problem in the two and three dimensional cases without any restriction on the size of the initial data. Moreover, regular and singular potentials for the phase-field equation are considered.
Distribution Shift Matters for Knowledge Distillation with Webly Collected Images

Knowledge distillation aims to learn a lightweight student network from a pre-trained teacher network. In practice, existing knowledge distillation methods are usually infeasible when the original training data is unavailable due to some privacy issues and data management considerations. Therefore, data-free knowledge distillation approaches proposed to collect training instances from the Internet. However, most of them have ignored the common distribution shift between the instances from original training data and webly collected data, affecting the reliability of the trained student network. To solve this problem, we propose a novel method dubbed "Knowledge Distillation between Different Distributions" (KD3), which consists of three components. Specifically, we first dynamically select useful training instances from the webly collected data according to the combined predictions of teacher network and student network. Subsequently, we align both the weighted features and classifier parameters of the two networks for knowledge memorization. Meanwhile, we also build a new contrastive learning block called MixDistribution to generate perturbed data with a new distribution for instance alignment, so that the student network can further learn a distribution-invariant representation. Intensive experiments on various benchmark datasets demonstrate that our proposed KD3 can outperform the state-of-the-art data-free knowledge distillation approaches.
Quantum Software Analytics: Opportunities and Challenges

Quantum computing systems depend on the principles of quantum mechanics to perform multiple challenging tasks more efficiently than their classical counterparts. In classical software engineering, the software life cycle is used to document and structure the processes of design, implementation, and maintenance of software applications. It helps stakeholders understand how to build an application. In this paper, we summarize a set of software analytics topics and techniques in the development life cycle that can be leveraged and integrated into quantum software application development. The results of this work can assist researchers and practitioners in better understanding the quantum-specific emerging development activities, challenges, and opportunities in the next generation of quantum software.
On the mini-symposium problem

The generalized Oberwolfach problem asks for a factorization of the complete graph $K_v$ into prescribed $2$-factors and at most a $1$-factor. When all $2$-factors are pairwise isomorphic and $v$ is odd, we have the classic Oberwolfach problem, which was originally stated as a seating problem: given $v$ attendees at a conference with $t$ circular tables such that the $i$th table seats $a_i$ people and ${\sum_{i=1}^t a_i = v}$, find a seating arrangement over the $\frac{v-1}{2}$ days of the conference, so that every person sits next to each other person exactly once. In this paper we introduce the related {\em minisymposium problem}, which requires a solution to the generalized Oberwolfach problem on $v$ vertices that contains a subsystem on $m$ vertices. That is, the decomposition restricted to the required $m$ vertices is a solution to the generalized Oberwolfach problem on $m$ vertices. In the seating context above, the larger conference contains a minisymposium of $m$ participants, and we also require that pairs of these $m$ participants be seated next to each other for $\left\lfloor\frac{m-1}{2}\right\rfloor$ of the days. When the cycles are as long as possible, i.e.\ $v$, $m$ and $v-m$, a flexible method of Hilton and Johnson provides a solution. We use this result to provide further solutions when $v \equiv m \equiv 2 \pmod 4$ and all cycle lengths are even. In addition, we provide extensive results in the case where all cycle lengths are equal to $k$, solving all cases when $m\mid v$, except possibly when $k$ is odd and $v$ is even.
An Analysis of Multi-Agent Reinforcement Learning for Decentralized Inventory Control Systems

Most solutions to the inventory management problem assume a centralization of information that is incompatible with organisational constraints in real supply chain networks. The inventory management problem is a well-known planning problem in operations research, concerned with finding the optimal re-order policy for nodes in a supply chain. While many centralized solutions to the problem exist, they are not applicable to real-world supply chains made up of independent entities. The problem can however be naturally decomposed into sub-problems, each associated with an independent entity, turning it into a multi-agent system. Therefore, a decentralized data-driven solution to inventory management problems using multi-agent reinforcement learning is proposed where each entity is controlled by an agent. Three multi-agent variations of the proximal policy optimization algorithm are investigated through simulations of different supply chain networks and levels of uncertainty. The centralized training decentralized execution framework is deployed, which relies on offline centralization during simulation-based policy identification, but enables decentralization when the policies are deployed online to the real system. Results show that using multi-agent proximal policy optimization with a centralized critic leads to performance very close to that of a centralized data-driven solution and outperforms a distributed model-based solution in most cases while respecting the information constraints of the system.
Explicit Constraints on the Geometric Rate of Convergence of Random Walk Metropolis-Hastings

Convergence rate analyses of random walk Metropolis-Hastings Markov chains on general state spaces have largely focused on establishing sufficient conditions for geometric ergodicity or on analysis of mixing times. Geometric ergodicity is a key sufficient condition for the Markov chain Central Limit Theorem and allows rigorous approaches to assessing Monte Carlo error. The sufficient conditions for geometric ergodicity of the random walk Metropolis-Hastings Markov chain are refined and extended, which allows the analysis of previously inaccessible settings such as Bayesian Poisson regression. The key technical innovation is the development of explicit drift and minorization conditions for random walk Metropolis-Hastings, which allows explicit upper and lower bounds on the geometric rate of convergence. Further, lower bounds on the geometric rate of convergence are also developed using spectral theory. The existing sufficient conditions for geometric ergodicity, to date, have not provided explicit constraints on the rate of geometric rate of convergence because the method used only implies the existence of drift and minorization conditions. The theoretical results are applied to random walk Metropolis-Hastings algorithms for a class of exponential families and generalized linear models that address Bayesian Regression problems.
Design of the offline test electronics for the nozzle system of proton therapy

A set of nozzle equipment for proton therapy is now being developed at China Institute of Atomic Energy. To facilitate the off-line commissioning of the whole equipment, a set of ionization chamber signal generation system, the test electronics, is designed. The system uses ZYNQ SoC as the main control unit and outputs the beam dose analog signal through DAC8532. The dual SPDT analog switch, DG636, is used to simulate the beam position signals according to Gaussian distribution. The results show that the system can simulate the beam position, dose, and other related analog signals generated by the proton beam when passing through the ionization chamber. Moreover, the accuracy of the simulated beam position is within +/-0.33mm, and the accuracy of the simulated dose signal is within +/-1%. At the same time, it can output analog signals representing environmental parameters. The test electronics meets the design requirements, which can be used to commission the nozzle system as well as the treatment control system without the proton beam.
Solving flows in porous media with a POD-Galerkin reduced order model coupled with multilayer perceptron

This paper deals with the numerical modeling of flow around and through a porous obstacle by a reduced order model (ROM) obtained by Galerkin projection of the Navier-Stokes equations onto a Proper Orthogonal Decomposition (POD) reduced basis. In the few existing works dealing with model reduction techniques applied to flows in porous media, flows were described by Darcy's law and the non linear Forchheimer term was neglected. This last term cannot be expressed in reduced form during the Galerkin projection phase. Indeeed, at each new time step, the norm of the velocity needs to be recalculated and projected, which significantly increases the computational cost, rendering the reduced model inefficient. To overcome this difficulty, we propose to model the projected Forchheimer term with artificial neural networks. Moreover in order to build a stable ROM, the influence of unresolved modes and pressure variations are also modeled using a neural network. Instead of separately modeling each term, these terms were combined into a single residual, which was modeled using the multilayer perceptron method (MLP). The validation of this approach was carried out for laminar flow past a porous obstacle in an unconfined channel. The proposed ROM coupled with MLP approach is able to accurately predict the dynamics of the flow while the standard ROM yields wrong results. Moreover, the ROM MLP method improves the prediction of flow for Reynols number that are not included in the sampling and for times longer than sampling times.
Multi-agent Deep Covering Skill Discovery

The use of skills (a.k.a., options) can greatly accelerate exploration in reinforcement learning, especially when only sparse reward signals are available. While option discovery methods have been proposed for individual agents, in multi-agent reinforcement learning settings, discovering collaborative options that can coordinate the behavior of multiple agents and encourage them to visit the under-explored regions of their joint state space has not been considered. In this case, we propose Multi-agent Deep Covering Option Discovery, which constructs the multi-agent options through minimizing the expected cover time of the multiple agents' joint state space. Also, we propose a novel framework to adopt the multi-agent options in the MARL process. In practice, a multi-agent task can usually be divided into some sub-tasks, each of which can be completed by a sub-group of the agents. Therefore, our algorithm framework first leverages an attention mechanism to find collaborative agent sub-groups that would benefit most from coordinated actions. Then, a hierarchical algorithm, namely HA-MSAC, is developed to learn the multi-agent options for each sub-group to complete their sub-tasks first, and then to integrate them through a high-level policy as the solution of the whole task. This hierarchical option construction allows our framework to strike a balance between scalability and effective collaboration among the agents. The evaluation based on multi-agent collaborative tasks shows that the proposed algorithm can effectively capture the agent interactions with the attention mechanism, successfully identify multi-agent options, and significantly outperforms prior works using single-agent options or no options, in terms of both faster exploration and higher task rewards.
Approximation of functions on a compact set by solutions of elliptic equations. Quantitative results

We establish that a generalized H\"{o}lder continuous function on an $(m-2)$-Ahlfors regular compact set in $\mathbb{R}^m$ can be approximated by solutions of an elliptic equation, with the rate of approximation determined by the continuity modulus of the function.
The evolutionary impact and influence of oestrogens on adipose tissue structure and function

Oestrogens are sex steroid hormones that have gained prominence over the years owing to their crucial roles in human health and reproduction functions which have been preserved throughout evolution. One of oestrogens actions, and the focus of this review, is their ability to determine adipose tissue distribution, function and adipose tissue ‘health’. Body fat distribution is sexually dimorphic, affecting males and females differently. These differences are also apparent in the development of the metabolic syndrome and other chronic conditions where oestrogens are critical. In this review, we summarize the different molecular mechanisms, pathways and resulting pathophysiology which are a result of oestrogens actions in and on adipose tissues. This article is part of a discussion meeting issue ‘Causes of obesity: theories, conjectures and evidence (Part I)’.
Characteristic length scales of strongly rotating Boussinesq flow in variable-aspect-ratio domains

We quantify the variability of the characteristic length scales of isotropically forced Boussinesq flows with stratification and frame rotation, as functions of the ratio $N/f$ of the Brunt–Väisälä frequency to the Coriolis frequency. The parameter ranges $0<N<f$ , domain aspect ratio $1\leqslant \unicode[STIX]{x1D6FF}_{d}\leqslant 32$ and Burger number $Bu=\unicode[STIX]{x1D6FF}_{d}N/f\leqslant 1$ are explored for two values of $f$ , one resulting in linear potential vorticity and the other in nonlinear potential vorticity. Characteristic length scales of the wave and vortical linear eigenmodes are separately quantified using $n$ th-order spectral moments in both horizontal and vertical directions, for integer $n\leqslant 3$ . In flows with linear potential vorticity, the horizontal vortical length scale $L_{0}$ , characterizing a typical width of columnar structures, grows as ${\sim}(N/f)^{1/2}$ at all orders of $n$ , regardless of domain aspect ratio. In unit-aspect-ratio domains, when intermediate scales are measured by filtering out the largest scales and using higher-order moments $n>1$ , the vortical-mode aspect ratio $\unicode[STIX]{x1D6FF}_{0}$ asymptotes to a scaling of ${\sim}(N/f)^{-1}$ , in agreement with quasi-geostrophic estimates. In contrast, the $\unicode[STIX]{x1D6FF}_{0}$ in tall-aspect-ratio domain flows yields a decay rate of at most ${\sim}(N/f)^{-1/2}$ after large-scale filtering. Flows with nonlinear potential vorticity display consistently weaker dependence of the characteristic scales on $N/f$ than the corresponding ones with linear potential vorticity. The wave-mode aspect ratios for all flows are essentially independent of $N/f$ . We highlight the differences of these flow structure scalings relative to those expected for quasi-geostrophic flows, and those observed in strongly stratified, non-quasi-geostrophic flows.