Miracle-dz/UniFAR-FTUs huggingface dataset:
Implicit Bilevel Optimization: Differentiating through Bilevel Optimization Programming

Bilevel Optimization Programming is used to model complex and conflicting interactions between agents, for example in Robust AI or Privacy preserving AI. Integrating bilevel mathematical programming within deep learning is thus an essential objective for the Machine Learning community. Previously proposed approaches only consider single-level programming. In this paper, we extend existing single-level optimization programming approaches and thus propose Differentiating through Bilevel Optimization Programming (BiGrad) for end-to-end learning of models that use Bilevel Programming as a layer. BiGrad has wide applicability and can be used in modern machine learning frameworks. BiGrad is applicable to both continuous and combinatorial Bilevel optimization problems. We describe a class of gradient estimators for the combinatorial case which reduces the requirements in terms of computation complexity; for the case of the continuous variable, the gradient computation takes advantage of the push-back approach (i.e. vector-jacobian product) for an efficient implementation. Experiments show that the BiGrad successfully extends existing single-level approaches to Bilevel Programming.
Discrete-Time Optimal Covariance Steering via Semidefinite Programming

This paper addresses the optimal covariance steering problem for stochastic discrete-time linear systems subject to probabilistic state and control constraints. A method is presented to efficiently attain the exact solution of the problem based on a lossless convex relaxation of the original non-linear program using semidefinite programming. Both the constrained and the unconstrained versions of the problem with either equality or inequality terminal covariance boundary conditions are addressed. We first prove that the proposed relaxation is lossless for all of the above cases. Numerical examples are then provided to illustrate the proposed method. Finally, a comparative study is performed on systems of various sizes and steering horizons to illustrate the advantages of the proposed method in terms of computational resources compared to the state of the art.
Membership Analysis and 3D Kinematics of the Star-forming Complex around Trumpler 37 Using Gaia-DR3

Identifying and characterizing young populations of star-forming regions are crucial to unraveling their properties. In this regard, Gaia-DR3 data and machine-learning tools are very useful for studying large star-forming complexes. In this work, we analyze the ∼7.1 deg2 area of one of our Galaxy’s dominant feedback-driven star-forming complexes, i.e., the region around Trumpler 37. Using the Gaussian mixture and random-forest classifier methods, we identify 1243 highly probable members in the complex, of which ∼60% are new members and are complete down to the mass limit of ∼0.1–0.2 M ⊙. The spatial distribution of the stars reveals multiple clusters toward the complex, where the central cluster around the massive star HD 206267 reveals two subclusters. Of the 1243 stars, 152 have radial velocity, with a mean value of −16.41 ± 0.72 km s−1. We investigate stars’ internal and relative movement within the central cluster. The kinematic analysis shows that the cluster’s expansion is relatively slow compared to the whole complex. This slow expansion is possibly due to newly formed young stars within the cluster. We discuss these results in the context of hierarchical collapse and feedback-induced collapse mode of star formation in the complex.
A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking

The robustness of deep neural networks is usually lacking under adversarial examples, common corruptions, and distribution shifts, which becomes an important research problem in the development of deep learning. Although new deep learning methods and robustness improvement techniques have been constantly proposed, the robustness evaluations of existing methods are often inadequate due to their rapid development, diverse noise patterns, and simple evaluation metrics. Without thorough robustness evaluations, it is hard to understand the advances in the field and identify the effective methods. In this paper, we establish a comprehensive robustness benchmark called \textbf{ARES-Bench} on the image classification task. In our benchmark, we evaluate the robustness of 55 typical deep learning models on ImageNet with diverse architectures (e.g., CNNs, Transformers) and learning algorithms (e.g., normal supervised training, pre-training, adversarial training) under numerous adversarial attacks and out-of-distribution (OOD) datasets. Using robustness curves as the major evaluation criteria, we conduct large-scale experiments and draw several important findings, including: 1) there is an inherent trade-off between adversarial and natural robustness for the same model architecture; 2) adversarial training effectively improves adversarial robustness, especially when performed on Transformer architectures; 3) pre-training significantly improves natural robustness based on more training data or self-supervised learning. Based on ARES-Bench, we further analyze the training tricks in large-scale adversarial training on ImageNet. By designing the training settings accordingly, we achieve the new state-of-the-art adversarial robustness. We have made the benchmarking results and code platform publicly available.
Meta-Learning with Adaptive Weighted Loss for Imbalanced Cold-Start Recommendation

Sequential recommenders have made great strides in capturing a user's preferences. Nevertheless, the cold-start recommendation remains a fundamental challenge as they typically involve limited user-item interactions for personalization. Recently, gradient-based meta-learning approaches have emerged in the sequential recommendation field due to their fast adaptation and easy-to-integrate abilities. The meta-learning algorithms formulate the cold-start recommendation as a few-shot learning problem, where each user is represented as a task to be adapted. While meta-learning algorithms generally assume that task-wise samples are evenly distributed over classes or values, user-item interactions in real-world applications do not conform to such a distribution (e.g., watching favorite videos multiple times, leaving only positive ratings without any negative ones). Consequently, imbalanced user feedback, which accounts for the majority of task training data, may dominate the user adaptation process and prevent meta-learning algorithms from learning meaningful meta-knowledge for personalized recommendations. To alleviate this limitation, we propose a novel sequential recommendation framework based on gradient-based meta-learning that captures the imbalanced rating distribution of each user and computes adaptive loss for user-specific learning. Our work is the first to tackle the impact of imbalanced ratings in cold-start sequential recommendation scenarios. Through extensive experiments conducted on real-world datasets, we demonstrate the effectiveness of our framework.
Multi-Layer Attention-Based Explainability via Transformers for Tabular Data

We propose a graph-oriented attention-based explainability method for tabular data. Tasks involving tabular data have been solved mostly using traditional tree-based machine learning models which have the challenges of feature selection and engineering. With that in mind, we consider a transformer architecture for tabular data, which is amenable to explainability, and present a novel way to leverage self-attention mechanism to provide explanations by taking into account the attention matrices of all layers as a whole. The matrices are mapped to a graph structure where groups of features correspond to nodes and attention values to arcs. By finding the maximum probability paths in the graph, we identify groups of features providing larger contributions to explain the model's predictions. To assess the quality of multi-layer attention-based explanations, we compare them with popular attention-, gradient-, and perturbation-based explanability methods.
Investigating the Effectiveness of Task-Agnostic Prefix Prompt for Instruction Following

In this paper, we present our finding that prepending a Task-Agnostic Prefix Prompt (TAPP) to the input improves the instruction-following ability of various Large Language Models (LLMs) during inference. TAPP is different from canonical prompts for LLMs in that it is a fixed prompt prepended to the beginning of every input regardless of the target task for zero-shot generalization. We observe that both base LLMs (i.e. not fine-tuned to follow instructions) and instruction-tuned models benefit from TAPP, resulting in 34.58% and 12.26% improvement on average, respectively. This implies that the instruction-following ability of LLMs can be improved during inference time with a fixed prompt constructed with simple heuristics. We hypothesize that TAPP assists language models to better estimate the output distribution by focusing more on the instruction of the target task during inference. In other words, such ability does not seem to be sufficiently activated in not only base LLMs but also many instruction-fine-tuned LLMs.
VRA: Out-of-Distribution Detection with variational rectified activations

Detecting out-of-distribution (OOD) data is critical to building reliable machine learning systems in the open world. Among the existing OOD detection meth-ods, ReAct is famous for its simplicity and efﬁciency, and has good theoretical analysis. The gap between ID data and OOD data is enlarged by clipping the larger activation value. But the question is, is this operation optimal? Is there a better way to expand the spacing between ID samples and OOD samples in theory? Driven by these questions, we propose the Variational Reciﬁed Acitvations method. To verify the effectiveness of our method, we conduct experiments on many benchmark datasets. Experimental results demonstrate that our method out-performs existing state-of-the-art approaches. Meanwhile, our method is easy to implement and does not require additional OOD data or ﬁne-tuning process. We can realize OOD detection in only one forward pass.
Understanding The Robustness of Self-supervised Learning Through Topic Modeling

Self-supervised learning has significantly improved the performance of many NLP tasks. However, how can self-supervised learning discover useful representations, and why is it better than traditional approaches such as probabilistic models are still largely unknown. In this paper, we focus on the context of topic modeling and highlight a key advantage of self-supervised learning - when applied to data generated by topic models, self-supervised learning can be oblivious to the specific model, and hence is less susceptible to model misspecification. In particular, we prove that commonly used self-supervised objectives based on reconstruction or contrastive samples can both recover useful posterior information for general topic models. Empirically, we show that the same objectives can perform on par with posterior inference using the correct model, while outperforming posterior inference using misspecified models.
Out-of-Plane Behavior of One-Way Spanning Unreinforced Masonry Walls

An analytical model is developed to describe the out-of-plane response of one-way spanning unreinforced masonry (URM) walls by investigating the effects of various parameters. Horizontal crack height, masonry compressive strength, and diaphragm support stiffness properties are assumed as variables, and sensitivity analyses are performed to study the influence of these parameters on the cracked wall characteristic behavior. The parametric studies show that crack height significantly influences wall stability by affecting both the instability displacement and the wall lateral resistance. The reduction in cracked wall lateral resistance and in the instability displacement caused by finite masonry compressive strength is shown to be significantly amplified by the applied overburden. A study using the typical configuration of flexible diaphragms and URM walls indicates that the wall top support flexibility does not significantly influence cracked wall out-of-plane response. An existing simplified wall behavioral model is improved, and a procedure is proposed for calculation of the wall out-of-plane response envelope.
Pump-to-signal crosstalk in Raman amplifier systems using polarization-multiplexed pump lasers

We have analyzed the relative intensity noise (RIN) of the pump lasers transferred to the signal in a Raman amplifier system using polarization-multiplexed pump lasers. The result shows that this RIN transfer could be reduced by 3 dB using the polarization-multiplexed pump lasers instead of single pump laser. In a Raman amplifier system, the pump modulation could be used for line monitoring. We have also analyzed the power penalty caused by the modulated pump laser in such a system.
Spatial structure and benefits to hosts allow plasmids with and without post-segregational killing systems to coexist

To persist, a plasmid relies on being passed on to a daughter cell, but this does not always occur. Plasmids with post-segregational killing (PSK) systems kill a daughter cell if the plasmid has not been passed on. By killing the host, it also kills competing plasmids in the same host, something competing plasmids without a similar system cannot do. Accordingly, plasmids with PSK systems can displace other plasmids. In nature, plasmids with and without PSK systems coexist and prior theory has suggested this is expected to be very rare or unstable, such that one or the other type of plasmid eventually takes over. Here, we show that if there is spatial structure and plasmids confer benefits to hosts, coexistence of plasmids occurs broadly. Often plasmids confer benefits (even ones with a PSK system) and bacteria are often spatially structured. So, our results may be generally applicable.
Strain rate sensitivity of a nanocrystalline Cu synthesized by electric brush plating

A method for synthesizing bulk nanocrystalline Cu by an electric brush-plating technique is reported. This brush-plated nanocrystalline Cu has a fine (26nm) and quite uniform grain structure and predominant high-angle grain boundaries. A pronounced strain rate sensitivity of the stress with an m of 0.104 and the Coble creep and a subsequent transition to the power-law creep were observed in room temperature tensile and creep tests. The dominant grain boundary deformation due to the truly nanocrystalline structure of this nanocrystalline Cu is responsible for the observed strain rate sensitivity.
Stability of a layered reactive channel flow

We analyse the linear stability of a reactive plane Poiseuille flow, where a reactant fluid A overlies another reactant B in a layered fashion within a two-dimensional channel. Both reactants are miscible and have the same viscosity, while upon reaction, they produce either a less or more-viscous product fluid C. The reaction kinetics is of simple A+B→C type, and the production of C occurs across the initial contact line of reactants A and B in a mixed zone of small and finite width. All three fluids have the same density. We demonstrate the effects of various controlling parameters such as the log-mobility ratio, Damköhler number, Schmidt number, Reynolds number, position and thicknesses of the reactive zone on the stability characteristics. We show that a tiny viscosity stratification by the reaction destabilizes the flow at a moderate (10–1000) and even at low Reynolds numbers (0.01–1). The maximum growth occurs for shorter waves than for the Tollmien–Schlichting eigenmode, and the ranges of unstable wavenumbers are wider than that known for non-reactive channel flow systems. In most cases, the instability occurs due to the overlap of the critical layer with the viscosity-stratified layer. Surprisingly for some parameters, it is observed that the reaction can make σM decrease with increasing Reynolds number.
Non-trace full-F gyro-fluid interchange impurity advection

A full-F isothermal gyro-fluid model and code (which is based on the full distribution function F compared to only small fluctuations) is extended to handle self-consistent coupling of multiple quasi-neutral ion species via the polarisation equation in the long wavelength approximation. The numerical model is used to determine two-dimensional interchange driven ‘blob’ transport in a plasma with intrinsic impurity content for a range of impurity parameters. With the same model, the self-consistent advective interaction of a main plasma species blob with a non-trace impurity cloud is studied. For homogeneous impurity distributions an increased effective mass reduces blob transport, whereas it is found that localised impurity clouds can lead either to acceleration or slowing down of blob propagation depending on the alignment of the impurity density gradient during the acceleration phase of the main ion species blob.
Remarks on transition in a round tube

This article has a twofold purpose: (1) to analyse the available theoretical and experimental knowledge concerning flow in the inlet region of a smooth round tube, and (2) to point out that the e9 amplification factor method apparently predicts natural transition correctly over a significant fraction of the entire inlet lenght of the tube. The successful prediction indicates, but does not prove, that flow in a smooth round tube becomes turbulent at higher Reynolds numbers because transition occurs in the inlet lenght—not in the fully developed Poiseuille régime. The close agreement between theory and a test result obtained by Pfenninger indicates that the e9 method is valid for a wide variety of flows having x Reynolds numbers of transition ranging from 570,000 to 40 million. The results are applicable to both plane and axially symmetric flows.
Natural convection between two horizontal cylinders in an adiabatic circular enclosure

A numerical study of natural convection flow structure and heat transfer has been undertaken for air around two horizontal, differentially heated cylinders confined to an adiabatic circular enclosure. Parametric simulations were performed to assess the effects of gap width between cylinders as well as the inclination angle of the enclosure with respect to gravity. Flow visualization experiments were conducted for the physical configuration under consideration
Supercontinuum in integrated photonics: generation, applications, challenges, and perspectives

Abstract Frequency conversion in nonlinear materials is an extremely useful solution to the generation of new optical frequencies. Often, it is the only viable solution to realize light sources highly relevant for applications in science and industry. In particular, supercontinuum generation in waveguides, defined as the extreme spectral broadening of an input pulsed laser light, is a powerful technique to bridge distant spectral regions based on single-pass geometry, without requiring additional seed lasers or temporal synchronization. Owing to the influence of dispersion on the nonlinear broadening physics, supercontinuum generation had its breakthrough with the advent of photonic crystal fibers, which permitted an advanced control of light confinement, thereby greatly improving our understanding of the underlying phenomena responsible for supercontinuum generation. More recently, maturing in fabrication of photonic integrated waveguides has resulted in access to supercontinuum generation platforms benefiting from precise lithographic control of dispersion, high yield, compact footprint, and improved power consumption. This Review aims to present a comprehensive overview of supercontinuum generation in chip-based platforms, from underlying physics mechanisms up to the most recent and significant demonstrations. The diversity of integrated material platforms, as well as specific features of waveguides, is opening new opportunities, as will be discussed here.
Evidence of a fractional Debye-Stokes-Einstein law in supercooled o-terphenyl

The rotational dynamics of nearly spherical and cylindrical tracers dissolved in supercooled o-terphenyl is studied via the Electron Spin Resonance spectroscopy. On decreasing the temperature, a sharp crossover is observed from the regime where the Debye-Stokes-Einstein law (DSE) holds to a new one which is accounted for by a fractional DSE (FDSE) Dr (η/T)−ξ with ξ ≤ 1. Dr is the rotational diffusion constant, η the shear viscosity and T the temperature. Tumbling and spinning of the cylindrical tracer exhibit distinct crossovers. A simple model of FDSE based on the presence of dynamic heterogeneities is presented.
Correlated hard X-ray and ultraviolet variability in NGC 5548

Contemporaneous X-ray (from the Ginga satellite) and ultraviolet (IUE) observations of NGC 5548 in 1989-1990 show that the 1350 A and 2-10 keV flux both varied by a factor of 2.6 in a highly correlated fashion. Adding an additional simultaneous IUE-Exosat observation from 1984 significantly weakens the correlation, since the UV flux was 3.5 times larger then than its average 1990 value, while the X-ray flux was only 70 percent higher. In 1990 the UV and X-ray flux variations appear to be simultaneous to within no more than six days. These results, together with the simultaneity of the variations at 5000 and 1350 A, are incompatible with the standard geometrically thin accretion model. They may be explained, however, if part of the UV and optical radiation originates from reprocessing of X-rays emitted above the disk. Thus essentially all the UV flux in 1990 would be due to reprocessing. But the large UV outburst in 1984 probably represents a genuine increase in the disk throughput due to instability in the disk itself. The similar relationship between UV and X-ray emission in NGC 4151 suggests that this scheme may be applicable to Seyfert galaxies as a class.
Multi-principle preventive maintenance: a design-oriented scheduling study for mechanical systems

Preventive maintenance (PM) is very important for the safe, efficient, and reliable operation of mechanical systems. This paper focuses on one of the most challenging tasks for PM: PM scheduling. Two basic principles are integrated to support the PM scheduling of mechanical systems: (1) the cost principle, and (2) the reliability principle. These two PM scheduling principles are regarded as conflicting objectives, and the improved strength Pareto evolutionary algorithm is used to find the Pareto-optimal set within which the best compromise solution can be obtained according to fuzzy set theory. Both conceptual and mathematical models of the proposed multi-principle PM scheduling method are explained, and a case study is provided to illustrate the practical application of the new method. 概要研究目的为复杂机械产品提供满足整机可靠性指标和维护成本指标的预防性维护方案多准则规划方法。 创新要点1. 分析了检查、 维修、 更换等对复杂机械产品零部件工作寿命变化的作用机理; 2. 提出了复杂机械产品预防性维护多准则规划方法。 研究方法1. 基于非完美维修理论, 建立不同模式下零件间工作寿命模型, 定义维修效能因子, 表征检查、 维修、 更换对零件寿命的影响; 2. 通过求解获得复杂机械产品指定时间区间的预防性维护方案, 根据零部件工作寿命, 采取维修和更换等预防性维护措施, 减少零部件故障的发生。 重要结论零部件的预防性维护次数与其故障因子相关; 机械产品尤其是复杂机械产品实施定期预防性维护能够减少或消除故障的发生。
Osdes_net: oil spill detection based on efficient_shuffle network using synthetic aperture radar imagery

Abstract Synthetic Aperture Radar (SAR) imagery can be beneficial for segmenting oil spills, which are a common environmental hazard. Oil spill detection in SAR imagery faces several challenges, including speckle noise, heterogeneous backgrounds, blurred edges, and a lack of comprehensive datasets with multiple images. ShuffleNet is one of the deep networks, which has never been used for oil spill segmentation. In this article, ShuffleNet blocks are used to detect oil spills in SAR images, which is more effective than other methods. Besides, the main network design, six other blocks were evaluated, and the most valuable one was selected. We use group convolutions, shuffle channels, and atrous convolutions in this model with a minimum number of layers of ReLU. The methods are evaluated based on the Intersection Over Union (IoU) parameter so that the proposed method improved the mIoU by 7.1% over the best results of some previous methods.