Miracle-dz/UniFAR-FTUs huggingface dataset:
MoT: Memory-of-Thought Enables ChatGPT to Self-Improve

Large Language Models (LLMs) have shown impressive abilities in various tasks. However, fundamentally improving them depends on high-quality datasets or computationally expensive fine-tuning. On the contrary, humans can easily improve themselves by self-thinking and memory, without external resources. In this paper, we propose a framework, MoT, to let the LLM self-improve through Memory-of-Thought, without annotated datasets and parameter updates. Specifically, MoT is divided into two stages: 1. before the test stage, the LLM pre-thinks on the unlabeled dataset and saves the high-confidence thoughts as external memory; 2. During the test stage, given a test question, the LLM recalls relevant memory to help itself reason and answer it. Experimental results show that MoT can help ChatGPT significantly improve its abilities in arithmetic reasoning, commonsense reasoning, factual reasoning, and natural language inference. Further analyses show that each component contributes critically to the improvements and MoT can lead to consistent improvements across various CoT methods and LLMs.
On Measurement Disturbances in Distributed Least Squares Solvers for Linear Equations

This paper aims at distributed algorithms for solving a system of linear algebraic equations. Different from most existing formulations for this problem, we assume that the local data at each node is not accurately measured but subject to some disturbances. To be specific, the local measurement consists of two parts: a nominal value and a multiple sinusoidal disturbance. By introducing an identifier-enhanced observer to estimate the disturbance, we present a novel distributed least squares solver for the linear equations using noisy measurements. The proposed solver is proven to be able to recover the least squares solution to the linear equations associated with the nominal values irrespective of any multi-sinusoidal disturbance even with unknown frequencies. We also show the robustness of the distributed solvers under standard conditions against unstructured perturbations. The effectiveness of our design is verified by a numerical example.
GPT-NAS: Evolutionary Neural Architecture Search with the Generative Pre-Trained Model

Neural Architecture Search (NAS) has emerged as one of the effective methods to design the optimal neural network architecture automatically. Although neural architectures have achieved human-level performances in several tasks, few of them are obtained from the NAS method. The main reason is the huge search space of neural architectures, making NAS algorithms inefficient. This work presents a novel architecture search algorithm, called GPT-NAS, that optimizes neural architectures by Generative Pre-Trained (GPT) model with an evolutionary algorithm (EA) as the search strategy. In GPT-NAS, we assume that a generative model pre-trained on a large-scale corpus could learn the fundamental law of building neural architectures. Therefore, GPT-NAS leverages the GPT model to propose reasonable architecture components given the basic one and then utilizes EAs to search for the optimal solution. Such an approach can largely reduce the search space by introducing prior knowledge in the search process. Extensive experimental results show that our GPT-NAS method significantly outperforms seven manually designed neural architectures and thirteen architectures provided by competing NAS methods. In addition, our experiments also indicate that the proposed algorithm improves the performance of finely tuned neural architectures by up to about 12% compared to those without GPT, further demonstrating its effectiveness in searching neural architectures.
A Survey on AI/ML-Driven Intrusion and Misbehavior Detection in Networked Autonomous Systems: Techniques, Challenges and Opportunities

AI/ML-based intrusion detection systems (IDSs) and misbehavior detection systems (MDSs) have shown great potential in identifying anomalies in the network traffic of networked autonomous systems. Despite the vast research efforts, practical deployments of such systems in the real world have been limited. Although the safety-critical nature of autonomous systems and the vulnerability of learning-based techniques to adversarial attacks are among the potential reasons, the lack of objective evaluation and feasibility assessment metrics is one key reason behind the limited adoption of these systems in practical settings. This survey aims to address the aforementioned limitation by presenting an in-depth analysis of AI/ML-based IDSs/MDSs and establishing baseline metrics relevant to networked autonomous systems. Furthermore, this work thoroughly surveys recent studies in this domain, highlighting the evaluation metrics and gaps in the current literature. It also presents key findings derived from our analysis of the surveyed papers and proposes guidelines for providing AI/ML-based IDS/MDS solution approaches suitable for vehicular network applications. Our work provides researchers and practitioners with the needed tools to evaluate the feasibility of AI/ML-based IDS/MDS techniques in real-world settings, with the aim of facilitating the practical adoption of such techniques in emerging autonomous vehicular systems.
Quantitative phase retrieval for Zernike phase-contrast microscopy

We present a single-image numerical phase retrieval method for Zernike phase-contrast microscopy (ZPM) that addresses halo and shade-off artifacts, as well as the weak phase condition, without requiring hardware modifications. By employing a rigorous physical model of ZPM and a gradient descent algorithm for its inversion, we achieve quantitative ZPM imaging. Our approach is experimentally validated using biological cells and its quantitative nature is confirmed through comparisons with digital holography observations.
A Unified Evaluation Framework for Novelty Detection and Accommodation in NLP with an Instantiation in Authorship Attribution

State-of-the-art natural language processing models have been shown to achieve remarkable performance in 'closed-world' settings where all the labels in the evaluation set are known at training time. However, in real-world settings, 'novel' instances that do not belong to any known class are often observed. This renders the ability to deal with novelties crucial. To initiate a systematic research in this important area of 'dealing with novelties', we introduce 'NoveltyTask', a multi-stage task to evaluate a system's performance on pipelined novelty 'detection' and 'accommodation' tasks. We provide mathematical formulation of NoveltyTask and instantiate it with the authorship attribution task that pertains to identifying the correct author of a given text. We use Amazon reviews corpus and compile a large dataset (consisting of 250k instances across 200 authors/labels) for NoveltyTask. We conduct comprehensive experiments and explore several baseline methods for the task. Our results show that the methods achieve considerably low performance making the task challenging and leaving sufficient room for improvement. Finally, we believe our work will encourage research in this underexplored area of dealing with novelties, an important step en route to developing robust systems.
Computation of an unsteady complex geometry flow using novel non‐linear turbulence models

Non‐linear zonal turbulence models are applied to an unsteady complex geometry flow. These are generally found to marginally improve predicted turbulence intensities. However, relative to linear models, convergence is mostly difficult to achieve. Clipping of some non‐linear Reynolds stress components is required along with velocity field smoothing or alternative measures. Smoothing is naturally achieved through multilevel convergence restriction operators. As a result of convergence difficulties, generally, non‐linear model computational costs detract from accuracy gains. For standard Reynolds stress model results, again computational costs are prohibitive. Also, mean velocity profile data accuracies are found worse than for a simple mixing length model. Of the non‐linear models considered, the explicit algebraic stress showed greatest promise with respect to accuracy and stability. However, even this shows around a 30% error in total (the sum of turbulence and unsteadiness) intensity. In strong contradiction to measurements the non‐linear and Reynolds models predict quasi‐steady flows. This is probably a key reason for the total intensity under‐predictions. Use of LES in a non‐linear model context might help remedy this modelling aspect. Copyright © 2003 John Wiley & Sons, Ltd.
Explanation-based Finetuning Makes Models More Robust to Spurious Cues

Large Language Models (LLMs) are so powerful that they sometimes learn correlations between labels and features that are irrelevant to the task, leading to poor generalization on out-of-distribution data. We propose explanation-based finetuning as a general approach to mitigate LLMs’ reliance on spurious correlations. Unlike standard finetuning where the model only predicts the answer given the input, we finetune the model to additionally generate a free-text explanation supporting its answer. To evaluate our method, we finetune the model on artificially constructed training sets containing different types of spurious cues, and test it on a test set without these cues. Compared to standard finetuning, our method makes GPT-3 (davinci) remarkably more robust against spurious cues in terms of accuracy drop across four classification tasks: ComVE (+1.2), CREAK (+9.1), e-SNLI (+15.4), and SBIC (+6.5). The efficacy generalizes across multiple model families and scales, with greater gains for larger models. Finally, our method also works well with explanations generated by the model, implying its applicability to more datasets without human-written explanations.
Minimal Learning Machine for Multi-Label Learning

Distance-based supervised method, the minimal learning machine, constructs a predictive model from data by learning a mapping between input and output distance matrices. In this paper, we propose methods and evaluate how this technique and its core component, the distance mapping, can be adapted to multi-label learning. The proposed approach is based on combining the distance mapping with an inverse distance weighting. Although the proposal is one of the simplest methods in the multi-label learning literature, it achieves state-of-the-art performance for small to moderate-sized multi-label learning problems. Besides its simplicity, the proposed method is fully deterministic and its hyper-parameter can be selected via ranking loss-based statistic which has a closed form, thus avoiding conventional cross-validation-based hyper-parameter tuning. In addition, due to its simple linear distance mapping-based construction, we demonstrate that the proposed method can assess predictions' uncertainty for multi-label classification, which is a valuable capability for data-centric machine learning pipelines.
Cooperating Graph Neural Networks with Deep Reinforcement Learning for Vaccine Prioritization

This study explores the vaccine prioritization strategy to reduce the overall burden of the pandemic when the supply is limited. Existing methods conduct macro-level or simplified micro-level vaccine distribution by assuming the homogeneous behavior within subgroup populations and lacking mobility dynamics integration. Directly applying these models for micro-level vaccine allocation leads to sub-optimal solutions due to the lack of behavioral-related details. To address the issue, we first incorporate the mobility heterogeneity in disease dynamics modeling and mimic the disease evolution process using a Trans-vaccine-SEIR model. Then we develop a novel deep reinforcement learning to seek the optimal vaccine allocation strategy for the high-degree spatial-temporal disease evolution system. The graph neural network is used to effectively capture the structural properties of the mobility contact network and extract the dynamic disease features. In our evaluation, the proposed framework reduces 7% - 10% of infections and deaths than the baseline strategies. Extensive evaluation shows that the proposed framework is robust to seek the optimal vaccine allocation with diverse mobility patterns in the micro-level disease evolution system. In particular, we find the optimal vaccine allocation strategy in the transit usage restriction scenario is significantly more effective than restricting cross-zone mobility for the top 10% age-based and income-based zones. These results provide valuable insights for areas with limited vaccines and low logistic efficacy.
On the Structure of Higher Order MDS Codes

A code of length n is said to be (combinatorially) (ρ, L)-list decodable if the Hamming ball of radius ρn around any vector in the ambient space does not contain more than L codewords. We study a recently introduced class of higher order MDS codes, which are closely related (via duality) to codes that achieve a generalized Singleton bound for list decodability. For some ℓ ≥ 1, higher order MDS codes of length n, dimension k, and order ℓ are denoted as (n, k)-MDS(ℓ) codes. We present a number of results on the structure of these codes, identifying the ‘extend-ability’ of their parameters in various scenarios. Specifically, for some parameter regimes, we identify conditions under which (n1, k1)-MDS(ℓ1) codes can be obtained from (n2, k2)-MDS(ℓ2) codes, via various techniques. We believe that these results will aid in efficient constructions of higher order MDS codes. We also obtain a new field size upper bound for the existence of such codes, which arguably improves over the best known existing bound, in some parameter regimes.
Adaptive Cross Tubal Tensor Approximation

In this paper, we propose a new adaptive cross algorithm for computing a low tubal rank approximation of third-order tensors, with less memory and lower computational complexity than the truncated tensor SVD (t-SVD). This makes it applicable for decomposing large-scale tensors. We conduct numerical experiments on synthetic and real-world datasets to confirm the efficiency and feasibility of the proposed algorithm. The simulation results show more than one order of magnitude acceleration in the computation of low tubal rank (t-SVD) for large-scale tensors. An application to pedestrian attribute recognition is also presented.
Robust Implicit Regularization via Weight Normalization

Overparameterized models may have many interpolating solutions; implicit regularization refers to the hidden preference of a particular optimization method towards a certain interpolating solution among the many. A by now established line of work has shown that (stochastic) gradient descent tends to have an implicit bias towards low rank and/or sparse solutions when used to train deep linear networks, explaining to some extent why overparameterized neural network models trained by gradient descent tend to have good generalization performance in practice.However, existing theory for square-loss objectives often requires very small initialization of the trainable weights, which is at odds with the larger scale at which weights are initialized in practice for faster convergence and better generalization performance. In this paper, we aim to close this gap by incorporating and analyzing gradient flow (continuous-time version of gradient descent) with weight normalization, where the weight vector is reparameterized in terms of polar coordinates, and gradient flow is applied to the polar coordinates. By analyzing key invariants of the gradient flow and using Lojasiewicz Theorem, we show that weight normalization also has an implicit bias towards sparse solutions in the diagonal linear model, but that in contrast to plain gradient flow, weight normalization enables a robust bias that persists even if the weights are initialized at practically large scale. Experiments suggest that the gains in both convergence speed and robustness of the implicit bias are improved dramatically by using weight normalization in overparameterized diagonal linear network models.
UAV-RIS-Aided SAGIN: Interference Alignment Design and Degrees-of-Freedom Analysis

—In space-air-ground integrated networks (SAGIN), terminals face interference from various sources such as satellites and terrestrial transmitters. However, managing interference with traditional interference management schemes (IM) is challenging since different terminals have different channel state information (CSI). This paper introduces a UAV carrying an active RIS (UAV-RIS) to assist in the interference elimination process. Furthermore, a UAV-RIS-aided IM scheme is proposed, which takes into account the multiple types of CSIs present in SAGIN. In this scheme, the satellite, terrestrial transmitters, and UAV-RIS collaborate to design precoding matrices based on the speciﬁc type of CSI of each node. Additionally, the DoF gain obtained by the proposed IM scheme is thoroughly discussed for SAGIN conﬁgurations with different numbers of users and transceiver antennas. Simulation results demonstrate that the proposed IM scheme outperforms existing IM schemes without UAV-RIS for the same type of CSI. The results also showcase the capacity improvement of the network when the proposed IM scheme is adopted under different types of CSI.
Counting points by height in semigroup orbits

We improve known estimates for the number of points of bounded height in semigroup orbits of polarized dynamical systems. In particular, we give exact asymptotics for generic semigroups acting on the projective line. The main new ingredient is the Wiener-Ikehara Tauberian theorem, which we use to count functions in semigroups of bounded degree.
On the Limits of HARQ Prediction for Short Deterministic Codes with Error Detection in Memoryless Channels

We provide a mathematical framework to analyze the limits of Hybrid Automatic Repeat reQuest (HARQ) and derive analytical expressions for the most powerful test for estimating the decodability under maximum-likelihood decoding and t-error decoding. Furthermore, we numerically approximate the most powerful test for sum-product decoding. We compare the performance of previously studied HARQ prediction schemes and show that none of the state-of-the-art HARQ prediction is most powerful to estimate the decodability of a partially received signal vector under maximum-likelihood decoding and sum-product decoding. Furthermore, we demonstrate that decoding in general is suboptimal for predicting the decodability.
FAENet: Frame Averaging Equivariant GNN for Materials Modeling

Applications of machine learning techniques for materials modeling typically involve functions known to be equivariant or invariant to specific symmetries. While graph neural networks (GNNs) have proven successful in such tasks, they enforce symmetries via the model architecture, which often reduces their expressivity, scalability and comprehensibility. In this paper, we introduce (1) a flexible framework relying on stochastic frame-averaging (SFA) to make any model E(3)-equivariant or invariant through data transformations. (2) FAENet: a simple, fast and expressive GNN, optimized for SFA, that processes geometric information without any symmetrypreserving design constraints. We prove the validity of our method theoretically and empirically demonstrate its superior accuracy and computational scalability in materials modeling on the OC20 dataset (S2EF, IS2RE) as well as common molecular modeling tasks (QM9, QM7-X). A package implementation is available at https://faenet.readthedocs. io.
Effective Medical Code Prediction via Label Internal Alignment

The clinical notes are usually typed into the system by physicians. They are typically required to be marked by standard medical codes, and each code represents a diagnosis or medical treatment procedure. Annotating these notes is time consuming and prone to error. In this paper, we proposed a multi-view attention based Neural network to predict medical codes from clinical texts. Our method incorporates three aspects of information, the semantic context of the clinical text, the relationship among the label (medical codes) space, and the alignment between each pair of a clinical text and medical code. Our method is verified to be effective on the open source dataset. The experimental result shows that our method achieves better performance against the prior state-of-art on multiple metrics.
Solar Cycle Variation of 0.3–1.29 MeV Nucleon−1 Heavy Ion Composition During Quiet Times Near 1 au in Solar Cycles 23 and 24

We report on the annual variation of quiet-time suprathermal ion composition for C through Fe using Advanced Composition Explorer (ACE)/Ultra-Low Energy Isotope Spectrometer data over the energy range 0.3–1.28 MeV nuc−1 from 1998 through 2019, covering solar cycle 23's rising phase through Solar Cycle 24's declining phase. Our findings are: (1) quiet-time suprathermal abundances resemble CIR-associated particles during solar minima; (2) quiet-time suprathermals are M/Q fractionated in a manner that is consistent with M/Q fractionation in large gradual solar energetic particle events (GSEP) during solar maxima; and (3) variability within the quiet-time suprathermal pool increases as a function of M/Q and is consistent with the analogous variability in GSEP events. From these observations, we infer that quiet-time suprathermal ions are remnants of CIRs in solar minima and GSEP events in solar maxima. Coincident with these results, we also unexpectedly show that S behaves like a low FIP ion in the suprathermal regime, and is therefore drawn from low FIP solar sources.
Holonomic and Non-Holonomic Geometric Models Associated to the Gibbs–Helmholtz Equation

By replacing the internal energy with the free energy, as coordinates in a “space of observables”, we slightly modify (the known three) non-holonomic geometrizations from Udriste’s et al. work. The coefficients of the curvature tensor field, of the Ricci tensor field, and of the scalar curvature function still remain rational functions. In addition, we define and study a new holonomic Riemannian geometric model associated, in a canonical way, to the Gibbs–Helmholtz equation from Classical Thermodynamics. Using a specific coordinate system, we define a parameterized hypersurface in R4 as the “graph” of the entropy function. The main geometric invariants of this hypersurface are determined and some of their properties are derived. Using this geometrization, we characterize the equivalence between the Gibbs–Helmholtz entropy and the Boltzmann–Gibbs–Shannon, Tsallis, and Kaniadakis entropies, respectively, by means of three stochastic integral equations. We prove that some specific (infinite) families of normal probability distributions are solutions for these equations. This particular case offers a glimpse of the more general “equivalence problem” between classical entropy and statistical entropy.
The complete set of efficient vectors for a reciprocal matrix

Efficient vectors are the natural set from which to choose a cardinal ranking vector for a reciprocal matrix. Such vectors are the key to certain business project selection models. Many ways to construct specific efficient vectors have been proposed. Yet, no previous method to produce all efficient vectors was known. Here, using some graph theoretic ideas, as well as a numerical extension technique, we show how to generate all efficient vectors for any given reciprocal matrix. We apply this method to give explicitly all efficient vectors for a 4-by-4 reciprocal matrix. Several examples are provided.