Miracle-dz/UniFAR-FTUs huggingface dataset:
RefConv: Re-parameterized Refocusing Convolution for Powerful ConvNets

We propose Re-parameterized Refocusing Convolution (RefConv) as a replacement for regular convolutional layers, which is a plug-and-play module to improve the performance without any inference costs. Specifically, given a pre-trained model, RefConv applies a trainable Refocusing Transformation to the basis kernels inherited from the pre-trained model to establish connections among the parameters. For example, a depth-wise RefConv can relate the parameters of a specific channel of convolution kernel to the parameters of the other kernel, i.e., make them refocus on the other parts of the model they have never attended to, rather than focus on the input features only. From another perspective, RefConv augments the priors of existing model structures by utilizing the representations encoded in the pre-trained parameters as the priors and refocusing on them to learn novel representations, thus further enhancing the representational capacity of the pre-trained model. Experimental results validated that RefConv can improve multiple CNN-based models by a clear margin on image classification (up to 1.47% higher top-1 accuracy on ImageNet), object detection and semantic segmentation without introducing any extra inference costs or altering the original model structure. Further studies demonstrated that RefConv can reduce the redundancy of channels and smooth the loss landscape, which explains its effectiveness.
Simultaneous Cluster and CUTLASS Observations of FTEs on 11 February 2004

This paper analyzes the simultaneous observations of the Flux Gate Magnetometer (FGM) and Plasma Electron and Current Experiment (PEACE) on Cluster Spacecraft, and the CUTLASS Finland radar between 11:00 and 11:40 UT on 11 February 2004. In this interval, the Cluster spacecraft array encountered the northern hemisphere high altitude cusp, and crossed the magnetopause into the magnetosheath at about 11:18 UT. A large number of flux transfer events (FTEs) were observed by all 4 Cluster spacecraft. It is found that these FTEs appeared quasi-periodically with a period of about 130 s. Since all 4 Cluster spacecraft observed these FTEs, we applied the four-spacecraft techniques “Minimum Directional Derivative (or Difference)” (MDD) and “Spatio-temporal Difference” (STD) to calculate the dimension, motion and scale of these FTEs. The inferred northwardly reconnected flux tubes for these FTEs are shown to move northward or north-east and tailward with a quasi-2-D structure and a scale of (0.87 ~ 1.81)RE. The ionospheric footprint of the Cluster spacecraft during the boundary passage is shown to map centrally within the field-of-view of the CUTLASS Finland radar. It is shown that the backscatter power of CUTLASS data is enhanced in synchronously with the magnetospheric FTEs at the latitude of the Cluster footprint. These radar auroral features in the radar subsequently propagated poleward, forming classic “poleward-moving radar auroral form” structures at higher latitudes. The FTE motions meanwhile are shown to be consistent with the expected motion of reconnected magnetic flux tubes over the surface of the magnetopause, arising from a predominantly sub-solar reconnection site during the prevailing upstream conditions.
Biasymptotics of the M\"obius- and Liouville-signed partition numbers

For $n \in \mathbb{N}$ let $\Pi[n]$ denote the set of partitions of $n$, i.e., the set of positive integer tuples $(x_1,x_2,\ldots,x_k)$ such that $x_1 \geq x_2 \geq \cdots \geq x_k$ and $x_1 + x_2 + \cdots + x_k = n$. Fixing $f:\mathbb{N}\to\{0,\pm 1\}$, for $\pi = (x_1,x_2,\ldots,x_k) \in \Pi[n]$ let $f(\pi) := f(x_1)f(x_2)\cdots f(x_k)$. In this way we define the {signed partition numbers} \[ p(n,f) = \sum_{\pi\in\Pi[n]} f(\pi). \] Building on the author's previous work on the quantities $p(n,\mu)$ and $p(n,\lambda)$, where $\mu$ and $\lambda$ are the M\"obius and Liouville functions of prime number theory, respectively, on assumptions about the zeros of the Riemann zeta function we establish an alternation of the terms $p(n,\mu)$ between two asymptotic behaviors as $n\to\infty$. Similar results for the quantities $p(n,\lambda)$ are established. However, it is also demonstrated that if the Riemann Hypothesis (RH) holds, then it is possible that the quantities $p(n,\lambda)$ maintain a single asymptotic behavior as $n\to\infty$. In particular, this stable asymptotic behavior occurs if, in addition to RH, it holds that all zeros of $\zeta(s)$ in the critical strip $\{0<\Re(s)<1\}$ are simple and the residues of $1/\zeta(s)$ at these zeros are not too large. To formally describe these stable and alternating behaviors, the notions of asymptotic and biasymptotic sequences are introduced using a modification of the real logarithm.
SGOOD: Substructure-enhanced Graph-Level Out-of-Distribution Detection

Graph-level representation learning is important in a wide range of applications. However, existing graph-level models are generally built on i.i. d. assumption for both training and testing graphs, which is not realistic in an open world, where models can encounter out-of-distribution (OOD) testing graphs that are from different distributions unknown during training. A trustworthy model should not only produce accurate predictions for in-distribution (ID) data, but also detect OOD graphs to avoid unreliable prediction. In this paper, we present SGOOD, a novel graph-level OOD detection framework. We find that substructure differences commonly exist between ID and OOD graphs. Hence, SGOOD explicitly utilizes substructures to learn powerful representations to achieve superior performance. Specifically, we build a super graph of substructures for every graph, and design a two-level graph encoding pipeline that works on both original graphs and super graphs to obtain substructure-enhanced graph representations. To further distinguish ID and OOD graphs, we develop three graph augmentation techniques that preserve substructures and increase expressiveness. Extensive experiments against 10 competitors on numerous graph datasets demonstrate the superiority of SGOOD, often surpassing existing methods by a significant margin. The code is available at https://anonymous.4open. science/r/SGOOD-0958.
A Blockchain-empowered Multi-Aggregator Federated Learning Architecture in Edge Computing with Deep Reinforcement Learning Optimization

Federated learning (FL) is emerging as a sought-after distributed machine learning architecture, offering the advantage of model training without direct exposure of raw data. With advancements in network infrastructure, FL has been seamlessly integrated into edge computing. However, the limited resources on edge devices introduce security vulnerabilities to FL in the context. While blockchain technology promises to bolster security, practical deployment on resource-constrained edge devices remains a challenge. Moreover, the exploration of FL with multiple aggregators in edge computing is still new in the literature. Addressing these gaps, we introduce the Blockchain-empowered Heterogeneous Multi-Aggregator Federated Learning Architecture (BMA-FL). We design a novel light-weight Byzantine consensus mechanism, namely PBCM, to enable secure and fast model aggregation and synchronization in BMA-FL. We also dive into the heterogeneity problem in BMA-FL that the aggregators are associated with varied number of connected trainers with Non-IID data distributions and diverse training speed. We proposed a multi-agent deep reinforcement learning algorithm to help aggregators decide the best training strategies. The experiments on real-word datasets demonstrate the efficiency of BMA-FL to achieve better models faster than baselines, showing the efficacy of PBCM and proposed deep reinforcement learning algorithm.
ByteStack-ID: Integrated Stacked Model Leveraging Payload Byte Frequency for Grayscale Image-based Network Intrusion Detection

In the ever-evolving realm of network security, the swift and accurate identification of diverse attack classes within network traffic is of paramount importance. This paper introduces"ByteStack-ID,"a pioneering approach tailored for packet-level intrusion detection. At its core, ByteStack-ID leverages grayscale images generated from the frequency distributions of payload data, a groundbreaking technique that greatly enhances the model's ability to discern intricate data patterns. Notably, our approach is exclusively grounded in packet-level information, a departure from conventional Network Intrusion Detection Systems (NIDS) that predominantly rely on flow-based data. While building upon the fundamental concept of stacking methodology, ByteStack-ID diverges from traditional stacking approaches. It seamlessly integrates additional meta learner layers into the concatenated base learners, creating a highly optimized, unified model. Empirical results unequivocally confirm the outstanding effectiveness of the ByteStack-ID framework, consistently outperforming baseline models and state-of-the-art approaches across pivotal performance metrics, including precision, recall, and F1-score. Impressively, our proposed approach achieves an exceptional 81\% macro F1-score in multiclass classification tasks. In a landscape marked by the continuous evolution of network threats, ByteStack-ID emerges as a robust and versatile security solution, relying solely on packet-level information extracted from network traffic data.
On the Statistical Foundations of H-likelihood for Unobserved Random Variables

The maximum likelihood estimation is widely used for statistical inferences. This paper aims to reformulate Lee and Nelder's (1996) h-likelihood, so that the maximum h-likelihood estimator resembles the maximum likelihood estimator of the classical likelihood. We establish the statistical foundations of the new h-likelihood. This extends classical likelihood theories to embrace broader class of statistical models with random parameters. Maximization of the h-likelihood yields asymptotically optimal estimators for both fixed and random parameters achieving the generalized Cram\'er-Rao lower bound, while providing computationally efficient fitting algorithms. Furthermore, we explore asymptotic theory when the consistency of either fixed parameter estimation or random parameter prediction is violated. We also study how to obtain maximum h-likelihood estimators when the h-likelihood is not explicitly available.
Reformulating NLP tasks to Capture Longitudinal Manifestation of Language Disorders in People with Dementia

Dementia is associated with language disorders which impede communication. Here, we automatically learn linguistic disorder patterns by making use of a moderately-sized pre-trained language model and forcing it to focus on reformulated natural language processing (NLP) tasks and associated linguistic patterns. Our experiments show that NLP tasks that encapsulate contextual information and enhance the gradient signal with linguistic patterns benefit performance. We then use the probability estimates from the best model to construct digital linguistic markers measuring the overall quality in communication and the intensity of a variety of language disorders. We investigate how the digital markers characterize dementia speech from a longitudinal perspective. We find that our proposed communication marker is able to robustly and reliably characterize the language of people with dementia, outperforming existing linguistic approaches; and shows external validity via significant correlation with clinical markers of behaviour. Finally, our proposed linguistic disorder markers provide useful insights into gradual language impairment associated with disease progression.
Label-efficient Segmentation via Affinity Propagation

Weakly-supervised segmentation with label-efficient sparse annotations has attracted increasing research attention to reduce the cost of laborious pixel-wise labeling process, while the pairwise affinity modeling techniques play an essential role in this task. Most of the existing approaches focus on using the local appearance kernel to model the neighboring pairwise potentials. However, such a local operation fails to capture the long-range dependencies and ignores the topology of objects. In this work, we formulate the affinity modeling as an affinity propagation process, and propose a local and a global pairwise affinity terms to generate accurate soft pseudo labels. An efficient algorithm is also developed to reduce significantly the computational cost. The proposed approach can be conveniently plugged into existing segmentation networks. Experiments on three typical label-efficient segmentation tasks, i.e. box-supervised instance segmentation, point/scribble-supervised semantic segmentation and CLIP-guided semantic segmentation, demonstrate the superior performance of the proposed approach.
Flow Dynamics Correction for Action Recognition

Various research studies indicate that action recognition performance highly depends on the types of motions being extracted and how accurate the human actions are represented. In this paper, we investigate different optical flow, and features extracted from these optical flow that capturing both short-term and long-term motion dynamics. We perform power normalization on the magnitude component of optical flow for flow dynamics correction to boost subtle or dampen sudden motions. We show that existing action recognition models which rely on optical flow are able to get performance boosted with our corrected optical flow. To further improve performance, we integrate our corrected flow dynamics into popular models through a simple hallucination step by selecting only the best performing optical flow features, and we show that by 'translating' the CNN feature maps into these optical flow features with different scales of motions leads to the new state-of-the-art performance on several benchmarks including HMDB-51, YUP++, fine-grained action recognition on MPII Cooking Activities, and large-scale Charades.
A simple uniformly optimal method without line search for convex optimization

Line search (or backtracking) procedures have been widely employed into first-order methods for solving convex optimization problems, especially those with unknown problem parameters (e.g., Lipschitz constant). In this paper, we show that line search is superfluous in attaining the optimal rate of convergence for solving a convex optimization problem whose parameters are not given a priori. In particular, we present a novel accelerated gradient descent type algorithm called auto-conditioned fast gradient method (AC-FGM) that can achieve an optimal $\mathcal{O}(1/k^2)$ rate of convergence for smooth convex optimization without requiring the estimate of a global Lipschitz constant or the employment of line search procedures. We then extend AC-FGM to solve convex optimization problems with H\"{o}lder continuous gradients and show that it automatically achieves the optimal rates of convergence uniformly for all problem classes with the desired accuracy of the solution as the only input. Finally, we report some encouraging numerical results that demonstrate the advantages of AC-FGM over the previously developed parameter-free methods for convex optimization.
TS-ENAS: Two-Stage Evolution for Cell-based Network Architecture Search

Neural network architecture search provides a solution to the automatic design of network structures. However, it is difficult to search the whole network architecture directly. Although using stacked cells to search neural network architectures is an effective way to reduce the complexity of searching, these methods do not able find the global optimal neural network structure since the number of layers, cells and connection methods is fixed. In this paper, we propose a Two-Stage Evolution for cell-based Network Architecture Search(TS-ENAS), including one-stage searching based on stacked cells and second-stage adjusting these cells. In our algorithm, a new cell-based search space and an effective two-stage encoding method are designed to represent cells and neural network structures. In addition, a cell-based weight inheritance strategy is designed to initialize the weight of the network, which significantly reduces the running time of the algorithm. The proposed methods are extensively tested and compared on four image classification dataset, Fashion-MNIST, CIFAR10, CIFAR100 and ImageNet and compared with 22 state-of-the-art algorithms including hand-designed networks and NAS networks. The experimental results show that TS-ENAS can more effectively find the neural network architecture with comparative performance.
Private Synthetic Data Meets Ensemble Learning

When machine learning models are trained on synthetic data and then deployed on real data, there is often a performance drop due to the distribution shift between synthetic and real data. In this paper, we introduce a new ensemble strategy for training downstream models, with the goal of enhancing their performance when used on real data. We generate multiple synthetic datasets by applying a differential privacy (DP) mechanism several times in parallel and then ensemble the downstream models trained on these datasets. While each synthetic dataset might deviate more from the real data distribution, they collectively increase sample diversity. This may enhance the robustness of downstream models against distribution shifts. Our extensive experiments reveal that while ensembling does not enhance downstream performance (compared with training a single model) for models trained on synthetic data generated by marginal-based or workload-based DP mechanisms, our proposed ensemble strategy does improve the performance for models trained using GAN-based DP mechanisms in terms of both accuracy and calibration of downstream models.
Evaluation of laboratory dolomite core sample size using representative elementary volume concepts

The adequacy for laboratory testing of four dolomite cores from the Culebra Dolomite of the Rustler Formation at the Waste Isolation Pilot Plant near Carlsbad, New Mexico, were evaluated using representative elementary volume (REV) theory. Gamma ray computerized tomography created three‐dimensional grids of bulk density and macropore index over volumes from 1.4 × 10−7 to 1.6 L. Three different methods for both volume averaging and REV analysis were applied and compared. Both density and macropore index converged to single values with increasing volume, which meets the most common qualitative definition of a REV. Statistical test results for the relatively homogeneous samples indicate that volumes larger than 1 to 7 mL have constant properties. Contrarily, a highly varied sample required 250 and 373 mL to achieve invariant density and macropore characteristics, respectively. Prismatic volume averaging was found to be better than slice averaging, while a qualitative test for the REV provided similar results as a rigorous statistical method. All cores were larger than the REV but were significantly different from one another. This implies that multiple cores are necessary to determine the entire range of transport properties within the rock.
Enhancing Task Performance of Learned Simplified Models via Reinforcement Learning

In contact-rich tasks, the hybrid, multi-modal nature of contact dynamics poses great challenges in model representation, planning, and control. Recent efforts have attempted to address these challenges via data-driven methods, learning dynamical models in combination with model predictive control. Those methods, while effective, rely solely on minimizing forward prediction errors to hope for better task performance with MPC controllers. This weak correlation can result in data inefficiency as well as limitations to overall performance. In response, we propose a novel strategy: using a policy gradient algorithm to find a simplified dynamics model that explicitly maximizes task performance. Specifically, we parameterize the stochastic policy as the perturbed output of the MPC controller, thus, the learned model representation can directly associate with the policy or task performance. We apply the proposed method to contact-rich tasks where a three-fingered robotic hand manipulates previously unknown objects. Our method significantly enhances task success rate by up to 15% in manipulating diverse objects compared to the existing method while sustaining data efficiency. Our method can solve some tasks with success rates of 70% or higher using under 30 minutes of data. All videos and codes are available at https://sites.google. com/view/lcs-rl.
Molecular orbital formation and metastable short-range ordered structure in VO$_2$

The metal-insulator transition (MIT) in vanadium dioxide VO$_2$ due to V-V dimerization has been extensively discussed for decades. While it is widely acknowledged that electron correlations, Peierls instabilities, and molecular orbital formations are crucial for understanding the MIT of VO$_2$, the primary origin of the MIT remains controversial. In this study, we delve into the crystal structure and orbital state of VO$_2$ through synchrotron x-ray diffraction experiments. The molecular orbital formation corresponding to the V-V dimerization is directly observed in the low-temperature insulating monoclinic phase, called the M1 phase, as indicated by the valence electron density distribution. Moreover, diffuse scattering observed in the high-temperature metal phase of rutile structure suggests the presence of short-range correlation of V displacements, which is not directly attributed to the structural fluctuation of the M1 phase. The short-range order in the rutile phase will be the key to understanding the MIT in this system.
Harnessing Unipolar Threshold Switches for Enhanced Rectification

Phase transition materials (PTM) have drawn significant attention in recent years due to their abrupt threshold switching characteristics and hysteretic behavior. Augmentation of the PTM with a transistor has been shown to provide enhanced selectivity (as high as ~107 for Ag/HfO2/Pt) leading to unique circuit-level advantages. Previously, a unipolar PTM, Ag-HfO2-Pt, was reported as a replacement for diodes due to its polaritydependent high selectivity and hysteretic properties. It was shown to achieve ~50% higher DC output compared to a diode-based design in a Cockcroft-Walton multiplier circuit. In this paper, we take a deeper dive into this design. We augment two different PTMs (unipolar Ag-HfO2-Pt and bipolar VO2) with diodeconnected MOSFETs to retain the benefits of hysteretic rectification. Our proposed hysteretic diodes (Hyperdiodes) exhibit a low forward voltage drop owing to their volatile hysteretic characteristics. However, augmenting a hysteretic PTM with a transistor brings an additional stability concern due to their complex interplay. Hence, we perform a comprehensive stability analysis for a range of threshold voltages (-0.2<Vth<0.8) and transistor sizes to ensure operational stability and to choose the most optimum design parameters. We then test a standalone AgHfO2-Pt and an Ag-HfO2-Pt-based Hyperdiode in two different types of voltage multipliers and report ~500 and ~20 times lower settling time, respectively.
Inference with Mondrian Random Forests

Random forests are popular methods for classification and regression, and many different variants have been proposed in recent years. One interesting example is the Mondrian random forest, in which the underlying trees are constructed according to a Mondrian process. In this paper we give a central limit theorem for the estimates made by a Mondrian random forest in the regression setting. When combined with a bias characterization and a consistent variance estimator, this allows one to perform asymptotically valid statistical inference, such as constructing confidence intervals, on the unknown regression function. We also provide a debiasing procedure for Mondrian random forests which allows them to achieve minimax-optimal estimation rates with $\beta$-H\"older regression functions, for all $\beta$ and in arbitrary dimension, assuming appropriate parameter tuning.