Miracle-dz/UniFAR-FTUs huggingface dataset:
A Comparison of Modeling Preprocessing Techniques

This paper compares the performance of various data processing methods in terms of predictive performance for structured data. This paper also seeks to identify and recommend preprocessing methodologies for tree-based binary classification models, with a focus on eXtreme Gradient Boosting (XGBoost) models. Three data sets of various structures, interactions, and complexity were constructed, which were supplemented by a real-world data set from the Lending Club. We compare several methods for feature selection, categorical handling, and null imputation. Performance is assessed using relative comparisons among the chosen methodologies, including model prediction variability. This paper is presented by the three groups of preprocessing methodologies, with each section consisting of generalized observations. Each observation is accompanied by a recommendation of one or more preferred methodologies. Among feature selection methods, permutation-based feature importance, regularization, and XGBoost's feature importance by weight are not recommended. The correlation coefficient reduction also shows inferior performance. Instead, XGBoost importance by gain shows the most consistency and highest caliber of performance. Categorical featuring encoding methods show greater discrimination in performance among data set structures. While there was no universal"best"method, frequency encoding showed the greatest performance for the most complex data sets (Lending Club), but had the poorest performance for all synthetic (i.e., simpler) data sets. Finally, missing indicator imputation dominated in terms of performance among imputation methods, whereas tree imputation showed extremely poor and highly variable model performance.
DisCO: Portrait Distortion Correction with Perspective-Aware 3D GANs

Close-up facial images captured at short distances often suffer from perspective distortion, resulting in exaggerated facial features and unnatural/unattractive appearances. We propose a simple yet effective method for correcting perspective distortions in a single close-up face. We first perform GAN inversion using a perspective-distorted input facial image by jointly optimizing the camera intrinsic/extrinsic parameters and face latent code. To address the ambiguity of joint optimization, we develop starting from a short distance, optimization scheduling, reparametrizations, and geometric regularization. Re-rendering the portrait at a proper focal length and camera distance effectively corrects perspective distortions and produces more natural-looking results. Our experiments show that our method compares favorably against previous approaches qualitatively and quantitatively. We showcase numerous examples validating the applicability of our method on in-the-wild portrait photos. We will release our code and the evaluation protocol to facilitate future work.
Turbulent burning velocity and thermodiffusive instability of premixed flames.

Reported in the paper are results of unsteady three-dimensional direct numerical simulations of laminar and turbulent, lean hydrogen-air, complex-chemistry flames propagating in forced turbulence in a box. To explore the eventual influence of thermodiffusive instability of laminar flames on turbulent burning velocity, (i) a critical length scale Λ_{n} that bounds regimes of unstable and stable laminar combustion is numerically determined by gradually decreasing the width Λ of computational domain until a stable laminar flame is obtained, and (ii) simulations of turbulent flames are performed by varying the width from ΛΛ_{n} (in this case, the instability may grow). Moreover, simulations are performed either using mixture-averaged transport properties (low Lewis number flames) or setting diffusivities of all species equal to heat diffusivity of the mixture (equidiffusive flames), with all other things being equal. Obtained results show a significant increase in turbulent burning velocity U_{T} when the boundary Λ=Λ_{n} is crossed in weak turbulence, but almost equal values of U_{T} are computed at ΛΛ_{n} in moderately turbulent flames characterized by a Karlovitz number equal to 3.4 or larger. These results imply that thermodiffusive instability of laminar premixed flames substantially affects burning velocity in weak turbulence only, in line with a simple criterion proposed by Chomiak and Lipatnikov (Phys. Rev. E 107, 015102, (2023)10.1103/PhysRevE.107. 015102).
Evaluating Automatic Speech Recognition in an Incremental Setting

The increasing reliability of automatic speech recognition has proliferated its everyday use. However, for research purposes, it is often unclear which model one should choose for a task, particularly if there is a requirement for speed as well as accuracy. In this paper, we systematically evaluate six speech recognizers using metrics including word error rate, latency, and the number of updates to already recognized words on English test data, as well as propose and compare two methods for streaming audio into recognizers for incremental recognition. We further propose Revokes per Second as a new metric for evaluating incremental recognition and demonstrate that it provides insights into overall model performance. We find that, generally, local recognizers are faster and require fewer updates than cloud-based recognizers. Finally, we find Meta's Wav2Vec model to be the fastest, and find Mozilla's DeepSpeech model to be the most stable in its predictions.
FTM: A Frame-level Timeline Modeling Method for Temporal Graph Representation Learning

Learning representations for graph-structured data is essential for graph analytical tasks. While remarkable progress has been made on static graphs, researches on temporal graphs are still in its beginning stage. The bottleneck of the temporal graph representation learning approach is the neighborhood aggregation strategy, based on which graph attributes share and gather information explicitly. Existing neighborhood aggregation strategies fail to capture either the short-term features or the long-term features of temporal graph attributes, leading to unsatisfactory model performance and even poor robustness and domain generality of the representation learning method. To address this problem, we propose a Frame-level Timeline Modeling (FTM) method that helps to capture both short-term and long-term features and thus learns more informative representations on temporal graphs. In particular, we present a novel link-based framing technique to preserve the short-term features and then incorporate a timeline aggregator module to capture the intrinsic dynamics of graph evolution as long-term features. Our method can be easily assembled with most temporal GNNs. Extensive experiments on common datasets show that our method brings great improvements to the capability, robustness, and domain generality of backbone methods in downstream tasks. Our code can be found at https://github.com/yeeeqichen/FTM.
Personalized Privacy-Preserving Framework for Cross-Silo Federated Learning

Federated learning (FL) is recently surging as a promising decentralized deep learning (DL) framework that enables DL-based approaches trained collaboratively across clients without sharing private data. However, in the context of the central party being active and dishonest, the data of individual clients might be perfectly reconstructed, leading to the high possibility of sensitive information being leaked. Moreover, FL also suffers from the nonindependent and identically distributed (non-IID) data among clients, resulting in the degradation in the inference performance on local clients' data. In this paper, we propose a novel framework, namely Personalized Privacy-Preserving Federated Learning (PPPFL), with a concentration on cross-silo FL to overcome these challenges. Specifically, we introduce a stabilized variant of the Model-Agnostic Meta-Learning (MAML) algorithm to collaboratively train a global initialization from clients' synthetic data generated by Differential Private Generative Adversarial Networks (DP-GANs). After reaching convergence, the global initialization will be locally adapted by the clients to their private data. Through extensive experiments, we empirically show that our proposed framework outperforms multiple FL baselines on different datasets, including MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100.
Efficient Context Integration through Factorized Pyramidal Learning for Ultra-Lightweight Semantic Segmentation

Semantic segmentation is a pixel-level prediction task to classify each pixel of the input image. Deep learning models, such as convolutional neural networks (CNNs), have been extremely successful in achieving excellent performances in this domain. However, mobile application, such as autonomous driving, demand real-time processing of incoming stream of images. Hence, achieving efficient architectures along with enhanced accuracy is of paramount importance. Since, accuracy and model size of CNNs are intrinsically contentious in nature, the challenge is to achieve a decent trade-off between accuracy and model size. To address this, we propose a novel Factorized Pyramidal Learning (FPL) module to aggregate rich contextual information in an efficient manner. On one hand, it uses a bank of convolutional filters with multiple dilation rates which leads to multi-scale context aggregation; crucial in achieving better accuracy. On the other hand, parameters are reduced by a careful factorization of the employed filters; crucial in achieving lightweight models. Moreover, we decompose the spatial pyramid into two stages which enables a simple and efficient feature fusion within the module to solve the notorious checkerboard effect. We also design a dedicated Feature-Image Reinforcement (FIR) unit to carry out the fusion operation of shallow and deep features with the downsampled versions of the input image. This gives an accuracy enhancement without increasing model parameters. Based on the FPL module and FIR unit, we propose an ultra-lightweight real-time network, called FPLNet, which achieves state-of-the-art accuracy-efficiency trade-off. More specifically, with only less than 0.5 million parameters, the proposed network achieves 66.93\% and 66.28\% mIoU on Cityscapes validation and test set, respectively. Moreover, FPLNet has a processing speed of 95.5 frames per second (FPS).
Constrained Reinforcement Learning using Distributional Representation for Trustworthy Quadrotor UAV Tracking Control

—Simultaneously accurate and reliable tracking con-trol for quadrotors in complex dynamic environments is challenging. As aerodynamics derived from drag forces and moment variations are chaotic and difficult to precisely identify, most current quadrotor tracking systems treat them as simple ‘disturbances’ in conventional control approaches. We propose a novel, interpretable trajectory tracker integrating a Distributional Reinforcement Learning disturbance estimator for unknown aerodynamic effects with a Stochastic Model Predictive Controller (SMPC). The proposed estimator ‘Constrained Distributional Reinforced disturbance estimator’ (ConsDRED) accurately identifies uncertainties between true and estimated values of aerodynamic effects. Simplified Affine Disturbance Feedback is used for control parameterization to guarantee convexity, which we then integrate with a SMPC. We theoretically guarantee that ConsDRED achieves at least an Θ(1 / √ T ) global convergence rate and an Θ(1 / √ T ) sublinear rate if constraints are violated with an error Θ(1 /m H 4 ) decreases as the width and the layer of neural network, i.e., m and H , increase. To demonstrate practicality, we show convergent training in simulation and real-world experiments, and empirically verify that ConsDRED is less sensitive to hyperparameter settings compared with canonical constrained RL approaches. We demonstrate our system improves accumulative tracking errors by at least 70% compared with the recent art. Importantly, the proposed framework, ConsDRED-SMPC, balances the tradeoff between pursuing high performance and obeying conservative constraints for practical implementations 1 .
IlocA: An algorithm to Cluster Cells and form Imputation Groups from a pair of Classification Variables

We set out the novel bottom up procedure to aggregate or cluster cells with small frequency counts together, in a two way classification while maintaining dependence in the table. The procedure is model free. It combines cells in a table into clusters based on independent log odds ratios. We use this procedure to build a set of statistically efficient and robust imputation cells, for the imputation of missing values of a continuous variable using a pair classification variables. A nice feature of the procedure is it forms aggregation groups homogeneous with respect to the cell response mean. Using a series of simulation studies, we show IlocA only groups together independent cells and does so in a consistent and credible way. While imputing missing data, we show IlocAs generates close to an optimal number of imputation cells. For ignorable non-response the resulting imputed means are accurate in general. With non-ignorable missingness results are consistent with those obtained elsewhere. We close with a case study applying our method to imputing missing building energy performance data
Variable selection in linear regression models: Choosing the best subset is not always the best choice.

We consider the question of variable selection in linear regressions, in the sense of identifying the correct direct predictors (those variables that have nonzero coefficients given all candidate predictors). Best subset selection (BSS) is often considered the "gold standard," with its use being restricted only by its NP-hard nature. Alternatives such as the least absolute shrinkage and selection operator (Lasso) or the Elastic net (Enet) have become methods of choice in high-dimensional settings. A recent proposal represents BSS as a mixed-integer optimization problem so that large problems have become computationally feasible. We present an extensive neutral comparison assessing the ability to select the correct direct predictors of BSS compared to forward stepwise selection (FSS), Lasso, and Enet. The simulation considers a range of settings that are challenging regarding dimensionality (number of observations and variables), signal-to-noise ratios, and correlations between predictors. As fair measure of performance, we primarily used the best possible F1-score for each method, and results were confirmed by alternative performance measures and practical criteria for choosing the tuning parameters and subset sizes. Surprisingly, it was only in settings where the signal-to-noise ratio was high and the variables were uncorrelated that BSS reliably outperformed the other methods, even in low-dimensional settings. Furthermore, FSS performed almost identically to BSS. Our results shed new light on the usual presumption of BSS being, in principle, the best choice for selecting the correct direct predictors. Especially for correlated variables, alternatives like Enet are faster and appear to perform better in practical settings.
Sentence Simplification via Large Language Models

Sentence Simplification aims to rephrase complex sentences into simpler sentences while retaining original meaning. Large Language models (LLMs) have demonstrated the ability to perform a variety of natural language processing tasks. However, it is not yet known whether LLMs can be served as a high-quality sentence simplification system. In this work, we empirically analyze the zero-/few-shot learning ability of LLMs by evaluating them on a number of benchmark test sets. Experimental results show LLMs outperform state-of-the-art sentence simplification methods, and are judged to be on a par with human annotators.
Dermatological Diagnosis Explainability Benchmark for Convolutional Neural Networks

In recent years, large strides have been taken in developing machine learning methods for dermatological applications, supported in part by the success of deep learning (DL). To date, diagnosing diseases from images is one of the most explored applications of DL within dermatology. Convolutional neural networks (ConvNets) are the most common (DL) method in medical imaging due to their training efficiency and accuracy, although they are often described as black boxes because of their limited explainability. One popular way to obtain insight into a ConvNet's decision mechanism is gradient class activation maps (Grad-CAM). A quantitative evaluation of the Grad-CAM explainability has been recently made possible by the release of DermXDB, a skin disease diagnosis explainability dataset which enables explainability benchmarking of ConvNet architectures. In this paper, we perform a literature review to identify the most common ConvNet architectures used for this task, and compare their Grad-CAM explanations with the explanation maps provided by DermXDB. We identified 11 architectures: DenseNet121, EfficientNet-B0, InceptionV3, InceptionResNetV2, MobileNet, MobileNetV2, NASNetMobile, ResNet50, ResNet50V2, VGG16, and Xception. We pre-trained all architectures on an clinical skin disease dataset, and fine-tuned them on a DermXDB subset. Validation results on the DermXDB holdout subset show an explainability F1 score of between 0.35-0.46, with Xception displaying the highest explainability performance. NASNetMobile reports the highest characteristic-level explainability sensitivity, despite it's mediocre diagnosis performance. These results highlight the importance of choosing the right architecture for the desired application and target market, underline need for additional explainability datasets, and further confirm the need for explainability benchmarking that relies on quantitative analyses.
JT Gravity on a Finite Lorentzian Strip: Time dependent Quantum Gravity Amplitudes

We formulate JT quantum gravity on a finite Lorentzian strip. Due to the spatial boundaries of the strip, it is possible to define left and right proper times. With respect to these times we compute non-perturbatively the quantum gravity (QG) time dependent transition amplitude. Lagrangian and Hamiltonian formulations are presented. Special attention is paid to the four corner terms (Hayward terms) in the action that are needed in order to have a well defined variational problem. From a detailed analysis of the gravity boundary condition on the spatial boundary, we find that while the lapse and the shift functions are independent Lagrange multipliers on the bulk, on the spatial boundary, these two are related. This fact leads to an algebraic equation of motion for a particular degree of freedom that is conveniently introduced on the spatial boundaries whose solution can be plugged back into the action allowing to fully determine the time dependent transition amplitude. The final result suggests that time evolution is non-unitary for most of the boundary conditions. Interestingly enough, unitary could be recovered when spatial $\text{AdS}_2$ boundary conditions are imposed. Other wave functions for other topologies obtained from the strip by gluing its spatial boundaries are also presented. Remarkably these do not exhibit any non-unitary evolution behavior.
Novel Class Discovery: an Introduction and Key Concepts

Novel Class Discovery (NCD) is a growing field where we are given during training a labeled set of known classes and an unlabeled set of different classes that must be discovered. In recent years, many methods have been proposed to address this problem, and the field has begun to mature. In this paper, we provide a comprehensive survey of the state-of-the-art NCD methods. We start by formally defining the NCD problem and introducing important notions. We then give an overview of the different families of approaches, organized by the way they transfer knowledge from the labeled set to the unlabeled set. We find that they either learn in two stages, by first extracting knowledge from the labeled data only and then applying it to the unlabeled data, or in one stage by conjointly learning on both sets. For each family, we describe their general principle and detail a few representative methods. Then, we briefly introduce some new related tasks inspired by the increasing number of NCD works. We also present some common tools and techniques used in NCD, such as pseudo labeling, self-supervised learning and contrastive learning. Finally, to help readers unfamiliar with the NCD problem differentiate it from other closely related domains, we summarize some of the closest areas of research and discuss their main differences.
Mean-field dynamo due to spatio-temporal fluctuations of the turbulent kinetic energy

Abstract In systems where the standard $\alpha$ effect is inoperative, one often explains the existence of mean magnetic fields by invoking the ‘incoherent $\alpha$ effect’, which appeals to fluctuations of the mean kinetic helicity at a mesoscale. Most previous studies, while considering fluctuations in the mean kinetic helicity, treated the mean turbulent kinetic energy at the mesoscale as a constant, despite the fact that both these quantities involve second-order velocity correlations. The mean turbulent kinetic energy affects the mean magnetic field through both turbulent diffusion and turbulent diamagnetism. In this work, we use a double-averaging procedure to analytically show that fluctuations of the mean turbulent kinetic energy at the mesoscale (giving rise to $\eta$-fluctuations at the mesoscale, where the scalar $\eta$ is the turbulent diffusivity) can lead to the growth of a large-scale magnetic field even when the kinetic helicity is zero pointwise. Constraints on the operation of such a dynamo are expressed in terms of dynamo numbers that depend on the correlation length, correlation time and strength of these fluctuations. In the white-noise limit, we find that these fluctuations reduce the overall turbulent diffusion, while also contributing a drift term which does not affect the growth of the field. We also study the effects of non-zero correlation time and anisotropy. Turbulent diamagnetism, which arises due to inhomogeneities in the turbulent kinetic energy, leads to growing mean-field solutions even when the $\eta$-fluctuations are statistically isotropic.
Optical conductivity and vibrational spectra of the narrow-gap semiconductor FeGa$_3$

Intermetallic narrow-gap semiconductors have been intensively explored due to their large thermoelectric power at low temperatures and a possible role of strong electronic correlations in their unusual thermodynamic and transport properties. Here we study the optical spectra and vibrational properties of $\mathrm{FeGa_3}$ single crystal. The optical conductivity indicates that $\mathrm{FeGa_3}$ has a direct band gap of $\sim 0.7$\,eV, consistent with density functional theory (DFT) calculations. Most importantly, we find a substantial spectral weight also below 0.4~eV, which is the energy of the indirect (charge) gap found in resistivity measurements and ab initio calculations. We find that the spectral weight below the gap decreases with increasing temperature, which indicates that it originates from the impurity states and not from the electronic correlations. Interestingly, we did not find any signatures of the impurity states in vibrational spectra. The infrared and Raman vibrational lines are narrow and weakly temperature dependent. The vibrational frequencies are in excellent agreement with our DFT calculations, implying a modest role of electronic correlations. Narrow M\"ossbauer spectral lines also indicate high crystallinity of the sample.