Miracle-dz/UniFAR-FTUs huggingface dataset:
xMLP: Revolutionizing Private Inference with Exclusive Square Activation

Private Inference (PI) enables deep neural networks (DNNs) to work on private data without leaking sensitive information by exploiting cryptographic primitives such as multi-party computation (MPC) and homomorphic encryption (HE). However, the use of non-linear activations such as ReLU in DNNs can lead to impractically high PI latency in existing PI systems, as ReLU requires the use of costly MPC computations, such as Garbled Circuits. Since square activations can be processed by Beaver's triples hundreds of times faster compared to ReLU, they are more friendly to PI tasks, but using them leads to a notable drop in model accuracy. This paper starts by exploring the reason for such an accuracy drop after using square activations, and concludes that this is due to an"information compounding"effect. Leveraging this insight, we propose xMLP, a novel DNN architecture that uses square activations exclusively while maintaining parity in both accuracy and efficiency with ReLU-based DNNs. Our experiments on CIFAR-100 and ImageNet show that xMLP models consistently achieve better performance than ResNet models with fewer activation layers and parameters while maintaining consistent performance with its ReLU-based variants. Remarkably, when compared to state-of-the-art PI Models, xMLP demonstrates superior performance, achieving a 0.58% increase in accuracy with 7x faster PI speed. Moreover, it delivers a significant accuracy improvement of 4.96% while maintaining the same PI latency. When offloading PI to the GPU, xMLP is up to 700x faster than the previous state-of-the-art PI model with comparable accuracy.
When can we Approximate Wide Contrastive Models with Neural Tangent Kernels and Principal Component Analysis?

Contrastive learning is a paradigm for learning representations from unlabelled data that has been highly successful for image and text data. Several recent works have examined contrastive losses to claim that contrastive models effectively learn spectral embeddings, while few works show relations between (wide) contrastive models and kernel principal component analysis (PCA). However, it is not known if trained contrastive models indeed correspond to kernel methods or PCA. In this work, we analyze the training dynamics of two-layer contrastive models, with non-linear activation, and answer when these models are close to PCA or kernel methods. It is well known in the supervised setting that neural networks are equivalent to neural tangent kernel (NTK) machines, and that the NTK of infinitely wide networks remains constant during training. We provide the first convergence results of NTK for contrastive losses, and present a nuanced picture: NTK of wide networks remains almost constant for cosine similarity based contrastive losses, but not for losses based on dot product similarity. We further study the training dynamics of contrastive models with orthogonality constraints on output layer, which is implicitly assumed in works relating contrastive learning to spectral embedding. Our deviation bounds suggest that representations learned by contrastive models are close to the principal components of a certain matrix computed from random features. We empirically show that our theoretical results possibly hold beyond two-layer networks.
Nodal solutions to Paneitz-type equations

On a closed Riemannian manifold $(M^n ,g)$ with a proper isoparametric function $f$ we consider the equation $\Delta^2 u -\alpha \Delta u +\beta u = u^q$, where $\alpha$ and $\beta$ are positive constants satisfying that $\alpha^2 \geq 4 \beta$. We let ${\bf m}$ be the minimum of the dimensions of the focal varieties of $f$ and $q_f = \frac{n-{\bf m}+4}{n-{\bf m}-4}$, $q_f = \infty$ if $n\leq {\bf m}+4$. We prove the existence of infinitely many nodal solutions of the equation assuming that $1<q<q_f$. The solutions are $f$-invariant. To obtain the result, first we prove a $C^0-$estimate for positive $f$-invariant solutions of the equation. Then we prove the existence of mountain pass solutions with arbitrarily large energy.
Fully discrete finite difference schemes for the Fractional Korteweg-de Vries equation

In this paper, we present and analyze fully discrete finite difference schemes designed for solving the initial value problem associated with the fractional Korteweg-de Vries (KdV) equation involving the fractional Laplacian. We design the scheme by introducing the discrete fractional Laplacian operator which is consistent with the continuous operator, and posses certain properties which are instrumental for the convergence analysis. Assuming the initial data (u_0 \in H^{1+\alpha}(\mathbb{R})), where (\alpha \in [1,2)), our study establishes the convergence of the approximate solutions obtained by the fully discrete finite difference schemes to a classical solution of the fractional KdV equation. Theoretical results are validated through several numerical illustrations for various values of fractional exponent $\alpha$. Furthermore, we demonstrate that the Crank-Nicolson finite difference scheme preserves the inherent conserved quantities along with the improved convergence rates.
DrivAerNet: A Parametric Car Dataset for Data-Driven Aerodynamic Design and Graph-Based Drag Prediction

This study introduces DrivAerNet, a large-scale high-fidelity CFD dataset of 3D industry-standard car shapes, and RegDGCNN, a dynamic graph convolutional neural network model, both aimed at aerodynamic car design through machine learning. DrivAerNet, with its 4000 detailed 3D car meshes using 0.5 million surface mesh faces and comprehensive aerodynamic performance data comprising of full 3D pressure, velocity fields, and wall-shear stresses, addresses the critical need for extensive datasets to train deep learning models in engineering applications. It is 60\% larger than the previously available largest public dataset of cars, and is the only open-source dataset that also models wheels and underbody. RegDGCNN leverages this large-scale dataset to provide high-precision drag estimates directly from 3D meshes, bypassing traditional limitations such as the need for 2D image rendering or Signed Distance Fields (SDF). By enabling fast drag estimation in seconds, RegDGCNN facilitates rapid aerodynamic assessments, offering a substantial leap towards integrating data-driven methods in automotive design. Together, DrivAerNet and RegDGCNN promise to accelerate the car design process and contribute to the development of more efficient vehicles. To lay the groundwork for future innovations in the field, the dataset and code used in our study are publicly accessible at \url{https://github.com/Mohamedelrefaie/DrivAerNet}
Palynological and petroleum geochemical assessment of the Lower Oligocene Mezardere Formation, Thrace Basin, NW Turkey

The Oligocene clastic sequence of the Mezardere Formation (MF) with laterally variable organic richness has long been known as a proven source of gas with minor oil accumulations across the Thrace Basin of northwest Turkey. However, based on well data for the thick MF, neither detailed work in relation to age dating and stratigraphy nor a close linkage between the depositional facies/ environments, organic richness/organic proxies, and cyclicity has been established yet. In the present study, the MF was informally subdivided into Lower MF (LMF) and Upper MF (UMF) based on the distinct differences in palynological and geochemical data. Based on the common occurrences of Glaphyrocysta cf. semitecta and absence of Wetzeliella gochtii , the LMF is considered to be deposited during the earliest Oligocene (? Pshekian) under the prevailing marine conditions. The UMF is characterized by a very rich and diverse dinocyst assemblage having abundant occurrences of age-diagnostic Wetzeliella gochtii and a Solenovian age is assigned. Common Pediastrum occurrences in the UMF may suggest fresh water input as is the case for many source rocks of the Central and Eastern Paratethys. The UMF shows the geochemical characteristics of a typical transgressive sequence such as higher TOC, hydrogen index (HI), and relative hydrocarbon potential (RHP) values than those for the regressive LMF. On the RHP basis, three short-term transgressive to regressive cycles are recognized in the entire MF in the wells studied. The early mature UMF samples showed a fair to good source rock potential (average TOC = 1.14 wt. %; HI = 283 mg oil/g TOC) and low to moderate genetic petroleum potential (GP = 3.65 mg oil/g rock) and source potential index (SPI = 1.44 t oil/m 2 ). The LMF samples were not evaluated due to their apparently low TOC, HI, and S2 values. Better understanding of the MF will eventually aid a better understanding of the paleoenvironment of the Eastern Paratethys.
The effects of topography and forest management on water storage in catchments in south‐central Chile

Our work analyses the intra‐annual variability of the volume of water stored in 15 forested headwater catchments from south‐central Chile, aiming at understanding how forest management, hydrology, and climate influence the dynamic components of catchment storage. Thus, we address the following questions: (a) How does the annual water storage vary in catchments located in diverse hydroclimatic conditions and subject to variable forest management? (b) Which natural (i.e., hydrologic regime and physiographic setting) and anthropogenic factors explain the variance in water storage? Results show that the annual catchment storage increases at the beginning of each hydrological year in direct response to increases in rainfall. The maximum water storage ranges from 666 to 1,272 mm in these catchments. The catchments with Pinus or Eucalyptus spp. cover store less water than the catchments with mixed forest species cover. Forest cover (biomass volume, plantation density, and percentage of plantation and age) has the primary control on dynamic storage in all catchments. These results indicate that forest management may alter the catchment water storage.
Generating Synthetic Computed Tomography for Radiotherapy: SynthRAD2023 Challenge Report

Radiation therapy plays a crucial role in cancer treatment, necessitating precise delivery of radiation to tumors while sparing healthy tissues over multiple days. Computed tomography (CT) is integral for treatment planning, offering electron density data crucial for accurate dose calculations. However, accurately representing patient anatomy is challenging, especially in adaptive radiotherapy, where CT is not acquired daily. Magnetic resonance imaging (MRI) provides superior soft-tissue contrast. Still, it lacks electron density information while cone beam CT (CBCT) lacks direct electron density calibration and is mainly used for patient positioning. Adopting MRI-only or CBCT-based adaptive radiotherapy eliminates the need for CT planning but presents challenges. Synthetic CT (sCT) generation techniques aim to address these challenges by using image synthesis to bridge the gap between MRI, CBCT, and CT. The SynthRAD2023 challenge was organized to compare synthetic CT generation methods using multi-center ground truth data from 1080 patients, divided into two tasks: 1) MRI-to-CT and 2) CBCT-to-CT. The evaluation included image similarity and dose-based metrics from proton and photon plans. The challenge attracted significant participation, with 617 registrations and 22/17 valid submissions for tasks 1/2. Top-performing teams achieved high structural similarity indices (>0.87/0.90) and gamma pass rates for photon (>98.1%/99.0%) and proton (>99.0%/97.3%) plans. However, no significant correlation was found between image similarity metrics and dose accuracy, emphasizing the need for dose evaluation when assessing the clinical applicability of sCT. SynthRAD2023 facilitated the investigation and benchmarking of sCT generation techniques, providing insights for developing MRI-only and CBCT-based adaptive radiotherapy.
Preserving Automotive Heritage: A Blockchain-Based Solution for Secure Documentation of Classic Cars Restoration

Classic automobiles are an important part of the automotive industry and represent the historical and technological achievements of certain eras. However, to be considered masterpieces, they must be maintained in pristine condition or restored according to strict guidelines applied by expert services. Therefore, all data about restoration processes and other relevant information about these vehicles must be rigorously documented to ensure their verifiability and immutability. Here, we report on our ongoing research to adequately provide such capabilities to the classic car ecosystem. Using a design science research approach, we have developed a blockchain-based solution using Hyperledger Fabric that facilitates the proper recording of classic car information, restoration procedures applied, and all related documentation by ensuring that this data is immutable and trustworthy while promoting collaboration between interested parties. This solution was validated and received positive feedback from various entities in the classic car sector. The enhanced and secured documentation is expected to contribute to the digital transformation of the classic car sector, promote authenticity and trustworthiness, and ultimately increase the market value of classic cars.
Heterogeneous Nucleation and Growth of Sessile Chemically Active Droplets

Droplets are essential for spatially controlling biomolecules in cells. To work properly, cells need to control the emergence and morphology of droplets. On the one hand, driven chemical reactions can affect droplets profoundly. For instance, reactions can control how droplets nucleate and how large they grow. On the other hand, droplets coexist with various organelles and other structures inside cells, which could affect their nucleation and morphology. To understand the interplay of these two aspects, we study a continuous field theory of active phase separation. Our numerical simulations reveal that reactions suppress nucleation while attractive walls enhance it. Intriguingly, these two effects are coupled, leading to shapes that deviate substantially from the spherical caps predicted for passive systems. These distortions result from anisotropic fluxes responding to the boundary conditions dictated by the Young-Dupr\'e equation. Interestingly, an electrostatic analogy of chemical reactions confirms these effects. We thus demonstrate how driven chemical reactions affect the emergence and morphology of droplets, which could be crucial for understanding biological cells and improving technical applications, e.g., in chemical engineering.
Thermal Hall effect in the van der Waals ferromagnet 
<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow><mml:mi>Cr</mml:mi><mml:msub><mml:mi mathvariant="normal">I</mml:mi><mml:mn>3</mml:mn></mml:msub></mml:mrow></mml:math>

CrI3 is a prototypical van der Waals ferromagnet with a magnetic honeycomb lattice. Previous inelastic neutron scattering studies have suggested topological nature of its magnetic excitations with a magnon gap at the Dirac points, which are anticipated to give rise to magnon thermal Hall effect. Here we report thermal transport properties of CrI3 and show that the long-sought thermal Hall signal anticipated for topological magnons is fairly small. In contrast, we find that CrI3 exhibits an appreciable anomalous thermal Hall signal at lower temperature which may arise from magnon-phonon hybridization or magnon-phonon scattering. These findings are anticipated to stimulate further neutron scattering studies on CrI3 single crystal, which can shed light not only on the intrinsic nature of magnetic excitations but also on the magnon-phonon interaction.
Asymptotic behaviour of integer programming and the $\text{v}$-function of a graded filtration

The $\text{v}$-function of a graded filtration $\mathcal{I}=\{I_{[k]}\}_{k\ge0}$ is introduced. Under the assumption that $\mathcal{I}$ is Noetherian, we prove that the $\text{v}$-function $\text{v}(I_{[k]})$ is an eventually quasi-linear function. This result applies to several situations, including ordinary powers, and integral closures of ordinary powers, among others. As another application, we investigate the asymptotic behaviour of certain integer programming problems. Finally, we present the Macaulay2 package $\texttt{VNumber}$.
LAFS: Landmark-based Facial Self-supervised Learning for Face Recognition

In this work we focus on learning facial representations that can be adapted to train effective face recognition models, particularly in the absence of labels. Firstly, compared with existing labelled face datasets, a vastly larger magnitude of unlabeled faces exists in the real world. We explore the learning strategy of these unlabeled facial images through self-supervised pretraining to transfer generalized face recognition performance. Moreover, motivated by one recent finding, that is, the face saliency area is critical for face recognition, in contrast to utilizing random cropped blocks of images for constructing augmentations in pretraining, we utilize patches localized by extracted facial landmarks. This enables our method - namely LAndmark-based Facial Self-supervised learning LAFS), to learn key representation that is more critical for face recognition. We also incorporate two landmark-specific augmentations which introduce more diversity of landmark information to further regularize the learning. With learned landmark-based facial representations, we further adapt the representation for face recognition with regularization mitigating variations in landmark positions. Our method achieves significant improvement over the state-of-the-art on multiple face recognition benchmarks, especially on more challenging few-shot scenarios.
Physics-Inspired Deep Learning Anti-Aliasing Framework in Efficient Channel State Feedback

Acquiring downlink channel state information (CSI) at the base station is vital for optimizing performance in massive Multiple input multiple output (MIMO) Frequency-Division Duplexing (FDD) systems. While deep learning architectures have been successful in facilitating UE-side CSI feedback and gNB-side recovery, the undersampling issue prior to CSI feedback is often overlooked. This issue, which arises from low density pilot placement in current standards, results in significant aliasing effects in outdoor channels and consequently limits CSI recovery performance. To this end, this work introduces a new CSI upsampling framework at the gNB as a post-processing solution to address the gaps caused by undersampling. Leveraging the physical principles of discrete Fourier transform shifting theorem and multipath reciprocity, our framework effectively uses uplink CSI to mitigate aliasing effects. We further develop a learning-based method that integrates the proposed algorithm with the Iterative Shrinkage-Thresholding Algorithm Net (ISTA-Net) architecture, enhancing our approach for non-uniform sampling recovery. Our numerical results show that both our rule-based and deep learning methods significantly outperform traditional interpolation techniques and current state-of-the-art approaches in terms of performance.
Capturing electronic correlations in electron-phonon interactions in molecular systems with the GW approximation

Electron-phonon interactions are of great importance to a variety of physical phenomena, and their accurate description is an important goal for first-principles calculations. Isolated examples of materials and molecular systems have emerged where electron-phonon coupling is enhanced over density functional theory (DFT) when using the Green's-function-based ab initio GW method, which provides a more accurate description of electronic correlations. It is however unclear how general this enhancement is, and how employing high-end quantum chemistry methods, which further improve the description of electronic correlations, might further alter electron-phonon interactions over GW or DFT. Here, we address these questions by computing the renormalization of the highest occupied molecular orbital energies of Thiel's set of organic molecules by harmonic vibrations using DFT, GW and equation-of-motion coupled-cluster calculations. We find that GW can increase the magnitude of the electron-phonon coupling across this set of molecules by an average factor of 1.1-1.8 compared to DFT, while equation-of-motion coupled-cluster leads to an increase of 1.4-2. The electron-phonon coupling predicted with the ab initio GW method is generally in much closer agreement to coupled cluster values compared to DFT, establishing GW as an accurate way of computing electron-phonon phenomena in molecules and beyond at a much lower computational cost than higher-end quantum chemistry techniques.
Random Search as a Baseline for Sparse Neural Network Architecture Search

Sparse neural networks have shown similar or better generalization performance than their dense counterparts while having higher parameter efficiency. This has motivated a number of works to learn or search for high performing sparse networks. While reports of task performance or efficiency gains are impressive, standard baselines are lacking leading to poor comparability and unreliable reproducibility across methods. In this work, we propose Random Search as a baseline algorithm for finding good sparse configurations and study its performance. We apply Random Search on the node space of an overparameterized network with the goal of finding better initialized sparse sub-networks that are positioned more advantageously in the loss landscape. We record the post-training performances of the found sparse networks and at various levels of sparsity, and compare against both their fully connected parent networks and random sparse configurations at the same sparsity levels. First, we demonstrate performance at different levels of sparsity and highlight that a significant level of performance can still be preserved even when the network is highly sparse. Second, we observe that for this sparse architecture search task, initialized sparse networks found by Random Search neither perform better nor converge more efficiently than their random counterparts. Thus we conclude that Random Search may be viewed as a reasonable neutral baseline for sparsity search methods.
Ab initio Density Response and Local Field Factor of Warm Dense Hydrogen

We present quasi-exact ab initio path integral Monte Carlo (PIMC) results for the partial static density responses and local field factors of hydrogen in the warm dense matter regime, from solid density conditions to the strongly compressed case. The full dynamic treatment of electrons and protons on the same footing allows us to rigorously quantify both electronic and ionic exchange--correlation effects in the system, and to compare with earlier incomplete models such as the archetypal uniform electron gas [Phys. Rev. Lett. 125, 235001 (2020)] or electrons in a fixed ion snapshot potential [Phys. Rev. Lett. 129, 066402 (2022)] that do not take into account the interplay between the two constituents. The full electronic density response is highly sensitive to electronic localization around the ions, and our results constitute unambiguous predictions for upcoming X-ray Thomson scattering (XRTS) experiments with hydrogen jets and fusion plasmas. All PIMC results are made freely available and can directly be used for a gamut of applications, including inertial confinement fusion calculations and the modelling of dense astrophysical objects. Moreover, they constitute invaluable benchmark data for approximate but computationally less demanding approaches such as density functional theory or PIMC within the fixed-node approximation.