# Quaternion-Based Vision-Transformer for Polycrystalline EBSD Scans Pre-Trained on Large-Scale Synthetic Data

Pierre Belamri, Henry Proudhon, Damien Texier, David Ryckelynck

► To cite this version:

Pierre Belamri, Henry Proudhon, Damien Texier, David Ryckelynck. Quaternion-Based Vision-Transformer for Polycrystalline EBSD Scans Pre-Trained on Large-Scale Synthetic Data. Materials & Design, 2025, 258, pp.114599. 10.1016/j.matdes.2025.114599. hal-05229470

HAL Id: hal-05229470

https://imt-mines-albi.hal.science/hal-05229470v1

Submitted on 16 Sep 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

# Quaternion-based vision-transformer for polycrystalline EBSD scans pre-trained on large-scale synthetic data

Pierre Belamri<sup>a,b</sup>, Henry Proudhon<sup>b</sup>, Damien Texier<sup>a, c</sup>, David Ryckelynck<sup>c</sup>

## ARTICLE INFO

**Keywords:**

Autoencoder

Vision transformer

Electron backscattered diffraction

Quaternion

Microstructural latent space

Multimodal analyses

## ABSTRACT

Dimensionality reduction is crucial in materials science for extracting patterns from high-dimensional data. This is vital for optimizing material design spaces by cutting computational complexity and for integrating diverse data sources for advanced anomaly detection, identifying structural or functional material deviations. Traditional convolutional autoencoders focus on local features, failing to capture global contextual information essential for predicting material behavior under stress, where overall material properties, grain orientations, morphologies and sizes are key. To address this, a Quaternion-Based Vision Transformer Masked Autoencoder (ViT-MAE) is proposed for polycrystalline materials. This model processes quaternion-valued EBSD orientation maps with a 65% masking ratio, capturing both local and global microstructural features efficiently. Pre-trained on synthetic data using self-supervision, the model achieves robust generalization on real-world EBSD scans by minimizing a misorientation loss function on the quaternion manifold. This work lays the groundwork for advanced multimodal data analysis in materials science, offering an open dataset and pre-trained vision-transformer weights. The goal is to provide a reliable latent space for EBSD orientation maps, advancing future multimodal characterization through latent space merging.

## 1. Introduction

In material science for polycrystalline materials, digital images are required to analyze global material properties at the microstructure scale, such as the orientation texture of the material, but also local properties such as the grain shape and misorientation across grain boundaries. Therefore, image-based models are more and more of a concern to better define and predict materials in terms of microstructure and functional properties (mechanical, electrical, optical, etc. properties). Similarly to the field of medical imaging [1], both engineering and scientific communities face a lack of annotated images that hinders the deployment of deep learning algorithms for polycrystalline materials. As in Ref. [1], the present study intends to develop a large synthetic EBSD dataset and a masked autoencoder model for self-supervised pre-training. A particular focus has been paid on the domain of digital images for polycrystalline materials. In this paper, the model has been pre-trained by exploiting the advantage of numerical microstructure generation models. The efficacy of the synthetic dataset for model pre-

training has been systematically evaluated, wherein the transferability of learned representations has been assessed using real-world anomaly detection tasks in polycrystalline materials.

Electron Backscatter Diffraction (EBSD) is a scanning electron microscope (SEM)-based technique that provides crystallographic information about the microstructure of crystalline materials [2,3]. The Kikuchi patterns obtained during EBSD acquisition are characteristic of the local crystalline orientation of the sample. As a result, Kikuchi patterns are used to locally determine crystal orientation, distinguish between different phases, characterize grain boundaries (including twin boundaries), and provide information on crystal defects or imperfect surface preparation prior to image acquisition. Clustering of individual orientation measurements aims also to segment grains, phases, microstructural features for size and morphological descriptions of the materials. For minimizing data storage, Kikuchi patterns are not necessarily saved during acquisition and the orientation information is kept as triplets of euler angles and diffraction information (confidence index, band contrast, indexed bands, etc.). The present paper will focus on the assimilation of the eu-

<footnote><sup>a</sup> Institut Clement Ader (ICA) - UMR CNRS 5312, Université de Toulouse, CNRS, INSA, UPS, Mines Albi, ISAE-SUPAERO, Campus Jarlard, 81013 Albi Cedex 09, France</footnote>

<footnote><sup>b</sup> MINES Paris, PSL University, MAT - Centre des matériaux, CNRS UMR 7633, BP 87, 91003 Evry, France</footnote>

<footnote><sup>c</sup> MINES Paris, PSL University, CEMEF, CNRS UMR 7635, BP 207, 06904, Sophia Antipolis, France</footnote>

<footnote>* Corresponding author.
E-mail address: damien.texier@mines-albi.fr (D. Texier).</footnote>
ler angle information converted and processed into quaternions which avoids the gimbal lock problem and is more numerically stable for loss computation [4]. However, Kikuchi patterns, in terms of band sharpness, contrast, etc., hide much more information than crystal orientation only. For instance, Kikuchi patterns were also used as sub-grain microstructural descriptors using machine learning approaches and spatial mapping of diffraction latent space features, showing dendrite structures in additive manufacturing materials that eular angles could not depict [5]. While this highlights the potential for richer microstructural analysis, the present study deliberately focuses only on the crystallographic orientation data (represented by quaternions). The goal is to first establish a robust baseline model for orientation maps, which can serve as a foundation for future work incorporating the full spectrum of information available in Kikuchi patterns.

Multimodal characterization of materials often leverage EBSD orientation map [6–12]. However, EBSD scans are subject to the most common challenges depicted in Ref. [13] when it comes to merging them with other modalities: Alignment and registration [14–16]; Different resolutions; Noise [17,18]; Errors [19]. Assuming that the alignment problem was solved, having a reliable latent representation of the EBSD map can help merging data modalities into common spaces and improve our ability to characterize materials. Modeling a latent representation of an EBSD scan is not trivial, as it involves capturing both the local features (e.g., grain boundaries, orientation gradients within grains) and potentially global features (e.g., overall texture, grain size distribution), all this with particular symmetry conditions inherent to the crystal structure.

Convolutional Neural Networks (CNNs) have proven to be a powerful approach for detecting local patterns in image-like data, such as EBSD scans. However, they do have limitations when it comes to capturing global features or understanding the overall structure of the input data. These limitations are explained by the nature of convolution filters that operate on local receptive fields. Additionally, they are designed to be spatially invariant, and this property can make it difficult to capture spatial relationships that are crucial for understanding global arrangements of data or layouts like EBSD orientation maps. Vision-Transformers (ViTs) [20] have emerged as a promising alternative to conventional CNNs for capturing global context in image data. Unlike CNNs, ViTs divide the image into patches and process them through self-attention mechanisms, enabling the model to assess the relative importance of different image regions. This ability to capture long-range dependencies is particularly valuable in materials science applications where stochastic microstructures do not follow a stationary process [21,22], with local probabilities that are not translationally invariant. Materials exposed to thermal, mechanical, or chemical processing gradients develop spatially varying statistical properties. These variations produce microstructural heterogeneities that necessitate analysis within a global spatial context. The self-attention mechanism in transformers effectively addresses these non-stationary patterns by establishing direct connections between all spatial positions, enabling the model to capture the complex spatial dependencies characteristic of heterogeneous materials with process-induced global gradients.

In this work, a Vision-Transformer Masked Autoencoder (ViT-MAE) [23] is presented, specifically adapted and trained on quaternion-based EBSD orientation maps. Using synthetically generated microstructures, the model is trained on a reconstruction task to learn robust and compact latent representations of EBSD data. This training strategy enables the model to effectively generalize to real-world scenarios while addressing the limitations posed by small experimental datasets commonly used in the field. To assess the approach, a downstream task of crystallographic twin detection, a particular case of grain boundary character distinctively observed in EBSD orientation maps with specific relationship between grains and morphology (Σ3 twin boundary) is demonstrated. Specifically, out-of-distribution (OOD) feature detection in the latent space using Mahalanobis distance is employed, showcasing the ability of the ViT-MAE to discriminate microstructural

features that were not present in the training distribution. This approach not only highlights the potential of self-supervised Vision Transformers in microstructure analysis but also provides a scalable solution for automating defect detection in polycrystalline materials. The trained ViT-MAE model weights and the synthetic dataset used in this study are released. These resources are intended to lay the foundation for future multimodal approaches by providing a robust and scalable framework for processing spatially resolved crystal orientation data as a first step toward integrated material characterization in general foundation models.

## 2. Related-work

An important trend in machine learning involves foundation models [24], which are general-purpose backbone models pre-trained on large datasets using self-supervised approaches, then adapted for various downstream tasks. Examples include GPT-4 [25] and Qwen-VL [26]. Inspired by this paradigm and the recent advances in local features extraction techniques for microstructural data [5], this effort aims to establish a pre-trained model for further mechanical properties of polycrystalline materials. The model emphasizes anomaly detection as a downstream task, enabling additional applications that leverage general representations learned from extensive unlabeled data.

Pre-trained ViT architectures have already been used for anomaly detection in several works related to materials or, more generally, to industrial data. The scarcity of data in the field of anomaly detection in industrial images has led to the development of specific coordinate attention for ViT architectures [27] in the framework of a reconstruction-based approach [28]. Pre-trained Vision Transformer models have been proposed for diagnosing material corrosion, where steel corrosion represents a critical anomalous condition in industrial components [29]. The authors addressed the challenge of limited training data through a two-stage methodology: initially employing masked self-supervised learning for pre-training, followed by a targeted fine-tuning phase. This framework enabled robust identification of diverse steel corrosion states while maintaining high diagnostic accuracy despite training data constraints.

ViT-MAE [23] are also beneficial for self-supervised few-shot learning technique [30], when limited labels are available. This approach can be generalized to out-of-domain classes by fine-tuning the model in a fully self-supervised method. The aforementioned papers are encouraging results for developing an out-of-distribution (OOD) anomaly detection in the latent space of a ViT-MAE by using Mahalanobis distance. For OOD detection using Mahalanobis distance applied to pre-trained deep features based on convolutional neural networks, the reader is referred to Refs. [31,32].

The present paper extends the OOD detection approach to pre-trained ViT-MAE by implementing a physically-based loss function for grain orientation in polycrystals. A rotational distance loss was proposed in Ref. [33] to quantify the misorientation angle between predicted and ground truth electron backscatter diffraction (EBSD) maps while properly accounting for the quaternion manifold. Quaternions represent orientation data with 4 channels. If $q_1$ and $q_2$ denote two quaternions of grain orientations, it is important to note that within this manifold, $q_1 = -q_2$ represents equivalent three-dimensional orientations. The rotational distance $\theta$ between these quaternions is defined as follows:

$$ \theta(q_1, q_2) = \arccos \left(1 - \frac{1}{2}d^2\right) = 2 \arcsin \left(\frac{d}{2}\right), \text{ where } d = \|q_1 - q_2\|_2 \quad (1) $$

The proposed method relies on the orientation map reconstruction task, but could be extended to use tokenized input from Kikuchi patterns using latent projections of the proposed encoder suggested in Ref. [5] and a cross-attention approach.
# 3. Materials and methods

## 3.1. Dataset

The dataset used in this study consists of synthetic EBSD orientation maps generated using the Neper<fnref n="1" /> software [34]. Three-dimensional synthetic face centered cubic (FCC) crystals were created using Voronoi tessellation with random seed distribution in non-cubic domains (1.0×1.2×1.4) with translation (−0.5, −0.6, −0.7). The generated cubes contained between 300 and 1000 grains with corresponding voxel dimensions of 300, 400, 500, and 900 pixels, respectively. Random 3D orientations with cubic crystal symmetry were assigned to each grain from a uniform distribution, ensuring the generated microstructures are texture-free. This approach encourages the model to learn general microstructural features without being biased towards any specific crystallographic texture. The resulting grain size distribution, an emergent property of the Voronoi tessellation process, follows a log-normal distribution with $μ = 2.084$ and $σ = 0.314$. The orientations are represented as quaternions, captured in $H \times W \times 4$ arrays (where Height and Width correspond to the spatial dimensions of the orientation map) that encode the local crystallographic orientations in a 4-channels format. To prepare the dataset for self-supervised training, the three-dimensional synthetic crystalline aggregates were sliced to generate two-dimensional quaternion maps. Each resulting slice was subsequently partitioned into smaller $128 \times 128$ tiles using a spatial step of 2 pixels for the training set and 4 pixels for the validation set. This methodical approach yielded a substantial dataset comprising 3.2 million images for training and 850,000 images for validation, thereby encompassing a diverse range of features.

For validation purposes, two real EBSD datasets containing twins were additionally utilized. Grain segmentation was performed on these real datasets using the pymicro<fnref n="2" /> Python package [35]. Two versions of each real dataset were generated: one preserving the original twin structures and another with these twins removed. This systematic preparation resulted in four distinct validation datasets—two real datasets with twins and their corresponding twin-free variants—thereby providing comprehensive test cases for thorough evaluation.

## 3.2. ViT-MAE architecture

### Patch embedding

The patch embedding process is implemented through a singular 2D Convolution layer with both kernel size and stride parameters set to match the patch dimensions (16 pixels) without bias terms. This configuration ensures that the convolutional filter encompasses the entire patch simultaneously (kernel) and advances precisely one patch width during each convolution operation (stride). This design intentionally processes each patch as a discrete unit, effectively partitioning the image into non-overlapping segments. The convolutional layer outputs are channels numbering equivalent to the transformer's embedding dimension (1024). This transformation enables each patch to be represented within a space of consistent dimensionality ($16 \times 16 \times 4$). Experimental results indicate that employing either a Quaternion Convolution filter [36] or a standard 2D Convolution filter produces comparable convergence characteristics, as the attention mechanisms in subsequent layers operate independently of the quaternion manifold's specific properties.

### Positional Encoding

Transformer treats input data as an unordered set, necessitating the encoding of the sequence order of input patches to preserve spatial context. The implementation utilized 2D sine-cosine positional encoding [37]. This method formalizes the spatial relationships inherent to

the structural organization of image data. The encoding generates embeddings based on the grid height and width of the image patches. Each position within the grid (both height and width) is encoded separately, and the results are combined to form a complete positional embedding for each patch. These positional embeddings are integrated into the ViT-MAE model as fixed parameters, initialized during the setup of the model and subsequently frozen. This decision was informed by empirical observations which indicated that learnable positional embeddings did not offer significant improvements over fixed embeddings in this specific setting. Notably, the sine-cosine encoding enables interpolation for processing images larger than those used during training, as its continuous mathematical properties allow meaningful representation of arbitrary spatial coordinates without requiring model retraining.

### Transformer Autoencoder

The encoder processes the embedded image patches through a stack of 10 transformer encoder layers. Each layer has an embedding dimension of 1024 and employs 16 attention heads, allowing the model to capture complex relationships within the data. The multi-head self-attention mechanism enables simultaneous attention to different parts of the patch sequence, enhancing both local feature extraction and global pattern recognition. Each layer also includes a position-wise feed-forward network with an intermediate size of 3072 and uses the Gaussian Error Linear Unit (GELU) activation function for smooth and effective learning.

The decoder aims to reconstruct the masked patches using the encoded representations. It consists of 8 transformer decoder layers with a reduced embedding dimension of 512, encouraging the model to distill essential features during decoding. Similar to the encoder, each decoder layer employs 16 attention heads and includes both self-attention and cross-attention mechanisms. The cross-attention attends to the encoder's output, allowing the decoder to leverage contextual information from the unmasked patches.

A high masking ratio of 65% is applied during pre-training. The standard level is often set at 75% [23], but a lower ratio was selected since EBSD serves as a reference modality anchoring grain boundaries and geometric features during multimodal analysis. Retaining a greater portion of visible patches supports these features. Mask tokens represent the hidden regions, and positional embeddings maintain spatial relations across the sequence. The schematic illustration of the whole model is shown in Fig. 1.

## 3.3. Training setup

### Loss

To train the model, the rotational distance loss proposed in Ref. [33] is used, denoted by $\theta$. Although $d$ in Equation (1) satisfies the Lipschitz condition, it must be ensured that it remains bounded away from 2 as $\arcsin'(x) \to \infty$ when $x \to 1$. To achieve this, a linear approximation is implemented when $d > 1.9$, as proposed by Jangid et al. in [38]. Crystal symmetries were not included during pre-training for two main reasons:

* The memory footprint of applying 24 FCC symmetries in parallel is significant. Instead of computing just two operations – i.e., $\min[d(q_1, q_2), d(-q_1, q_2)]$ – it becomes necessary to evaluate $2 \times 24^2 = 1152$ possibilities per pixel. This creates a major computational bottleneck when pre-training on several million samples.

* A general model that was symmetry agnostic was desired. For symmetry-specific use cases, symmetries can always be included during fine-tuning.

### Configurations

The model was pre-trained over 150 epochs using a batch size of 128. The patch embedding dimension was set to 1024 as patch-wise compression was undesired. The tile-wise compression is done through the masking process.

<footnote n="1">¹ https://neper.info/.</footnote>

<footnote n="2">² https://github.com/heprom/pymicro.</footnote>
<figure><img src="image_3.png" /><figcaption>Fig. 1. Schematic illustration of the EBSD ViT-MAE.</figcaption></figure>

<table><caption>**Table 1**
Pretraining Settings.</caption><thead><tr><td>Config</td><td>Value</td></tr></thead><tbody><tr><td>Optimizer</td><td>AdamW</td></tr><tr><td>Base Learning Rate</td><td>1e-4</td></tr><tr><td>Weight Decay</td><td>0.05</td></tr><tr><td>Optimizer Momentum</td><td>(β₁, β₂) = (0.9, 0.95)</td></tr><tr><td>Batch Size</td><td>128</td></tr><tr><td>Learning Rate Schedule</td><td>Cosine Annealing [39]</td></tr><tr><td>Warmup Epochs</td><td>40</td></tr><tr><td>Augmentation</td><td>RandomHorizontalFlip<br/>RandomVerticalFlip</td></tr></tbody></table>

<figure><img src="image_4.png" /><figcaption>Fig. 2. Training and validation losses.</figcaption></figure>

## Computational Resources and Runtime

The pre-training was conducted on a compute cluster equipped with 8 NVIDIA A100 (40 GB) GPUs using a *DistributedDataParallel* (DDP) strategy. The model was replicated on each GPU, while the entire dataset, consisting of several million samples, was pre-loaded into system RAM and partitioned across the distributed processes. After each backward pass, gradients were synchronized and averaged across all GPUs using the ring-AllReduce algorithm. The primary computational bottlenecks were the data transfer latency from system RAM to each GPU's VRAM at every training step and the computation of the linear approximation in the loss. Each of the 150 epochs took approximately 30 to 45 minutes to complete, leading to a total training time of about 3 days. The subsequent zero-shot evaluation was performed on a single A100 GPU, with a runtime on the order of minutes for the entire test set (Table 1, Fig. 2).

### 3.4. Downstream task: twin detection via anomaly detection in latent space

To verify that the model has learned meaningful latent representations beyond simple orientation reconstruction, the utility of its latent space is assessed via a downstream task. This task, demonstrated on the ViT-MAE, is focused on twin detection in real-world EBSD datasets. Twins in FCC crystals, such as the Alloy 718 Ni-based superalloy, are characterized by Σ3 boundaries corresponding to a 60° misorientation around the (011) crystallographic direction and sharing a common

{111} plane. These Σ3 relations, which define the crystallographic characteristics of twins, were automatically annotated in the dataset using the pymicro package (pymicro.readthedocs.io [35]). Twins are treated as anomalies in this context because the ViT-MAE model was explicitly trained on twin-free microstructural datasets, making twin features out-of-distribution (OOD) elements with respect to the model's learned representation space.

The downstream task uses an out-of-distribution (OOD) anomaly detection approach in the latent space based on the Mahalanobis distance [40]. First, the encoder's last hidden states of size = [seq_length, embedding] = [23, 1024]) were inferred from EBSD samples without defects (in-distribution, ID). The model was found to experience dimensional collapse (Fig. 3) with these hidden states tensors, whereby the embedding vectors end up spanning a lower-dimensional subspace instead of the entire available latent space. This behavior is consistent with previous findings that establish a connection between masked autoencoding and contrastive learning [41], the latter being associated with dimensional collapse [42].

To address this and enable computation of the inverse covariance matrix needed for the Mahalanobis distance [40], latent vectors have to be extracted from the hidden states tensors, so that the resulting covariance matrix of the latent space only contains non-zero singular values.
<figure><img src="image_5.png" /><figcaption>Fig. 3. Singular value spectrum of the embedding space. The embedding vectors are computed from the pretrained ViT-MAE Encoder part on a validation set of Synthetic EBSD scans. Each embedding vector has a dimension of 23 × 1024. The spectrum contains the singular values of the covariance matrix of these embedding vectors in sorted order and logarithmic scale. A number of singular values drop to zero, indicating collapsed dimensions.</figcaption></figure>

A principal component analysis (PCA) model with 1024 components – well below the matrix’s apparent rank – was fitted on a prior synthetic dataset’s last hidden state to perform the latent projection. The same PCA model was then used to reduce the dimensions of all other hidden distributions, allowing for the computation of Mahalanobis distances in the latent space. This approach was motivated by empirical observations which indicated that other aggregation techniques such as min/max/mean pooling and attention-based weighted pooling over visible tokens in the sequence axis yielded poorer downstream performance.

To establish the in-distribution (ID) and out-of-distribution (OOD) datasets:

* ID Samples: two real EBSD scans of FCC crystals with no twins (manually removed twins).

* OOD Samples: the same two real EBSD scans but containing twins.

The twin removal procedure was implemented by segmenting all grain boundaries, identifying Σ3 twin boundaries using the Brandon criterion, reindexing twin orientations to their parent grain ID, and merging the grains by assigning the parent orientation to the former twin.

# 4. Results

## 4.1. Reconstruction performance on synthetic dataset

The reconstruction process involved partitioning the original EBSD scan into tiles of dimension 128 × 128 × 4, applying random masking to 65% of each tile, and subsequently reconstructing the masked regions using the trained ViT MAE. Fig. 4 presents the reconstruction results for synthetic quaternion-based orientation maps. The figure displays the original synthetic EBSD scan (Fig. 4(a)), the masked input provided to the model (Fig. 4(b)), and the reconstructed output (Fig. 4(c)). The inverse pole figure (IPF-Z) coloring scheme was employed to facilitate qualitative assessment of the orientation reconstruction accuracy.

A higher magnification analysis was performed on a selected region containing multiple grain boundaries. Fig. 5 presents this detailed comparison, showing the original (Fig. 5(a)) and reconstructed (Fig. 5(b)) orientation maps in a boundary-rich region. The orientation profiles along the indicated transect line are presented in Fig. 5(c).

Quantitative evaluation was performed through misorientation analysis between the original and reconstructed quaternion maps for the synthetic dataset, as illustrated in Fig. 6. The misorientation map (Fig. 6)

reveals a spatial distribution of disorientation values across the microstructure. Grain interiors exhibit disorientation values generally below 3 degrees, while grain boundary regions show disorientation values in the range of 10-15 degrees. The misorientation map is computed by evaluating the pixel-wise angular difference between the original EBSD orientation map $X \in \mathbb{R}^{H \times W \times 4}$ and the autoencoder-reconstructed orientation map $X^* \in \mathbb{R}^{H \times W \times 4}$, where both are represented in quaternion format. For each spatial position $(i, j)$, the misorientation angle $\theta_{i,j}$ is calculated according to:

$$ \theta_{i,j} = 2 \arccos \left( \min \left( |\mathbf{q}_{i,j} \cdot \mathbf{q}_{i,j}^*|, 1.0 \right) \right) \cdot \frac{180}{\pi} $$

where $\mathbf{q}_{i,j}$ and $\mathbf{q}_{i,j}^*$ represent the unit quaternions at position $(i, j)$ in the original and reconstructed maps, respectively. The resulting misorientation map $\Theta \in \mathbb{R}^{H \times W \times 1}$ provides a quantitative measure of the local reconstruction accuracy across the microstructural features.

Fig. 6 presents the misorientation map for the synthetic data, displaying the misorientation in degrees between the original and reconstructed quaternion maps.

## 4.2. Reconstruction performance on real-world EBSD data

To evaluate the Synthetic to Real domain transfer, the encoding-decoding process was applied to quaternion-based orientation maps derived from real EBSD scans of the Alloy 718, as illustrated in Fig. 7. Fig. 7 presents a comparative visualization of the original EBSD scan (Fig. 7(a)), the masked input provided to the model (Fig. 7(b)), and the reconstructed output (Fig. 7(c)). The inverse pole figure (IPF-Z) coloring scheme was employed for visualization of crystallographic orientations.

To further validate the reconstruction fidelity beyond visual comparison, comparative texture analyses were performed between the original and reconstructed orientation maps. Fig. 8 presents a comprehensive textural comparison through three complementary representations: the original microstructure (Fig. 8(a), 8(d)), the reconstructed masked regions only (Fig. 8(b), 8(e)), and a composite representation (Fig. 8(c), 8(f)) combining original orientations from unmasked regions with reconstructed orientations from masked regions. The pole figure analysis demonstrates substantial preservation of crystallographic texture, with primary texture components exhibiting minimal deviation in intensity and spatial distribution. Similarly, the inverse pole figure comparison reveals consistent orientation density distributions, confirming that the reconstructed data maintains the fundamental crystallographic character of the original microstructure. Importantly, the composite representation illustrates how reconstructed orientations integrate seamlessly with preserved orientations, maintaining texture continuity across the entire microstructure.

The textural analyses offer corroborating visual evidence for the fidelity of the ViT-MAE reconstruction. Nevertheless, the pole figures of the masked inputs would exhibit a comparable texture. Consequently, this method serves only as a visual assessment and does not elucidate potentially learned textual features.

The observed grain boundary-specific reconstruction challenges can be attributed to two primary factors: (1) the inherent difficulty in modeling sharp discontinuities in orientation space, and (2) the relative underrepresentation of boundary configurations in the synthetic training dataset compared to grain interior orientations. Nevertheless, the overall low misorientation across the majority of the microstructure demonstrates the effectiveness of the ViT MAE architecture for crystallographic orientation reconstruction from partially masked EBSD data.

## 4.3. Twin detection

The encoder part of the ViT-MAE model, originally pre-trained on the reconstruction task using a synthetic dataset, was used to extract latent representations from multiple datasets. Initially, 1024-dimensional latent vectors were extracted from a holdout synthetic dataset (Synthetic 1) containing no twin boundaries to establish the in-distribution
<figure><img src="image_6.png" /><figcaption>Fig. 4. Reconstruction results for synthetic data: (a) Original synthetic EBSD scan in IPF-Z coloring, (b) masked input, and (c) reconstructed EBSD scan.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 5. Local reconstruction analysis: (a) High-magnification view of original synthetic orientation map, (b) corresponding reconstructed orientation map, and (c) orientation profiles across the transect line indicated in (a) and (b). The transect line crosses a single grain boundary to examine reconstruction characteristics at crystallographic discontinuity.</figcaption></figure>

(ID) baseline. A dimensionality reduction procedure was implemented by fitting a Principal Component Analysis (PCA) model to these latent projections, with 1024 principal components retained to preserve the complete variance structure. The PCA transformation matrix was subsequently applied to latent projections from five additional datasets: (1) Synthetic 2—a second holdout synthetic dataset without twins, (2) Real EBSD dataset 1 with twin lamellae present, (3) Real EBSD dataset 1 with twins algorithmically removed, (4) Real EBSD dataset 2 with twin lamellae, and (5) Real EBSD dataset 2 with twins removed. From the Synthetic 1 latent projections, the mean vector $\mu$ and covariance matrix $\Sigma$ were calculated, which served as reference parameters for computing the Mahalanobis distance distributions across all six datasets. The Mahalanobis distance, defined as $d(x) = \sqrt{(x - \mu)^T \Sigma^{-1} (x - \mu)}$, provides a multivariate generalization of statistical distance that accounts for correlations between features. The resulting distance distributions, illustrated in Fig. 9, reveal distinct separation patterns that align with the progressive deviation from the baseline distribution. Notably, a rightward shift is observed between the synthetic datasets and the real-world datasets without twins, indicating the model's sensitivity to the synthetic-to-real domain gap. A further significant rightward shift is

observed between the twin-free real datasets and their counterparts containing twin lamellae, demonstrating the model's capacity to identify twin boundaries as Out-of-Domain microstructural features despite never explicitly being trained on this classification task.

The effectiveness of the downstream task was evaluated through comprehensive statistical analysis using the Receiver Operating Characteristic-Area Under Curve (ROC-AUC) methodology Fig. 10. The ROC-AUC metric quantifies the model's discriminative capability by plotting the true positive rate against the false positive rate across various threshold settings, thereby providing a threshold-independent assessment of classification performance. Values approaching 1.0 indicate near-perfect discrimination between in-distribution (ID) and out-of-distribution (OOD) samples.

The anomaly detection performance was further evaluated through a comparative analysis of two distinct methodologies, as illustrated in Fig. 10. The Mahalanobis distance-based approach achieved superior performance, with an AUC of 0.93, compared to the reconstruction error method, which yielded an AUC of 0.82. This metric effectively captures Out-Of-Distribution samples by detecting deviations in quater-nion orientation patterns that correspond to twin boundaries in the mi-
<figure><img src="image_8.png" /><figcaption>Fig. 6. Misorientation analysis of synthetic data: Misorientation map (in degrees) between original and reconstructed quaternion maps.</figcaption></figure>

crostructure. In contrast, the reconstruction error-based method, which relies on the autoencoder's ability to faithfully reproduce in-distribution crystallographic orientations, yielded a lower AUC of 0.82. The reconstruction error approach quantifies the mean misorientation between the original and reconstructed EBSD maps, where higher misorientation values indicate potential crystallographic anomalies. The superior performance of the Mahalanobis distance approach suggests that the latent space representation more effectively encapsulates the complex high-dimensional crystallographic relationships present in EBSD orientation maps compared to direct pixel-wise reconstruction comparison. However, the model was sensitive only to the geometric characteristics of twin lamellae rather than explicit Σ3 misorientation boundaries.

To provide an intuitive understanding of the downstream task in latent space, standard dimensionality reduction techniques such as t-SNE and UMAP were initially employed. However, these methods, which tend to preserve global data structure dominated by high-variance components, failed to produce a clear separation between the in-distribution and out-of-distribution datasets.

This observation aligns with recent findings in anomaly detection, which suggest that the suitability of an eigencomponent for class discrimination does not correlate with its variance. The work by Gula et al. [43], for example, demonstrates that a targeted, greedy selection of eigencomponents significantly outperforms methods based on variance. Their research indicates that many high-variance components may be redundant or even contain spurious features that are detrimental to the detection task.

Motivated by this, an alternative visualization was created by projecting the latent vectors onto a 2D subspace optimized for separation. Instead of selecting the principal axes with the largest eigenvalues, a greedy search algorithm was implemented to identify the two principal components that maximally discriminate the in-distribution from the out-of-distribution samples. The resulting projection is displayed in Fig. 11. The visualization helps to understand the efficacy of the Mahalanobis distance metric, showing the in-distribution samples clustering tightly within the 2σ and 3σ boundaries, while the out-of-distribution samples are clearly segregated. This demonstrates the existence of a highly discriminative low-dimensional subspace, reinforcing the idea that targeted feature selection is crucial for effective anomaly detection.

This anomaly detection pipeline demonstrates the robustness of the latent space for downstream tasks, even though twin detection itself can be efficiently achieved without machine learning. This validation confirms that the ViT-MAE model's learned representations are meaningful

and could be leveraged for transfer learning for application to other defect types and multimodal data integration.

### 4.4. Latent projection methods

While the 'Flattening + PCA' method was ultimately selected for the results presented in this paper, it was chosen following a comparative study of various techniques for projecting the encoder's final hidden state into a global feature vector. This projection is a critical step that transforms the encoder's output tensor, which has a shape of $[N_{vis} \times D]$, into a single vector, $z_{global} \in \mathbb{R}^D$, suitable for the Mahalanobis distance calculation. Here, $N_{vis}$ is the number of visible (unmasked) patches and $D$ is the embedding dimension. To determine the optimal approach, we systematically evaluated several aggregation strategies, quantifying the performance of each using the ROC AUC score on the twin detection task.

The investigated methods fall into three main categories: pooling-based, attention-based, and a flattening approach with dimensionality reduction.

* **Pooling Strategies:** These methods apply a permutation-invariant function across the sequence of token embeddings. We evaluated element-wise mean, max, min, and median pooling over the $N_{vis}$ token embeddings. For an encoder output $Z \in \mathbb{R}^{N_{vis} \times D}$, mean pooling is defined as:

$$z_{\text{global}} = \frac{1}{N_{\text{vis}}} \sum_{i=1}^{N_{\text{vis}}} Z_i$$

We also explored variants where ReLU or Softmax activations were applied to the token embeddings prior to averaging, but these did not yield improved performance.

* **Attention-based Pooling:** These strategies compute a weighted average of the token embeddings, with weights derived from attention mechanisms. The hypothesis was that focusing on, or away from, the most salient features could improve OOD detection.

- **Self-Attention Pooling:** Attention weights, $w_i$, are generated from the token embeddings themselves by applying a softmax function to the mean of each token's feature vector. The aggregated vector is then computed as:

$$w_i = \mathrm{softmax}(\mathrm{mean}(Z_i)) , \quad z_{\text{global}} = \sum_{i=1}^{N_{\text{vis}}} w_i Z_i$$

- **Inverse & Complement Attention-Weighted Aggregation:** These methods leverage the attention maps from the final layer of the transformer. The core idea is that for an anomaly detection task, the most informative signals may reside in patches the model paid less attention to during pre-training. We derive normalized per-token attention scores, $\alpha_{norm,i}$, from the final layer's attention matrix. We then compute aggregation weights, $\beta_i$, that are either inversely proportional to these scores ($\beta_i = 1/\alpha_{norm,i}$) or based on their complement ($\beta_i = 1 - \alpha_{norm,i}$). The final representation is a weighted sum using these new weights.

* **Flattening with Dimensionality Reduction:** This approach, which yielded the best performance, avoids any immediate summarization. The entire last hidden state $Z$ is first flattened into a high-dimensional vector $z_{flat} \in \mathbb{R}^{N_{vis} \times D}$. Subsequently, Principal Component Analysis (PCA) is applied to project this vector into a more compact and discriminative latent space. The final representation is obtained by projecting $z_{flat}$ onto the first $k$ principal components. For our experiments, we set $k = 1024$.

$$z_{\text{flat}} = \text{Flatten}(Z) , \quad z_{\text{global}} = \text{PCA}_{k=1024}(z_{\text{flat}})$$

The comparative results of these methods are summarized in Table 2. The Flattening + PCA method significantly outperformed all other
<figure><img src="image_9.png" /><figcaption>Fig. 7. (a) Original, (b) model input and (c) reconstructed tiles forming the overall validation dataset containing observational data.</figcaption></figure>

<table><caption>**Table 2**

Comparison of ROC AUC scores for different latent projection methods on the twin detection task.</caption><thead><tr><th>Aggregation Method</th><th>ROC AUC Score (%)</th></tr></thead><tbody><tr><td>Flattening + PCA (Retained Method)</td><td>93.0</td></tr><tr><td>Max Pooling</td><td>88.0</td></tr><tr><td>Min Pooling</td><td>88.0</td></tr><tr><td>Median Pooling</td><td>88.0</td></tr><tr><td>Mean Pooling</td><td>84.0</td></tr><tr><td>Attention-based Methods</td><td>83.0 – 84.0</td></tr></tbody></table>

strategies. We attribute its success to its ability to preserve the complete feature set from all visible patches, which is in contrast to pooling techniques that aggressively summarize and lose spatial information. The subsequent PCA step effectively distills this high-dimensional data into a latent space optimized to capture the most significant modes of variation, proving most effective for differentiating between in-distribution and out-of-distribution samples.

### 4.5. Loss choice

The present model was pre-trained on minimizing the pixel-wise misorientation between the reconstructed and original images. While original implementations of the ViT-MAE rely on the Mean Squared Error (MSE), this latter is not necessarily intuitively the most ideal loss for reducing misorientation between two quaternions as it is not capturing the specifics of the quaternions manifold (such as $q = -q$ in terms of 3D rotations). To evaluate that, a Gaussian noise of random intensity ([0, 100%]) was generated and applied to an EBSD scan batch of size (10, 4, 128, 128). The three losses MSE, misorientation and misorientation with FCC symmetries were computed between the batch and its noisy version 1000 times for random noise intensities to obtain the Fig. 12.

Analysis demonstrated that Mean Squared Error (MSE) constitutes an inherently unsuitable optimization metric when applied to quaternion representations of crystallographic orientations. This limitation stems from the fundamental mathematical properties of quaternions as a representation of the rotation group SO(3) on a non-Euclidean manifold. The quaternion space exhibits a spherical topology where the distance
<figure><img src="image_10.png" /><figcaption>Fig. 10. Receiver Operating Characteristic (ROC) curves comparing the anomaly detection performance of two methodologies: the Mahalanobis distance approach, which evaluates statistical deviations in latent space representations, and the reconstruction error approach, which quantifies mean misorientation between original and reconstructed EBSD orientation maps. The curves demonstrate the relative efficacy of each method in distinguishing twin boundaries from twin-free microstructures.</figcaption></figure>

ratio was adjusted from 0% (no masking) to 95% (heavy occlusion) during inference, with the latent projection being recomputed at each iteration.

The results, summarized in Fig. 13, reveal that the choice of latent projection method significantly influences performance, with the **flattening + PCA** approach consistently outperforming all other tested methods. For this superior method, a distinct and non-monotonic relationship between the masking ratio and discriminative capability is observed. Starting from a ROC-AUC of 0.9019 with no masking, the performance progressively improves, reaching an optimal value of 0.9729 at a masking ratio of 35%. In contrast, the other projection methods (Mean Pooling, Max Pooling, and Inverse Attention Weighted Average) exhibited a similar trend of performance plateauing before decreasing, but their peak ROC-AUC scores remained lower than those achieved by the PCA-based method across all masking ratios. This result can be explained by the interaction between inference-time masking and the PCA projection method. A low masking ratio appears to create a more distributed variance profile in the final hidden state, causing the PCA step to potentially discard important, lower-variance features needed for twin detection. The performance peak at 35% masking suggests an optimal trade-off, where the model is compelled to encode the most critical discriminative information into the high-variance principal components that are preserved after projection.

Beyond this 35% peak, increasing the masking ratio further results in a steady decline in performance. As more significant portions of the input data are obscured, the loss of critical crystallographic information begins to outweigh the regularizing benefits of masking, impairing the model's ability to extract the features necessary for accurate classification.

## 4.7. Twins features

An increase in the Mahalanobis distance relative to the prior distribution was observed for the out-of-distribution (OOD) samples containing twins. To determine whether this increase was influenced by the specific geometry of thin twin domains (morphology), the Σ3 orientation relationship between grains, or a combination of these factors, a synthetic dataset with three variations was generated:

* **No thin twin domains:** A dataset where grains had no thin twin domains. This dataset aims to document the influence of the grain morphology on the robustness of the EBSD reconstruction.

* **Random thin twin domains:** A dataset where random thin twin domains were introduced within the grains, with each thin twin domain assigned a random orientation, i.e., without the 60° misorientation around the ⟨011⟩ crystallographic direction and the common {111} shared plane. This dataset aims to document the influence of introducing thin and sharp grains (specific morphological character) on the robustness of the EBSD reconstruction.

* **Σ3 relation:** A dataset where the thin twin domain orientations adhered to the Σ3 relation relative to the parent grain's orientation, closely mimicking real twins. This dataset aims to evaluate the influence of the both crystallographic and morphological considerations on the robustness of the EBSD reconstruction.

As shown in Fig. 14, the analysis revealed an increase in the Mahalanobis distance distributions for the two datasets containing thin twin domains, regardless of whether they had random orientations or adhered to the Σ3 relationship. This suggests that the OOD latent projections of the model are primarily linked to the morphological features of the thin twin domains, rather than the Σ3 orientation relationship.

## 5. Discussion

The present study introduces a Quaternion-Based Vision Transformer Masked Autoencoder (ViT-MAE) pre-trained on large-scale synthetic Electron Backscatter Diffraction (EBSD) data. This approach is justified by the need to capture both local and global contextual information within polycrystalline microstructures, a capability often limited in traditional convolutional methods that primarily focus on local receptive fields. The major results demonstrate the model effectiveness in reconstructing masked EBSD orientation maps with high fidelity, preserving key textural information even when transferring from synthetic training data to real-world scans. Furthermore, the utility of the learned latent space was validated through a downstream task, successfully detecting crystallographic twins as out-of-distribution features using Mahalanobis distance analysis. This discussion will elaborate on two primary aspects: the contributions and limitations of the proposed ViT-MAE approach in relation to the existing literature, and the potential benefits of leveraging the generated latent space for multimodal material characterizations. The adoption of a Vision Transformer architecture addresses a key limitation of CNNs in microstructural analysis. While CNNs excel at identifying local patterns like grain boundaries or specific phases, their inherent locality, stemming from convolutional filters, restricts their ability to model long-range spatial dependencies critical for understanding global phenomena such as texture gradients or the influence of processing history on microstructure [21,22]. The self-attention mechanism within the ViT allows the model to weigh the importance of different regions across the entire EBSD map, enabling a more holistic representation that captures these global characteristics. The pre-training on a large synthetic dataset using a masked autoencoding strategy aligns with trends in self-supervised learning [23,24], mitigating the common challenge of limited annotated data in materials science. Indeed, the random selection of masked patches at each training epoch generates a large volume of training data, although this process does not simulate actual experimental sparsity. This pre-training step facilitates subsequent fine-tuning of the model to specific experimental conditions using a substantially smaller number of training examples. The use of a quaternion representation and a physics-informed rotational distance loss [33,38] further refines the model by respecting the underlying crystallographic manifold, proving more suitable than standard MSE loss for orientation data, as demonstrated in the loss comparison analysis. However, the model exhibits limitations. Reconstruction accuracy, while generally high within grain interiors, degrades near grain boundaries, likely due to the sharp orientation discontinuities and potential underrepresentation of diverse
<figure><img src="image_11.png" /><figcaption>Fig. 11. Projection of latent vectors onto the two most discriminative principal axes. The axes were identified via a greedy search to maximize the separation between In-Distribution (blue) and Out-of-Distribution (red) data. The ellipses represent the 1σ, 2σ, and 3σ Mahalanobis distance boundaries derived from the in-distribution training data.</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>Fig. 12. Relation between the Misorientation losses (with and without symmetries) and the Mean Squared Error on random orientation pairs.</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>Fig. 13. The effect of inference-time masking on twin detection performance. The ROC-AUC score is plotted as a function of the masking ratio for various latent projection strategies, with the optimal performance observed at a ratio of 35% for PCA projection.</figcaption></figure>

boundary types in the training data. The observed dimensional collapse in the latent space [41,42], necessitating PCA for the Mahalanobis distance calculation, suggests potential redundancy in the learned features or limitations in the MAE pre-training objective for this specific data type. Additionally, the OOD detection of twins appeared more sensitive to the morphological features of thin lamellae than the specific Σ3 crystallographic relationship, indicating that the model primarily learned geometric anomaly detection in this context. The exclusion of explicit crystal symmetries during pre-training, while computationally pragmatic, means the current latent space is symmetry-agnostic, although symmetries could be incorporated during fine-tuning for specific applications. The training dataset also excluded many features that would be extremely valuable for better generalization, such as more diverse EBSD scan resolutions or larger images. A significant potential application of this work lies in multimodal materials characterization [13,6,11]. EBSD provides fundamental crystallographic information, but understanding material behavior often requires integrating data from various sources, such as mechanical testing (micromechanics), chemical analysis (corrosion mapping, EDS), or other imaging modalities (e.g., SEM, TEM). Establishing a robust, compressed latent representation of the EBSD microstructure, as achieved here, provides a foundational layer for data fusion. This latent space can serve as a common framework where information from different modalities, potentially acquired at different resolutions or with different noise characteristics, can be aligned and correlated [5]. For instance, correlating the EBSD latent features with local mechanical properties derived from nanoindentation maps or with corrosion susceptibility patterns could reveal complex structure-property relationships that are difficult to discern from individual data sources. The comparison of latent spaces derived from EBSD and other techniques could identify synergistic or conflicting information, leading to a more comprehensive understanding of material heterogeneity and performance. The success of the OOD detection task, even if primarily morphology-driven in the twin example, demonstrates the latent space's sensitivity to microstructural variations, suggesting its potential utility for broader anomaly detection (e.g., identifying unexpected phases, defect clusters, or processing inconsistencies) and as input for predictive models linking microstructure to functional properties. The public release of the model and dataset aims to facilitate such future multimodal investigations.

## 6. Conclusions

This research successfully developed and validated a Quaternion-Based Vision Transformer Masked Autoencoder (ViT-MAE) for analyzing EBSD orientation maps of polycrystalline materials. Pre-trained in a self-supervised manner on a large-scale synthetic dataset (3.2 million images), the model demonstrated effective reconstruction of masked orien-
<figure><img src="image_14.png" /><figcaption>Fig. 14. Mahalanobis distances distributions are similar for the two twinned datasets (random orientations and Σ3 relations).</figcaption></figure>

tation data, preserving key microstructural features and crystallographic texture, even when applied to real-world EBSD scans not seen during training. A physics-informed quaternion misorientation loss function was employed, proving more suitable for orientation data than standard metrics like MSE. The utility of the learned latent space was highlighted via a downstream OOD detection task, where Mahalanobis distance effectively distinguished microstructures containing twins (treated as anomalies) from twin-free samples, outperforming reconstruction error methods.

## CRediT authorship contribution statement

**Pierre Belamri:** Writing – original draft, Visualization, Software, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. **Henry Proudhon:** Writing – review & editing, Validation, Supervision, Project administration, Methodology, Investigation, Funding acquisition, Conceptualization. **Damien Texier:** Writing – review & editing, Validation, Supervision, Project administration, Methodology, Investigation, Funding acquisition, Conceptualization. **David Ryckelynck:** Writing – review & editing, Validation, Supervision, Project administration, Methodology, Investigation, Funding acquisition, Conceptualization.

## Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Damien TEXIER reports financial support was provided by European Research Council. Henry Proudhon reports financial support was provided by Fondation Mines Paris by Safran. Pierre Belamri reports equipment, drugs, or supplies was provided by GENCI at IDRIS. If there are other authors, they declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

## Acknowledgements

This work was supported by the European Research Council [project HT-S₄DefOx - Grant number 948007]. The authors acknowledge the funding of the BIGMECA chair of Fondation Mines Paris by Safran. This project was provided with computing AI and storage resources by GENCI at IDRIS thanks to the grant 20XX-AD011015549 on the supercomputer Jean Zay's V100/A100 partition.

## Data availability

Data will be made available on request.

The model definition code and trained weights are publicly available to facilitate reproducibility and further research. These resources provide a foundation for leveraging our Vision-Transformer Masked Autoencoder in tasks beyond those explored in this study.

Beyond the model itself, the newly created synthetic EBSD dataset represents a key contribution of this work. Prior to this, no large-scale open dataset existed that combines quaternion-based orientation maps and physically informed symmetry constraints. By releasing both the full dataset generation code and the resulting data, this work aims to foster broader adoption of AI-driven methods in materials science and encourage future research on large-scale orientation-based tasks.

## References

[1] D. Wolf, T. Payer, C.S. Lisson, C.G. Lisson, M. Beer, M. Götz, T. Ropinski, Self-supervised pre-training with contrastive and masked autoencoder methods for dealing with small datasets in deep learning for medical imaging, Sci. Rep. 13 (2023) 20260, https://doi.org/10.1038/s41598-023-46433-0.

[2] F. Humphreys, Review grain and subgrain characterisation by electron backscatter diffraction, J. Mater. Sci. 36 (2001) 3833-3854.

[3] A. Schwartz, M. Kumar, B. Adams, D. Field, Electron Backscatter Diffraction in Materials Science, Springer, New York, NY, 2009.

[4] O. Alvarez Tunon, Y. Brodskiy, E. Kayacan, Loss it right: Euclidean and Riemannian metrics in learning-based visual odometry, in: ISR Europe 2023, VDE, in: ISR Europe 2023, 56th International Symposium on Robotics, ISR 2023, Conference Date, Verlag GmbH, 2023, Conference date: 27-09-2023.

[5] M. Calvat, C. Bean, D. Anjaria, H. Park, H. Wang, K. Vecchio, J.C. Stinville, Learning metal microstructural heterogeneity through spatial mapping of diffraction latent space features, https://arxiv.org/abs/2501.18064, arXiv:2501.18064, 2025.

[6] M. Charpagne, J. Stinville, A. Polonsky, M. Echlin, T. Pollock, A multi-modal data merging framework for correlative investigation of strain localization in three dimensions, JOM 73 (2021) 3263-3271, https://doi.org/10.1007/s11837-021-04894-6.

[7] G. Wille, C. Lerouge, U. Schmidt, A multimodal microcharacterisation of trace-element zonation and crystallographic orientation in natural cassiterite by combining cathodoluminescence, ebsd, epma and contribution of confocal Raman-in-sem imaging, J. Microsc. 270 (2018) 309-317, https://doi.org/10.1111/jmi.12684.

[8] J. Aspinall, D.E. Armstrong, M. Pasta, Ebsd-coupled indentation: nanoscale mechanics of lithium metal, Mater. Today Energy 30 (2022) 101183, https://doi.org/10.1016/j.mtener.2022.101183.

[9] C.M. Magazzeni, H.M. Gardner, I. Howe, P. Gopon, J.C. Waite, D. Rugg, D.E. Armstrong, A.J. Wilkinson, Nanoindentation in multi-modal map combinations: a correlative approach to local mechanical property assessment, J. Mater. Res. 36 (2021) 2235-2250, https://doi.org/10.1557/s43578-020-00035-y.

[10] E. Rossi, J.M. Wheeler, M. Sebastiani, High-speed nanoindentation mapping: a review of recent advances and applications, Curr. Opin. Solid State Mater. Sci. 27 (2023) 101107, https://doi.org/10.1016/j.cossms.2023.101107.
[11] D. Texier, T. Richeton, H. Proudhon, A. Dziri, Q. Sirvin, M. Legros, Increase in elastic and hardness anisotropy of titanium with oxygen uptake due to high temperature oxidation: a multimodal framework using high speed nanoindentation mapping, Mater. Charact. 216 (2024) 114244, https://doi.org/10.1016/j.matchar.2024.11424.

[12] Y. Idrissi, T. Richeton, D. Texier, S. Berbenni, J.-S. Lecomte, Robust determination of cubic elastic constants via nanoindentation and Bayesian inference, Acta Mater. (2024) 120406, https://doi.org/10.1016/j.actamat.2024.120406.

[13] D. Lahat, T. Adali, C. Jutten, Multimodal data fusion: an overview of methods, challenges, and prospects, Proc. IEEE 103 (2015) 1449–1477, https://doi.org/10.1109/JPROC.2015.2460697, https://apisemanticscholar.org/CorpusID:26071662.

[14] Y. Zhang, A. Elbrond, F. Lin, A method to correct coordinate distortion in ebsd maps, Mater. Charact. 96 (2014) 158–165, https://doi.org/10.1016/j.matchar.2014.08.003.

[15] M.-A. Charpagne, F. Strub, T.M. Pollock, Accurate reconstruction of ebsd datasets by a multimodal data approach using an evolutionary algorithm, Mater. Charact. 150 (2019) 184–198, https://doi.org/10.1016/j.matchar.2019.01.033, https://www.sciencedirect.com/science/article/pii/S1044580318329073.

[16] V.S. Tong, T. Ben Britton, Trueebsd: correcting spatial distortions in electron backscatter diffraction maps, Ultramicroscopy 221 (2021) 113130, https://doi.org/10.1016/j.ultramic.2020.113130.

[17] S.I. Wright, M.M. Nowell, S.P. Lindeman, P.P. Camus, M. De Graef, M.A. Jackson, Introduction and comparison of new ebsd post-processing methodologies, Ultramicroscopy 159 (2015) 81–94, https://doi.org/10.1016/j.ultramic.2015.08.001.

[18] R. Hielscher, C.B. Silbermann, E. Schmidl, J. Ihlemann, Denoising of crystal orientation maps, J. Appl. Crystallogr. 52 (2019) 984–996, https://doi.org/10.1107/S1600576719009075.

[19] F. Ram, S. Wright, S. Singh, M. De Graef, Error analysis of the crystal orientations obtained by the dictionary approach to ebsd indexing, Ultramicroscopy 181 (2017) 17–26, https://doi.org/10.1016/j.ultramic.2017.04.016.

[20] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhang, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al., An image is worth 16x16 words: transformers for image recognition at scale, arXiv preprint arXiv:2010.11929, 2020, https://doi.org/10.48550/arXiv.2010.11929.

[21] D. Jeulin, Morphological Models of Random Structures, Interdisciplinary Applied Mathematics, Springer International Publishing, 2021, https://books.google.fr/books?id=jzx5jEAAAABQAJ.

[22] A.E. Robertson, C. Kelly, M. Buzzy, S.R. Kalindiki, Local–global decompositions for conditional microstructure generation, Acta Mater. 253 (2023) 118966, https://www.sciencedirect.com/science/article/pii/S1359645423002975, https://doi.org/10.1016/j.actamat.2023.118966.

[23] K. He, X. Chen, S. Xie, Y. Li, P. Dollár, R.B. Girshick, Masked autoencoders are scalable vision learners, CoRR, arXiv:2111.06377, 2021, https://doi.org/10.48550/arXiv.2111.06377, arXiv:2111.06377.

[24] R. Bommasanis, D.A. Hudson, E. Adeli, R.B. Altman, S. Arora, S. von Arx, M.S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill, E. Brynjolfsson, S. Buch, D. Card, R. Castellon, N.S. Chatterji, A.S. Chen, K. Creel, J.Q. Davis, D. Demszky, C. Donahue, M. Doumbouya, E. Durmus, S. Ermon, J. Etchemendy, K. Ethayarajh, L. Fei-Fei, C. Finn, T. Gale, L.E. Gillespie, K. Goel, N.D. Goodman, S. Grossman, N. Guha, T. Hashimoto, P. Henderson, J. Hewitt, D.E. Ho, J. Hong, K. Hsu, J. Huang, T. Icai, S. Jain, D. Jurafsky, P. Kalluri, S. Karamcheti, G. Keeling, F. Khani, O. Khattab, P.W. Koh, M.S. Krass, R. Krishna, R. Kuditipudi, et al., On the opportunities and risks of foundation models, CoRR, arXiv:2108.07258, 2021, https://doi.org/10.48550/arXiv.2108.07258.

[25] OpenAI, J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F.L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, R. Avila, I. Babuschkin, S. Balaji, V. Balcom, P. Baltescu, H. Bao, R. Avellan, R. Belgum, I. Bello, J. Berdine, G. Bernadett-Shapiro, C. Berner, L. Bogdonoff, O. Boiko, M. Boyd, A.-L. Brakman, G. Brockman, T. Brooks, M. Brundage, K. Button, T. Cai, R. Campbell, A. Cann, C. Carey, C. Carlson, R. Carmichael, B. Chan, C. Chang, F. Chantzis, D. Chen, S. Chen, R. Chen, J. Chen, M. Chen, B. Chess, C. Cho, C. Chu, H.W. Chung, D. Cummings, J. Currier, Y. Dai, C. Decareaux, T. Degry, N. Deutsch, T. Deville, A. Dhar, D. Dohan, S. Dowling, S. Dunning, A. Ecoffet, A. Eleti, T. Eloundou, D. Farhi, L. Fedus, N. Felix, S.P. Fishman, J. Forte, I. Fulford, L. Gao, E. Georges, C. Gibson, V. Goel, T. Gogineni, G. Goh, R. Gontijo-Lopes, J. Gordon, M. Grafstein, S. Gray, R. Greene, J. Gross, S.S. Gu, Y. Guo, C. Hallacy, J. Han, J. Harris, Y. He, M. Heaton, J. Heidecke, C. Hesse, A. Hickey, W. Hickey, P. Hoeschele, B. Houghton, K. Hsu, S. Hu, X. Hu, J. Huizinga, S. Jain, S. Jain, J. Jang, A. Jiang, R. Jiang, H. Jin, D. Jin, S. Jomoto, B. Jonn, H. Jun, T. Taftan, Lukasz Kaiser, A. Kamali, I. Kanitscheider, N.S. Keskar, T. Khan, L. Kilpatrick, J.W. Kim, C. Kim, Y. Kim, J.H. Kirchner, J. Kiros, M. Knight, D. Kokotajlo, Łukasz Kondraciuk, A. Kondrich, A. Konstantinidis, K. Kosic, G. Krueger, V. Kuo, M. Lampe, I. Lan, T. Lee, J. Leike, J. Leung, D. Levy, C.M. Li, R. Lim, M. Lin, S. Lin, M. Litwin, T. Lopez, R. Lowe, P. Lue, A. Makanju, K. Malfacini, S. Manning, T. Markov, Y. Markovski, B. Martin, K. Mayer, A. Mayne, B. McGrew, S.M. McKinney, C. McLeavey, P. McMillan, J. McNeil, D. Medina, A. Mehta, J. Menick, L. Metz, A. Mishchenko, P. Mishkin, V. Monaco, E. Morikawa, D. Mossing, T. Mu, M. Murati, O. Murk, D. Mély, A. Nair, R. Nakano, R. Nayak, A. Neelakantan, R. Ngo, H. Noh, L. Ouyang, C. O’Keefe, J. Pachocki, A. Paino, J. Palermo, A. Pantuliano, G. Parascandolo, J. Parish, E. Parparita, A. Passos, M. Pavlov, A. Peng, A. Perelman, F. de Avila Belbutte Peres, M. Petrov, H.P. de Oliveira Pinto, Michael Pokorny, M. Pokrass, V.H. Pong, T. Powell, A. Power, B. Power, E. Proehl, R. Puri, A. Radford, J. Rae, A. Ramesh, C. Raymond, F. Real, K.

Rimbach, C. Ross, B. Rotsted, H. Roussez, N. Ryder, M. Saltarelli, T. Sanders, S. Santurkar, G. Sastry, H. Schmidt, D. Schnurr, J. Schulman, D. Selsam, K. Sheppard, T. Sherbakov, J. Shieh, S. Smoker, P. Shyam, S. Sidor, E. Sigler, M. Simens, J. Sitkin, K. Slama, I. Sohl, B. Sokolowsky, Y. Song, N. Staudacher, F.P. Such, N. Summers, I. Sutskever, J. Tang, N. Tezak, M.B. Thompson, P. Tillet, A. Tootoonchian, E. Tseng, P. Tuggle, N. Turkey, J. Tworek, J.F.C. Uribe, A. Vallone, A. Vijayvergiya, C. Voss, C. Wainwright, J.J. Wang, A. Wang, B. Wang, J. Ward, J. Wei, C. Weinmann, A. Welihinda, P. Welinder, J. Weng, L. Weng, M. Wiethoff, D. Willner, C. Winter, S. Wolrich, H. Wong, L. Workman, S. Wu, J. Wu, M. Wu, K. Xiao, T. Xu, S. Yoo, K. Yu, Q. Yuan, W. Zaremba, R. Zellers, C. Zhang, M. Zhang, S. Zhao, T. Zheng, J. Zhuang, W. Zhuk, C. Zoph, Gpt-4 technical report, 2024. URL: https://arxiv.org/abs/2303.08774, https://doi.org/10.48550/arXiv.2303.08774, arXiv:2303.08774.

[26] J. Bai, S. Bai, S. Yang, S. Wang, S. Tan, P. Wang, J. Lin, C. Zhou, J. Zhou, Qwen-vl: a versatile vision-language model for understanding, localization, text reading, and beyond, https://arxiv.org/abs/2308.12966, 2023, https://doi.org/10.48550/arXiv.2308.12966, arXiv:2308.12966.

[27] Q. Yang, R. Guo, An Unsupervised Method for Industrial Image Anomaly Detection with Vision Transformer-Based Autoencoder, Sensors, vol. 24, 2024, https://www.scopus.com/inward/record.uri?eid=2-s.2.0-85191406757&doi=10.3390/2f524082440&partnerID=40&md5=936708bf2e49ac11a428c35eab94764a, cited by: 4, All Open Access, Gold Open Access, Green Open Access.

[28] J. An, S. Cho, Variational autoencoder based anomaly detection using reconstruction probability, in: Special Lecture on IE, SNU Data Mining Center 2015, 2015, pp. 1–18.

[29] J. Zhang, S. Chu, Z. Lai, G. Peng, H. Lin, Z. Tan, P. Liu, Masked self-supervised pre-training for steel corrosion recognition via vision transformer under limited sample conditions, in: L. Yang (Ed.), Seventh International Conference on Advanced Electronic Materials, Computers, and Software Engineering (AEMCSE 2024), in: International Society for Optics and Photonics, vol. 13229, SPIE, 2024, p. 132290T.

[30] R. Walsh, I. Osman, O. Abdelaziz, M.S. Shehata, Fully self-supervised out-of-domain few-shot learning with masked autoencoders, J. Imag. 10 (2024), https://www.scopus.com/inward/record.uri?eid=2-s.2.0-85183078767&doi=10.3390/2fjimaging10010023&partnerID=40&md5=60beff7b602e3a903b6c4d014868ca98, cited by: 0, All Open Access, Gold Open Access, Green Open Access.

[31] O. Rippel, P. Mertens, D. Merhof, Modeling the distribution of normal data in pretrained deep features for anomaly detection, in: 2020 25th International Conference on Pattern Recognition (ICPR), IEEE Computer Society, Los Alamitos, CA, USA, 2021, pp. 6726–6733, https://doi.ieeecomputersociety.org/10.1109/ICPR48806.2021.9412109.

[32] Z. Jiang, J.P.C. Bertoldo, E. Decencière, Heuristic hyperparameter choice for image anomaly detection, in: 2023 Twelfth International Conference on Image Processing Theory, Tools and Applications (IPTA), 2023, pp. 1–5.

[33] D. Jangid, N. Brodnik, M. Goebel, A. Khan, S. Majeti, M. Echlin, S. Daly, T. Pollock, B. Manjunath, Adaptive physics-based super-resolution for electron backscatter diffraction maps, npj Comput. Mater. 8 (2022), https://doi.org/10.1038/s41524-022-00924-2.

[34] R. Quey, A. Villani, C. Maurice, Nearly uniform sampling of crystal orientations, J. Appl. Crystallogr. 51 (2018) 1162–1173, https://doi.org/10.1107/S1600576718009019.

[35] A. Marano, C. Ribart, H. Proudhon, Towards a data platform for multimodal 4d mechanics of material microstructures, Mater. Des. 246 (2024) 113306, https://doi.org/10.1016/j.matdes.2024.113306.

[36] T. Parcollet, Y. Zhang, M. Morchid, C. Trabelsi, G. Linarès, R. De Mori, Y. Bengio, Quaternion convolutional neural networks for end-to-end automatic speech recognition, arXiv preprint, arXiv:1806.07789, 2018, https://doi.org/10.21437/Interspeech.2018-1898.

[37] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, Adv. Neural Inf. Process. Syst. 30 (2017), https://doi.org/10.48550/arXiv:1706.03762.

[38] D.K. Jangid, N.R. Brodnik, M.P. Echlin, C. Gudavalli, C. Levenson, M.T. Pollock, S.H. Daly, B. Manjunath, Q-rbsa: high-resolution 3d ebsd map generation using an efficient quaternion transformer network, npj Comput. Mater. 10 (2024) 27, https://doi.org/10.1038/s41524-024-01209-6.

[39] I. Loshchilov, F. Hutter, Sgdr: stochastic gradient descent with warm restarts, https://arxiv.org/abs/1608.03983, https://doi.org/10.48550/arXiv.1608.03983, arXiv:1608.03983, 2017.

[40] K. Lee, K. Lee, H. Lee, J. Shin, A simple unified framework for detecting out-of-distribution samples and adversarial attacks, https://arxiv.org/abs/1807.03888, https://doi.org/10.48550/arXiv.1807.03888, arXiv:1807.03888, 2018.

[41] Q. Zhang, Y. Wang, Y. Wang, How mask matters: towards theoretical understandings of masked autoencoders, https://arxiv.org/abs/2210.08344, https://doi.org/10.48550/arXiv.2210.08344, arXiv:2210.08344, 2023.

[42] L. Jing, P. Vincent, Y. LeCun, Y. Tian, Understanding dimensional collapse in contrastive self-supervised learning, CoRR, arXiv:2110.09348, 2021, https://doi.org/10.48550/arXiv.2110.09348.

[43] T. Gula, J.a.P. Bertoldo, Gaussian image anomaly detection with greedy eigencomponent selection, in: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023, pp. 4110–4118.