# Automatic detection of grains in partially recrystallized microstructures using deep learning

Fengxiang Lin, Haixing Fang, Hong Liu, Yubin Zhang, Dorte Juul Jensen, Emil Hovad

► To cite this version:

Fengxiang Lin, Haixing Fang, Hong Liu, Yubin Zhang, Dorte Juul Jensen, et al.. Automatic detection of grains in partially recrystallized microstructures using deep learning. Materials Characterization, 2025, 219, pp.114576. 10.1016/j.matchar.2024.114576. hal-05099701

HAL Id: hal-05099701

https://hal.science/hal-05099701v1

Submitted on 10 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

# Automatic detection of grains in partially recrystallized microstructures using deep learning

Fengxiang Lin<sup>a,b,*</sup>, Haixing Fang<sup>c</sup>, Hong Liu<sup>d</sup>, Yubin Zhang<sup>a</sup>, Dorte Juul Jensen<sup>a,*</sup>, Emil Hovad<sup>e</sup>

## ARTICLE INFO

### Keywords:

EBSD

Instance segmentation

Convolutional neural network

Microstructure

Computer vision

Deep learning

Recrystallization

## ABSTRACT

Precise identification of recrystallizing grains in partially recrystallized microstructures is essential to obtain quantitative information regarding the recrystallization process. Automatic, robust, user-friendly, and unbiased identification methods that do not rely on hard-coded, preselected values would be highly advantageous. In this study, we test convolutional neural network instance segmentation models to achieve automatic segmentation of individual recrystallizing grains in partially recrystallized microstructures. Our training dataset includes micrographs obtained using electron backscattered diffraction from five alloys with different thermal-mechanical histories and more than 100,000 recrystallizing grains. We adapt and train two state of the art deep learning models, namely Mask R-CNN and PointRend. Both models provide instance segmentation results of good quality, enabling quantitative determination of the microstructural parameters. The PointRend model demonstrates better performance for grains with irregular shapes than Mask R-CNN. Compared to conventional methods, the trained deep learning approach is easier to use, more flexible, and applicable to a wide range of materials.

## 1. Introduction

Recrystallization takes place during annealing of deformed metals and alloys. It involves the formation of new, nearly defect-free grains (known as recrystallizing grains) that nucleate and grow within the previously deformed matrix. Recrystallization is widely used to tailor microstructures and textures to enhance a range of material properties. Deep insight into the kinetics and microstructural changes during recrystallization is of interest for both research purposes, such as developing more accurate recrystallization models [1–5], and industrial applications, such as controlling annealing temperature and duration for optimal microstructures [6].

Electron backscattered diffraction (EBSD) analysis has been extensively used to investigate recrystallization. With EBSD data, recrystallizing (Rex) grains may be distinguished from the deformed matrix in partially recrystallized microstructures to deduce statistical information, such as grain sizes, crystallographic orientations, and spatial

distributions. The approaches used to differentiate Rex grains from the deformed matrix can be categorized into two types. The first approach utilizes values of image quality (IQ) or band contrast (BC) of the EBSD Kikuchi pattern as a criterion [7]. This choice is based on the fact that nearly defect-free Rex grains have higher IQ or BC values than the deformed matrix. However, the IQ or BC values of Rex grains depend strongly on the type of material, the surface quality, and the EBSD acquisition settings (e.g., working distance, distance of the detector). The second approach starts with the reconstruction of grains from the EBSD data, using a specified threshold misorientation angle. Subsequently, it distinguishes Rex grains from the deformed matrix by evaluating grain properties, such as grain orientation spread (GOS), grain average misorientation (GAM), and grain size. This differentiation is based on the fact that each Rex grain has a nearly uniform orientation and a low number of internal dislocations [8]. The results are however affected by the chosen grain properties, the threshold misorientation angle employed for grain reconstruction, and by factors including the

<footnote><sup>a</sup> Department of Civil and Mechanical Engineering, Technical University of Denmark, 2800 Kgs. Lyngby, Denmark</footnote>

<footnote><sup>b</sup> Department of Engineering and Physics, Karlstad University, 65188 Karlstad, Sweden</footnote>

<footnote><sup>c</sup> European Synchrotron Radiation Facility, 71 Av. Des Martyrs, 38000 Grenoble, France</footnote>

<footnote><sup>d</sup> National Engineering Research Centre of Light Alloy Net Forming and State Key Laboratory of Metal Matrix Composite, Shanghai Jiao Tong University, Shanghai 200240, PR China</footnote>

<footnote><sup>e</sup> Department of Applied Mathematics and Computer Science, Technical University of Denmark, 2800 Kgs. Lyngby, Denmark</footnote>

<footnote>* Corresponding authors at: Department of Civil and Mechanical Engineering, Technical University of Denmark, 2800 Kgs. Lyngby, Denmark. E-mail addresses: linfx.cn@gmail.com, fengxiang.lin@kau.se (F. Lin), doje@dtu.dk (D. Juul Jensen).</footnote>
microstructure of the materials (e.g. whether the deformed matrix undergoes recovery), the step size of EBSD measurements, and the quality of sample surface preparation. Wu and Juul Jensen [9] employed a 1° threshold to reconstruct grains in a partially recrystallized high-purity aluminum sample. They identified Rex grains as those having a minimum grain size of 3 µm and being partially surrounded by high-angle boundaries. In contrast, Field et al. [10] utilized a 5° threshold for grain reconstruction and introduced a maximum grain orientation spread (GOS) value of 2°. It is worth noting that many GOS threshold values ranging from 1° to 3° have been reported for various materials [11,12]. Ayad et al. [13] incorporated both GOS and GOS normalized over grain size as parameters to distinguish Rex grains in interstitial-free steel samples. More recently, Zhu et al. [14] constructed a neural network to determine weight ratios of grain size, aspect ratio, and GOS for recognizing dynamically Rex grains in a hot deformed nickel-based superalloy. However, this model was trained only for this particular alloy. Although the method based on the properties of grains reconstructed from EBSD data has greatly advanced the automation of Rex grain detection, adjusting the parameters still largely depends on the expertise of the researchers: Researchers need to inspect EBSD maps, including orientation maps and BC or IQ maps, to evaluate the reasonableness of the set parameters and fine-tune them manually or with the assistance of machine learning methods on a case-by-case basis. In short, there is currently no straightforward method to directly differentiate Rex grains based on EBSD data, regardless of the material condition and the image quality. Without careful parameter selections, very misleading results can be obtained.

Recently, deep learning techniques have undergone significant advancements and found widespread application in computer vision domains, such as rail damage detection [15]. The success of deep learning methods in computer vision has attracted the interest of material scientists to explore the application of these techniques for the automatic identification of microstructural features [16,17] and diffraction spots [18,19]. For example, Azimi et al. [20] trained a Fully Convolutional Neutral Network (FCNN) for segmentation of martensite, tempered martensite, bainite, and pearlite phases within scanning electron microscopy (SEM) and optical microscopy (LOM) images of low carbon steel. Similarly, DeCost et al. [21] used FCNN to segment microstructural constituents within ultrahigh carbon steel. FCNN has also been used for segmentation of dislocation lines, precipitates, and voids [22]. All these works belong to the category of semantic segmentation, i.e. each pixel within an image is assigned a label denoting the corresponding phase or feature type it belongs to. But semantic segmentation models cannot distinguish two objects of the same types from each other, such as two Rex grains. They are just included in the category Rex, so one can for example not quantify the grain size or number of Rex grains. The task of detecting individual grains can however be approached through the method known as instance segmentation. In 2017, Facebook AI Research introduced Mask R-CNN [23], which remains one of the most popular network architectures for instance segmentation. Mask R-CNN extends the capabilities of Faster R-CNN [24], a network designed for object detection tasks, by incorporating supplementary convolution layers to predict segmentation masks for every Region of Interest (ROI). The efficiency of Mask R-CNN has been demonstrated in diverse applications of materials science, including identification of individual powder particles within SEM images of metal powders [25] and segmentation of diffraction spots for lab-based X-ray diffraction contrast tomography [26]. However, the segmentation masks predicted by Mask R-CNN may exhibit an over-smoothed appearance, because predictions are often made on an under-sampled image of low resolution. To overcome this limitation, an alternative network termed PointRend [27] was developed, and has been shown to exhibit better performance in capturing fine details at the boundaries of instances.

Inspired by the capability of deep learning in computer vision to mimic human visual observation, we in the present work aim to test the potential of existing relevant deep learning models to enable fast,

unbiased and automatic detection of individual Rex grains in partially recrystallized microstructures. Such a tool with broad adaptability will significantly facilitate recrystallization analysis. Considering that Rex grains might exhibit irregular grain shapes [28–30], both Mask R-CNN and PointRend are used, and their performance on detecting Rex grains is compared. A series of materials with diverse thermal mechanical processing histories are included as input for the training process. While light optical micrographs or scanning electron micrographs have often been used to quantify recrystallization, images generated from EBSD data are used here. This is because (i) EBSD maps contains richer information and (ii) it is easier to acquire a substantial amount of high-quality labeled training data from EBSD data, which is critical for the accuracy of the model [31,32]. The trained models are expected to function similarly to an experienced researcher in identifying Rex grains through micrograph inspection.

## 2. Experimental

EBSD data from previous studies of different materials, deformed to different strains, annealed to different recrystallization fractions, and characterized with different EBSD scanning step sizes are used. The materials include nickel (Ni), aluminum (Al), and tantalum (Ta) alloys. A summary is listed in Table 1.

All EBSD scans were conducted on longitudinal sample sections (containing the rolling and normal directions) using the HKL system. Noise reduction was accomplished using the HKL Channel 5 software, involving the removal of incorrectly indexed single pixels and the replacement of non-indexed pixels with the most common neighboring orientation, provided that the non-indexed pixel has at least 6 indexed neighboring pixels from the same grain.

## 3. Deep learning strategy

### 3.1. Data preprocessing

For both Mask R-CNN and PointRend, square-shaped images are preferred; otherwise, images will be resized to square-shaped within the model. To prevent distortion in grain shapes due to such resizing, the EBSD datasets were cropped to a square shape following a specific procedure: for EBSD datasets with the maximum length less than 800 pixels, a square subset was cropped, with the side length corresponding to the shorter length. In cases where the maximum length exceeded 800 pixels, multiple equal-sized square subsets (typically 2 or 4) were cropped, each having a maximum size of 800 × 800 pixels. Overall, 120 square-shaped EBSD subsets were utilized for this study. Among these, 5% (6 EBSD subsets) were reserved for validation, with two subsets randomly chosen from the Ni samples and one subset randomly selected from each of the other materials. The remaining 95% (114 EBSD subsets) were used for training.

To generate images from the EBSD data, we used the inverse pole figure (IPF) color map superimposed by the grayscale band contrast (BC) map in a semi-transparent manner (termed IPF + BC maps). This combination contains both orientation and BC information. Our preliminary results indicate that the models trained on IPF + BC maps performs comparably to the one trained on IPF maps, and surpasses the performance of the model trained on BC maps (refer to Figs. S2 and S3 in the supplementary materials). Therefore, in the following, we will focus on models trained on IPF + BC maps. As a form of data augmentation, three maps were generated for each EBSD subset, according to IPF color coding of the X, Y, and Z directions, noted as IPF-X, IPF-Y and IPF-Z maps in the following. Consequently, a total of 342 images (114 EBSD subsets × 3 maps each) were used as the training input.

### 3.2. Data labeling

To train the model, it is required to label the Rex grains in the EBSD
<table><caption>**Table 1**
Summary of materials and conditions used in this work. (ARB: accumulative roll bonding, CR: cold rolling).</caption><thead><tr><th>Material</th><th>Deformation</th><th>Strain</th><th>Annealing condition</th><th>Recrystallized fraction</th><th>Number of samples</th><th>Step size (μm)</th><th>Mapped area in total (μm²)</th></tr></thead><tbody><tr><td>Ta-10%W [33]</td><td>CR</td><td>1.4</td><td>1200 °C 10 min to 16 h</td><td>1%-75%</td><td>8</td><td>0.5-3</td><td>5,500,000</td></tr><tr><td>AA5657 [34]</td><td>CR</td><td>1.6</td><td>300 °C 50 min to 6 h</td><td>8%-90%</td><td>6</td><td>1</td><td>4,700,000</td></tr><tr><td>AA5182 [35]</td><td>CR</td><td>1.6</td><td>245 °C 100 min to 24 h</td><td>3%-88%</td><td>6</td><td>0.3-0.7</td><td>4,000,000</td></tr><tr><td>AA1200 [35]</td><td>CR</td><td>2.0</td><td>270 °C 10 min to 6 h</td><td>2%-83%</td><td>4</td><td>0.1-1</td><td>900,000</td></tr><tr><td>AA1200 [36]</td><td>CR</td><td>4.0</td><td>250 °C 40 min to 20 h</td><td>1%-70%</td><td>9</td><td>0.1-1</td><td>900,000</td></tr><tr><td>Pure Ni [37]</td><td>ARB</td><td>4.8</td><td>220 °C 10 min to 150 min</td><td>3%-94%</td><td>9</td><td>0.5</td><td>4,200,000</td></tr></tbody></table>

maps. This entails assigning a pixel mask to each Rex grain, indicating its precise position. The labeled training data serve as the ground-truth during the training process for updating the model's weights, whereas the labeled validation dataset was exclusively used to assess whether the model is sufficiently trained and to avoid overfitting.

The process of labeling Rex grains was in this work conducted in a semi-automatic approach: First, grain reconstruction was performed utilizing a threshold value of 2°. Subsequently, Rex grains were separated from the deformed matrix by considering a set of parameters. These included a minimum number of pixels within the grain ($P_{Rex}$), a maximum Grain Orientation Spread (GOS), and a maximum normalized GOS value (GOSN), where GOSN is normalized with respect to the equivalent diameter of the grain. The default parameter configuration was as follows: $P_{Rex} \ge 25$, GOS $\ge 0.5°$. For certain EBSD maps, the GOS value was increased to 1.5°, and to prevent mislabeling of small sub-grains of the deformed matrix as Rex grains, the extra criterion GOSN $\ge 0.5°/\mu m$ was applied. In Ni samples, Rex grains often exhibit annealing twins. Annealing twins larger than 25 pixels fulfill all the criteria to be classified as Rex grains, but those less than 25 pixels need special treatment to be included as Rex grains. This is done by checking neighboring grains of each identified Rex grain. If a neighboring grain has a twin relationship (less than 0.5° deviation from the ideal twin relationship) with the identified Rex grain, it was also considered as a Rex grain (a twin and its parent grain are treated as two Rex grains). Finally, manual visual examination was conducted to remove instances where deformed matrix was inaccurately labeled as Rex grains or to add instances where Rex grains were overlooked. This approach involving a fine-tuning of parameters for the different samples and a final manual inspection and correction, is indeed very labor intensive, but this only has to be done once. When the models are properly trained, this procedure is not needed again. However, it needs to be noted that labeling all the Rex grains correctly is very challenging, even with the tedious manual inspection, we used here. The labeled data are the best we can do, but might include a small fraction of mislabeling. Instead of calling the labeled data the 'ground truth', we will therefore in the following refer to them as the 'master truth'.

This iterative semi-automatic process led to a total of 39,363 labeled Rex grains within the training dataset, as detailed in Table 2. Considering that three IPF images from each EBSD subset were utilized for training, the resulting dataset contains 118,089 (39,363 × 3) labeled Rex grains. As listed in Table 2, the EBSD subsets within the validation

dataset contained 1625 Rex grains, which are exclusively used for evaluating the performance of the model without contributing to the training process. More details on the Rex grains, including the grain size distribution and the average grain size of each sample, are provided in Table S1 in the supplementary material.

### 3.3. Data augmentation

Rex grains, identified within each EBSD subset, were encoded into labeled images. In the labeled image, pixels corresponding to Rex grains were assigned specific grain numbers, while the deformed matrix was designated with a zero value. An example of a labeled image can be seen in Fig. 1b. The labeled Rex grains are represented using random colors to facilitate visualization. These labeled images were transformed into the COCO dataset format [38], wherein each individual Rex grain was denoted as a distinct instance. Each instance was accompanied by a corresponding mask and bounding box, as exemplified in Fig. 1c and e. To amplify the quantity of training data, a series of image augmentation techniques were applied. These include random flipping, random rotation, and random resizing with scaling factors between 0.5 and 1.5. The rotation operation aims to remove the potential bias in the training dataset, where the Rex grain may be elongated along a specific direction (e.g., rolling direction), while the resizing operation aims to make the trained model capable of dealing with images of varying magnifications. Fig. 1d provides an example of an augmented image, along with the corresponding masks and bounding boxes after these augmentations. In this work, the Mask R-CNN and PointRend models, which are designed to handle multiple categories, are used to detect only one category of instances. Thus, data augmentation is important to reduce the risk of overfitting.

### 3.4. Models and training

Mask R-CNN is a two-stage model with a backbone network to extract image feature maps from the input image at different scales, and a regional proposal network to detect object regions from the multi-scale features and to generate corresponding cropped features [23]. The cropped image features are re-used in the second stage to find the classes, the bounding boxes and the masks for instance segmentation. PointRend shares the same fundamental architectural components but introduces an additional approach to mask prediction by selectively predicting masks at adaptive points within object regions, thereby preserving fine details [27].

Training the models involves adjusting the model weights to minimize the loss function, which quantifies differences between model predictions and the ground truth. Both Mask R-CNN and PointRend use a multi-task loss function, including components for classification, bounding box position, and binary segmentation masks for each instance, represented as

$$L = L_{cls} + L_{box} + L_{mask}$$

The first term, $L_{cls}$, measures the error in the predicted class label for each instance. In this study, there is only one class, namely "Rex grain". The second term, $L_{box}$, assesses the difference between the position and size of the predicted and true bounding boxes. The final term, $L_{mask}$,

<table><caption>**Table 2**
The number of EBSD subsets and labeled Rex grains within the training and validation datasets.</caption><thead><tr><th rowspan="2">Material</th><th colspan="2">Training</th><th colspan="2">Validation</th></tr><tr><th>Nr. EBSD subsets</th><th>Nr. Rex grains</th><th>Nr. EBSD subsets</th><th>Nr. Rex grains</th></tr></thead><tbody><tr><td>Ta-10%W</td><td>26</td><td>3161</td><td>1</td><td>90</td></tr><tr><td>AA5657</td><td>17</td><td>5780</td><td>1</td><td>177</td></tr><tr><td>AA5182</td><td>25</td><td>8897</td><td>1</td><td>101</td></tr><tr><td>AA1200</td><td>18</td><td>3719</td><td>1</td><td>201</td></tr><tr><td>Pure Ni</td><td>28</td><td>17,806</td><td>2</td><td>1056</td></tr><tr><td>Total</td><td>114</td><td>39,363</td><td>6</td><td>1625</td></tr></tbody></table>
<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Example of training data showing a partially recrystallized microstructure of the Ta sample. a) Electron Backscatter Diffraction (EBSD) maps color-coded according to the inverse pole figure (IPF) of the direction perpendicular to the characterization plane (see the insert in a), with a semi-transparently superimposed grayscale band contrast map, i.e. IPF-Z map. b) Labeled map highlighting Rex grains in random colors. c) Image before augmentation. d) Image after data augmentation. The regions inside the dashed squares in c) and d) are enlarged in e) and f), respectively. Masks and bounding boxes for individual Rex grains are shown as contour lines and rectangles surrounding each grain, respectively, in c)-f).</figcaption></figure>

quantifies the differences between the predicted and ground-truth segmentation masks.

Both the Mask R-CNN and PointRend models were in this work implemented using the PyTorch framework with the Detectron2 model (https://github.com/facebookresearch/detectron2). Training was

performed using 1 GPU (NVIDIA A100) with a batch size of 2. A small batch size was used due to the large number of instances per image. The image area and instances per image for training are similar to those used in recrystallization kinetics analysis, a typical application where Rex grains need to be identified. To ensure high prediction quality, we
maintained consistency between the training and target data. During each training iteration, the loss was calculated for a single batch of training data using forward propagation. Subsequently, the gradients of each weight of the model with respect to the loss were computed using backpropagation, and these gradients were used to update the model weights to minimize the loss. The loss was calculated for the validation dataset in a similar manner as for the training dataset, but without backpropagation and model weights updates. Training was considered complete when both the training and validation losses became stable. In this study, both the Mask R-CNN and PointRend models achieved stable validation losses after 10,000 iterations (See Fig. S1 in the supplementary material), with a training time of 15 h.

### 3.5. Non-maximum suppression (NMS) and prediction settings

Both Mask R-CNN and PointRend are built upon the object detection model Faster R-CNN, which typically generates multiple detections per object with varying confidence scores, and some detections may only capture parts of the object. Duplicated predictions are removed in a process called non-maximum suppression (NMS). Mask R-CNN and PointRend employ a default NMS method based on the intersection over union (IOU) of bounding boxes from two predictions:

$$IOU(A, B) = \frac{A \cap B}{A \cup B}$$

Here, A and B represent the bounding boxes of the two predictions. In other words, it calculates the ratio of the overlapped area of two bounding boxes relative to the total area covered by the two bounding boxes. If the IOU of two bounding boxes exceeds a threshold value, the prediction with the lower confidence score is discarded. Setting the IOU threshold correctly is crucial because a too-high threshold may retain some duplicated predictions, while a too-low threshold could incorrectly remove neighboring instances. The default setting of 0.5 generally works well in many applications [25]. However, in the case of Rex grains studied in this work, the grains may exhibit clustering and irregular shapes. Consequently, using the bounding box IOU as the threshold value may be problematic. Fig. 2 shows an example where some duplicated predictions, which only partially cover the Rex grain, are retained after NMS. To address this issue, we implemented a pixel based NMS using a mask IOU instead of a bounding box IOU. Similar NMS has been previously used for instance segmentation of diffraction spots [26]. A threshold value of 0.1 for the mask IOU was applied for NMS in this

study.

In addition to the NMS threshold, the confidence score threshold for each prediction is another hyper-parameter influencing the predictions. Specifically, only predictions with a confidence score exceeding the threshold value are incorporated into the final prediction. In this study, the default setting of 0.5 is applied.

### 3.6. Model evaluation

In this study, with only one instance class, i.e. Rex grains, we adopted a modified model evaluation method introduced in [25], and utilized four sets of metrics, namely segmentation precision, segmentation recall, detection precision, and detection recall, to assess the predictions of the models.

Segmentation precision and recall are used to evaluate the accuracy of the predicted masks. A pixel is classified as a true-positive pixel if it belongs to both the predicted and ground-truth masks. Segmentation precision is calculated as the ratio of true-positive pixels to the total number of pixels in the predicted masks, while segmentation recall is the ratio of true-positive pixels to the total number of pixels in the ground-truth masks.

Detection precision and recall serve to assess the correspondence between predicted and ground-truth instances. For each predicted instance, we computed the IOU of its segmentation mask with the mask of each ground-truth instance to identify the prediction-ground-truth instance pair with the highest IOU. If this highest IOU exceeded 0.5, the predicted instance was considered a true-positive instance. Detection precision is calculated as the ratio of true-positive instances to the total number of predicted instances, while detection recall is the ratio of true-positive instances to the total number of ground-truth instances.

Indeed, segmentation/detection precision measures the fraction of correctly predicted Rex pixels/grains out of the total predicted pixels/grains, while segmentation/detection recall measures the fraction of correctly predicted Rex pixels/grains out of the total Rex pixels/grains in the ground/master truth. For the latter, one minus the value of the segmentation/detection recall represents the fraction of missing Rex pixels/grains.

## 4. Results

The trained Mask R-CNN and PointRend models were employed to predict Rex grains for the six EBSD subsets in the validation dataset. The

<figure><img src="image_4.png" /><figcaption>Fig. 2. Examples illustrating duplicated predictions retained after NMS using the bounding box IOU. a) A submap from the Ni sample with the same color coding as Fig. 1a. b)-d) Predictions for the large Rex grain (seen in Fig. 2a) retained after NMS. The predictions in c) and d) only partially represent the target grain, and they were not removed by NMS when the bounding box IOU threshold of 0.5 was used. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)</figcaption></figure>
four sets of metrics, namely segmentation precision, segmentation recall, detection precision, and detection recall, as defined in Section 3.6, were calculated with respect to the master truth, and are shown in Fig. 3. The scatter bars in Fig. 3 represent the standard deviation of predictions from the three IPF images of each EBSD subset. The standard deviations for all four sets of metrics are generally quite small. This suggests that predictions from different IPF images are consistent. Only AA1200 and AA5182 exhibit slightly higher standard deviations for segmentation and detection precision.

Both models give high segmentation recall for all six samples, with Ta, AA5657, and AA1200 exhibiting segmentation recall exceeding 0.9 (see Fig. 3a). The detection recall is slightly lower for most samples (around 0.8), except the two Ni samples (named Ni A and Ni B) which have significantly lower values (less than 0.5) (see Fig. 3b). The high segmentation recall indicates accurate prediction of the majority of Rex pixels. The low detection recalls of the two Ni samples suggest that some Rex grains are missing in the predictions. Nevertheless, considering that the segmentation recall of the two samples is high, it implies that the missing grains are likely small, and thus do not significantly affect the overall segmentation accuracy. Ta, AA5657, and the two Ni samples exhibit higher segmentation and detection precision than AA5182 and AA1200 (see Fig. 3c and d). Notably, AA1200 has a detection precision lower than 0.5. These low segmentation and detection precisions for AA1200 suggest that the model misclassifies some deformed matrix as Rex grains. Across all samples, Mask R-CNN and PointRend exhibit similar values in the four sets of metrics.

In the following sections, we will analyze predictions for representative samples, focusing on Ta, AA1200, and Ni B, to better understand these statistical results.

## 4.1. Analysis of the predictions

Fig. 4 illustrates an example of the predictions for the Ta sample, a case where both models give predictions of high recall and precision values. By comparing the maps highlighting individual Rex grains using random colors, namely Fig. 4b, e, and h, it is seen that both models accurately predict most of the individual Rex grains. Only in very few cases, as marked by the white arrows in Fig. 4h, there exist two Rex grains that are mis-predicted as one grain or one Rex grain that is mis-predicted as two grains. The former occurs because the two grains have a similar color in the IPF map, and the latter occurs because the grain shape is very irregular, and the predictions, that only cover a part of the grain, have the highest confidence score.

A more detailed comparison of the model predictions and the master truth can be seen in the maps showing their differences, where the false-positive pixels are in magenta, and the false-negative pixels are in green (Fig. 4f and i). There are very few of these. The majority of false-positive pixels are small and isolated mispredicted Rex grains, which are, in reality, subgrains of the deformed matrix, and these false-positives have a low confidence score value. A few false-positive pixels appear inside Rex grains, such as the one marked by the red arrow in Fig. 4f. These false-positive pixels can be a small deformed subgrain or a small Rex grain that is fully surrounded by a large Rex grain. These pixels are not considered as recrystallized in the master truth, but are included as a part of the surrounding Rex grain in the predictions. False-negative pixels are primarily observed for small Rex grains within regions containing clustered Rex grains, such as the area enlarged in the inset in Fig. 4f and i. In addition, for some Rex grains, an edge in magenta or in green is observed, which is usually one or two pixels beyond or inside

<figure><img src="image_5.png" /><figcaption>**Fig. 3.** Evaluation of the models for the six samples in the validation dataset, where the solid and hatched bars represent results from the Mask R-CNN and PointRend models, respectively. Precision and recall values were computed with reference to the master truth. Scatter bars represent the standard deviation of predictions obtained from the three IPF images generated from the same EBSD subsets.</figcaption></figure>
<figure><img src="image_6.png" /><figcaption>**Fig. 4.** Comparison of model predictions and master truth for the Ta sample. (a) The IPF-Y map used as input for prediction (600 × 600 pixels, 1 pixel = 0.5 µm). Rex grains in the master truth map are shown with random colors in (b), and with the IPF-Y colors in (c). Predictions from Mask R-CNN and PointRend are presented in (d-f) and (g-i), respectively. Predicted Rex grains in (d) and (g) are color-coded based on their confidence scores, while those in (e) and (h) are displayed in random colors. Panels (f) and (i) illustrate the differences between the predictions and the master truth. In these maps, regions in gray reveal overlapping areas, green pixels denote false negatives (pixels not included in the prediction), and magenta pixels represent false positives (pixels incorrectly labeled as Rex grains in the prediction). A subregion in c), f) and i) is enlarged to facilitate visualization. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)</figcaption></figure>

the correct grain boundaries. This is due to small errors in predicting the accurate grain boundaries. For this particular sample, PointRend does not exhibit a noticeable difference from Mask R-CNN in tracking the boundaries of individual Rex grains.

The aforementioned different types of false-positive and false-negative situations are representative also for the other samples. Additionally, there is however one special type of “false negative” seen in the AA5657 sample. As illustrated by two examples in Fig. 5c and g, the false-negative regions appear as extensions of Rex grains, resembling tails. This type of false negative is not truly “false”: the tails are some subgrains of the deformed matrix, which in the master truth map are incorrectly identified as a part of a Rex grain. These subgrains typically have very low misorientation angles to the Rex grain, and there is a high possibility that the Rex grains nucleated via strain-induced boundary migration (SIBM) [39,40] from the subgrains. Removing these subgrains from the master truth requires decreasing the grain reconstruction threshold to a very low value, which however often leads to fragmentation of many other Rex grains. Therefore, it is challenging to remove these tails in the master truth. On the other hand, both deep learning models can remove the tails correctly. Thus, in terms of separating the

subgrains related to SIBM from the Rex grains, the predictions generated by the trained models outperform the master truth.

The AA1200 sample is a typical example of a case with lower detection precision, where a significant number of false positive Rex grains are predicted (see Fig. 6f and i). In this high-purity aluminum sample deformed to a high strain, significant recovery occurs simultaneously with recrystallization during annealing [35,36]. Recovered subgrains resemble Rex grains, leading to the false positives. Recovery is also the main reason for the false positive predictions in AA5657, although the issue is less significant in AA5657. The recovered subgrains often exhibit low confidence scores, as shown Fig. 6d and g. Unambiguous Rex grains typically receive scores close to 1, while false positives often receive scores less than 0.8. This may be used in a possible further optimization of the predictions, which will be discussed in Section 5.2.

The Ni B sample, as shown in Fig. 7, is a typical example of a case with lower detection recall. Similar to the situation in the Ta sample, smaller Rex grains within clusters have a higher tendency to be overlooked in the prediction. The clustering phenomenon is much more pronounced in this Ni sample than in any of the other samples considered here. In addition to the false negatives caused by clustering,
<figure><img src="image_7.png" /><figcaption>**Fig. 5.** Comparison of PointRend model predictions and the master truth for the AA5657 sample. Two subregions of the IPF-X map are shown in (a) and (e), and grain boundaries with a misorientation larger than 2° are added to enhance visualization. The three Rex grains in each of the master truth maps (b) and (f) are shown in red, blue, and green. Predictions are shown in (c) and (g) with difference maps, where regions in gray reveal overlapping areas, while green and magenta pixels represent false negatives and false positives pixels, respectively. The Rex grains in the predictions (d) and (h) are shown with the same corresponding colors as those in the master truth maps. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)</figcaption></figure>

another type of false negative arises due to the irregular shape of Rex grains. These irregularly shaped Rex grains, usually of the Cube orientation, are frequently observed in highly strained materials like Ni and Cu [30,41]. In Fig. 7f, a typical example is highlighted by a dashed rectangle. The lower part of this grain is not included in the prediction by Mask R-CNN. Upon reviewing all predictions for this grain, we find that one prediction with a confidence score of 0.68 approximates the complete grain better but is excluded by NMS since the prediction that only covers partially the grain has a higher confidence score. PointRend exhibits better performance for this irregularly shaped grain, although some pixels at the lower part of the grain are still missing. Another complication arises from the fact that Ni samples develop annealing twins. On one hand, the annealing twins may partially cut through the parent grain, making one parent grain more likely to be predicted as several grains. On the other hand, some small annealing twins are incorporated into the predicted parent grain. Although not seen as false negatives in Fig. 7f and i, these small annealing twins do not have a one-to-one correspondence between the master truth and predicted grains, and they thus also contribute to the low detection recall.

# 5. Discussion

## 5.1. *Challenges detecting Rex grains*

The task of detecting Rex grains using the deep learning approach is indeed demanding because of two major challenges. The first is associated with detection of small Rex grains, particularly in cases with severe clustering. In the field of computer vision, small objects are typically defined as those less than 35 × 35 pixels or with a width and height both less than 1/10 that of the original image [38]. According to this definition, detecting Rex grains often involves identifying a large number of small objects, and it is thus a challenging task even for the state-of-art models. To overcome this challenge, a simple but effective strategy is to divide a single image into several sub-images and make predictions on these sub-images. As demonstrated in the results part, the omission of small Rex grains (i.e. false negatives) primarily occurs when these grains are located within clusters of Rex grains. The deep learning model generates many anchor boxes on feature maps to detect instances, and when both a tiny instance and a larger instance fall at least partially within the same anchor box, the model tends to prioritize the larger
<figure><img src="image_8.png" /><figcaption>**Fig. 6.** Comparison of model predictions and the master truth for the AA1200 sample. (a) The IPF-X map used as input for prediction (631 × 631 pixels, 1 pixel = 1 µm). Rex grains in the master truth map are shown with random colors (b) and with IPF-X colors (c). Predictions from Mask R-CNN and PointRend are presented in (d-f) and (g-i), respectively. Predicted Rex grains in (d) and (g) are color-coded based on their confidence scores, while those in (e) and (h) are displayed in random colors. Panels (f) and (i) illustrate the differences between the predictions and the master truth. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)</figcaption></figure>

instance, and thus the tiny instance is somewhat obstructed by the larger one. Using sub-images can, on one hand, reduce the number of instances in the image, thus reducing the obstruction between instances. On the other hand, it can increase the area fraction of each tiny instance per image, thereby 'enlarging' the instances in a certain sense. For the present models, if Rex grains are clustered, it is recommended to divide one image into smaller images to limit the number of instances per image to fewer than 250. Fig. 8d illustrates predictions based on a sub-image of the bottom-right corner, approximately half the width and length of the initial image, while Fig. 8c shows predictions based on the entire image (i.e., Fig. 8c is a cut from Fig. 7i). It is evident that fewer false-negative pixels appear when using the sub-image. For this sub-image, the detection recall increases significantly from 0.37 to 0.65, and the segmentation recall increases from 0.89 to 0.94. It is noteworthy that when using a sub-image, the input image size is reduced, while the image resolution remains unchanged. Therefore, further dividing the sub-image into even smaller sizes does not significantly contribute to improving predictions. Instead, it is necessary to enhance the actual resolution of the image, i.e., reduce the step size of EBSD data collection, to further improve the prediction of small Rex grains. Effects of the EBSD step size will be discussed in Section 5.2.

The second challenge is associated with the detection of grains with

irregular shapes. For Rex grains with smooth boundaries, both models achieve an accuracy of one to two pixels in tracking the grain boundaries, which is generally acceptable. However, in samples such as the present Ni samples, some Rex grains exhibit very irregular shapes, attributed to non-uniform boundary migration during recrystallization and the formation of annealing twins. While the PointRend model demonstrated better results for Rex grains with irregular shapes, it still did not precisely track the boundaries. One possible reason is that the training data did not include a sufficient number of irregular Rex grains, because even in the Ni samples, irregularly shaped grains are only very few, which leads to the issue of unbalanced distribution of ground truth within category. If target data contains many irregularly shaped grains, and accurate tracking of grain boundaries is desired (e.g., for calculating boundary roughness or tracking the movement of grain boundaries), the current model may be improved by oversampling the irregularly shaped grains (e.g. cutting sub-images around irregularly shaped grains) and conducting further training.

## 5.2. Effects of parameters

Compared to the conventional method based on properties of grains reconstructed from EBSD, the trained deep learning models require
<figure><img src="image_9.png" /><figcaption>**Fig. 7.** Comparison of model predictions and the master truth for the Ni B sample. (a) The IPF-Z map used as input for prediction (479 × 479 pixels, 1 pixel = 0.5 µm). Rex grain in the master truth map are shown with random colors (b) and with IPF-Z colors (c). Predictions from Mask R-CNN and PointRend are presented in panels (d-f) and (g-i), respectively. Predicted Rex grains in (d) and (g) are color-coded based on their confidence scores, while those in (e) and (h) are displayed in random colors. Panels (f) and (i) illustrate the differences between the predictions and the master truth. The dashed rectangle in (f) highlights a region where a Rex grain with an irregular shape is not correctly predicted. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)</figcaption></figure>

fewer adjustments to parameters for prediction, making them more user-friendly and robust. Nevertheless, effects of some parameters might need to be considered to achieve optimal predictions, including the threshold score and the step size of the EBSD data.

Each potential prediction is associated with a confidence score, indicating how likely this prediction represents a real Rex grain. A threshold score value is set to remove predictions with a score lower than the threshold, which by default is 0.5. This default value works well for some samples (e.g. Ta), but for some other samples, it needs to be increased to remove the false positives, such as the AA1200 sample shown in Fig. 6. Upon analyzing the effect of threshold score values on precision and recall, we for the present set of samples observe that they can be divided into two groups. All Al samples belong to the first group, characterized by a certain fraction of predicted Rex grains with low scores, as shown in Fig. 6g. These low-score predictions are attributed to simultaneous recovery and recrystallization, leading to ambiguity between recovered subgrains and Rex grains. Fig. 9a illustrates how the recall and precision values change as the threshold score value increases for AA1200. It is observed that as the threshold score value increases, both segmentation and detection precision increase rapidly, while segmentation and detection recall initially decrease at a slower rate.

Therefore, increasing the threshold score value can effectively enhance both detection and segmentation precision, with a lesser impact on recall. The issue of low detection and segmentation precision of AA1200 can be resolved by setting a high confidence score threshold. Fig. 9c shows the predictions with a score threshold of 0.9. Compared with Fig. 6i, false positive predictions have been significantly reduced. Although there is a slight increase in the number of false negative predictions, these false negatives are primarily related to small grains. It is worth noting that the present false positives and negatives are calculated with respect to the master truth. However, in cases with ambiguity between recovered subgrains and Rex grains, the master truth may contain errors. As a result, some false negatives and false positives in the prediction may not truly be “false”; it may well be the deep learning model, which is correct, and not the master truth.

The Ta sample belongs to the second group, where the majority of predicted Rex grains exhibit high scores, close to 1, with only a few grains having scores less than 0.9 (see Fig. 4g). Fig. 9b illustrates how precision and recall change as the threshold score value increases. It is observed that the threshold score has minor effects on segmentation precision and recall. Detection precision increases slightly, whereas detection recall decreases slightly with increasing threshold score
<figure><img src="image_10.png" /><figcaption>**Fig. 8.** Comparison of model predictions on the entire image and sub-image for the Ni B sample. (a) The complete IPF-Z map, with a square indicating the sub-image used for prediction. (b) The sub-image used for prediction (250 × 250 pixels, 1 pixel = 0.5 µm). Panels (c) and (d) show predictions of PointRend based on the entire image and the sub-image, respectively, where regions in gray reveal overlapping areas, green pixels denote false negatives (pixels not included in the prediction), and magenta pixels represent false positives. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)</figcaption></figure>

values. When comparing the predictions using a threshold score of 0.9 (Fig. 9d) and 0.5 (Fig. 4i), only minor changes are noticed. The two Ni samples also belong to this second group but with slightly more low-score predictions, which are usually related to irregular grain shapes. Increasing the threshold score value will remove some of these low-score predictions.

In practice, different strategies for the threshold score should be considered for different materials. For materials that can experience significant recovery, such as Al cold rolled to high strains, recovered subgrains may have a confidence score higher than 0.5. Therefore, increasing the threshold score is helpful in removing false positives. This strategy is also applicable to materials that have been hot deformed. For other samples, the default threshold score value should be kept.

The step size is another parameter that need to be considered. To illustrate how model predictions depend on step sizes, two artificial EBSD maps were created by selecting every second or fourth pixel along the X and Y directions. Fig. 10a and b show two maps generated from Fig. 4a. The initial step size for Fig. 4a was 0.5 µm, so the step sizes for Fig. 10a and b are 1 µm and 2 µm, respectively. Fig. 10c and d show the predictions of the PointRend model. Compared to Fig. 4i, the predictions in Fig. 10c are of similar quality. In contrast, Fig. 10d contains more green pixels with a segmentation recall of only 0.85, whereas that for

Fig. 4i and Fig. 10c are both 0.95. This indicates that more small Rex grains are missed when the step size increases to 2 µm. These small Rex grains are challenging to identify even with visual inspection, and it is not surprising that the deep learning model misses them. Moreover, a large step size can also lead to the mislabeling of some deformed matrix as Rex grains when color variations are small, and these false-positive predictions are indeed problematic since they might have confidence scores higher than some real Rex grains, as marked by the arrow in Fig. 10f.

The comparison of predictions based on maps with different step sizes indicates that the deep learning approach can handle maps with step sizes varying within a certain range, but a too coarse step size results in many false predictions. Whether a step size is properly chosen or not depends on the Rex grain sizes in the sample. In this particular sample, the minimum equivalent circular diameters of the correct predictions are 3.1 µm (31 pixels), 3.6 µm (10 pixels), and 5.0 µm (5 pixels) for maps with 0.5 µm, 1 µm and 2 µm step sizes, respectively. It is noteworthy that highly confident Rex grain predictions (e.g. with score higher than 0.95) usually have a size larger than 25 pixels (5 × 5 pixels). Therefore, it is recommended to maintain a ratio of at least 5 between the typical Rex grain size and the EBSD step size.
<figure><img src="image_11.png" /><figcaption>**Fig. 9.** Effects of threshold score on PointRend model predictions. a) and b) show the changes in detection precision, detection recall, segmentation precision, and segmentation recall as the threshold score value increases for the AA1200 (a) and Ta (b) samples. c) and d) show the predictions using a threshold score value of 0.9, where regions in gray reveal overlapping areas, green pixels denote false negatives (pixels not included in the prediction), and magenta pixels represent false positives. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)</figcaption></figure>

### 5.3. Performance of the models for determination of microstructural parameters

For statistical analyses of recrystallization kinetics, the most commonly used microstructural parameters include the recrystallized volume fraction ($V_V$), the interface area between Rex grains and the deformed matrix per unit volume ($S_V$), and the mean intercept length of the Rex grains ($\lambda_{Rex}$). All these values depend critically on a proper detection of Rex grains in partially recrystallized microstructures. Fig. 11 compares the values of $V_V$, $S_V$, and $\lambda_{Rex}$ calculated from the master truth and the Rex maps predicted by the models. Regarding $V_V$, the predicted values from both models are in excellent agreement with the master truth. For $S_V$, the predicted values are in reasonably good agreement with the master truth, with only a slight underestimation. This underestimation may be attributed to the fact that the *predicted* boundaries of Rex grains tend to be less rough than the master truth. As expected, the PointRend model performs slightly better than the Mask R-CNN model in this respect. For $\lambda_{Rex}$, the predicted values are typically 1–2 µm larger than the master truth, mainly due to some small Rex grains being missed.

### 5.4. Comparison between deep learning and the conventional automatic method

Compared with conventional automatic method of detecting Rex grains that rely on grain reconstruction, an advantage of the deep learning approach is that, when the model has been trained, it is easy to apply it for further detection tasks. Conventional methods often need simultaneous adjustments of a series of parameters and this adjustment is different for different samples. Fig. 12a and b illustrate two examples of results from the conventional method using the default parameters outlined in Section 3.2. While these default parameters work well for Ta and Ni samples, several large Rex grains are missed for AA1200 and AA5657 due to GOS values larger than the default threshold of 0.5°. Larger GOS values for Rex grains may have two causes: The first is associated with the sample surface quality, EBSD acquisition quality (e. g. focus), or the characteristics of the Rex grains. The second cause is linked to a misorientation angle threshold that may be too high for grain reconstruction, thus including neighboring grains or deformed matrix into one Rex grain. To avoid missing these Rex grains when using the conventional method, the threshold values of the GOS or the misorientation angle for grain reconstruction must be adjusted. However, fine-tuning of other parameters is also required, demanding efforts and experience.

When using the deep learning approach, false predictions, whether
<figure><img src="image_12.png" /><figcaption>**Fig. 10.** Model predictions for maps with different step sizes. (a) and (b) show IPF maps generated from Fig. 4a by selecting every 2nd and 4th pixel, respectively, along both the X and Y directions. Corresponding master truth maps are generated from Fig. 4b in a similar manner. (c) and (d) show the model predictions compared to the corresponding master truth maps. (e) and (f) show the predicted Rex grains color-coded according to their scores. The arrow in (f) marks a false-positive Rex grain with a confidence score of 0.9. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>**Fig. 11.** Microstructural parameters calculated for Rex grains predicted by the Mask R-CNN and the PointRend models as a function of those calculated from the master truth. The dashed lines have a slope of 1 and are added to facilitate visualization. a) Recrystallized volume fraction ($V_v$). b) Interface area between Rex grains and the deformed matrix per unit volume ($S_v$). c) Mean intercept length of the Rex grains ($\lambda_{Rex}$). For the AA1200 sample, the score threshold was set to 0.9 (same as Fig. 9c), while the default score threshold of 0.5 was used for all the other samples. For the Ni B sample, values were obtained using the sub-image shown in Fig. 8.</figcaption></figure>

false positives or false negatives, are typically associated with small grains. In contrast, the conventional approach tends to miss large Rex grains. Considering that larger grains have a more significant impact on $V_v$ and $S_v$, missing big grains may pose a more serious issue at least for these values. Another advantage of the deep learning method is that it is less affected by sample surface quality. Non-indexed noisy points inside Rex grains, such as those shown in Fig. 5, due to insufficient sample preparation or surface contamination, may be mistaken as un-recrystallized points by the conventional method. The deep learning method incorporates these noisy points within the Rex grains, thus avoiding the generation of artificial boundaries inside the grains.

## 6. Conclusions and outlook

In this study, we used the Mask R-CNN and PointRend networks to train models for the automatic detection of individual recrystallizing (Rex) grains in EBSD images of partially recrystallized microstructures. By including EBSD data from various materials with diverse thermo-mechanical processing histories into the training dataset, we successfully developed models with broad adaptability that can detect and segment Rex grains with sufficient accuracy for extracting key microstructural parameters, such as the recrystallized volume fraction ($V_v$) and the interfacial area of un-impinged recrystallizing boundaries per unit volume ($S_v$). Evaluation of the models reveal that the deep learning
<figure><img src="image_14.png" /><figcaption>**Fig. 12.** Rex grains identified using the conventional method for AA1200 (a) and AA5657 samples. The green and magenta pixels represent false negatives and false positives compared with the master truth. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)</figcaption></figure>

models may ignore some small Rex grains within clusters of larger Rex grains. This results in a slight overestimation of the average Rex grain size by the models. While the Mask R-CNN and PointRend models demonstrate similar performance in the overall predictions, the PointRend model shows more potential when dealing with Rex grains of irregular shapes.

Compared with the conventional automatic method that detects Rex grains based on properties of reconstructed grains, the trained deep learning models are more user-friendly and require less parameter fine-tuning efforts after training. For many materials, satisfactory predictions may be achieved by directly using the default setting of the deep learning models. The models function well based on EBSD maps acquired with different step sizes, but it is recommended to maintain a ratio of at least 5 between the typical size of Rex grains and the step size during EBSD acquisition. For microstructures with a significantly recovered deformed matrix, increasing the threshold value for confidence scores is recommended to avoid subgrains becoming misclassified as Rex grains. For microstructures with severe clustering of a large number of Rex grains, using sub-images with a smaller number of Rex grains per image is recommended.

Since the deep learning models are primarily image-based, they could potentially be extended to detect Rex grains or other types of grains (e.g., proeutectoid ferrite grains) from SEM images using the electron channeling contrast (ECC) technique. ECC images are easier and faster to obtain than EBSD, but labeling Rex grains from ECC images requires manual selection of the contour of each Rex grain, and it is challenging to acquire a high enough number of Rex grains with labeled masks of pixel level accuracy for training. If a future effort is to be devoted to using ECC as the basis for the grain identification, the models trained in this work using EBSD images may serve as an excellent starting point for the further training. This is because of the similarities between EBSD and ECC images, and the novel ECC based model(s) would thus requiring less training data.

### CRediT authorship contribution statement

**Fengxiang Lin:** Methodology, Investigation, Formal analysis, Data curation, Conceptualization, Writing – review & editing, Writing – original draft. **Haixing Fang:** Methodology, Investigation, Conceptualization, Writing – review & editing. **Hong Liu:** Resources, Funding acquisition, Writing – review & editing. **Yubin Zhang:** Investigation, Data curation, Conceptualization, Writing – review & editing, Writing – original draft. **Dorte Juul Jensen:** Supervision, Resources, Project

administration, Funding acquisition, Conceptualization, Writing – review & editing, Writing – original draft. **Emil Hovad:** Software, Methodology, Conceptualization, Writing – review & editing.

### Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

### Data availability

The trained models and the raw data required to reproduce these findings are available upon request.

### Acknowledgement

This work is initiated during a project funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (M4D – grant agreement No. 788567), while it was finished during a second project funded by VIL-LUM FONDEN (MicroAM VIL-54495). Furthermore, FXL acknowledges the funding from the KK-Foundation (No. 20190033) and from the Vice-Chancellor, Karlstad University. HL acknowledges the funding from the National Natural Science Foundation of China (No. 52101018) and from Shanghai Pujiang Program (No. 21PJ1405800). The authors gratefully acknowledge Prof. Guilin Wu, Prof. J.F. Nie, Dr. Xiuchuan Lei, Dr. Chunlei Zhang for generously providing the EBSD data used in this study.

## Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.matchar.2024.114576.

### References

[1] A.D. Rollett, D. Raabe, A hybrid model for mesoscopic simulation of recrystallization, Comput. Mater. Sci. 21 (1) (2001) 69–78.

[2] J.J. Sidor, R.H. Petrov, L.A. Kestens, Modeling the crystallographic texture changes in aluminum alloys during recrystallization, Acta Mater. 59 (14) (2011) 5735–5748.

[3] T. Takaki, Y. Hisakuni, T. Hirouchi, A. Yamanaka, Y. Tomita, Multi-phase-field simulations for dynamic recrystallization, Comput. Mater. Sci. 45 (4) (2009) 881–888.
[4] F. Lin, M. Sitko, L. Madej, L. Delannay, Non-uniform grain boundary migration during static recrystallization: a cellular automaton study, Metallurgical and Materials Transactions A 53 (5) (2022) 1630–1644.

[5] R.A. Vandermeer, X.C. Lei, E.F.F. Knipschildt-Okkels, F. Lin, R.E. Sanders, D. Juul Jensen, Advanced microstructural path modeling of primary recrystallization in aluminum alloys AA5182 and AA5657, MetalMat e15 (2024).

[6] Q.N. Guo, X.C. Lei, R.E. Sanders Jr., X.F. Yang, Y.X. Liang, L. Wang, Z.Z. Fan, Effect of annealing conditions on recrystallization of AA5182 sheet, in: Materials Science Forum vol. 877, Trans Tech Publications Ltd., 2017, pp. 264–271.

[7] S. Wronski, J. Tarasiuk, B. Bacroix, A. Baczmański, C. Braham, Investigation of plastic deformation heterogeneities in duplex steel by EBSD, Mater Charact 73 (2012) 52–60.

[8] S.R. Ahl, H. Simons, C. Detlefs, D. Juul Jensen, H.F. Poulsen, Subgrain dynamics during recovery of partly recrystallized aluminum, Acta Mater. 185 (2020) 142–148.

[9] G. Wu, D. Juul Jensen, Automatic determination of recrystallization parameters based on EBSD mapping, Mater Charact 59 (6) (2008) 794–800.

[10] D.P. Field, L.T. Bradford, M.M. Nowell, T.M. Lillo, The role of annealing twins during recrystallization of Cu, Acta Mater. 55 (12) (2007) 4233–4241.

[11] S. Mandal, A.K. Bhaduri, V. Subramanya Sarma, A study on microstructural evolution and dynamic recrystallization during isothermal deformation of a Ti-modified austenitic stainless steel, Metallurgical and Materials Transactions A 42 (2011) 1062–1072.

[12] A. Hadadzadeh, F. Mokdad, M.A. Wells, D.L. Chen, A new grain orientation spread approach to analyze the dynamic recrystallization behavior of a cast-homogenized mg-Zn-Zr alloy using electron backscattered diffraction, Mater. Sci. Eng. A 709 (2018) 285–289.

[13] A. Ayad, M. Ramoul, A.D. Rollett, F. Wagner, Quantifying primary recrystallization from EBSD maps of partially recrystallized states of an IF steel, Mater Charact 171 (2021) 110773.

[14] Y. Zhu, Y. Cao, Q. He, R. Luo, J. Zhang, H. Di, J. Xiao, Machine learning neural-network identification for dynamic recrystallization grains during hot deformation of nickel-based superalloy, Mater Charact 191 (2022) 112108.

[15] E. Hovad, T. Wix, M. Khomiakov, G. Vassos, A.F. da Silva Rodrigues, A. de Miguel Tejada, L.H. Clemmensen, Deep learning for automatic railway maintenance, in: Intelligent Quality Assessment of Railway Switches and Crossings, 2021, pp. 207–228.

[16] K. Choudhary, B. DeCost, C. Chen, A. Jain, F. Tavazza, R. Cohn, C. Wolverton, Recent advances and applications of deep learning methods in materials science, npj Comput. Mater. 8 (1) (2022) 59.

[17] E.A. Holm, R. Cohn, N. Gao, A.R. Kitahara, T.P. Matson, B. Lei, S.R. Yarasi, Overview: computer vision and machine learning for microstructural characterization and analysis, Metall. Mater. Trans. A 51 (2020) 5985–5999.

[18] H. Fang, E. Hovad, Y. Zhang, L.K.H. Clemmensen, B.K. Ersbøll, D. Juul Jensen, Deep learning for improving non-destructive grain mapping in 3D, IUCrJ 8 (5) (2021) 719–731.

[19] E. Hovad, H. Fang, Y. Zhang, L.K.H. Clemmensen, B.K. Ersbøll, D. Juul Jensen, Unsupervised deep learning for laboratory-based diffraction contrast tomography, Integr. Mater. Manuf. Innov. 9 (2020) 315–321.

[20] S.M. Azimi, D. Britz, M. Engstler, M. Fritz, F. Mücklich, Advanced steel microstructural classification by deep learning methods, Sci. Rep. 8 (1) (2018) 2128.

[21] B.L. DeCost, B. Lei, T. Francis, E.A. Holm, High throughput quantitative metallography for complex microstructures using deep learning: a case study in ultrahigh carbon steel, Microsc. Microanal. 25 (1) (2019) 21–29.

[22] G. Roberts, S.Y. Haile, R. Sainju, D.J. Edwards, B. Hutchinson, Y. Zhu, Deep learning for semantic segmentation of defects in advanced STEM images of steels, Sci. Rep. 9 (1) (2019) 12744.

[23] K. He, G. Gkioxari, P. Dollár, R. Girshick, Mask R-CNN, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2961–2969.

[24] R. Girshick, Fast R-CNN, in: Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 1440–1448.

[25] R. Cohn, I. Anderson, T. Prost, J. Tiarks, E. White, E. Holm, Instance segmentation for direct measurements of satellites in metal powders and automated microstructural characterization from image data, Jom 73 (7) (2021) 2159–2172.

[26] H. Fang, E. Hovad, Y. Zhang, D. Juul Jensen, Application of mask R-CNN for lab-based X-ray diffraction contrast tomography, Mater Charact 201 (2023) 112983.

[27] A. Kirillov, Y. Wu, K. He, R. Girshick, Pointrend: image segmentation as rendering, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 9799–9808.

[28] Y. Zhang, A. Godfrey, D. Juul Jensen, Local boundary migration during recrystallization in pure aluminium, Scr. Mater. 64 (4) (2011) 331–334.

[29] K. Huang, K. Zhang, K. Marthinsen, R.E. Logé, Controlling grain structure and texture in Al-Mn from the competition between precipitation and recrystallization, Acta Mater. 141 (2017) 360–373.

[30] F.X. Lin, Y.B. Zhang, W. Pantleon, D. Juul Jensen, Supercube grains leading to a strong cube texture and a broad grain size distribution after recrystallization, Philos. Mag. 95 (22) (2015) 2427–2449.

[31] T.M. Ostormujof, R.P.R. Purohit, S. Breumier, N. Gey, M. Salib, L. Germain, Deep learning for automated phase segmentation in EBSD maps. A case study in dual phase steel microstructures, Mater Charact 184 (2022) 111638.

[32] A.R. Durmaz, M. Müller, B. Lei, A. Thomas, D. Britz, E.A. Holm, P. Gumbsch, A deep learning approach for complex microstructure inference, Nat. Commun. 12 (1) (2021) 6272.

[33] G. Ma, Q. He, X. Luo, G. Wu, Q. Chen, Effect of recrystallization annealing on corrosion behavior of ta-4% W alloy, Materials 12 (1) (2018) 117.

[34] X. Lei, R.E. Sanders, X. Yang, F. Lin, D. Juul Jensen, Quantitative comparison of the recrystallization kinetics of two industrially processed 5xxx aluminum alloys, Metallurgical and Materials Transactions A 52 (2021) 4827–4840.

[35] G. Wu, D. Juul Jensen, Recrystallisation kinetics of aluminium AA1200 cold rolled to true strain of 2, Mater. Sci. Technol. 21 (12) (2005) 1407–1411.

[36] G.L. Wu, D. Juul Jensen, Effect of annealing temperature on recrystallisation in Al (AA1200) cold rolled to a true strain of 4, in: Materials Science Forum vol. 558, Trans Tech Publications Ltd., 2007, pp. 395–400.

[37] Y.B. Zhang, O.V. Mishin, Stored energy and recrystallized microstructures in nickel processed by accumulative roll bonding to different strains, Mater Charact 129 (2017) 323–328.

[38] T.Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, C.L. Zitnick, Microsoft coco: common objects in context, in: Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, Springer International Publishing, 2014, pp. 740–755.

[39] P.A. Beck, P.R. Sperry, Strain induced grain boundary migration in high purity aluminum, J. Appl. Phys. 21 (2) (1950) 150–152.

[40] E.F. Ferdinand Knipschildt, Nucleation of recrystallization, Mater. Sci. Technol. 38 (12) (2022) 765–779.

[41] Y.B. Zhang, A. Godfrey, Q. Liu, W. Liu, D. Juul Jensen, Analysis of the growth of individual grains during recrystallization in pure nickel, Acta Mater. 57 (9) (2009) 2631–2639.