AMB2022-03 Benchmark Measurements and Challenge Results – PDF Copy

1

AMB2022-03 Benchmark Measurements and Challenge Results

Last updated on 07/21/2022

Overview

The following describes compiled benchmark challenge measurement results which are used to judge 

submissions to the 2022 AM-Bench modeling challenges. The results presented here are summarized 

and formatted similar to how modelers were asked to submit their modeling results by July 15, 2022. 

Additional context, description, or measurement results may also be provided where necessary. 

Additional information may become available later so updated versions of these documents may be 

posted. Please check back occasionally.

Please note that the measurement results presented here are focused on the challenge problems and 

reflect only part of the validation measurement data which will be provided by AM Bench for each set of 

benchmarks.

AMB2022-03: Single laser tracks using different processing conditions and 2D arrays of laser tracks 

(Pads) on solid plates of IN718. Detailed descriptions of the experiment design, measurement methods, 

and data analysis are given here.

Challenges

● Track Solid Cooling Rate (CHAL-AMB2022-03-TSCR): Cooling rate immediately following 

complete solidification (below solidus) at the center of each track for all processing conditions.

● Track Liquid Cooling Rate (CHAL-AMB2022-03-TLCR): Liquid cooling rate immediately before 

start of solidification (above liquidus) at the center of each track for all processing conditions.

● Track Time Above Melt (CHAL-AMB2022-03-TTAM): Time above the midpoint between solidus 

and liquidus temperature (assumed to be 1 298 °C) for single laser-scanned tracks.

● Track Melt Pool Geometry (CHAL-AMB2022-03-TMPG): The laser track width and depth near the 

center of each track for all processing conditions.

● Pad Time Above Melting (CHAL-AMB2022-03-PTAM): Time above the midpoint between solidus 

and liquidus temperature (assumed to be 1 298 °C) for specified locations of the X-Pads and YPads.

● Pad Solid Cooling Rate (CHAL-AMB2022-03-PSCR): Cooling rate immediately following complete 

solidification (below solidus) at the center of each track for all processing conditions.

● Pad Melt Pool Geometry (CHAL-AMB2022-03-PMPG): Laser track depth and geometrical 

measurements describing the overlapping laser tracks near the center and near the edge of both 

X-Pads and Y-Pads. 

_____________________________________________________________________________________

1. Description of Benchmark Challenge Problems

1.1 Single Laser Track Benchmarks

Seven combinations of spot size, scan speed, and laser power were tested with three repeats of each 

melt track. The process parameters for the thermography tests are summarized in Table 1.
AMB2022-03 Benchmark Measurements and Challenge Results – PDF Copy

2

Table 1: Summary of single-track thermography process parameter cases for quick reference

Case 

Number

Laser 

Power 

[W]

Scan 

Speed 

[mm/s]

Spot size, 

D4σ [μm]

Baseline 0 285 960 67

Change spot 1.1 285 960 49

1.2 285 960 82

Change Speed 2.1 285 1200 67

2.2 285 800 67

Change Power 3.1 325 960 67

3.2 245 960 67

The time above melt (TAM) and cooling rate (CR) values are determined based on 30 adjacent pixels at 

the track centerline at a nominally steady-state location along the track length. In order to determine 

the TAM and CRs, the apparent temperature at the solidification inflection is determined. It is assumed 

that no undercooling occurs, and the emissivity is used to convert the apparent temperature to ‘true’ 

temperature based on the midpoint of the solidus and liquidus table data for IN718. The apparent 

solidification temperature and corresponding emissivity values for each case (with three repeats) are 

shown in Figure 1. 

As discussed in the measurements and challenge description documentation, the measurable TAM and 

CRs are based on the apparent temperature distribution, which includes any and all measurement error 

induced by the laser-matter interaction and byproducts thereof. It can be observed that, with 

decreasing volumetric energy density (VED) of each parameter, the apparent solidification temperature 

and emissivity increase. Although a change in undercooling may occur with differing cooling rates, it is 

also evident from the image data that the size, intensity, and position of the radiating process 

byproducts changes with VED, which also affects the measured temperature and associated TAM and 

CRs. It can be observed that the change in spot size had the most significant effect on the apparent 

solidification temperature. 
AMB2022-03 Benchmark Measurements and Challenge Results – PDF Copy

3

Figure 1: Single track apparent solidification temperature and emissivity with three repeats of seven 

process parameter combinations.

The reported TAM and CRs are based on the average of three individual tracks, and the results are 

shown graphically in Figure 2. Increasing scan speed and decreasing power (both decreasing VED) causes 

the CRs to increase and the TAM to decrease approximately linearly. Increasing spot size shows a 

nonlinear decrease in TAM and a decreasing trend in SCR, which is converse of the other process 

parameters with respect to decreasing VED. These results combined with the substantial increase in 

apparent solidification temperature at the largest spot size suggest that the measurement error in TAM 

and SCR may be greatest at the largest spot size. The results should be interpreted accordingly.

Figure 2: Graphical summary of the measured TAM and CRs as a function of each process parameter.

VED decreases from left to right in each graph, and so the power axis is decreases from left to right. 

The results are tabulated in the following sections.
AMB2022-03 Benchmark Measurements and Challenge Results – PDF Copy

4

1.1.1 CHAL-AMB2022-03-TTAM, -TSCR, and -TLCR

Table 2 provides the summary results for the track time above melting (TTAM), track solid cooling rate 

(TSCR), and track liquid cooling rate (TLCR) results for the CHAL-AMB2022-03 set of modeling challenges:

Table 2: CHAL-AMB2022-03-TTAM, -TSCR, and -TLCR measurement results

Case 

Number

Track Time 

Above 

Melting 

(TTAM) [s]

Track Solid 

Cooling 

Rate (TSCR) 

[°C/s]

Track 

Liquid

Cooling 

Rate (TLCR) 

[°C/s]

0 1.22E-03 6.99E+05 4.14E+05

1.1 1.30E-03 7.32E+05 4.06E+05

1.2 1.04E-03 5.11E+05 3.65E+05

2.1 8.96E-04 7.85E+05 5.06E+05

2.2 1.59E-03 6.11E+05 3.25E+05

3.1 1.38E-03 7.05E+05 3.81E+05

3.2 1.03E-03 7.49E+05 4.33E+05

1.1.2 Supplementary Data

The following measurements were also obtained during the AMB2022-03 set of tests, and described in 

the AMB2022-03 modeling challenge description. These were not part of the 2022 modeling challenge 

competition, but are provided here for future reference:

1.1.2.1 Track Transition Cooling Rate (TTCR)

Table 3: Measured TTCR for reference

Case 

Number

Track 

Transition 

Cooling 

Rate (TTCR) 

[°C/s]

0 1.93E+05

1.1 1.38E+05

1.2 1.93E+05

2.1 3.01E+05

2.2 1.46E+05

3.1 1.57E+05

3.2 2.32E+05
AMB2022-03 Benchmark Measurements and Challenge Results – PDF Copy

5

1.1.3 CHAL-AMB2022-03-TMPG

Dark field micrographs for single tracks are shown in Figure 3 for the track melt pool geometry (TMPG) 

challenge. The challenge measurements for melt pool depth and width are provided in Table 4. Trends 

with laser diameter, velocity, and power are plotted in Figure 4.

Figure 3. Representative cross-sectional dark field optical micrographs of single tracks. Image labels 

include the part number-line condition-line number (e.g., P3-L1.2-1).

Table 4. CHAL-AMB2022-03-TMPG Results: track depth (dt) and track width (wt). There are 6 

measurements for each condition (Sample ID).

Sample ID and Case Number

Mean dt

(μm)

Std. dev. dt

(μm)

Mean wt

(μm)

Std. dev. 

wt

(μm)

AMB2022-718-SH1-BP1-L0 139.7 1.9 136.3 2.9

AMB2022-718-SH1-BP1-L1.1 227.2 3.2 106.2 3.6

AMB2022-718-SH1-BP1-L1.2 102.4 1.1 141.7 1.8

AMB2022-718-SH1-BP1-L2.1 109.7 1.7 112.9 1.7

AMB2022-718-SH1-BP1-L2.2 176.5 2.6 156.1 4.9

AMB2022-718-SH1-BP1-L3.1 166.1 2.0 134.3 2.5

AMB2022-718-SH1-BP1-L3.2 116.9 1.2 129.4 1.6
AMB2022-03 Benchmark Measurements and Challenge Results – PDF Copy

6

Figure 4. Melt pool size (depth and width) trends with (a) laser spot diameter, (b) laser velocity, and (c) 

laser power for single track laser scans on BP1. Data are the mean ± 95% confidence interval (n = 6).

1.2 Laser Pad Benchmarks

1.2.1 CHAL-AMB2022-03-PTAM 

The algorithm for analyzing the pad high speed thermal imaging is based on that described in the 

AMB2022-01 challenges for the 3D builds, as was mentioned in the AMB2022-03 challenge description. 

Pre-processing was conducted in a slightly different order than the algorithm described in the AMB2022-

03 challenge description for TAM and SCR. Here, spatter removal processing was first conducted on the 
AMB2022-03 Benchmark Measurements and Challenge Results – PDF Copy

7

raw thermal video signal prior to conversion to temperature. However, this is inconsequential to the 

resulting PTAM and PSCR values.

The following algorithmic steps were taken to arrive at PTAM and PSCR results described in this 

document:

1. Perform spatter-removal algorithm on raw thermal camera signal.

a. Frame-by-frame spatter removal using threshold mask at 100 digital levels 

(approximately equivalent to T = 1077 °C at ε = 0.5).

b. Masking of part boundaries using threshold of 760 digital levels (approximately 

equivalent to T = 1260 °C at ε = 0.5).

2. Convert the spatter-removed thermal images to temperature using the camera calibration and 

estimated emissivity of ε = 0.5.

3. Proceed with calculation of TAM and SCR for each pixel as described in the AMB2022-03 

challenge description using transition temperature of Ttrans = 1298 °C.

Further details on the exact algorithms will be clarified in a future publication. It should be noted that 

the selection of spatter-removal threshold parameters as well as the assumed emissivity affect the 

measured PTAM and PSCR results. 

The choice in spatter-removal algorithm parameters, and emissivity value where chosen which provided 

similar results to the track time above melt (TTAM) along the centerline of the individual tracks. 

1.2.1.1 PTAM Results

Figure 5 shows the PTAM results derived from the high-speed thermal imaging for the X-pads and Ypads, including histograms of each pixel’s PTAM value. It should be noted that histograms showed dual 

peaks, and non-normal distributions. The left peaks constitute lower PTAM pixel values which largely 

stem from residual spatter, plume, or other spurious features which depend highly on the spatterremoval preprocessing. While these features are present in the thermographic measurement, it is 

recognized they may not be present in computational simulation. 
AMB2022-03 Benchmark Measurements and Challenge Results – PDF Copy

8

Figure 5: Pad time-above-melt (PTAM) results for X pads and X pads. 

X pad 1 and X pad 2 were measured separately resulting in separate thermographic videos, as were 

Y pad 1 and Y pad 2. The CHAL-AMB2022-03-PTAM modelling challenge requested summary statistics 

for each X pad and Y pad pairs combined. Table 5 provides statistical results for each separate pad, and 

X pad and Y pad pairs combined. Additional metrics, including mode, skew, and kurtosis results are 

provided for reference.

Table 5: PTAM statistical results. Results that were requested for the CHAL-AMB2022-03-PTAM

challenge are in bold.

N [pixels] Mean [s] Median [s] Mode [s] Std. Dev. [s] Skew [-] Kurtosis [-]

X Pad 1 87020 4.83E-04 1.15E-04 1.01E-05 5.61E-04 0.72 1.72

X Pad 2 81042 5.06E-04 1.31E-04 1.01E-05 5.64E-04 0.61 1.56

X Pads combined 168062 4.94E-04 1.22E-04 1.01E-05 5.63E-04 0.66 1.64

Y Pad 1 39230 1.04E-03 1.24E-03 1.17E-03 5.02E-04 -0.80 2.27

Y Pad 2 39174 1.04E-03 1.24E-03 1.36E-03 5.04E-04 -0.82 2.27

Y Pads combined 78404 1.04E-03 1.24E-03 1.30E-03 5.03E-04 -0.81 2.27

1.2.1.2 PTAM Results, thresholded to > 0.5 ms.

Since spatter-removal parameters affect the prevalence of lower measured PTAM values, the left-most 

peaks in the histograms in Figure 5 were removed by removing PTAM values < 0.5 ms. Figure 6 gives 

examples of the resultant PTAM value histograms with values < 0.5 ms removed for X pad 1 and Y pad 2, 

and Table 6 gives summary statistics for the resultant distributions.

X pad 1, PTAM X pad 2, PTAM Y pad 1, PTAM Y pad 2, PTAM
AMB2022-03 Benchmark Measurements and Challenge Results – PDF Copy

9

Figure 6: Example histograms of PTAM results which exclude values below 0.5 ms.

Table 6: PTAM statistical results for values above 0.5 ms. Results that were requested for the CHALAMB2022-03-PTAM challenge are in bold.

N [pixels] Mean [s] Median [s] Mode [s] Std. Dev. [s] Skew [-] Kurtosis [-]

X Pad 1 31008 1.21E-03 1.25E-03 1.16E-03 2.27E-04 -0.80 3.30

X Pad 2 30985 1.20E-03 1.24E-03 1.21E-03 2.21E-04 -0.83 3.34

X Pads combined 61993 1.20E-03 1.24E-03 1.22E-03 2.24E-04 -0.81 3.32

Y Pad 1 30552 1.28E-03 1.31E-03 1.17E-03 2.46E-04 -0.61 3.73

Y Pad 2 30525 1.28E-03 1.32E-03 1.36E-03 2.45E-04 -0.68 3.72

Y Pads, combined 61077 1.28E-03 1.31E-03 1.30E-03 2.46E-04 -0.64 3.72

1.2.1.3 Note on modeling challenge evaluation

Measurement results were shown to depend on various parameters required for processing the thermal 

images. Additionally, residual features stemming from spatter or plume are not expected to be 

representative of the surface thermal history, and may skew summary statistics presented. Other 

thermal image processing parameters may provide improved relation to real surface thermal history, 

and more complex or locally variant quantitative metrics for comparing measurement and simulation 

results may be more appropriate. These issues will be considered when evaluating model challenge 

submission results.

1.2.2 CHAL-AMB2022-03-PSCR

The algorithm for analyzing the pad high speed thermal imaging is based on that described in the 

AMB2022-01 challenges for the 3D builds, as was mentioned in the AMB2022-03 challenge description. 

Similar thermal image pre-processing steps described above for pad time-above-melt challenge (CHALAMB2022-03-PTAM) were also used to evaluate pad solid cooling rate (PSCR). Figure 7 shows the PSCR 

results derived from the high-speed thermal imaging for the X-pads and Y-pads, including histograms of 

each pixel’s PSCR value. Similar to PTAM results, it should be noted that the selection of spatter-removal 

threshold parameters as well as the assumed emissivity affect the measured PTAM and PSCR results.

X Pad 1

PTAM > 0.5E-3 s

Y Pad 1

PTAM > 0.5E-3 s
AMB2022-03 Benchmark Measurements and Challenge Results – PDF Copy

10

Figure 7: Pad solid cooling rate (PSCR) results for X pads and X pads. Note that colorbar limits are 

selected for optimal image contrast, and values above 1 x 106°C/s are removed from the histograms.

PSCR results were non-normally distributed and included a small number of spuriously high cooling rate 

values which skewed mean values. Values above 1 x 106°C/s were removed prior to calculating 

statistical metrics in Table 7. Table 7 provides statistical results for each separate pad, and X pad and Y 

pad pairs combined. Additional metrics, including mode, skew, and kurtosis results are provided for 

reference.

Table 7: PSCR statistical results for values below 1 x 106°C/s. Results that were requested for the CHALAMB2022-03-PSCR challenge are in bold. 

N [pixels]

Mean 

[°C/s]

Median 

[°C/s]

Mode 

[°C/s]

Std. Dev. 

[°C/s] Skew [-] Kurtosis [-]

X Pad 1 47572 3.26E+05 2.65E+05 1.65E+05 1.73E+05 1.81 5.92

X Pad 2 45397 3.21E+05 2.61E+05 1.61E+05 1.70E+05 1.88 6.25

X Pads, combined 92969 3.24E+05 2.63E+05 1.61E+05 1.72E+05 1.84 6.08

Y Pad 1 37269 2.96E+05 2.36E+05 1.35E+05 1.67E+05 1.70 5.79

Y Pad 2 37146 2.87E+05 2.35E+05 1.37E+05 1.56E+05 1.77 6.28

Y Pads, combined 74415 2.91E+05 2.35E+05 1.35E+05 1.62E+05 1.74 6.04

1.2.2.1 Note on modeling challenge evaluation

Measurement results were shown to depend on various parameters required for processing the thermal 

images. Additionally, residual features stemming from spatter or plume are not expected to be 

representative of the surface thermal history, and may skew summary statistics presented. Other 

thermal image processing parameters may provide improved relation to real surface thermal history, 

and more complex or locally variant quantitative metrics for comparing measurement and simulation 

results may be more appropriate. These issues will be considered when evaluating model challenge 

submission results.

X pad 1, PSCR X pad 2, PSCR Y pad 1, PSCR Y pad 2, PSCR
AMB2022-03 Benchmark Measurements and Challenge Results – PDF Copy

11

1.2.3 CHAL-AMB2022-03-PMPG

Pad scan results were analyzed in terms of track number and cross-section position within the pad. The 

X-pad (BP2) contained 47 tracks and two cross-section positions. The Y-pad (BP3) contained 23 tracks 

and four cross-section positions. Odd and Even refer to odd and even track numbers. Representative 

micrographs for the two pad types are shown in Figure 8. The challenge results for depths and widths 

are provided in Table 8. Melt pool depths and widths versus track number are plotted for each pad type 

at one cross-section position in Figure 9. Qualitatively, the first three tracks in each pad had smaller 

depths. These were included in the Odd and Even averages. The Y-pad had a significant difference in 

melt pool height for Odd and Even track numbers. We note that the Y-pad, Even track numbers 

correspond to a parallel laser scan direction and gas flow, whereas, the Y-Pad, Odd track numbers 

correspond to an antiparallel configuration. The X-pad scan direction and gas flow direction are 

perpendicular for Odd and Even track numbers. The Even and Odd melt pool depth and width trends 

with cross-section position are plotted in Figure 10. A position of 0 mm corresponds to the starting side 

of the pad.

Figure 8. Representative cross-sectional dark field optical micrographs of pad scans: (a) X-pad, BP2-P2 

and (b) Y-pad, BP3-P3. Track numbers are listed for the final five tracks in each pad/image. The terms 

Odd and Even correspond to odd and even track numbers.
AMB2022-03 Benchmark Measurements and Challenge Results – PDF Copy

12

Table 8. CHAL-AMB2022-03-PMPG Results: pad depth (dp), pad width (wp), overlap depth (do), and 

overlap width (wo).

Sample ID and Track Group Mean 

dp (μm)

Std. 

dev. dp

(μm)

Mean 

do (μm)

Std. 

dev. do

(μm)

Mean 

wp (μm)

Std. 

dev. wp

(μm)

Mean 

wo (μm)

Std. 

dev. wo

(μm)

AMB-2022-718-SH1-BP2-P2-Odd 163.5 6.1 76.0 17.6 100.0 5.0 122.0 26.8

AMB-2022-718-SH1-BP2-P2-Even 155.8 4.7 91.8 7.8 86.2 4.2 87.1 4.2

AMB-2022-718-SH1-BP2-P4-Odd 166.0 7.0 79.2 18.3 95.1 6.6 113.3 25.5

AMB-2022-718-SH1-BP2-P4-Even 162.0 3.9 95.1 11.8 90.0 5.4 95.7 6.8

AMB-2022-718-SH1-BP3-P1-Odd 174.0 7.1 83.3 9.3 98.0 7.8 122.2 7.8

AMB-2022-718-SH1-BP3-P1-Even 124.7 10.7 89.2 8.9 83.0 3.5 89.0 4.2

AMB-2022-718-SH1-BP3-P2-Odd 175.2 5.9 76.1 7.8 102.3 6.7 131.4 6.2

AMB-2022-718-SH1-BP3-P2-Even 120.0 10.2 88.1 6.7 82.3 4.5 81.3 6.3

AMB-2022-718-SH1-BP3-P3-Odd 170.1 7.2 87.6 6.8 91.8 9.7 109.7 7.3

AMB-2022-718-SH1-BP3-P3-Even 137.1 7.7 91.1 8.8 86.6 5.1 101.7 6.9

AMB-2022-718-SH1-BP3-P4-Odd 175.1 6.7 82.3 6.7 100.8 9.0 123.9 8.1

AMB-2022-718-SH1-BP3-P4-Even 123.5 9.8 88.0 6.3 84.6 7.2 87.2 6.3

Figure 9. Melt pool depths and widths versus track number for pad scans: (a-b) Y-Pad, BP3-P3 and (c-d) 

X-pad, BP2-P2. Each data point is a single measurement.
AMB2022-03 Benchmark Measurements and Challenge Results – PDF Copy

13

Figure 10. Melt pool depth and width trends for pad scan Even and Odd tracks versus position within the 

pad: (a, c) X-Pad, BP2 and (b,d) Y-Pad, BP3. Data are the mean ± 95% confidence interval. n = 23 and 24 

for BP2 Even and Odd, respectively. n = 11 and 12 for BP3 Even and Odd, respectively.

2. Description and Links to Associated Data

Measurement results data associated with the AMB2022-03 challenges are provide in separate NIST 

Public Data Repository (PDR) datasets, accessible via the following DOI links to the PDR landing pages: 

• AM Bench 2022 Measurement Results Data: In-situ Thermography and Scan Strategy for Laserscanned Single Tracks and Pads on Bare In718 (AMB2022-03)” – https://doi.org/10.18434/mds2-

2716

• AM Bench 2022 Measurement Results Data: Optical and Scanning Electron Microscopy of Laserscanned Single Tracks and Pads (AMB2022-03) - https://doi.org/10.18434/mds2-2718

New data files, updates, and/or changes to download URLs may be made periodically. Users should refer 

to the README text file in each dataset page which will record all updates. Additionally, the NIST Public 

Data Repository (PDR) undergoes frequent updates. If file downloads fail or are unavailable, users 

should wait several hours before contacting the technical support listed on the PDR dataset webpage.
AMB2022-03 Benchmark Measurements and Challenge Results – PDF Copy

14

 

References

Citations are provided throughout this document as hyperlinked URLs to the associated digital object 

identifier (DOI). Clicking these hyperlinked text should open the associated publication or cited source.

†Disclaimer

The National Institute of Standards and Technology (NIST) uses its best efforts to deliver high-quality 

copies of the AM Bench database and to verify that the data contained therein have been selected on 

the basis of sound scientific judgment. However, NIST makes no warranties to that effect, and NIST shall 

not be liable for any damage that may result from errors or omissions in the AM Bench databases.

Certain commercial equipment, instruments, or materials are identified in this paper in order to specify 

the experimental procedure adequately. Such identification is not intended to imply recommendation 

or endorsement by NIST, nor is it intended to imply that the materials or equipment identified are 

necessarily the best available for the purpose.