AMB2022-01 Benchmark Measurements and Challenge Results

Last updated on 07/18/2022

Overview

The following describes compiled benchmark challenge measurement results which are used to judge 

submissions to the 2022 AM-Bench modeling challenges. The results presented here are summarized 

and formatted similar to how modelers were asked to submit their modeling results by July 15, 2022. 

Additional context, description, or measurement results may also be provided where necessary. 

Additional information may become available later so updated versions of these documents may be 

posted. Please check back occasionally.

Please note that the measurement results presented here are focused on the challenge problems and 

reflect only part of the validation measurement data which will be provided by AM Bench for each set of 

benchmarks.

AMB2022-01: Laser powder bed fusion (LPBF) 3D builds of nickel-based superalloy IN718 test objects. 

Detailed descriptions are found here (last updated 4/22/2022). An informational webinar for AMB2022-

01 and AMB2022-02 was held on May 5, 2022 and a link to the presentation is here.

Challenges

• Time Above Melting Temperature (CHAL-AMB2022-01-TAM): Time above solidus temperature 

for the melt pool at specified locations within the build volume. This metric is closely related to 

melt pool length but is explicitly location specific. See section 1.1.1 for more details.

• Solid Cooling Rate (CHAL-AMB2022-01-SCR): Cooling rate immediately following complete 

solidification (below solidus) at specified locations within the build. See section 1.1.2

• Residual Elastic Strains (CHAL-AMB2022-01-RS): Residual elastic strain components at select 

locations internal to the bridge structure, corresponding to synchrotron X-ray diffraction 

measurements. See section 1.2.1 for more details.

• Part Deflection (CHAL-AMB2022-01-PD): Deflection of the as-built (no heat treatment) bridge 

structure after it is partially separated from the build plate. See section 1.2.2 for more details.

• Microstructure (CHAL-AMB2022-01-MS): Histograms of direction-specific grain sizes from 

specified regions within as-built and heat-treated samples. See section 1.2.3 for more details.

• Phase Evolution (CHAL-AMB2022-01-PE): Formation and evolution of phases and phase 

fractions, including major precipitates, as a function of time for heat treatments of IN718 from a 

2.5 mm leg. See section 1.3.1 for more details.

_____________________________________________________________________________________

1. Description of Benchmark Challenge Results

1.1 In Situ Build Benchmarks

1.1.1 CHAL-AMB2022-01-TAM 
AMB2022-01 Benchmark Measurements and Result Descriptions – PDF Copy

2

The following algorithmic steps were taken to arrive at TAM and SCR results described in this document:

1. Perform spatter-removal algorithm on raw thermal camera signal.

a. Frame-by-frame spatter removal using threshold mask at 100 digital levels 

(approximately equivalent to T = 1077 °C at ε = 0.5).

b. Masking of part boundaries using threshold of 760 digital levels (approximately 

equivalent to T = 1260 °C at ε = 0.5).

2. Convert the spatter-removed thermal images to temperature using the camera calibration and 

estimated emissivity of ε = 0.5.

3. Proceed with calculation of TAM and SCR for each pixel as described in the AMB2022-01 

challenge description using transition temperature of Ttrans = 1298 °C.

Further details on the exact algorithms will be clarified in a future publication. It should be noted that 

the selection of spatter-removal threshold parameters as well as the assumed emissivity affect the 

measured TAM and SCR results. 

The choice in spatter-removal algorithm parameters, and emissivity value where chosen which provided 

similar results to the track time above melt (CHAL-AMB2022-03-TTAM) challenge along the centerline of 

the individual tracks. 

1.1.1.1 TAM Results

TAM results for the select layers are shown in Figure 1.

Figure 1: TAM values for each of the selected odd (left) and even (right) layers, aligned and superimposed on the part geometry 

stereolithography file (transparent red).

Similar to the no-powder ‘Pad’ thermographic challenge results in CHAL-AMB2022-03-PTAM, the 3D 

build TAM value distributions from multiple layers of the 3D build also showed a significant number of 

values less than ≈ 0.5 ms, as shown in Figure 2. Upper layers, starting at the bridge closure at layers 176 

and above, showed singular, more normal (e.g., Gaussian) distributions. For this reason, no values were 

Odd layers Even layers

Layer 5

Layer 75

Layer 175

Layer 251

Layer 6

Layer 76

Layer 176

Layer 250
AMB2022-01 Benchmark Measurements and Result Descriptions – PDF Copy

3

omitted, and the entire set of pixel TAM values are used to derive summary statistics, given in Table 1, 

without thresholding or removing any values as was done for CHAL-AMB2022-03-PTAM. 

Figure 2: Histograms for TAM values calculated from selected layers.

Table 1: Statistical results for select layers for the CHAL-AMB2022-01-TAM challenge.

Layer N-pixels1

TAM

Mean1

TAM

Median1

TAM

Mode2

TAM Std. 

Dev.1

TAM

Skew2

TAM

Kurtosis2

5 122935 1.26E-03 1.43E-03 3.62E-05 6.87E-04 -0.40 2.34

6 127614 9.90E-04 1.16E-03 3.62E-05 5.58E-04 -0.46 2.10

75 96809 9.41E-04 9.97E-04 3.62E-05 6.27E-04 0.20 2.42

76 100372 8.12E-04 8.70E-04 3.62E-05 5.63E-04 0.29 2.77

175 167758 1.29E-03 1.32E-03 3.62E-05 5.53E-04 0.00 4.88

176 160726 1.26E-03 1.25E-03 5.43E-05 4.02E-04 1.70 31.93

250 161030 9.42E-04 9.47E-04 3.62E-05 2.97E-04 0.12 6.99

251 167700 1.22E-03 1.25E-03 3.62E-05 5.23E-04 -0.12 4.26

1 Results requested for modelling challenge 

2 Additional results

1.1.1.2 Note on modeling challenge evaluation

Measurement results were shown to depend on various parameters required for processing the thermal 

images. Additionally, residual features stemming from spatter or plume are not expected to be 

representative of the surface thermal history, and may skew summary statistics presented. Other 

thermal image processing parameters may provide improved relation to real surface thermal history, 

and more complex or locally variant quantitative metrics for comparing measurement and simulation 

results may be more appropriate. These issues will be considered when evaluating model challenge 

submission results.

1.1.2 CHAL-AMB2022-01-SCR 

Layer 5 Layer 6

Layer 75 Layer 76

Layer 175 Layer 176

Layer 250 Layer 251
AMB2022-01 Benchmark Measurements and Result Descriptions – PDF Copy

4

Similar thermal image pre-processing steps described above for the 3D build time-above-melt challenge 

(CHAL-AMB2022-01-TAM) were also used to evaluate 3D build solid cooling rate (SCR). Figure 3 shows 

the SCR results derived from select layers of the 3D build. Similar to TAM results, it should be noted that 

the selection of spatter-removal threshold parameters as well as the assumed emissivity affect the 

measured TAM and SCR results.

Figure 3: SCR values for each of the selected odd (left) and even (right) layers, aligned and superimposed on the part geometry 

stereolithography file (transparent red).

Similar to the AMB2022-03-PSCR pad challenge results, the distribution of the 3D build SCR values was

non-normal, and included a small number of spuriously high cooling rate values that skewed the 

distribution mean. Figure 3 shows the histograms for SCR values for each select layer of the 3D build. 

For this reason, values above 1E6 °C/s are removed, with resulting statistical metrics given in Table 2. 

Additional metrics, including mode, skew, and kurtosis results are provided for reference.

Odd layers Even layers

Layer 5

Layer 75

Layer 175

Layer 251

Layer 6

Layer 76

Layer 176

Layer 250
AMB2022-01 Benchmark Measurements and Result Descriptions – PDF Copy

5

Figure 4: Histogram for TAM values for select layers of the 3D build.

Table 2: Statistical results for select layers for the CHAL-AMB2022-01-TAM challenge. Note that values above 1 x 106°C/s are 

removed from the histograms.

Layer N-pixels1SCR Mean1

SCR 

Median1

SCR 

Mode2

SCR Std. 

Dev.1

SCR 

Skew2

SCR 

Kurtosis2

5 106074 2.17E+05 1.63E+05 2.26E+05 1.65E+05 2.24 8.47

6 106085 2.50E+05 1.79E+05 7.96E+05 1.90E+05 1.81 5.89

75 82051 2.10E+05 1.27E+05 5.29E+05 2.06E+05 1.84 5.91

76 79966 2.45E+05 1.69E+05 1.01E+05 2.05E+05 1.66 5.30

175 154548 1.49E+05 8.97E+04 7.39E+04 1.61E+05 2.78 11.28

176 155732 1.68E+05 1.28E+05 1.43E+05 1.31E+05 2.92 13.87

250 152382 2.00E+05 1.31E+05 1.55E+05 1.80E+05 2.17 7.64

251 152301 1.57E+05 9.06E+04 1.04E+05 1.70E+05 2.63 10.12

1 Results required for modelling challenge 

2 Additional results

1.1.2.1 Note on modeling challenge evaluation

Measurement results were shown to depend on various parameters required for processing the thermal 

images. Additionally, residual features stemming from spatter or plume are not expected to be 

representative of the surface thermal history, and may skew summary statistics presented. Other 

thermal image processing parameters may provide improved relation to real surface thermal history, 

and more complex or locally variant quantitative metrics for comparing measurement and simulation 

results may be more appropriate. These issues will be considered when evaluating model challenge 

submission results.

Layer 5 Layer 6

Layer 75 Layer 76

Layer 175 Layer 176

Layer 251 Layer 250
AMB2022-01 Benchmark Measurements and Result Descriptions – PDF Copy

6

1.2 Ex Situ Benchmarks

1.2.1 CHAL-AMB2022-01-RS

Although elastic strains and stresses were characterized using synchrotron X-ray diffraction at both the 

APS and CHESS, neutron diffraction at HIFR, and the contour method by UC Davis and Hill Engineering, 

the CHAL-AMB2022-01-RS challenge problem targeted only the CHESS measurements on part 

AMB2022-718-AMMT-B7-P3. Figure 5 shows the 2248 synchrotron X-ray sample measurement positions 

in the XZ plane for the part. The sampled volume was centered on the midplane of the part along the Y 

axis as described in section 3.3 of the AMB2022-01 Challenge Problem Description Document. The 

unstrained lattice parameter was measured using two small specimens extracted from another as-built 

specimen from regions with low strain gradient. 

Figure 5: Diagram showing the 2248 synchrotron X-ray energy dispersive diffraction measurement positions. The 

measurement volume was centered midplane of the sample along the Y axis. Note that measurements were made 

on the 1st thick leg from the left, averaging the lattice spacings of the internal thin walls and powder. 

Elastic strains in both the X and Z directions were measured for each sample location. Following the 

same format as the original submission document, columns 1, 2, and 3 provide the nominal X, Y, and Z 

coordinates for each sample location. The X and Z origins are specified in Figure 5, and the origin for Y 

is defined as the plane closest to the reader based on Figure 5. The mid-plane of the sample has a value 

Y = + 2.5 mm. The measured XX elastic strain (xx) is provided in Column 4 and the measured ZZ elastic 

strain (zz) is provided in Column 5. Provided data files include:

AMB22_EDD_results_V2.xlsx: Spreadsheet including all the measured elastic strain values

AMB22_EDD_XXstrain.tif: Image file showing measured distribution of xx across the specimen

AMB22_EDD_ZZstrain.tif: Image file showing measured distribution of zz across the specimen

Figure 6 shows the measured elastic strains in the XX and ZZ directions for the as-built specimen.
AMB2022-01 Benchmark Measurements and Result Descriptions – PDF Copy

7

Figure 6: XX and ZZ elastic strains measured using energy dispersive diffraction.

All residual strain files may be found at https://doi.org/10.18434/mds2-2711.

1.2.2 CHAL-AMB2022-01-PD 

Results for the part defection measurements are presented in Figure . The points on the plot are the 

deflection calculated as the difference between the initial and final part measurements and the 

expanded uncertainty (k=2) of the deflection measurement results was 4 μm. A summary of the part 

deflection measurement results is available here: https://doi.org/10.18434/mds2-2711.

Figure 7: Deflection measurement results. The embedded picture shows the part deflection of the part after the legs are 

separated from the baseplate via EDM with the individual ridges labeled.

1.2.3 CHAL-AMB2022-01-MS 

1.2.3.1 2D Cross Sections 

1.2.3.1.1 Measurement locations: 
AMB2022-01 Benchmark Measurements and Result Descriptions – PDF Copy

8

2D cross sections through the as built and heat-treated samples were examined using Scanning Electron 

Microscopy (SEM) methods including Electron BackScatter Diffraction (EBSD), Energy Dispersive 

Spectroscopy (EDS), and Large Area Mapping (LAM). The cross sections include XZ and XY planes of the 

solid material, and YZ and XY planes of some thin walls internal to leg L10. Figure shows the XZ midplane cross section AMB2022-718-B7-P1-L7-L8-L9-O1 (as built). The regions outlined by the green, red, 

and yellow boxes designate locations of EBSD montages. The red and green regions extend at least 500 

μm into the baseplate. Table 3 shows the sample IDs for the various cross sections that were measured. 

For CHAL-AMB2022-01-MS, only the LAM data are required and example data are shown below, with 

the full datasets available at https://doi.org/10.18434/mds2-2692.

Figure 8: Approximate SEM measurement locations for AMB2022-718-AMMT-B7-P1-L7-L8-L9-O1

Table 3: Sample IDs for the measured cross sections

Sample ID Cross section plane Sample condition

AMB2022-718-AMMT-B7-P1-L7-L8-L9-O1 XZ As built

AMB2022-718-AMMT-B6-P2-L7-L8-L9-O1 XZ Heat treated

AMB2022-718-AMMT-B8-P3-L9 XY As built

AMB2022-718-AMMT-B8-P3-L10-W3 XY As built

1.2.3.1.2 Sample preparation:

After EDM cutting, samples were prepared by progressive polishing from 600 grit SiC paper down to 1 

m diamond suspension. Final surface preparation used vibro-polishing with 0.05 m non-colloidal silica 

for 16 h. Samples were cleaned in an ultrasonic bath in three solutions: soap/water, water, and ethanol.

1.2.3.1.3 Large area mapping:

L9 L8 L7
AMB2022-01 Benchmark Measurements and Result Descriptions – PDF Copy

9

Imaging and data acquisition were performed using a JEOL Field Emission JSM7100 with the Oxford 

Symmetry S2 EBSD detector with fore-scatter diodes and the Oxford Ultim Max EDS 100 mm2silicon 

drift detector. Oxford Aztec software Ver 6.0 was used for EDS/EBSD analysis and to create Large Area 

Maps. The measurement conditions are listed in Table 4. 

Table 4: SEM settings used for the 2D cross section measurements

SEM 

settings

20 kV 10 nA 70 deg tilt Working distance ~ 

25 mm

400 x magnification

Field of view = 300 

pixels x 226 pixels 

(0.3 mm x 0.23 

mm)

1.2.3.1.4 Example data:

Figure 9 shows an inverse pole figure map in the X direction (IPF-X) of the L9 region (red box in Fig. 8) for 

the as-built condition. Measurements started deep enough into the baseplate to characterize the 

interface between the markedly different microstructures. The sampling resolution is 1 m so the small 

grains within the baseplate are fully resolved. In contrast, Fig. 10 shows the corresponding IPF-X map 

for the fully heat-treated sample. The marked difference in grain growth behavior between the AMproduced material and the rolled substrate is likely due to grain boundary pinning rather than stress 

annealing. As demonstrated by the residual strain measurements described above, the elastic strains 

and stresses in the as-built material are very large and the baseplate underwent a stress relieving heat 

treatment prior to the build. 
AMB2022-01 Benchmark Measurements and Result Descriptions – PDF Copy

10

Figure 9: IPF-X montage covering the entire L9 from an as-built specimen
AMB2022-01 Benchmark Measurements and Result Descriptions – PDF Copy

11

Figure 10: IPF-X montage covering the entire L9 from a fully heat-treated specimen
AMB2022-01 Benchmark Measurements and Result Descriptions – PDF Copy

12

The published data at https://doi.org/10.18434/mds2-2692 include IPF-X, IPF-Y, and IPF-Z images for all 

three regions of both the as-built and heat treated samples, along with all of the angular EBSD data in 

different formats. Some data from XY cuts through the midplane of L9 are also included.

1.2.3.1.5 Grain boundary intercept histograms:

There are many possible metrics that can be used to evaluate the degree of similarity between 

measured and simulated microstructures. For this challenge problem, we used a histogram of chord 

lengths obtained from multiple parallel lines intersecting grain boundaries along the orthogonal 

directions. Detailed descriptions of this algorithm are given in section 4.2.3 of the AMB2022-01

Challenge Problem Description Document. 

The measurement data used for this challenge problem are the extended XZ-plane large area EBSD 

maps acquired from AMB2022-718-AMMT-B7-P1-L7-L8-L9-O1 (as built) and AMB2022-718-AMMT-B6-

P2-L7-L8-L9-O1 (heat treated) as shown in Fig. 8. 

Three 1 mm x 1 mm square regions from the as built sample and two 1 mm x 1 mm square regions from 

the heat-treated sample were used for the challenge problem. As shown in Figure 11, these include a 

region at the base of a 2.5 mm leg, a second region toward the top of the same 2.5 mm leg, and a third 

region within the bridge section.

Figure 11: Diagram showing the sizes and locations of the modeling challenge regions for CHAL-AMB2022-01-MS

For this challenge problem, adjacent pixels with at least a 10° misorientation angle define a grain 

boundary. The line spacing between the parallel lines is 5 m and the width of the bins for the linear 

intercept length histograms is 100 m. The provided measurement data include histograms in the same 

format as the challenge problem submission template found in the /ChallengeSubmissionTemplates/ 

directory of the AMB2022-01 challenge description dataset.

Solution histograms for this challenge problem may be found with the microstructure datasets available 

at https://doi.org/10.18434/mds2-2692.

1.3 In Situ Heat Treatment Benchmark (includes as built and final heated conditions)

1.3.1 CHAL-AMB2022-01-PE
AMB2022-01 Benchmark Measurements and Result Descriptions – PDF Copy

13

Phase fractions were measured in situ during thermal processing using a combination of in situ 

synchrotron X-ray measurements on beamline 9ID-C at the Advanced Photon Source and ex situ phase 

identification measurements using transmission electron microscopy (TEM). With a time resolution of 

about four minutes, the USAXS instrument at the APS uses small-angle scattering to obtain information 

about a sample’s microstructure over a size range from micrometers to below one nanometer while 

simultaneously using diffraction to probe the phases present. The instrument uses ultra-small-angle Xray scattering (USAXS) to study the larger length scales, pin-hole small-angle X-ray scattering (SAXS) to 

study the smaller length scales, and wide-angle scattering (X-ray diffraction) to obtain the phase 

information, including the most common precipitates. Detailed information on this unique instrument 

and its use may be found in Ilavsky et al. 2018, and Ilavsky et al. 2012. These transmission 

measurements use thin transverse specimens cut from 2.5 mm legs. Conventional metallographic 

polishing methods are used to thin these specimens to approximately 35 μm. Measurements were 

conducted in situ during both homogenization and precipitation thermal processing.

Definitive Identification of the various diffraction peaks in the synchrotron X-ray data requires detailed 

ex situ TEM measurements. These measurements were delayed due to license issues that interrupted 

operation of numerous NIST TEMs for several months. Thus, the current phase identification results 

should be considered preliminary. In particular, a phase that appears in the as-built material and 

dissolves during the homogenization treatment has not yet been positively identified. Another caveat is 

that the measured diffraction peaks of the γ and γ’ phases overlap to the extent that these phases 

cannot be reliably distinguished. Therefore, we report the combination γ + γ’ in the CHAL-AMB2022-01-

PE answer key. We expect to release updated results for this challenge problem in the next few weeks.

Solution results for this challenge problem may be found with the microstructure datasets available at 

https://doi.org/10.18434/mds2-2692.

2. Description and Links to Associated Data

Measurement results data associated with the AMB2022-01 challenges are provided in separate NIST 

Public Data Repository (PDR) datasets, accessible via the following DOI links to the PDR landing pages: 

• AM Bench 2022 Measurement Results Data: In-situ thermography and Scan Strategy for 3D Builds 

(AMB2022-01) – https://doi.org/10.18434/mds2-2715

• AM Bench 2022 Microstructure Measurements for IN718 3D Builds –

https://doi.org/10.18434/mds2-2692

• AM Bench 2022 Residual Elastic Strain, Residual Stress, and Part Deflection Measurements for IN718 

3D Builds – https://doi.org/10.18434/mds2-2711

New data files, updates, and/or changes to download URLs may be made periodically. Users should refer 

to the README text file in each dataset page which will record all updates. Additionally, the NIST Public 

Data Repository (PDR) undergoes frequent updates. If file downloads fail or are unavailable, users 

should wait several hours before contacting the technical support listed on the PDR dataset webpage. 

Note that the measurement results datasets are separate from the “Challenge Description” datasets 

previously made available, but may contain similar data. Challenge description dataset “AM Bench 2022 

3D Build Modeling Challenge Description Data (AMB2022-01)” is available here: 

https://doi.org/10.18434/mds2-2607. 
AMB2022-01 Benchmark Measurements and Result Descriptions – PDF Copy

14

References

Citations are provided throughout this document as hyperlinked URLs to the associated digital object 

identifier (DOI). Clicking these hyperlinked text should open the associated publication or cited source.

†Disclaimer

The National Institute of Standards and Technology (NIST) uses its best efforts to deliver high-quality 

copies of the AM Bench database and to verify that the data contained therein have been selected on 

the basis of sound scientific judgment. However, NIST makes no warranties to that effect, and NIST shall 

not be liable for any damage that may result from errors or omissions in the AM Bench databases.

Certain commercial equipment, instruments, or materials are identified in this paper in order to specify 

the experimental procedure adequately. Such identification is not intended to imply recommendation 

or endorsement by NIST, nor is it intended to imply that the materials or equipment identified are 

necessarily the best available for the purpose.