# DIC Challenge 2.0: Developing Images and Guidelines for Evaluating Accuracy and Resolution of 2D Analyses
## Focus on the Metrological Efficiency Indicator

Philipp L Reu, Benoît Blaysat, Edward Andò, Kaushik Bhattacharya, Cyrille Couture, Vincent Couty, Debasis Deb, Samuel S Fayad, Mark A Iadicola,

Stéphanie Jaminion, et al.

► To cite this version:

Philipp L Reu, Benoît Blaysat, Edward Andò, Kaushik Bhattacharya, Cyrille Couture, et al. DIC Challenge 2.0: Developing Images and Guidelines for Evaluating Accuracy and Resolution of 2D Analyses Focus on the Metrological Efficiency Indicator. Strain, 2022, 10.1007/s11340-021-00806-6. [https://hal.science/hal-03427288](https://hal.science/hal-03427288)

HAL Id: hal-03427288

https://hal.science/hal-03427288

Submitted on 13 Nov 2021

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# DIC Challenge 2.0: Developing Images and Guidelines for Evaluating Accuracy and Resolution of 2D Analyses
## Focus on the Metrological Efficiency Indicator

P.L. Reu · B. Blaysat · E. Andò · K. Bhattacharya ·

C. Couture · V. Couty · D. Deb · S.S. Fayad ·

M.A. Iadicola · S. Jaminion · M. Klein · A.K. Landauer ·

P. Lava · M. Liu · L.K. Luan · S.N. Olufsen · J. Réthoré ·

E. Roubin · D.T. Seidl · T. Siebert · O. Stamati ·

E. Toussaint · D. Turner · C.S.R. Vemulapati ·

T. Weikert · J.F. Witz · O. Witzel · J. Yang ·

Received: date / Accepted: date

P.L. Reu · S.S. Fayad · D.T. Seidl · D. Turner ·
Sandia National Laboratory, Albuquerque, NM, USA

B. Blaysat · E. Toussaint ·
Université Clermont Auvergne, CNRS, SIGMA Clermont, Institut Pascal, France
Tel.: +33-473288131 - E-mail: benoit.blaysat@uca.fr

E. Andò · E. Roubin · O. Stamati · C. Couture ·
Université Grenoble Alpes, CNRS, Grenoble INP, 3SR, F-38000 Grenoble, France

K. Bhattacharya ·
Division of Engineering and Applied Science, California Institute of Technology, Pasadena, CA, USA

V. Couty · JF. Witz ·
Univ. Lille, CNRS, Centrale Lille, UMR 9013 LaMcube Laboratoire de mécanique multiphysique et multiéchelle, F-59000, Lille, France

D. Deb · C.S.R. Vemulapati ·
Indian Institute of Technology, Kharagpur, West Bengal, India

M.A. Iadicola ·
Material Measurement Laboratory, National Institute of Standards and Technology, Gaithersburg, MD, USA, 20899

S. Jaminion ·
CorreliSTC, France

M. Klein · O. Witzel · T. Weikert ·
GOM, GmbH, Braunschweig, Germany

A.K. Landauer ·
School of Engineering, Brown University, Providence, RI, USA, 20912 and Material Measurement Laboratory, National Institute of Standards and Technology, Gaithersburg, MD, USA, 20899

P. Lava ·
MatchID NV, Gent, Belgium

M. Liu ·
Physics and Engineering Department, Washington and Lee University, Lexington, VA 24450

L.K. Luan ·
Dantec Dynamics GmbH, Ulm, Germany

S.N. Olufsen ·
Centre for Advanced Structural Analysis (CASA), Department of Structural Engineering, NTNU, Norwegian University of Science and Technology, NO-7491, Trondheim, Norway

J. Réthoré ·
Research Institute in Civil Engineering and Mechanics (GeM), UMR 6183, CNRS, Ecole Centrale de Nantes, Université de Nantes, France

T. Siebert ·
LaVision GmbH, Goettingen, Germany
Correspondence to: B. Blaysat - E-mail: benoit.blaysat@uca.fr

**Abstract** Background: The DIC Challenge 2.0 follows on from the work accomplished in the first Digital Image Correlation (DIC) Challenge [1]. The second challenge was required to better quantify the spatial resolution of 2D-DIC codes. Objective: The goal of this paper is to outline the methods and images for the 2D-DIC community to use to evaluate the performance of their codes and improve the implementation of 2D-DIC. Methods: This paper covers the creation of the new challenge images and the analysis and discussion of the results. It proposes a method of unambiguously defining spatial resolution for 2D-DIC and explores the tradeoff between displacement and strain noise (measurement resolution) and spatial resolution for a wide variety of DIC codes by a combination of the images presented here and a performance factor called Metrological Efficiency Indicator (MEI). Results: The performance of the 2D codes generally followed the expected theoretical performance, particularly in the measurement of the displacement. The comparison did however show that even with fairly uniform displacement performance, the calculation of the strain spatial resolution varied widely. Conclusions: This work provides a useful framework for understanding the tradeoff and analyzing the performance of the DIC software using the provided images. It details some of the unique errors associated with the analysis of these images, such as the Pattern Induced Bias (PIB) and imprecision introduced through the strain calculation method. Future authors claiming improvements in 2D accuracy are encouraged to use these images for an unambiguous comparison.

**Keywords** Digital Image Correlation · DIC · DIC Challenge · metrology · full-field measurement

# 1 Introduction

Digital Image Correlation (DIC) is a full-field optical measurement technique widely used in experimental mechanics for quantitative measurements in material testing. 2D-DIC uses a single camera to acquire an image used to measure in-plane displacement by correlation of the grey levels between a reference and a deformed image. Strain is subsequently calculated from this displacement field. In engineering, DIC is often used in material testing, where calculating the displacement and strain are important in characterizing the material. The focus is made here on the assessment of the metrological performance of the DIC codes themselves. The proposed method relies on synthetic images to focus only on the error associated with the image processing. Other experimental sources of error, such as the camera or misalignment and lens aberrations, are thus not taken into account. They are, however, expected to impact all the full-field measurement techniques studied here in the same manner. In practice, the experimentalist must be aware of these error sources and minimize them with meticulous experiments to reach the metrological performance limits shown in this work. The first DIC challenge [1] generated 19 image sets that can be used for DIC software verification and validation. Verification determines whether the DIC software has been implemented correctly. Validation checks whether DIC returns measurements that are an accurate reflection of the physical reality. For the first DIC Challenge, verification images were synthetically generated with two different numerical methods to test common issues associated with 2D-DIC, specifically for measuring displacement or strain. The validation images were gathered from experiments with supplementary displacement measurements to compare with the DIC displacement measurements. All are encouraged to use the images for publications, particularly when claiming improvements in DIC algorithms.

While useful, the first DIC Challenge did not provide images with spatial variations of the displacement or the strain that were sufficient to thoroughly evaluate the displacement and strain filtering associated with 2D-DIC codes. This deficiency led to the creation of the DIC Challenge 2.0, with images designed to easily visualize and assess the spatial resolution of DIC.

## Spatial Resolution and Metrological Efficiency Indicator

Because both the DIC displacement calculation and the strain calculation are low-pass spatial filtering operations, attenuation bias can occur if the Quantity of Interest (QOI) varies significantly over a small region of the image. Considering harmonic displacements as an illustration, the amount of attenuation
<figure><img src="image_2.png" /><figcaption>Fig. 1: Illustration of DIC acting as a low-pass filter on a 1D example. Blue is the original unfiltered signal (ground truth or GT) and red is the filtered DIC results. The filtering is calculated using a Savitzky-Golay (SG) filter approximation of the first order. See appendix for more information. The SG filter is defined to represent a subset size of 17 pixels and linear shape function. Signal (a) represents a rapid QOI change over 50 pixels. The two dashed lines represent the summation of either the Fourier GT signals or the SG signals. Figures (b) to (g) show the first six frequencies of its Fourier series. In other words, the sum of right-hand side data represents the ramp function in (a). Quantity “Att.” is peak-to-peak attenuation relative to the GT. High frequency signals are undeniably attenuated by the low-pass filtering effect of DIC.</figcaption></figure>

bias is related to the spatial period and magnitude of the displacements. The shorter the spatial period of the displacement the higher the frequency and the more attenuation occurs. This is illustrated in a 1D example in Figure 1(a) using a displacement change of 1 pixel over 50 pixels to represent a rapidly changing QOI. The Fourier transform represents this ramp transition as a series of sine waves. The first six frequencies of the transform are shown in Figure 1(b-g) progressing from the lowest frequency (b) to the highest frequency in (g). The blue curve is the ground-truth signal, representing an unfiltered signal. The red curve represents the filtered DIC signal of a 17 pixels subset DIC code using a first-order shape function. The DIC filtering is calculated using a first order Savitzky-Golay (SG) filter with a width of 17 pixels (see appendix for more details) [2]. The progressive attenuation of the signals, from no attenuation in (b) to a highly attenuated signal in (g) clearly illustrates the low-pass filtering effect of the DIC calculation. The peak-to-peak attenuation amount is shown in the figure title as a percent of the ground-truth value. The lower frequency (Mode 1) exhibits almost no attenuation at 5%, while the highest frequency (Mode 11) exhibits 92% attenuation. This filtering effect is what causes the sharp displacement change in the GT (Figure 1(a)) to exhibit a rounded transition in the SG filter data. The sharp displacement change at the beginning and end of the transition contain the information with the highest frequencies, while the more gradually sloped portion the lowest frequencies. DIC low-pass filtering fundamentally smooths high frequencies and thus limits rapid changes in the QOI. In other words, it limits the spatial resolution of the measurement, or “The minimum distance between two localized features that can be independently resolved.” DIC low-pass filtering fundamentally limits the spatial resolution of the measurement, or “The
minimum distance between two localized features that can be independently resolved.” as defined in the
DIC Good Practices Guide (GPG) [3] available from (www.idics.org).

The strain spatial resolution is also affected because of both the bias in the displacement measurements
as just discussed, but also because the strain approximations explicitly include spatial filtering during their
calculation from the displacement fields. This calculation varies depending on the implementation of the
code, but for many methods, the strain calculation functions as a second low-pass filter.

It is worth noting that DIC deals with images that are always impaired with pixelwise acquisition noise. DIC spatial filtering does have the benefit of reducing the propagation of this noise to the kinematic measurement but may create biases as just discussed. The ability of DIC to resolve ever smaller periods of the QOI is, therefore, in competition with the measurement resolution also called displacement or strain noise. Indeed, more filtering of the DIC data results in a lower overall noise of the QOI, but at the expense of decreased spatial resolution. It is thus of great interest to understand the relationship between the spatial resolution, bias, and the measurement resolution or QOI noise. The first DIC challenge explored this using Sample 14 and Sample 15 that were analyzed in detail in that paper [1]. However, Sample 14 did not provide small enough displacement or strain periods to adequately challenge the 2D-DIC codes. While Sample 15 had extremely small periods, i.e., a ramp function, it did not help visualize the filtering of the DIC code. The 2D Challenge 2.0 images were created to fix this problem by providing displacement and strain gradient images that have appropriate spatial periods to investigate the tradeoff between bias and measurement resolution in DIC. We will express the results using the Metrological Efficiency Indicator (MEI) to score the measurements. The MEI was originally proposed in [4] for the Localized Spectral Analysis (LSA, or Grid Method) and is the product of the spatial resolution at a given bias amount and the measurement resolution, where measurement resolution is defined as $1\sigma$ of the noise floor. The lower the value of the MEI, the better the performance of the code. It has been extended to local DIC [5] and global DIC [6]. Since it does not depend on the DIC analysis parameters associated which each measurement technique, such as subset or element size, this indicator enables direct comparison between different DIC codes [6].

## Paper content and organization

The paper presents the results of the DIC Challenge 2.0. Section 2 describes the data, i.e., images, that constitute the DIC Challenge 2.0, the participants, the collected data, i.e., displacement and strain measurements, and illustrates the full-field maps. Section 3 details the proposed strategy for the assessment of the metrological performances of the collected measurements. Section 4 assesses and discusses the metrological performances of the provided DIC outputs. Conclusions and future work are then discussed in Section 5.

# 2 DIC Challenge 2.0: Image Creation, Participation and Collected data

## 2.1 Image Creation

Table 1 lists the DIC Challenge 2.0 image sets available at the website [7]. These images are freely downloadable and can be used with attribution in any journal publication. Their maintenance is supported by the Society for Experimental Mechanics (www.sem.org) and the International DIC Society (www.idics.org). There are six image sets, including images with and without added image noise.

Star 1 is a ±0.5 pixel sinusoidal vertical displacement amplitude “star” pattern with varying spatial frequency. The images are noise free. Noise-free in this context means that no image noise was added to the simulation, beyond the truncation of the pixel intensity when converting it from double-precision to discrete 8-bit integers as part of the image generation process (see Section 2.2 “Image generation – the Boolean model”). The inset figure for Star 1 (Table 1) illustrates the frequency variation across the image. The left of the image has a period of 10 pixels for the sinusoid decreasing towards the right hand of the image to a period of 150 pixels<fnref n="1" />. Period, defined as 1/frequency, is more natural to use for discussing the variation of the displacement gradients in the image as it gives the distance between points that can be measured, i.e., spatial resolution. The Star 1 data set contains only a reference and a deformed image. Star 2 is the same

<footnote n="1">¹ Period $\lambda$ exactly linearly evolves from 10.07 pixels (left-hand side) to 149.93 pixels (right-hand side). It satisfies the formula $\lambda = 10 + \frac{150-10}{2000}x$, with $x \in [1, 2000]$.</footnote>
<figure><img src="image_3.png" /><figcaption>Fig. 2: Heteroscedastic image acquisition noise profile used for the 2D Challenge images for Star 2 and Star 4. This is constructed specifically using the noise profile on data from a PointGrey Grasshopper 5 megapixel, 8-bit, monochrome CCD camera. The left axis shows the grey level variance superimposed for each pixel value found in an image, while on the right axis the variance is normalized by the signal intensity (i.e., grey level). Simulated camera noise is elaborated as detailed in [15].</figcaption></figure>

deformation field as Star 1, but with a heteroscedastic image acquisition noise, modeled specifically on a FLIR (formerly PointGrey) 5 megapixel camera (GRAS-50S5C-C). The noise profile is shown in Figure 2. In what follows, image noise refers to this heteroscedastic noise. Star 2 contains three images, a noisy reference, an undeformed noise floor image, and a deformed noisy image. The undeformed noise floor image is used to calculate the measurement resolution of the QOI in the absence of motion, while the deformed image is used to calculate the spatial resolution. Star 3 is a noise-free ±5% Lagrangian vertical ($ε_{yy} = ±0.05$ [–]) sinusoidal strain field with varying spatial period. The spatial period also linearly varies from 10 pixels to 150 pixels across the image. Only two images are provided, a reference and a deformed image. Star 4 is the same strain deformation image with noise added (Figure 2). Three images are again provided, a reference, an undeformed noise floor image and a deformed image to measure the spatial resolution. After a preliminary analysis revealed that the DIC filtering was creating too much attenuation at even the largest period, two more image sets were added that were twice the width in order to include lower periods. Star 5 is like Star 2, with noise and a ±0.5 pixel vertical displacement field. Star 6 is Star 4 with noise and a vertical strain field of ±5% (Lagrangian). Star 5 and Star 6 both have three images, a noisy reference image, a noise floor image and a noisy deformed image used to calculate the spatial resolution. Star 5 and Star 6 in Table 1 are 4000 × 500 pixels and their underlaying displacements have periods, $λ$, varying linearly between 10 pixels at the first pixel, $x = 1$ pixels, to 300 pixels at $x=4000$ pixels according to Eq. 1.

$$ \lambda = 10 \frac{300 - 10}{4000} (x - 1) \qquad (1) $$

All the inset results in Table 1 are “typical” DIC results and not the theoretical displacement or strain fields. The applied fields all continue smoothly across the image. The results shown in this paper illustrate the typical effect of filtering of the DIC codes (Figure 4) – and are the prime topic of this paper.

## 2.2 Image generation - the Boolean model

The methods used for image generation are important when using them to test the accuracy of DIC codes and assess their metrological performance. At a minimum, they should not introduce any significant bias and preferably have realistic patterns and image noise applied. In the literature, several synthetic image generation techniques exist [8,9,10,11,12]. They begin with a reference image $I_{ref}$ defined over a domain $Ω$. Next, a deformed image, $I_{cur}$, is created that corresponds to the reference image but deformed with a
<figure><img src="image_4.png" /><figcaption>Table 1: DIC Challenge 2.0 Image sets are available for download at: https://sem.org/dicchallenge/. There are 6 image sets as outlined in the table. The images show example computed displacements from DIC.</figcaption></figure>

ground truth displacement $u_{GT}$. This deformed image is mathematically represented by:

$$ \forall x \in \Omega, I_{cur}(x + u_{GT}(x)) = I_{ref}(x) \Rightarrow I_{cur}(x) = I_{ref}(\phi^{-1}(x)), \text{ with } \phi(x) = x + u_{GT}(x). \quad (2) $$

$I_{ref}$ is defined pixelwise. When the displacement is a rigid body motion, the calculation of $I_{cur}$ can be performed by a single phase shift of the image in the Fourier domain [13]. However, when the displacement varies across the image (i.e., there is strain), more complex image interpolation is required. To reduce the bias that this interpolation introduces, some approaches use a binning approach [8, 14], where $\tilde{I}_{ref}$ and $\tilde{I}_{cur}$ are defined on a super-resolved pixel grid. The calculations of $I_{ref}$ and $I_{cur}$ are then realized by summation.

The synthetic image generation adopted here relies on a closed-form expression of the image displacements and corresponding grey values to prevent any bias caused by the interpolation. The closed-form model corresponds to a Boolean model, which creates a realistic pattern consisting of black speckles on a white background. The black speckles are circles, whose center coordinates were uniformly distributed on the image domain, and whose radii follow an exponential distribution. An integration is then performed from this infinite spatial resolution image to the pixelwise image. A Monte Carlo scheme is introduced, with a number of realizations ensuring that integration error is lower than the rounding error that corresponds to the gray value truncation when converting to 8-bit images. The code used for the generation of the data set is available here: https://members.loria.fr/FSur/software/BSpeckleRender/ [15]. The parameters used for defining the Boolean model and its generation are given in Table 2.
<table><caption>Table 2: Parameters chosen for the image generations.</caption><thead><tr><th></th><th>Star 1 & 4</th><th>Star 5 & 6</th></tr></thead><tbody><tr><td>Image size (pixel × pixel)</td><td>501 × 2000</td><td>501 × 4000</td></tr><tr><td>Sample size to estimate</td><td>1001</td><td>1001</td></tr><tr><td>Quantization error probability</td><td>0.1</td><td>0.1</td></tr><tr><td>Size of the largest Monte Carlo sample</td><td>2<sup>24</sup></td><td>2<sup>24</sup></td></tr><tr><td>Sensor bit depth</td><td>8</td><td>8</td></tr><tr><td>Standard deviation of the Gaussian PSF (pixel)</td><td>1</td><td>1</td></tr><tr><td>Probability distribution of the radii</td><td>Exponential</td><td>Exponential</td></tr><tr><td>Average radius of disks (pixel)</td><td>0.5</td><td>0.5</td></tr><tr><td>Average number of disks per image</td><td>556667</td><td>1113333</td></tr><tr><td>Contrast of the speckle</td><td>0.6</td><td>0.6</td></tr></tbody></table>

<table><caption>Table 3: DIC Challenge 2.0 Image participants (in no particular order) and type of code. Codes in the text are ordered Local.Affine.A, etc. and do not correspond to the order of the table.</caption><thead><tr><th>Participant</th><th>Code Name</th><th>Code type</th><th>Weblink</th></tr></thead><tbody><tr><td>D. Turner<br>D.T. Seidl</td><td>DICe</td><td>Local and<br>global</td><td>DICe on github</td></tr><tr><td>E. Toussaint</td><td>SeptD</td><td>Local</td><td></td></tr><tr><td>B. Blaysat</td><td>BibDIC</td><td>Local</td><td></td></tr><tr><td>J. Yang</td><td>ALDIC [17]</td><td>Miscellaneous</td><td>ALDIC on Matlab central</td></tr><tr><td>T. Siebert</td><td>LaVision</td><td>Local</td><td>https://www.lavision.de/en</td></tr><tr><td>P. Lava</td><td>MatchID</td><td>Local</td><td>https://www.matchid.eu/</td></tr><tr><td>M. Simonsen</td><td>Vic2D</td><td>Local</td><td></td></tr><tr><td>L.K. Luan</td><td>Dantec</td><td>Local</td><td>Dantec website</td></tr><tr><td>C. Boo</td><td>Image<br>Systems</td><td>Local</td><td></td></tr><tr><td>S. Jaminion</td><td>CorreliSTC</td><td>Global</td><td>https://www.correli-stc.com</td></tr><tr><td>T. Weikert</td><td>GOM</td><td>Miscellaneous</td><td>GOM website</td></tr><tr><td>M. Liu</td><td>NCORR [16]</td><td>Miscellaneous</td><td>http://www.ncorr.com</td></tr><tr><td>J. Réthoré</td><td>UFreckles</td><td>Global</td><td>UFreckles on zenodo</td></tr><tr><td>V. Couty</td><td>DISflow</td><td>Miscellaneous</td><td>DISflow on opencv</td></tr><tr><td>C. Sai Reddy</td><td>FEDIC</td><td>Global</td><td>https://www.fe-dic.com</td></tr><tr><td>A. Landauer</td><td>qDIC [18]</td><td>Miscellaneous</td><td>qDIC on github</td></tr><tr><td>S.N. Olufsen</td><td>μDIC [19]</td><td>Global</td><td>μDIC on github</td></tr><tr><td>E. Andò</td><td></td><td></td><td></td></tr><tr><td>E. Roubin</td><td>Spam 0.5.3.3</td><td>Local</td><td>Spam on pypi</td></tr><tr><td>O. Stamati</td><td></td><td></td><td></td></tr></tbody></table>

## 2.3 Participation

The DIC challenge was open to all participants with the goal to ensure that the commercial DIC codes were well represented as well as open source (e.g., DICe and Ncorr [16]), university codes (e.g., ALDIC [17]), and global codes (e.g., Correli). In addition to the variety of code implementations, there were variations on the analysis that were used, including subset weighting and higher-ordered shape functions, that are listed without attribution. These variations are important to include as they influence the DIC results. Table 3 lists the codes participating. The codes are not listed in order and are uniquely identified throughout the paper with a code identifier. Sample code identifiers are Local.Affine.A (indicating a local DIC code with affine subset shape functions), Local.Quad.A (local DIC code with quadratic shape functions), Misc.A (not enough information provided to categorize), and Global.A (global DIC code with varying implementations).
<figure><img src="image_5.png" /><figcaption>Fig. 3: (a) Vertical displacement obtained with Local.Affine.B along the line-cut obtained with a subset size of 17 pixels. The spatial period of the vertical sine wave used for the image generation (top axis) is proportional to the image position across the width of the image (bottom axis). (b) Line-cut location in image at the center row with a fixed amplitude of ±0.5 pixel with the DIC results overlaid.</figcaption></figure>

## 2.4 Data collected

This section illustrates the results from the Star 5 and Star 6 images. The larger images were needed to ensure that the spatial period was long enough that all codes could process these images and almost reach the ground truth displacement using a range of software settings. Because there were multiple software settings that could be varied, we fixed the subset size for the local codes to between 9 × 9 pixels to 59 × 59 pixels in increments of 10 pixels. For most implementations, this range yielded a variety of filtering and noise results. The global codes were asked to vary the element size (or other similar filtering parameter) to produce a range of results from the highest possible spatial resolution to one with “reasonable” filtering. A similar directive was given for the virtual strain gauge (VSG) size: “Choose any subset or variety of subset or element sizes to produce results that range from the highest spatial resolution to the lowest”. This vague direction was required because of the wide variety of strain calculation implementations that were employed by the participating codes.

The results were created by the respective code owners/experts and submitted to the DIC Challenge committee via a comma separated file (.csv). The analyzed results were selected from the center row (referred to here as a line-cut) of the DIC image results and reported with a 1 pixel horizontal spacing for both the noisy undeformed image and the noisy deformed image. The file starts with a short header that contains information on the subset, element size, VSG size or other parameters deemed important by the participant. The code output was arranged in columns of data with the first row indicating the horizontal pixel position with 1 pixel spacing for both the noise floor image and for the deformed image. The 1 pixel spacing was important in the regions with highest spatial frequency to avoid interpolating the DIC results and, therefore, introducing more filtering. An interested reader can refer to [5] where the advantages of such dense DIC on metrological performances is discussed. Figure 3 shows a typical displacement line-cut obtained here with technique Local.Affine.B.
## 2.5 Illustration of full-field results

All participants submitted a line slice in the horizontal direction of the vertical displacement obtained for both the deformed image and the noise image. These are shown in Figure 4(a) for a typical local DIC code for the 5 subset sizes. This plot illustrates the tradeoff between filtering of the DIC code and measurement resolution. That is, results with the highest spatial resolution also have the highest measurement resolution, and vice versa for the largest subset size. Figure 4(b) similarly shows typical results for a global DIC code for a variety of length parameters (element sizes). Figure 5 shows the strain results for a local DIC code. These results are representative of nearly all the submitted results. Figure 5 provides results from a typical local DIC approach, for several subset choices, with the attenuated strain fields shown on the left and the corresponding noise field on the right.

## 3 Assessment of the Metrological Performances

The thorough assessment of the metrological performances necessitates the evaluation of the measurement resolution, *n*, and of the spatial resolution, *ℓ*, to calculate, in turn, the Metrological Efficiency Indicator MEI. The noise image line-cut was used to calculate the standard deviation (n = 1σ) of the displacement measurement resolution and is reported as the QOI measurement resolution for the results. The deformed image line-cut was used to calculate the spatial resolution, defined here as a cutoff period. Indeed, the central row is at a constant amplitude of ±0.5 pixel vertical displacement or ±5% ε<sub>yy</sub> strain across the entire image. For both the displacement and strain results, an attenuation of the signal of 10% was chosen as a reasonable cutoff for defining the spatial resolution, denoted thus *ℓ*<sub>10%</sub>. The value chosen for the attenuation is crucial for the evaluation of the metrological performances. It has been shown in [6] how its value impacts the results for local and global versions of DIC. The attenuation of 10% chosen here corresponds to a middle regime which does not give an advantage to one specific version of DIC, global versions being particularly robust for high attenuation and whereas local versions behave well with low attenuation, cf. [6].

The jagged line in Figure 3(a) corresponds to the Pattern Induced Bias (PIB). PIB is the consequence of the interaction between the shape function, the image intensity gradients and the displacement field, which causes the algorithm to converge away from the lowpass filtered version of the true answer [20,21]. Synthetic data is use in Figure 6 to highlight the PIB effect. For this purpose, two sets of 50 pairs of images are using the displacement field for the Star 1 image. The first set relies on 50 independent speckle patterns on top of which independent pixel noise is added to each image. The second set relies on one unique speckle pattern on top of which noise independent noise is added to each image. Figure 6 illustrates line cuts of the retrieved displacement maps from the image pairs, from which the red curve corresponds to the average value. The dashed lines shown in Figure 6 are the theoretical displacement-bias result caused by the theoretical shape function attenuation. The averaged results from second set, i.e. from the 50 independent speckle patterns, is smoother due to the averaging out the PIB error. Such results clearly show that image pattern drives the DIC solution. The analytical link between shape function, image pattern and displacement field is proposed and discussed in [20].

Because the line cut from the results crossed the 10% fractional attenuation of the displacement magnitude point at multiple locations (see Figure 7) due to the PIB, it was impossible to simply locate the spatial period *ℓ*<sub>10%</sub> at which the signal first crossed the 10% fractional attenuation line from the raw data. Furthermore, as it would not be practical to ask all participants to submit results for 50 unique patterns for averaging to yield a smoother curve, we decided to fit a polynomial to the results to find a unique spatial period *ℓ*<sub>10%</sub> where the DIC results first cross the 10% fractional attenuation. We determined that a 12<sup>th</sup> order polynomial is a good compromise providing the “best-fit” to capture the shape of the results, without overfitting. This determination was made using 2 metrics.

First, we compared this 12^th^ order polynomial fit with the theoretical solution, which exists for non-weighted local DIC approach, cf. Appendix. Figure 7 (a-b) shows DIC measurements obtained with a local DIC approach (Local.Affine.B, with a subset width of 17 pixels (a) or 45 pixels (b)), on top of which the theoretical solution (red line) and different fits: the yellow line is the 5^th^ order polynomial fit and finally the purple line corresponds to the 12^th^ order polynomial fit. Measured displacement is plotted versus the period of the vertical sine wave of the GT displacement used for the image generation. This clearly highlights the lowpass filtering effect of DIC and thus facilitates the calculation of the cutoff period *ℓ*<sub>10%</sub>. It is worth noting
<figure><img src="image_6.png" /><figcaption>Fig. 4: (a) Typical local DIC code (Local.Affine.B) results showing on the (left) deformed image results all with the same scaling. (Right) noise image results, all with the same scaling. Deformed images and noise images have different scales. (b) Typical global DIC code (Global.C) results showing on the (left) deformed image results all with the same scaling. (Right) noise image results, all with the same scaling. Deformed images and noise images have different scales.</figcaption></figure>
<figure><img src="image_7.png" /><figcaption>Fig. 5: Typical local DIC code (Local.Affine.B) results showing strain results on the (left) deformed image results all with the same scaling. (Right) noise image results, all with the same scaling. Deformed images and noise images have different scales.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 6: Line cuts from 50 displacements obtained when applying DIC to 50 image pairs defined with (a) different speckle patterns or (b) one unique speckle but different image noises. The red curve is the average of the 50 results. The dashed line is the expected DIC results attenuated due to the shape function attenuation bias.</figcaption></figure>

that the 12^th^ order polynomial fit is closer to the theoretical solution, whereas the 5^th^ order polynomial fails for short periods. The inset emphasizes the values reached by these two fits close to the 10% fractional bias threshold. The spatial resolution that corresponds to where the fits cross the 10% bias threshold are 67.8 pixels (theoretical solution), 67.2 pixels (12^th^ order polynomial fit) or 68.1 pixels (5^th^ order polynomial fit). The difference is thus relatively small showing that the polynomial fits behave well.

Second, versatility of the fitting method was also studied. Indeed, the theoretical solution is method dependent. For instance, it does not exist for all DIC approaches, e.g. global and miscellaneous. It thus
<figure><img src="image_9.png" /><figcaption>Fig. 7: Examples of different fitting methods using the DIC measurements. Measurement given by the Local.Affine.B technique (a-b), by the Global.A technique (c) and the Misc.B technique (d). In all plots, measurements are plotted in blue, theoretical solutions in red and the 5^th^ and 12^th^ order polynomial fits introduced for the calculation of the spatial resolution are in yellow and purple. Since the theoretical solution does not exist for global and miscellaneous approaches, the red line corresponds to the local closed-form best fit, in the least square sense.</figcaption></figure>

seemed prudent to choose a polynomial fit that reasonably matches the data while also reproducing the correct theoretical fit for the local affine solution shown Figure 7 (a-b). Figure 7 (c-d) illustrates the fitting results when the proposed fits are applied to data provided from Global.A with elements of 75 pixels width (c) and Misc.B with length parameter set to 9 pixels (d). A fit is proposed, which best identifies the coefficients of the theoretical solution, in the least square sense. As expected, the theoretical fit, relevant for the local affine version of DIC, fails to describe the alternate DIC measurement techniques. Because the measurement techniques are based on a wide variety of algorithms, versatility offered by polynomial fits is necessary. The 5^th^ order polynomial does not provide enough degrees of freedom to correctly represent the trend of the measured data, with the fit over-estimating the period by about 20 pixels in Figure 7(d). This leaves the 12^th^ order polynomial fit as the best compromise for determining the spatial resolution. This fit has been used for all data, both strain and displacement in the rest of the article.
<figure><img src="image_10.png" /><figcaption>Fig. 8: Spatial resolution versus the length parameter (subset/element size, or else) for all codes. Displacement results are considered here for the calculation of the spatial resolution. Black lines correspond to the theoretical solution for affine (continuous line) and quadratic (dashed line) local versions of DIC.</figcaption></figure>

# 4 Study of the Metrological Performance of full-field measurements

## 4.1 Spatial resolution versus length parameter, e.g., subset/element size

After fitting with the 12th order polynomial, the cutoff period $ℓ_{10%}$ was found for different subset sizes (as reported by each participant) to define the spatial resolution for each code. The results for all codes are shown in Figure 8. The theoretical spatial resolution for local DIC with an affine shape function using uniformly weighted square subsets was calculated and is shown as the solid black line with the dotted black line showing the local quadratic theoretical spatial resolution. Several remarks can be drawn from these results:

- We observe similar performance despite the wide spread of DIC algorithm formulations and codes with different intrinsic definitions of the length parameters. All perform close to the theory. Since the Local codes separate clearly into the affine or quadratic theory value/trends, one could assume that Global.A-B and D-G and Misc. C, B, and D can be categorized based on where they fall in Figure 8, however it is not exactly clear for Misc.A, Misc.E, Misc.F or Global.C.

- All the curves feature a similar slope equaling 1 on a logarithmic scale, which indicates that the DIC length parameter such as subset or element size and the spatial resolution are proportional.

- Except for approaches Misc.C-D and Global.G, the spatial resolution obtained with the theoretical solution and a length parameter corresponding to the subset width appears as a generous estimate of a DIC code's expected resolution. We expect the DIC versions that perfectly match theoretical solutions to correspond to the straightforward implementations, whereas specific implementations can offer better performance.
<figure><img src="image_11.png" /><figcaption>Fig. 9: Measurement resolution or displacement noise ($n = 1\sigma$) versus spatial resolution for all codes.</figcaption></figure>

## 4.2 Spatial resolution versus measurement resolution

We investigated the tradeoff between spatial resolution and measurement resolution by plotting the QOI noise versus cutoff period for all codes in Figure 9. The noise was calculated as the standard deviation of the displacement along the same line-cut for the undeformed, but noisy image. The different measurement resolution and resolution results corresponded to different subset sizes as shown for Local.Affine.B in Figure 4. The other subset-based codes have the leftmost point for their smallest subset reported. Global codes had the “smallest” practical element size on the left to largest element size on the right, generally corresponding to the trends observed in filtering for the subset size in the local codes. All codes showed the expected improved measurement resolution corresponding to an increase in spatial filtering as subset and/or element size increased. Key observations from this figure include:

- As in Figure 8, two clusters are easily identified, which correspond to affine and quadratic kinematics. In Figure 9, clusters are defined by the spatial resolution as opposed to being defined by the length parameter.

- For all methods, plotting the measurement resolution versus the spatial resolution for all given subset or element sizes highlights the fact that these quantities are inversely proportional. The slope is indeed almost exactly -1, with the exception of assessing the lowest spatial resolution for Quad approaches (for the smallest length parameters) or Misc.F (also for the smallest parameters). This proportionality directly yields the definition of the Metrological Efficiency Indicator (MEI), defined as the product between the spatial resolution and the measurement resolution.

## 4.3 Metrological Efficiency Indicator

We rely here on the Metrological Efficiency Indicator (MEI) for comparing the metrological performances of the different measurement tools [5,4]. When displacements are considered, MEI is the measurement
<figure><img src="image_12.png" /><figcaption>Fig. 10: Metrological Efficiency Indicator (MEI) versus index of submission. The index of submission corresponds to the requested element or subset size ranges ordered from the smallest to the largest. Except when it corresponds to the smallest element/subset size, submission index have a relatively low impact to the MEI for all but the miscellaneous codes.</figcaption></figure>

resolution times the spatial resolution, i.e., the spatial period determined at a 10% cutoff via a 12th order polynomial. For strain, MEI corresponds to the square of the measurement resolution times the spatial resolution. It writes thus:

$$MEI = n^o \times \ell_{10\%} \quad \text{where } o = \begin{cases} 1 & \text{for displacement measurements} \\ 2 & \text{for strain measurements} \end{cases}, \qquad (3)$$

with n the measurement resolution, and $\ell_{10\%}$ the spatial resolution. The lower the value of the MEI, the better the performance of the code. Moreover, as already discussed in the literature, the MEI should remain constant when changing the subset size. A constant MEI is analytically demonstrated for another full-field measurement technique, i.e. the Localized Spectrum Analysis (LSA), and empirically observed for Local and Global DIC, cf [4,5,6]. The MEI values for the codes considered here are shown in Figure 10. It is worth noting that except for the few miscellaneous approaches, for which the property is not proven, all measurement techniques exhibit a constant Metrological Efficiency Indicator, when considering larger subset or element sizes, with the exception of the very smallest sizes (or lowest index submission number) which might be the result of degenerate cases.

In what follows, MEI is calculated as the average of the three lowest MEI values for each code.

Figure 11 shows the MEI for all the codes plotted together. A wide variation can be seen between some of the codes, but most of the codes lay between ±15% of each other for a given kinematic approach (affine or quadratic). Relative differences are computed with respect to the identified “straightforward” local DIC implementations (labeled “Ref” in Figure 11), that match the theory. They correspond to Local.Affine.B (bar with red edge) for the affine approaches and to Local.Quad.A (bar with blue edge) for the quadratic approaches. Because the MEI is the product of the displacement measurement resolution and the spatial resolution, we can easily calculate the spatial resolution at a uniform noise level for a fixed measurement resolution ±0.01 pixel, a common displacement noise floor. The local quadratic codes performed slightly better than the global codes as expected [22,23], which does not reflect the conclusion proposed in [24].
<figure><img src="image_13.png" /></figure>

Fig. 11: MEI of all the codes broken up by code type (lower is better). The y-axis of the right-hand side highlights the corresponding spatial resolution for a fixed measurement noise of ±0.01 pixel. Code Local.Affine.G is an outlier and not used in the calculation of standard deviation. Bar colors correspond to the same as in Figures 8, 9 or 10. References used for calculating the percentage are defined by the edged bar, red for the approaches submitted or identified as affine and blue for the quadratic ones. Finally, background colors highlight the identified approach: affine corresponds to red and quadratic to blue.

The difference is primarily because the earlier comparisons did not take into account spatial resolution. This figure reveals that:

- Methods cluster in two categories: affine and quadratic kinematics. These clusters are represented with the red backgrounds (for approaches submitted of identified as affine) and blue backgrounds (for approaches submitted or identified as quadratic). Since the MEI incorporates the measurement resolution, the categorization is slightly different than the one proposed considering the spatial resolution only, as illustrated in Figure 8.

- The metrological performances of the second order approaches are very close. The extrema scores are only a few percentages different when compared with the straightforward implementation. The latter thus seems to correspond to the implementation that features the best metrological performance.

- Measurement technique Local.Affine.C, whose associated participant submitted as affine kinematics, performs as well as quadratic kinematics. Specific post-processing may be implemented, such as deconvolution [25], which increases the metrological performance of affine strategies

## 4.4 Strain results analysis

For Star 6 with a Lagrangian ±5% strain in the vertical direction ($ε_{yy}$), the participants were requested to submit results over a range of virtual strain gauge sizes (VSG) from the smallest possible VSG up to one with “reasonable” filtering, but less than 10% decrease at the largest spatial periods on the right of the image. Initial results were checked and when needed, smaller VSG’s were supplied. The VSG as defined in the GPG [3] depends on the local code’s step size, subset size, and strain window used in the calculation of the strain. While this calculation is valid for local codes, it has no application for global or other miscellaneous implementations of both DIC and the strain calculation. The VSG is therefore used in this context as shorthand for “the region of the image over which the strain is calculated.” Because of the variety of codes used, the actual VSG size is not reported. The results, however, do include the smallest feasible VSG for that code implementation. For this evaluation, knowing the VSG size is not important. Instead, like the displacement, the strain noise (measurement resolution) and spatial resolution can be plotted as in Figure 12. Nevertheless the MEI for the strain is defined as the squared of the spatial resolution times the measurement resolution. There is a larger spread in the results due to the extra choices that can be made in the strain calculation, however, the same trends occur, with the same tradeoff in spatial resolution and measurement resolution. The curves are not smooth and not all are monotonically decreasing, because multiple parameters can be changed at the same time causing jumps in either the spatial resolution or the measurement resolution. Nevertheless, almost all curves have a slope of -2. This indicates that the spatial resolution is proportional to the square of the measurement resolution.
<figure><img src="image_14.png" /><figcaption>Fig. 12: Star 6 spatial resolution versus strain measurement resolution or measurement resolution. Increasing VSG size going to the right.</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>Fig. 13: MEI for the strain (lower is better). Order of codes the same as in Figure 11 and broken up by code type. Bar colors correspond to the same as in Figures 8, 9, 10 or 12. Percentage are calculated here with respect to the mean value.</figcaption></figure>

The MEI for the strain is summarized in Figure 13, where Local.Affine.C and Local.Affine.G are not included because they were not submitted. In the figure, both the mean value and the standard deviation obtained with all the codes are shown in the red text and background. Relative percentages of the MEI with respect to the mean value are also shown for each code. One might expect that if the displacement MEI is approximately the same between two codes using the same strain calculation, they would produce about the same strain MEI. However, this does not seem to be the case when comparing Figures 11 and 13. The large impact of the strain calculation is clearly highlighted with the variation in affine strain MEI and its departure from the displacement MEI trends. For example, Local.Affine.D has the smallest strain MEI in the local affine category, but the largest displacement MEI in that same category (with the exception
of Local.Affine.G). There is smaller variations in strain MEI for the quadratic codes (except Global.F). Generally, the quadratic kinematics outperformed the affine when calculating strain. The straightforward implementations Local.Affine.B and Local.Quad.A correspond to the DIC approaches that are the closest implementations to theory, as illustrated with the displacement results. With these implementations, strain gradients are numerically computed by central difference of the displacement pixel-wise maps. Obviously other affine methods must calculate strain in a different manner to achieve the lower strain MEI. Similarly, some quadratic approaches clearly offer better results than the simple central derivation used with Local.Quad.A.

## 5 Conclusions

Rather than functioning as a competition, it has always been the mission of the DIC Challenge board to encourage a healthy code development environment by sharing how the various codes compare with each other. Furthermore, each participant is given the code letter for their code, allowing them to see where they sit relative to all other codes, and allowing them to improve their code, or implement new approaches as needed. An important link between the measurement resolution and the spatial resolution has been highlighted in this work and is quantified via the MEI, emphasizing its relevancy. This clearly shows the direct tradeoff between spatial resolution and the measurement resolution for the displacement results. Tuning DIC parameters for improving the measurement resolution may impair the spatial resolution depending on the gradients of the measured QOI. The results also show a much wider variation in the strain MEI, indicating that the method of calculation of the strain can have a very large impact on the results. While it seems that 2D-DIC is well understood, there is clearly a substantial amount of variation in the performance among the different codes. While spatial resolution is important, it is not the only criterion for deciding what methods to implement in a code. For example, analysis speed may be more important than bias, or tying directly to model results in the case of global DIC. However, when looking at full-field spatially varying data, this tradeoff between the measurement resolution and the spatial resolution is key. This work provides a useful framework for understanding the tradeoff and analyzing the performance of the DIC software using the provided images. It details some of the unique errors associated with the analysis of these images, such as the PIB and imprecision introduced through the strain calculation method. Our hope is that the images and techniques provided in the DIC Challenge become a standard verification and validation technique for new codes and developers to improve their DIC algorithms.

**Acknowledgements** The 2D-DIC Challenge is dedicated to Dr. Laurent Robert. An active and important board member in the early years of the project, who passed away in 2016. He has been sorely missed in the experimental mechanics community. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology and Engineering Solutions of Sandia, LLC., a wholly owned subsidiary of Honeywell International, Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA-0003525. B. Blaysat is grateful to the French National Research Agency (ANR) and to the French government research program "Investissements d'Avenir" for their financial support (ICARES project, No ANR-18-CE08-0028-01 & IDEX-ISITE initiative 16-IDEX-0001, CAP 20-25). S. N. Olufsen gratefully appreciate the financial support from the Research Council of Norway through the Centre for Advanced Structural Analysis, Project No. 237885 (SFI-CASA).

*Conflict of interest* Certain commercial equipment, instruments, or materials are identified in this paper in order to specify the experimental procedure adequately. Such identification is not intended to imply recommendation or endorsement by NIST, nor is it intended to imply that the materials or equipment identified are necessarily the best available for the purpose.

## Appendix: Comparison with theory

For most implementations of DIC, the optimal displacement minimizes the SSD criterion on each subset. The corresponding optimal parameters $q^*$ satisfies:

$$q^* = \underset{q}{\operatorname{Argmin}} \sum_{x \in \Omega} (I_{ref}(x) - I_{cur}(x + v(x, q)))^2 \quad (4)$$

where $I_{ref}$ (resp. $I_{cur}$) is the reference (resp. current) image and $v(x, q)$ is a polynomial basis function that is either 1st order (affine) or 2nd order (quadratic). Let $u_{GT}$ be the ground truth displacement. Considering an infinite resolution
<figure><img src="image_16.png" /><figcaption>Fig. 14: Expected DIC results for the DIC Challenge 2.0 Star images using a SSD criterion and affine or quadratic shape functions. Notation h corresponds to the subset size.</figcaption></figure>

texture that tracks the underlying material, $I_{ref}(x) = I_{cur}(x + u_{GT}(x))$. Equation 4 can be rewritten as

$$q^* = \underset{q}{\operatorname{Argmin}} \sum_{x \in \Omega} (I_{def}(x + u_{GT}(x)) - I_{cur}(x + v(x, q)))^2. \qquad (5)$$

For most implementations of DIC, assuming that the image pattern corresponds to the perfect information carrier, it drives the DIC algorithm to the best solution. The expected DIC solution simply results to the projection of the ground truth displacement to the DIC kinematics, thus neglecting the effect of $I_{cur}$ in Equation 5. This leads to

$$q^* = \underset{q}{\operatorname{Argmin}} \sum_{x \in \Omega} (u_{GT}(x) - v(x, q))^2. \qquad (6)$$

This result was introduced in [26], experimentally validated in [27,13] and thoroughly mathematically demonstrated in [20]. It simply corresponds to the effect of the well-known Savitzky-Golay filter. Even though these results are shown only for the SSD, most unweighted criteria lead to the same solution at convergence [28].

These equations produced the expected DIC results for the DIC Challenge 2.0 Star image in Figure 3, 6 and 8 and produced the curves in the Figure 14 for affine (1st order) and quadratic (2nd order) shape functions.

## References

1. P.L. Reu, E. Toussaint, E. Jones, H.A. Bruck, M. Iadicola, R. Balcaen, D.Z. Turner, T. Siebert, P. Lava, M. Simon-sen, Experimental Mechanics 58(7), 1067 (2018). DOI 10.1007/s11340-017-0349-0. URL https://doi.org/10.1007/s11340-017-0349-0

2. A. Savitzky, M.J.E. Golay, Analytical Chemistry 36(8), 1627 (1964). DOI 10.1021/ac60214a047. URL http://dx.doi.org/10.1021/ac60214a047

3. E. Jones, M. Iadicola (eds.), A Good Practices Guide for Digital Image Correlation (International Digital Image Correlation Society, 2018). URL https://doi.org/10.32720/idics/gpg.ed1/print.format

4. M. Grédiac, F. Sur, Strain 50(1), 1 (2014)

5. M. Grédiac, B. Blaysat, F. Sur, Experimental Mechanics 57(6), 871 (2017). DOI 10.1007/s11340-017-0279-x. URL http://dx.doi.org/10.1007/s11340-017-0279-x

6. B. Blaysat, J. Neggers, M. Grédiac, F. Sur, Experimental Mechanics 60(3), 393 (2020). DOI 10.1007/s11340-019-00566-4. URL https://link.springer.com/article/10.1007/s11340-019-00566-4

7. URL https://sem.org/dicchallenge/

8. Y. Wang, P. Lava, P. Reu, D. Debruyne, Strain 52(2), 110 (2016). DOI 10.1111/str.12173. URL http://dx.doi.org/10.1111/str.12173

9. Y. Barranger, P. Doumalin, J.C. Dupré, A. Germaneau, Strain 48(5), 357 (2012). DOI https://doi.org/10.1111/j.1475-1305.2011.00831.x. URL https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1475-1305.2011.00831.x

10. J.J. Orteu, D. Garcia, L. Robert, F. Bugarin, in Proceedings SPIE, vol. 6341 (2006), vol. 6341, pp. 1–6. URL http://dx.doi.org/10.1117/12.695280
11. M. Rossi, P. Lava, F. Pierron, D. Debruyne, M. Sasso, Strain 51(3), 206 (2015). DOI https://doi.org/10.1111/str.12134. URL https://onlinelibrary.wiley.com/doi/abs/10.1111/str.12134

12. R. Balcaen, L. Wittevrongel, P.L. Reu, P. Lava, D. Debruyne, Experimental Mechanics 57(5), 703 (2017). DOI 10.1007/s11340-017-0259-1. URL https://doi.org/10.1007/s11340-017-0259-1

13. P.L. Reu, Experimental Mechanics 51(4), 443 (2011). DOI 10.1007/s11340-010-9417-4. URL https://doi.org/10.1007/s11340-010-9417-4

14. B. Blaysat, M. Grédiac, F. Sur, International Journal for Numerical Methods in Engineering 108(3), 213 (2016). DOI 10.1002/nme.5212. URL http://dx.doi.org/10.1002/nme.5212

15. F. Sur, B. Blaysat, M. Grédiac, Journal of Mathematical Imaging and Vision 60(5), 634 (2018). DOI 10.1007/s10851-017-0779-4. URL https://doi.org/10.1007/s10851-017-0779-4

16. J. Blaber, B. Adair, A. Antoniou, Experimental Mechanics 55(6), 1105 (2015). DOI 10.1007/s11340-015-0009-1. URL http://dx.doi.org/10.1007/s11340-015-0009-1

17. J. Yang, K. Bhattacharya, Experimental Mechanics 59(2), 187 (2019). DOI 10.1007/s11340-018-00457-0. URL https://doi.org/10.1007/s11340-018-00457-0

18. A. Landauer, M. Patel, D. Henann, C. Franck, Experimental Mechanics 58(815-830) (2018). DOI 10.1007/s11340-018-0377-4. URL https://doi.org/10.1007/s11340-018-0377-4

19. S.N. Olufsen, M.E. Andersen, E. Fagerholt, SoftwareX 11, 100391 (2020). DOI https://doi.org/10.1016/j.softx.2019.100391. URL https://www.sciencedirect.com/science/article/pii/S2352711019301967

20. F. Sur, B. Blaysat, M. Grédiac, Journal of Mathematical Imaging and Vision (Available online)

21. S.S. Fayad, D.T. Seidl, P.L. Reu, Experimental Mechanics 60(2), 249 (2020). DOI 10.1007/s11340-019-00553-9. URL https://doi.org/10.1007/s11340-019-00553-9

22. B. Pan, B. Wang, G. Lubineau, A. Moussawi, Experimental Mechanics 55(5), 887 (2015). DOI 10.1007/s11340-015-9989-0. URL http://dx.doi.org/10.1007/s11340-015-9989-0

23. L. Wittevrongel, P. Lava, S.V. Lomov, D. Debruyne, Experimental Mechanics 55(2), 361 (2015). DOI 10.1007/s11340-014-9946-3. URL http://dx.doi.org/10.1007/s11340-014-9946-3

24. F. Hild, S. Roux, Experimental Mechanics 52, 1503 (2012). DOI 10.1007/s11340-012-9603-7. URL http://dx.doi.org/10.1007/s11340-012-9603-7

25. M. Grédiac, B. Blaysat, F. Sur, Experimental Mechanics 59(2), 219 (2019). DOI 10.1007/s11340-018-00461-4. URL https://doi.org/10.1007/s11340-018-00461-4

26. H.W. Schreier, M.A. Sutton, Experimental Mechanics 42(3), 303 (2002). DOI 10.1007/BF02410987. URL http://dx.doi.org/10.1007/BF02410987

27. A. Baldi, F. Bertolino, Strain pp. 248-263 (2015). DOI 10.1111/str.12137. URL http://dx.doi.org/10.1111/str.12137

28. B. Pan, H. Xie, Z. Wang, Appl. Opt. 49(28), 5501 (2010). DOI 10.1364/AO.49.005501. URL http://ao.osa.org/abstract.cfm?URI=ao-49-28-5501