# Deformation Monitoring System Based on 2D-DIC for Cultural Relics Protection in Museum Environment with Low and Varying Illumination

Authors: Zhiqi Wang; Jian Zhao; Lihua Fei; Yanglei Jin; Dong Zhao
Journal: Mathematical Problems in Engineering
 (2018)

---

## Abstract

Quantifying the condition of large cultural relics, such as marine archaeological shipwreck, is important to verify stability and reliability. Deformation monitoring system plays a key role in the preservation and long-term conservation of cultural relics. Two-dimensional digital image correlation (2D-DIC) method has proven its efficiency in being able to provide accurate quantitative information of structural deformations. In this study, a deformation monitoring system with four cameras based on 2D-DIC is developed to perform noncontact, optically based measurement to monitor the deformation of shipwreck in museum environment with low and varying illumination. Because the consistency of the accuracy of 2D-DIC measurements for different locations is the most basic requirement in the application of structural deformation monitoring, selecting the appropriate exposure time and quantifying the bias errors on 2D-DIC measurements should be helpful to the optimal use of this optical nondestructive testing technique. A theoretical criterion is deduced to quantitatively characterize the dependence of interpolation bias upon natural patterns and illuminations. Then, an exposure adjustment scheme is built based on the aforementioned criterion. Numerical experiments reveal that the exposure adjustment scheme is able to provide consistency interpolation error for different natural patterns even though the environmental illumination is different as well. The deformation monitoring system with the proposed exposure adjustment scheme is promising for developing flexible and robust in situ structural health monitoring for use in museum environment with low and varying illumination, making 2D-DIC technique a really useful tool for in situ long-term monitoring of large cultural relics.


---

## Body

## 1. Introduction

Structurally intact wooden shipwrecks on the seabed constitute time capsules carrying significant historic, archaeological, social, and scientific information [1]. In 1974, after more than 700 years buried in marine clay sediments off the Quanzhou Bay (24°37′N, 118°37′E), the Song Dynasty shipwreck was raised from the seabed with the aim of restoration, preservation, and long-term conservation and has since been kept at the Quanzhou Maritime Museum. The Song Dynasty shipwreck (Figure 1) provides a unique environment for research into microbial wood deterioration, strength and stiffness degradation, biogeochemical cycling of iron and sulfur, and the medium to long-term impacts of conservation of marine archaeological wood. From a conservation or structural perspective waterlogged archaeological wood differs from modern wood because it has been degraded by various irreversible changes of physical, chemical, and biological processes in the marine environment [2]. Environmental degradation [3] of marine archaeological wood results in loss of cellulose [4, 5], which provides longitudinal strength and stiffness. These changes result in a very porous material, due to the formation of cavities and cracks, and the result is a significant loss in mechanical strength [6]. The major challenges faced during conservation of marine archaeological wood include the nonuniform degradation of timbers and the seasonal variation of relative humidity (RH) in museum. The marine archaeological wood substance strives for equilibrium with the ambient relative humidity. In the environment of high relative humidity, the cavities within the wood will fill with water, which if descended can result in severe shrinkage of the wood due to an inability to resist the strong capillary forces during the water-desorption process. Uncontrolled water-desorption process can lead to collapse, shrinkage, distorted shape and surfaces (twists, warping, cracks, and splits), disintegration, precipitation of salts, and corrosion products in the marine archaeological wood [7]. Therefore, it must be achieved while avoiding the destructive forces of water that can result in shrinking, warping, and cracking of the wood in the current preservation environment. In addition, the process such as shrinking and warping could result in further local deformation and subsidence of the wooden shipwreck. An easy-to-use, effective deformation monitoring system for shipwreck condition assessment, would allow the detection of potentially dangerous situations at an early stage and providing an efficient basis to control preservation environment or reinforce the shipwreck, while not interfering with exhibition.Figure 1
The Song Dynasty shipwreck.Digital image correlation (DIC) among other optical nondestructive testing (NDT) methods is becoming more widely accepted for materials and structural monitoring. The two-dimensional DIC is applied to a series of images of a deforming planar object acquired by a single camera [8]. The surface of the object is speckled to present a randomly distributed intensity pattern. The principle of the 2D-DIC method lies in determining the local correlation of two speckle images. The randomness of the speckle pattern is essential for obtaining a unique solution in the correlation process. Reference subsets in the reference image are compared to subsets in a deformed image to find the target subset, which is the subset in the deformed image that shows the maximum similarity with the reference subset.To achieve high-accuracy measurements, various aspects involved in the implementation of 2D-DIC must be given careful consideration. As an optical NDT method based on image matching, the measurement of 2D-DIC relies heavily on the quality of the acquired images. Various researchers have demonstrated that the speckle pattern has an important influence on DIC measurements. A number of speckle image assessment criteria have been presented in recent literature [9–18]. Among them, the mean intensity gradient (MIG) [10] and the sinusoidal approximate formula for noise-induced bias [17] are typical representatives. In addition, efficient speckle pattern optimized algorithms and corresponding bias error estimation and correction approaches have been proposed [19–23]. In practical application of DIC measuring and/or monitoring, a continuous and even illumination on the test object with stable and controllable white light source is an important means to ensure sufficient and constant contrast for reliable and accurate speckle image matching and to produce a series of quality speckle images for subsequent DIC analysis. However, in an actual museum environment, especially in situ long-term monitoring of cultural relics in the exhibition hall where medium and/or high light-sensitivity objects are on display, the requirement of continuous and even illumination is unable to be achieved, as ambient light adopts general lighting and has requirements for total amount of illumination (annual lighting exposure). In 2009, China issued a National Standard for the Design of Museum Lighting (GB/T 23863-2009). This code articulates the different deterioration phenomena caused by light and establishes the highest levels of illumination to limit damage [24]. China has divided museum objects into three categories, low, medium, and high sensitivity. For Song Dynasty shipwreck that belongs to high sensitivity objects, the total amount of illumination should be less than 360 000 lux-hours per year, which limits the number of display days to less than 300 per year when the illuminance is 150 lux for eight hours.On-site investigation and measurement result show that the environmental illumination of Quanzhou Maritime Museum below 100 lux with adoption of natural lighting. On the other hand, the environmental illuminations fluctuate from the medium illumination of daytime to low illumination of dark night and vary from location to location on shipwreck. Speckle patterns markedly affect the accuracy of the correlation results because they are associated with the similarity before and after deformation. When the ambient light illumination changes significantly, the contrast of speckle patterns may change accordingly. In practice, the most detrimental lighting conditions are the ones in which the illumination gradient change over the image. As a result, the similarity between the deformed images and the originally recorded reference image may decrease notably, resulting in a failure of 2D-DIC analysis. In order to suppress influences of varying ambient light on speckle patterns and acquire quality speckle images suitable for 2D-DIC analysis, Pan et al. [25] proposed a monochromatic light illuminated active imaging DIC method for high fidelity deformation measurement. Simončič et al. [26] modified the inverse compositional Gauss-Newton algorithm to achieve high-accuracy measurement in image sequences with varying lighting intensity by DIC. Xu et al. [27] proposed a weighted normalized gradient-based algorithm to decrease the influence of illumination and got better results by DIC in case of nonlinear gray-value-based illumination variation.The speckle pattern serving as the information carrier is a key issue in relation to the accuracy in using DIC. The speckle patterns cannot be fabricated artificially for protection reasons, so natural wood texture distribution inherent in shipwreck surface was used as natural pattern to perform correlation calculation. Monitoring of multiple locations is necessary for the deformation monitoring system. It should be noted that the natural patterns on different locations of the shipwreck may lead to distinctly different bias errors. The bias error of 2D-DIC includes the random error, interpolation bias caused by interpolation, and the noise-induced bias caused by coupling of interpolation and noise. Among them, interpolation error is the main bias error form. In order to synthetically evaluate the result of deformation monitoring, the same bias errors are required for different monitoring regions. This requires that the deformation monitoring system based on 2D-DIC must have the capability to estimate bias errors for natural patterns and function to adjust the quality of natural patterns in order to meet the consistency requirements of bias errors. In fact, to ensure constant bias errors of different natural patterns captured by ordinary industrial cameras, an alternative approach is to adjust exposure time. The industrial camera's exposure time determine the amount of light that reaches the image sensor. Changing exposure time can change the brightness and contrast of speckle patterns. A good speckle pattern on the test sample surface should have high contrast. The decrease in illuminance causes the decrease in the grayscale gradient of the speckle pattern, and then it will lead to lower measurement accuracy significantly [28]. In order to acquire effective speckle patterns with the constant bias error suitable for DIC monitoring in low and varying illumination, how to choose appropriate exposure time is a problem to solve.In this work, a deformation monitoring system based on 2D-DIC was established to investigate the deformation of the Song Dynasty shipwreck. A quantitative criterion for bias error estimation of natural pattern is proposed based on theoretical analysis and experimental verification. Next, we apply the quantitative criterion as the basis for exposure adjustments. The remainder of present paper is organized as follows. Section2 briefly explains the 2D-DIC measurement principle and the influence of different natural patterns with different illumination conditions on measuring accuracy. In Section 3, a quantitative criterion based on the theoretical analysis of interpolation bias is presented, and adjustment scheme of exposure is experimented and analyzed. In Section 4, the influence of image noise is discussed. Finally, conclusions are summarized in Section 5.


## 2. The Influence of Speckle Pattern on 2D-DIC Measurements

### 2.1. Principle of 2D-DIC

2D-DIC uses image registration algorithms to retrieve full-field relative displacements between undeformed (reference) image and deformed images [29]. Each pixel of these images stores a grayscale value matrix because of a pattern at the surface of the specimen called speckle pattern [30]. DIC relies on the speckle pattern to obtain surface displacement fields [10]. In subset-based 2D-DIC algorithms, the reference image is divided into smaller regions referred to as subsets. Reference subsets are initially a contiguous rectangular group of points that are on integer pixel locations. The deformation is assumed to be homogeneous inside each subset, and the deformed subsets are then tracked in the deformed image using the correlation matching algorithm. To find the deformation of a subset, 2D-DIC algorithms find the extremum of a correlation function. The practical and robust zero-mean normalized sum of squared difference (ZNSSD) correlation criterion is used to evaluate the similarity between the reference and deformed subsets [31, 32]. The grayscale intensity functions of the speckle pattern in each subset is represented by a function pair which refers to the reference configuration and to the deformed incremented state. Here are the coordinates of a reference subset point and the coordinates of a deformed subset point. The correlation function is the subset DIC residual, which is the zero-mean normalized sum of squared difference between the gray levels in subsets extracted in the deformed and reference images. For each subset, we have(1)CΩ,ZNSSD=∑i,j∈ΩIrefxi,yj-I-ref∑i,j∈ΩIrefxi,yj-I-ref2-Idefxi′,yj′-I-def∑i,j∈ΩIdefxi′,yj′-I-def22FunctionsI-ref=1/N∑(i,j)∈ΩIref(xi,yj) and I-def=1/N∑(i,j)∈ΩIdef(xi′,yj′) correspond to the mean grayscale values of the reference and current subset with N denoting the total number of points within subset Ω.The transformation of the coordinates(xi,yj) from the reference subset to the deformed configuration (xi′,yj′) is constrained to a linear, first-order transformation referred to as subset shape function W(Δx,Δy;P), which is utilized to depict the shape of the deformed subset relative to the reference subset, (Δx,Δy) is the local coordinates of the pixel point in each subset, and P is the displacement parameter vector. It can be written in the following form:(2)WΔx,Δy;P=1+uxuyuvx1+vyv001ΔxΔy1Here P=(ux,vx,uy,vy,u,v)T with u,v denoting the displacement components, and the other four parameters represent the displacement gradients. If the linear intensity variation is assumed, the ZNSSD correlation criterion together with an affine shape function W(x,y;P) and an incremental shape function W(x,y;ΔP) is defined by the following expression: (3)∑i,j∈ΩIrefx+Wx,y;ΔP-I-ref∑i,j∈ΩIrefx+Wx,y;ΔP-I-ref2-Idefx+Wx,y;P-I-def∑i,j∈ΩIdefx+Wx,y;P-I-def22where Iref(x) and Idef(x) denote the grayscale levels at x=(x,y,1)T of reference image and the deformed image.To accurately match the two subsets, the roles of the deformed subset and reference subset are reversed in inverse compositional Gauss-Newton (IC-GN) algorithm [32–34]. A backward matching strategy is used by exerting an incremental shape function W(x,y;ΔP) to the reference subset and comparing it with the deformed subset, where the affine shape function W(x,y;P) is defined. The incremental shape function W(x,y;ΔP) can be defined in the same manner as the affine shape function W(x,y;P).(4)Wx,y;ΔP=1+ΔuxΔuyΔuΔvx1+ΔvyΔv001ΔxΔy1where ΔP=(Δux,Δvx,Δuy,Δvy,Δu,Δv)T denotes the incremental deformation parameter vector, which is calculated by the minimization procedure. In order to solve for W(x,y;ΔP), a first-order Taylor expansion of (3) with respect to ΔP is performed. We obtain(5)∑i,j∈ΩIrefx+Wx,y;0+∇Iref∂W/∂PΔP-I-ref∑i,j∈ΩIrefx+Wx,y;ΔP-I-ref2-Idefx+Wx,y;P-I-def∑i,j∈ΩIdefx+Wx,y;P-I-def22where ∇Iref=∂Iref/∂x,∂Iref/∂y is the gradient in the x- and y-directions of the reference subset, which can be calculated by interpolation. In this work, quintic B-spline interpolation is employed. In (5), the term ∂W/∂P represents the Jacobian of the shape function, which can be written as (6)∂W∂P=1ΔxΔy0000001ΔxΔyMinimization of CΩ,ZNSSD with respect to ΔP gives the least-squares solution of ΔP:(7)ΔP=-∑Ω∇f∂W∂PT×∇f∂W∂P×∑Ω∇f∂W∂PT×Irefx+Wx,y;ΔP-I-ref-ΔfΔgIdefx+Wx,y;P-I-defwhere Δf=∑ΩIref(x+W(x,y;ΔP))-I-ref2 and Δg=∑ΩIdef(x+W(x,y;P))-I-def2.Based on the minimization procedure, the incremental shape functionW(x,y;ΔP) is inverted and composed with the current estimated shape function. The obtained result is the updated shape function of the deformed subset, from which the incremental parameter vector can be determined. The operation can be expressed as follows:(8)Wx,y;P←Wx,y;P∘W-1x,y;ΔP=1+uxuyuvx1+vyv0011+ΔuxΔuyΔuΔvx1+ΔvyΔv001-1The iteration is repeated until the preset convergence condition is satisfied. In this work, the convergence condition,ΔPi≤10-6, is predefined.


### 2.2. The Influence of Speckle Patterns on 2D-DIC Measurements

Speckle patterns have an important influence on the accuracy in the displacements measurement of DIC [23]. To demonstrate the influence of speckle patterns on 2D-DIC measurements, eight reference speckle images obtained from real experiments were utilized in the following numerical experiments. The image acquisition operation is done in a darkroom with an illuminance of 0. In this environment, the ambient illuminance can be adjusted by using a brightness-adjustable white artificial light, and the reading of the illuminance value is read by a digital illuminometer. We slowly increased the brightness of the artificial light in the condition that the room is completely dark, continuously read the ambient illuminance values by digital illuminometer under various brightness conditions, and captured image until the illuminometer reading is approximately equal to 60 lux or other values.Figure2 illustrates the reference images and their corresponding power spectrums. The pattern A was acquired by randomly spraying black and white paints on flat specimen surface under the condition of the illumination of 60 lux and exposure time of 100 ms. The patterns B, C, and D were obtained from the same specimen of the speckle pattern A but different illumination or exposure time. The pattern E was acquired another flat specimen surface under the same condition of the pattern A. The patterns F, G, and H were obtained from the same specimen of the pattern E but unequal illumination or exposure time. All of the speckle patterns were acquired by a CCD camera with a fixed aperture lens (f = 5.6). The specific parameters for different speckle patterns are listed in Table 1. The illuminance is adjusted by changing the brightness of the white artificial light. The exposure time is adjusted by the camera software.Table 1
Specific parameters for different speckle patterns.
Speckle pattern Illumination (lux) Exposure time (ms) A 60 100 B 60 50 C 40 100 D 80 50 E 60 100 F 60 50 G 40 100 H 80 50Figure 2
Eight reference images used in numerical experiment and their corresponding speckle spectrums.
(a) (b) (c) (d) (e) (f) (g) (h)For each speckle pattern, 20 subpixel translated images are generated by applying appropriate shifts in the Fourier domain of the reference speckle pattern according to the shift theorem [35]. The subpixel displacements are applied in the x-direction, ranging from 0 to 1 pixel with a spacing of 0.05 pixels between two successive speckle patterns. The displacements of each translated speckle pattern were computed by using IC-GN subpixel registration algorithm.To quantitatively evaluate the accuracy of different natural patterns adopted in this numerical experiment, the systematic error of the computed displacements associated with speckle pattern is represented as mean bias error [36]. The mean bias error of the measured displacement is defined as follows: (9)pe=pmean-pimpwhere pmean=1/N∑i=1Npi represents the mean of the N estimated displacements and pimp denotes the exact imposed displacement.Figure3 shows the mean bias error as a function of preassigned subpixel displacements for the eight speckle patterns (solid lines). The sinusoidal-shaped mean bias error can be attributed to the interpolation error. This observation agrees well with existing results reported in [37]. By inspection, it is seen that (1) speckle pattern has a significant influence on the bias error; the mean bias errors of different speckle patterns with the same illumination and exposure are different (compare the experimental results by DIC of speckle patterns A and E); (2) lighting condition has a significant influence on the bias error; the mean bias errors of different speckle patterns with different illumination and the same exposure are different (compare the experimental results by DIC of speckle patterns A and G); (3) for given speckle patterns, whether or not the patterns and illuminations differ, it is always possible to find appropriate exposure time to produce approximate bias errors.Figure 3
Interpolation biases by DIC (marks and solid lines), by the theoretical equation (dashed lines) for eight speckle patterns with different illuminations and exposure time.


## 2.1. Principle of 2D-DIC

2D-DIC uses image registration algorithms to retrieve full-field relative displacements between undeformed (reference) image and deformed images [29]. Each pixel of these images stores a grayscale value matrix because of a pattern at the surface of the specimen called speckle pattern [30]. DIC relies on the speckle pattern to obtain surface displacement fields [10]. In subset-based 2D-DIC algorithms, the reference image is divided into smaller regions referred to as subsets. Reference subsets are initially a contiguous rectangular group of points that are on integer pixel locations. The deformation is assumed to be homogeneous inside each subset, and the deformed subsets are then tracked in the deformed image using the correlation matching algorithm. To find the deformation of a subset, 2D-DIC algorithms find the extremum of a correlation function. The practical and robust zero-mean normalized sum of squared difference (ZNSSD) correlation criterion is used to evaluate the similarity between the reference and deformed subsets [31, 32]. The grayscale intensity functions of the speckle pattern in each subset is represented by a function pair which refers to the reference configuration and to the deformed incremented state. Here are the coordinates of a reference subset point and the coordinates of a deformed subset point. The correlation function is the subset DIC residual, which is the zero-mean normalized sum of squared difference between the gray levels in subsets extracted in the deformed and reference images. For each subset, we have(1)CΩ,ZNSSD=∑i,j∈ΩIrefxi,yj-I-ref∑i,j∈ΩIrefxi,yj-I-ref2-Idefxi′,yj′-I-def∑i,j∈ΩIdefxi′,yj′-I-def22FunctionsI-ref=1/N∑(i,j)∈ΩIref(xi,yj) and I-def=1/N∑(i,j)∈ΩIdef(xi′,yj′) correspond to the mean grayscale values of the reference and current subset with N denoting the total number of points within subset Ω.The transformation of the coordinates(xi,yj) from the reference subset to the deformed configuration (xi′,yj′) is constrained to a linear, first-order transformation referred to as subset shape function W(Δx,Δy;P), which is utilized to depict the shape of the deformed subset relative to the reference subset, (Δx,Δy) is the local coordinates of the pixel point in each subset, and P is the displacement parameter vector. It can be written in the following form:(2)WΔx,Δy;P=1+uxuyuvx1+vyv001ΔxΔy1Here P=(ux,vx,uy,vy,u,v)T with u,v denoting the displacement components, and the other four parameters represent the displacement gradients. If the linear intensity variation is assumed, the ZNSSD correlation criterion together with an affine shape function W(x,y;P) and an incremental shape function W(x,y;ΔP) is defined by the following expression: (3)∑i,j∈ΩIrefx+Wx,y;ΔP-I-ref∑i,j∈ΩIrefx+Wx,y;ΔP-I-ref2-Idefx+Wx,y;P-I-def∑i,j∈ΩIdefx+Wx,y;P-I-def22where Iref(x) and Idef(x) denote the grayscale levels at x=(x,y,1)T of reference image and the deformed image.To accurately match the two subsets, the roles of the deformed subset and reference subset are reversed in inverse compositional Gauss-Newton (IC-GN) algorithm [32–34]. A backward matching strategy is used by exerting an incremental shape function W(x,y;ΔP) to the reference subset and comparing it with the deformed subset, where the affine shape function W(x,y;P) is defined. The incremental shape function W(x,y;ΔP) can be defined in the same manner as the affine shape function W(x,y;P).(4)Wx,y;ΔP=1+ΔuxΔuyΔuΔvx1+ΔvyΔv001ΔxΔy1where ΔP=(Δux,Δvx,Δuy,Δvy,Δu,Δv)T denotes the incremental deformation parameter vector, which is calculated by the minimization procedure. In order to solve for W(x,y;ΔP), a first-order Taylor expansion of (3) with respect to ΔP is performed. We obtain(5)∑i,j∈ΩIrefx+Wx,y;0+∇Iref∂W/∂PΔP-I-ref∑i,j∈ΩIrefx+Wx,y;ΔP-I-ref2-Idefx+Wx,y;P-I-def∑i,j∈ΩIdefx+Wx,y;P-I-def22where ∇Iref=∂Iref/∂x,∂Iref/∂y is the gradient in the x- and y-directions of the reference subset, which can be calculated by interpolation. In this work, quintic B-spline interpolation is employed. In (5), the term ∂W/∂P represents the Jacobian of the shape function, which can be written as (6)∂W∂P=1ΔxΔy0000001ΔxΔyMinimization of CΩ,ZNSSD with respect to ΔP gives the least-squares solution of ΔP:(7)ΔP=-∑Ω∇f∂W∂PT×∇f∂W∂P×∑Ω∇f∂W∂PT×Irefx+Wx,y;ΔP-I-ref-ΔfΔgIdefx+Wx,y;P-I-defwhere Δf=∑ΩIref(x+W(x,y;ΔP))-I-ref2 and Δg=∑ΩIdef(x+W(x,y;P))-I-def2.Based on the minimization procedure, the incremental shape functionW(x,y;ΔP) is inverted and composed with the current estimated shape function. The obtained result is the updated shape function of the deformed subset, from which the incremental parameter vector can be determined. The operation can be expressed as follows:(8)Wx,y;P←Wx,y;P∘W-1x,y;ΔP=1+uxuyuvx1+vyv0011+ΔuxΔuyΔuΔvx1+ΔvyΔv001-1The iteration is repeated until the preset convergence condition is satisfied. In this work, the convergence condition,ΔPi≤10-6, is predefined.


## 2.2. The Influence of Speckle Patterns on 2D-DIC Measurements

Speckle patterns have an important influence on the accuracy in the displacements measurement of DIC [23]. To demonstrate the influence of speckle patterns on 2D-DIC measurements, eight reference speckle images obtained from real experiments were utilized in the following numerical experiments. The image acquisition operation is done in a darkroom with an illuminance of 0. In this environment, the ambient illuminance can be adjusted by using a brightness-adjustable white artificial light, and the reading of the illuminance value is read by a digital illuminometer. We slowly increased the brightness of the artificial light in the condition that the room is completely dark, continuously read the ambient illuminance values by digital illuminometer under various brightness conditions, and captured image until the illuminometer reading is approximately equal to 60 lux or other values.Figure2 illustrates the reference images and their corresponding power spectrums. The pattern A was acquired by randomly spraying black and white paints on flat specimen surface under the condition of the illumination of 60 lux and exposure time of 100 ms. The patterns B, C, and D were obtained from the same specimen of the speckle pattern A but different illumination or exposure time. The pattern E was acquired another flat specimen surface under the same condition of the pattern A. The patterns F, G, and H were obtained from the same specimen of the pattern E but unequal illumination or exposure time. All of the speckle patterns were acquired by a CCD camera with a fixed aperture lens (f = 5.6). The specific parameters for different speckle patterns are listed in Table 1. The illuminance is adjusted by changing the brightness of the white artificial light. The exposure time is adjusted by the camera software.Table 1
Specific parameters for different speckle patterns.
Speckle pattern Illumination (lux) Exposure time (ms) A 60 100 B 60 50 C 40 100 D 80 50 E 60 100 F 60 50 G 40 100 H 80 50Figure 2
Eight reference images used in numerical experiment and their corresponding speckle spectrums.
(a) (b) (c) (d) (e) (f) (g) (h)For each speckle pattern, 20 subpixel translated images are generated by applying appropriate shifts in the Fourier domain of the reference speckle pattern according to the shift theorem [35]. The subpixel displacements are applied in the x-direction, ranging from 0 to 1 pixel with a spacing of 0.05 pixels between two successive speckle patterns. The displacements of each translated speckle pattern were computed by using IC-GN subpixel registration algorithm.To quantitatively evaluate the accuracy of different natural patterns adopted in this numerical experiment, the systematic error of the computed displacements associated with speckle pattern is represented as mean bias error [36]. The mean bias error of the measured displacement is defined as follows: (9)pe=pmean-pimpwhere pmean=1/N∑i=1Npi represents the mean of the N estimated displacements and pimp denotes the exact imposed displacement.Figure3 shows the mean bias error as a function of preassigned subpixel displacements for the eight speckle patterns (solid lines). The sinusoidal-shaped mean bias error can be attributed to the interpolation error. This observation agrees well with existing results reported in [37]. By inspection, it is seen that (1) speckle pattern has a significant influence on the bias error; the mean bias errors of different speckle patterns with the same illumination and exposure are different (compare the experimental results by DIC of speckle patterns A and E); (2) lighting condition has a significant influence on the bias error; the mean bias errors of different speckle patterns with different illumination and the same exposure are different (compare the experimental results by DIC of speckle patterns A and G); (3) for given speckle patterns, whether or not the patterns and illuminations differ, it is always possible to find appropriate exposure time to produce approximate bias errors.Figure 3
Interpolation biases by DIC (marks and solid lines), by the theoretical equation (dashed lines) for eight speckle patterns with different illuminations and exposure time.


## 3. Deformation Monitoring System

### 3.1. Systematic Error Estimation

Sinusoidal-shaped systematic error in 2D-DIC was periodic with a period of 1 pixel. Schreier et al. attributed this periodic error to imperfect interpolation [35]. To obtain subpixel accuracies, grayscale must be evaluated at noninteger locations in 2D-DIC. Therefore, gray values and gray-value derivatives must be interpolated. On the other hand, the IC-GN algorithm must estimate the gradients of the reference image. Gradients can be estimated by interpolation. Nonideal interpolation will lead to this systematic error, which is also called interpolation bias [38]. The interpolation bias is inevitable because the ideal interpolation cannot be implemented in practice. If the dependence of interpolation bias upon the interpolation algorithm and the speckle pattern can be determined analytically, the quantitative criterion for interpolation bias estimation can be predicted, assessment of speckle patterns will be benefited, and appropriate exposure time for different patterns under different illumination will be gained based on this quantitative criterion.In the absence of noise, the measured displacementp(u,v) is not equal to the actual displacement p0(u0,v0) due to subpixel interpolation and gradient estimation. The imperfect interpolation will introduce systematic errors, which are referred to as interpolation bias pbias=p-p0 [35]. Reference function Iref(x,y) with domain Ω is sampled to produce a reference sequence Irefm,n=Iref(m,n). For two-dimensional situation, translate Iref(x,y) towards the positive direction of x by u0 units and towards the positive direction of y by v0 units to obtain the deformed function Iref(x-u0,y-v0), which is sampled to produce a deformed sequence Idef[m,n]=Iref(m-u0,n-v0). If convolution-based interpolation is employed, the reconstruction of the reference function Iref(x,y) and deformed function Idef(x,y) are (10)Irefx,y=∑m=-∞+∞∑n=-∞+∞Irefm,nϕx-m,y-n(11)Idefx,y=∑m=-∞+∞∑n=-∞+∞Idefm,nφx-m,y-nwhere ϕ(x,y) and φ(x,y) denote the interpolation basis for the reference sequence and deformed sequence, respectively. For convenient description, suppose a continuous function h(x,y) with continuous-time Fourier transform h^(νx,νy)=∫-∞+∞∫-∞+∞h(x,y)e-j2π(νxx+νyy)dxdy and its translated version h(x-u0,y-v0). The functions h(x,y) and h(x-u0,y-v0) are, respectively, sampled to produce the reference sequence Idef[m,n] and the deformed sequence Idef[m,n]; (10) and (11) can been rewritten as (12)Irefx,y=∑k=-∞+∞∑l=-∞+∞hk,lϕx-k,y-l(13)Idefx,y=∑k=-∞+∞∑l=-∞+∞hk-u0,l-v0φx-k,y-lThe corresponding continuous-time Fourier transform is(14)I^refνx,νy=ϕ^νx,νy∑k=-∞+∞∑l=-∞+∞h^νx-k,νy-l(15)I^defνx,νy=φ^νx,νy∑k=-∞+∞∑l=-∞+∞e-j2πνx-ku0+νy-lv0h^νx-k,νy-lFor translational situation, the following assumptions are used:(16)I-ref≈I-def(17)Δf≈ΔgAccording to (7), the measured displacement p(u,v) satisfies (18)∑m=-∞+∞∑n=-∞+∞Irefm,n-Idefm+u,n+v∇Irefm,n=0where Iref(m,n)-Idef(m+u,n+v) is the interpolation error sequence, ∇Iref(m,n) is the gradients of reference sequence, and their discrete-time Fourier transforms are X(νx,νy)=F(Iref(νx,νy))-F(Idef(νx,νy)) and Y(νx,νy)=∑m=-∞+∞∑n=-∞+∞∇Iref(m,n)e-j2π(νxm+νyn), respectively. F(Iref(νx,νy)) and F(Idef(νx,νy)) are the discrete-time Fourier transforms of Iref(x,y) and Idef(x,y), respectively. Applying Parseval’s theorem, the frequency representation of (18) can be written as (19)∫-1/21/2∫-1/21/2Xνx,νyY∗νx,νy+X∗νx,νyYνx,νydνxdνy=0The Poisson summation formulae are employed to present F(Iref(νx,νy)) in terms of I^ref(νx,νy) and F(Idef(νx,νy)) in terms of I^def(νx,νy):(20)FIrefν=∑m=-∞+∞∑n=-∞+∞Irefm,ne-j2πνxm+νyn=∑m=-∞+∞∑n=-∞+∞h^νx-m,νy-n(21)FIdefν=∑m=-∞+∞∑n=-∞+∞Idefm+u,n+ve-j2πνxm+νyn=∑m=-∞+∞∑n=-∞+∞ej2πνx-mu+νy-nvφ^νx-m,νy-n∑k=-∞+∞∑l=-∞+∞e-j2πνx-ku0+νy-lv0h^νx-k,νy-lTherefore, the form of X(νx,νy) is given by(22)Xνx,νy=∑m=-∞+∞∑n=-∞+∞h^νx-m,νy-n-∑m=-∞+∞∑n=-∞+∞ej2πνx-mu+νy-nvφ^νx-m,νy-n∑k=-∞+∞∑l=-∞+∞e-j2πνx-ku0+νy-lv0h^νx-k,νy-lThe continuous Fourier transform of ∇Iref(m,n) is denoted as ∇I^ref(νx,νy) provided that ∇I^ref(νx,νy)=j2π(νx+νy)ϕ^(νx,νy)∑k=-∞+∞∑l=-∞+∞h^(νx-k,νy-l). An application of the Poisson summation equation yields Y(νx,νy) in terms of ∇I^ref(νx,νy):(23)Yνx,νy=∑m=-∞+∞j2πνx-m+νy-nϕ^νx-m,νy-n∑k=-∞+∞∑l=-∞+∞h^νx-k,νy-lWith the expressions ofX(νx,νy) and Y(νx,νy), (19) can be rewritten as(24)I∫-1/21/2∫-1/21/2∑m1=-∞+∞∑n1=-∞+∞ej2πνx-m1u+νy-n1vφ^νx-m1,νy-n1∑m2=-∞+∞∑n2=-∞+∞νx-m2,νy-n2ϕ^νx-m2,νy-n2×∑k1=-∞+∞∑l1=-∞+∞e-j2πνx-k1u0+νy-l1v0h^νx-k1,νy-l1∑k2=-∞+∞∑l2=-∞+∞h^∗νx-k2,νy-l2dνxdνy=0Since in practical situationspbias≪1, it is reasonable to employ a linear approximation(25)ej2πνx-m1u+νy-n1v≈eνx-k1u0+νy-l1v01+j2πνx-m+νy-npbiasSubstituting (25) into (24), utilizing the band-limited hypothesis of h(x,y), and recognizing that ϕ^(νx,νy) and φ^(νx,νy) are approximation of an ideal low-pass filter, for simplicity, let actual displacement along the x-axis be u0; it is evident that there is no interpolation bias along the y-axis due to symmetry. Hence, pbias can be approximatively derived as follows: (26)pbias≈12π∫-1/21/2∫-1/21/2Bνx,νyh^νx,νy2dνxdνy∫-1/21/2∫-1/21/2νx2ϕ^νx,νyφ^νx,νyh^νx,νy2dνxdνysin⁡2πu0Bνx,νy=νx-1ϕ^νx-1,νy+νxϕ^νx,νy+νx+1ϕ^νx+1,νyφ^νx-1,νy-φ^νx+1,νyEquation (26) becomes identical to (41) given by Su et al. in [39]. Similarly, (26) explains the well-known sinusoidal-shaped curves of the interpolation bias pbias with respect to the prescribed subpixel displacement; it also explicitly presents the dependence of the interpolation bias pbias upon the reference function h(x,y) and interpolation basis ϕ(x,y) and φ(x,y) in the frequency domain. Within (26), h^(νx,νy)2 is the power spectrum of h(x,y),B(νx,νy) is exclusively determined by the subpixel and gradient interpolation algorithm, and it has been called the interpolation bias kernel by Su [38, 39]. In this work both the deformed image and the gradient of reference image were interpolated quintic B-spline interpolation (known as a convolution-based interpolation). The uniform quintic B-spline basis function can be expressed as(27)Φi,5x=15!0x∉xi,xi+6ξi5x∈xi,xi+1ξi5-6ξi-15x∈xi+1,xi+2ξi5-6ξi-15+15ξi-25x∈xi+2,xi+36-ξi5-65-ξi5+154-ξi5x∈xi+3,xi+46-ξi5-65-ξi5x∈xi+4,xi+56-ξi5x∈xi+5,xi+6where ξi=(x-xi)/(xi+1-xi).Equation (26) implies that the interpolation bias is mainly determined by the interpolation bias kernel B(νx,νy) and the power spectrum. The amplitude of interpolation bias pbias plays a central role in this work and is of importance because it quantifies the bias error response for different patterns, thus providing a criterion of exposure adjustments of the deformation monitoring system.The interpolation biases of the eight speckle patterns utilized in Section2.2 were estimated by theory (see (26)). The interpolation biases by DIC and by theory are illustrated in Figure 3. Figure 3 demonstrates that the interpolation biases by DIC and by theory (dashed lines) show good agreement. The correctness of theoretical estimations of interpolation bias is verified. Figure 4 indicates the standard deviation errors of computed displacements for these eight speckle patterns. It is observed from Figure 4 that the standard deviation errors of DIC are approximately stable and do not depend on the imposed subpixel displacement.Figure 4
Standard deviation error of measured displacement for the eight speckle patterns.


### 3.2. Exposure Adjustment Scheme and Experimental Verification

In order to acquire appropriate speckle patterns with constant systematic interpolation basis for multiple monitoring locations, the exposure time of cameras must be adjustable. Based on this consideration, a deformation monitoring system with multicamera based on 2D-DIC is established in this work. The deformation monitoring system consists of four illuminometers, four digital CMOS cameras, and four prime lenses. To minimize the negative impact of natural patterns and varying lighting condition and ensure the same accuracy at different location of the shipwreck, we develop an exposure adjustment scheme, which can adjust exposure time of the digital CMOS cameras according to the interpolation bias amplitude of patterns captured at different locations. For this purpose, an initial value of exposure time and interpolation bias amplitude should first be selected, and then the illuminance of regions of interest of the shipwreck is monitored in real time by illuminometers. Speckle images of the surface of the shipwreck are acquired by CMOS cameras. Afterwards, the amplitudes of interpolation biaspbias of each image are firstly calculated and then compared with the initial amplitude pbias∗; if pbias∗ approximately equals pbias∗, it means that the interpolation bias estimations of this pattern are identical with the predefined values, and the captured natural pattern can be used as a reference image. If pbias is not equal to pbias∗, it means that the interpolation bias estimations of this pattern are different from the predefined values and the exposure time should be adjusted. Both image collection and the value of interpolation bias kernel will be reperformed in the same way as stated in the foregoing step. The process is ended until pbias approximately equals pbias∗. During above process, if the illumination changes more than 50%, the exposure time should be adjusted as well. For clarity, Figure 5 presents a simple flowchart describing the fundamental principle of the proposed exposure adjustment scheme.Figure 5
Flow chart of the exposure adjustment scheme.Actual natural patterns of the Song Dynasty shipwreck are utilized to verify the correctness of proposed exposure adjustment scheme for deformation monitoring system. Four different locations are chosen as regions of interest, where the illuminations of surfaces are different. These four exact locations of the region of interest are marked in Figure6; the corresponding illuminations are listed in Table 2. Considering the characteristics of low contrast and anisotropy of natural grains of shipwreck, the initial value of exposure time is set to 200ms, and pbias∗ is set to the initial value of 1.3e-2. According to the exposure adjustment scheme (Figure 5) mentioned above, four images have been recorded by the CMOS cameras with different exposures. The exposure time of each appropriate reference natural pattern is shown in the final column of Table 2. The four natural patterns shown in Figure 7 are utilized for further analysis. Figure 8 illustrates their corresponding power spectrums.Table 2
Illumination and exposure for natural speckle patterns.
Natural pattern Illumination (lux) Exposure time (ms) A 40.4 800 B 59.5 400 C 80.6 400 D 100.3 800Figure 6
Four regions of interests extracted by the deformation monitoring system.Figure 7
Four natural patterns with different illuminations and exposure time.
(a) (b) (c) (d)Figure 8
Power spectrums of the four natural speckle patterns.
(a) (b) (c) (d)Firstly, the interpolation bias of DIC was evaluated. Because it is difficult to perform regular translation experiments for shipwreck, 20 deformed images were also produced by shifting in Fourier space, which is suggested to be used since it will not introduce extra bias. For each pattern, quintic B-spline was employed, respectively, to estimate the gradients of the reference image and to interpolate the subpixel registration of the deformed images. Then, the interpolation bias was estimated by (26). The value of pbias for each pattern is calculated and given in Table 3.Table 3
Value ofpbias∗ for natural speckle patterns.
Natural pattern p b i a s ∗ A 1.2952e-2 B 1.2956e-2 C 1.2960e-2 D 1.2961e-2The interpolation biases by DIC and by (26) are illustrated in Figure 9. Figure 9 indicates that (1) the theoretical estimations show good agreement with the DIC results; (2) the interpolation biases of the four natural patterns almost coincide with each other, which satisfy the requirements of the deformation monitoring system to the consistency of bias errors. Standard deviation errors of measured displacement for the four natural speckle patterns were shown in Figure 10.Figure 9
Interpolation biases by DIC (marks and solid lines) and by the theoretical equation (dashed lines) for four natural patterns.Figure 10
Standard deviation error of measured displacement for the four natural speckle patterns.


## 3.1. Systematic Error Estimation

Sinusoidal-shaped systematic error in 2D-DIC was periodic with a period of 1 pixel. Schreier et al. attributed this periodic error to imperfect interpolation [35]. To obtain subpixel accuracies, grayscale must be evaluated at noninteger locations in 2D-DIC. Therefore, gray values and gray-value derivatives must be interpolated. On the other hand, the IC-GN algorithm must estimate the gradients of the reference image. Gradients can be estimated by interpolation. Nonideal interpolation will lead to this systematic error, which is also called interpolation bias [38]. The interpolation bias is inevitable because the ideal interpolation cannot be implemented in practice. If the dependence of interpolation bias upon the interpolation algorithm and the speckle pattern can be determined analytically, the quantitative criterion for interpolation bias estimation can be predicted, assessment of speckle patterns will be benefited, and appropriate exposure time for different patterns under different illumination will be gained based on this quantitative criterion.In the absence of noise, the measured displacementp(u,v) is not equal to the actual displacement p0(u0,v0) due to subpixel interpolation and gradient estimation. The imperfect interpolation will introduce systematic errors, which are referred to as interpolation bias pbias=p-p0 [35]. Reference function Iref(x,y) with domain Ω is sampled to produce a reference sequence Irefm,n=Iref(m,n). For two-dimensional situation, translate Iref(x,y) towards the positive direction of x by u0 units and towards the positive direction of y by v0 units to obtain the deformed function Iref(x-u0,y-v0), which is sampled to produce a deformed sequence Idef[m,n]=Iref(m-u0,n-v0). If convolution-based interpolation is employed, the reconstruction of the reference function Iref(x,y) and deformed function Idef(x,y) are (10)Irefx,y=∑m=-∞+∞∑n=-∞+∞Irefm,nϕx-m,y-n(11)Idefx,y=∑m=-∞+∞∑n=-∞+∞Idefm,nφx-m,y-nwhere ϕ(x,y) and φ(x,y) denote the interpolation basis for the reference sequence and deformed sequence, respectively. For convenient description, suppose a continuous function h(x,y) with continuous-time Fourier transform h^(νx,νy)=∫-∞+∞∫-∞+∞h(x,y)e-j2π(νxx+νyy)dxdy and its translated version h(x-u0,y-v0). The functions h(x,y) and h(x-u0,y-v0) are, respectively, sampled to produce the reference sequence Idef[m,n] and the deformed sequence Idef[m,n]; (10) and (11) can been rewritten as (12)Irefx,y=∑k=-∞+∞∑l=-∞+∞hk,lϕx-k,y-l(13)Idefx,y=∑k=-∞+∞∑l=-∞+∞hk-u0,l-v0φx-k,y-lThe corresponding continuous-time Fourier transform is(14)I^refνx,νy=ϕ^νx,νy∑k=-∞+∞∑l=-∞+∞h^νx-k,νy-l(15)I^defνx,νy=φ^νx,νy∑k=-∞+∞∑l=-∞+∞e-j2πνx-ku0+νy-lv0h^νx-k,νy-lFor translational situation, the following assumptions are used:(16)I-ref≈I-def(17)Δf≈ΔgAccording to (7), the measured displacement p(u,v) satisfies (18)∑m=-∞+∞∑n=-∞+∞Irefm,n-Idefm+u,n+v∇Irefm,n=0where Iref(m,n)-Idef(m+u,n+v) is the interpolation error sequence, ∇Iref(m,n) is the gradients of reference sequence, and their discrete-time Fourier transforms are X(νx,νy)=F(Iref(νx,νy))-F(Idef(νx,νy)) and Y(νx,νy)=∑m=-∞+∞∑n=-∞+∞∇Iref(m,n)e-j2π(νxm+νyn), respectively. F(Iref(νx,νy)) and F(Idef(νx,νy)) are the discrete-time Fourier transforms of Iref(x,y) and Idef(x,y), respectively. Applying Parseval’s theorem, the frequency representation of (18) can be written as (19)∫-1/21/2∫-1/21/2Xνx,νyY∗νx,νy+X∗νx,νyYνx,νydνxdνy=0The Poisson summation formulae are employed to present F(Iref(νx,νy)) in terms of I^ref(νx,νy) and F(Idef(νx,νy)) in terms of I^def(νx,νy):(20)FIrefν=∑m=-∞+∞∑n=-∞+∞Irefm,ne-j2πνxm+νyn=∑m=-∞+∞∑n=-∞+∞h^νx-m,νy-n(21)FIdefν=∑m=-∞+∞∑n=-∞+∞Idefm+u,n+ve-j2πνxm+νyn=∑m=-∞+∞∑n=-∞+∞ej2πνx-mu+νy-nvφ^νx-m,νy-n∑k=-∞+∞∑l=-∞+∞e-j2πνx-ku0+νy-lv0h^νx-k,νy-lTherefore, the form of X(νx,νy) is given by(22)Xνx,νy=∑m=-∞+∞∑n=-∞+∞h^νx-m,νy-n-∑m=-∞+∞∑n=-∞+∞ej2πνx-mu+νy-nvφ^νx-m,νy-n∑k=-∞+∞∑l=-∞+∞e-j2πνx-ku0+νy-lv0h^νx-k,νy-lThe continuous Fourier transform of ∇Iref(m,n) is denoted as ∇I^ref(νx,νy) provided that ∇I^ref(νx,νy)=j2π(νx+νy)ϕ^(νx,νy)∑k=-∞+∞∑l=-∞+∞h^(νx-k,νy-l). An application of the Poisson summation equation yields Y(νx,νy) in terms of ∇I^ref(νx,νy):(23)Yνx,νy=∑m=-∞+∞j2πνx-m+νy-nϕ^νx-m,νy-n∑k=-∞+∞∑l=-∞+∞h^νx-k,νy-lWith the expressions ofX(νx,νy) and Y(νx,νy), (19) can be rewritten as(24)I∫-1/21/2∫-1/21/2∑m1=-∞+∞∑n1=-∞+∞ej2πνx-m1u+νy-n1vφ^νx-m1,νy-n1∑m2=-∞+∞∑n2=-∞+∞νx-m2,νy-n2ϕ^νx-m2,νy-n2×∑k1=-∞+∞∑l1=-∞+∞e-j2πνx-k1u0+νy-l1v0h^νx-k1,νy-l1∑k2=-∞+∞∑l2=-∞+∞h^∗νx-k2,νy-l2dνxdνy=0Since in practical situationspbias≪1, it is reasonable to employ a linear approximation(25)ej2πνx-m1u+νy-n1v≈eνx-k1u0+νy-l1v01+j2πνx-m+νy-npbiasSubstituting (25) into (24), utilizing the band-limited hypothesis of h(x,y), and recognizing that ϕ^(νx,νy) and φ^(νx,νy) are approximation of an ideal low-pass filter, for simplicity, let actual displacement along the x-axis be u0; it is evident that there is no interpolation bias along the y-axis due to symmetry. Hence, pbias can be approximatively derived as follows: (26)pbias≈12π∫-1/21/2∫-1/21/2Bνx,νyh^νx,νy2dνxdνy∫-1/21/2∫-1/21/2νx2ϕ^νx,νyφ^νx,νyh^νx,νy2dνxdνysin⁡2πu0Bνx,νy=νx-1ϕ^νx-1,νy+νxϕ^νx,νy+νx+1ϕ^νx+1,νyφ^νx-1,νy-φ^νx+1,νyEquation (26) becomes identical to (41) given by Su et al. in [39]. Similarly, (26) explains the well-known sinusoidal-shaped curves of the interpolation bias pbias with respect to the prescribed subpixel displacement; it also explicitly presents the dependence of the interpolation bias pbias upon the reference function h(x,y) and interpolation basis ϕ(x,y) and φ(x,y) in the frequency domain. Within (26), h^(νx,νy)2 is the power spectrum of h(x,y),B(νx,νy) is exclusively determined by the subpixel and gradient interpolation algorithm, and it has been called the interpolation bias kernel by Su [38, 39]. In this work both the deformed image and the gradient of reference image were interpolated quintic B-spline interpolation (known as a convolution-based interpolation). The uniform quintic B-spline basis function can be expressed as(27)Φi,5x=15!0x∉xi,xi+6ξi5x∈xi,xi+1ξi5-6ξi-15x∈xi+1,xi+2ξi5-6ξi-15+15ξi-25x∈xi+2,xi+36-ξi5-65-ξi5+154-ξi5x∈xi+3,xi+46-ξi5-65-ξi5x∈xi+4,xi+56-ξi5x∈xi+5,xi+6where ξi=(x-xi)/(xi+1-xi).Equation (26) implies that the interpolation bias is mainly determined by the interpolation bias kernel B(νx,νy) and the power spectrum. The amplitude of interpolation bias pbias plays a central role in this work and is of importance because it quantifies the bias error response for different patterns, thus providing a criterion of exposure adjustments of the deformation monitoring system.The interpolation biases of the eight speckle patterns utilized in Section2.2 were estimated by theory (see (26)). The interpolation biases by DIC and by theory are illustrated in Figure 3. Figure 3 demonstrates that the interpolation biases by DIC and by theory (dashed lines) show good agreement. The correctness of theoretical estimations of interpolation bias is verified. Figure 4 indicates the standard deviation errors of computed displacements for these eight speckle patterns. It is observed from Figure 4 that the standard deviation errors of DIC are approximately stable and do not depend on the imposed subpixel displacement.Figure 4
Standard deviation error of measured displacement for the eight speckle patterns.


## 3.2. Exposure Adjustment Scheme and Experimental Verification

In order to acquire appropriate speckle patterns with constant systematic interpolation basis for multiple monitoring locations, the exposure time of cameras must be adjustable. Based on this consideration, a deformation monitoring system with multicamera based on 2D-DIC is established in this work. The deformation monitoring system consists of four illuminometers, four digital CMOS cameras, and four prime lenses. To minimize the negative impact of natural patterns and varying lighting condition and ensure the same accuracy at different location of the shipwreck, we develop an exposure adjustment scheme, which can adjust exposure time of the digital CMOS cameras according to the interpolation bias amplitude of patterns captured at different locations. For this purpose, an initial value of exposure time and interpolation bias amplitude should first be selected, and then the illuminance of regions of interest of the shipwreck is monitored in real time by illuminometers. Speckle images of the surface of the shipwreck are acquired by CMOS cameras. Afterwards, the amplitudes of interpolation biaspbias of each image are firstly calculated and then compared with the initial amplitude pbias∗; if pbias∗ approximately equals pbias∗, it means that the interpolation bias estimations of this pattern are identical with the predefined values, and the captured natural pattern can be used as a reference image. If pbias is not equal to pbias∗, it means that the interpolation bias estimations of this pattern are different from the predefined values and the exposure time should be adjusted. Both image collection and the value of interpolation bias kernel will be reperformed in the same way as stated in the foregoing step. The process is ended until pbias approximately equals pbias∗. During above process, if the illumination changes more than 50%, the exposure time should be adjusted as well. For clarity, Figure 5 presents a simple flowchart describing the fundamental principle of the proposed exposure adjustment scheme.Figure 5
Flow chart of the exposure adjustment scheme.Actual natural patterns of the Song Dynasty shipwreck are utilized to verify the correctness of proposed exposure adjustment scheme for deformation monitoring system. Four different locations are chosen as regions of interest, where the illuminations of surfaces are different. These four exact locations of the region of interest are marked in Figure6; the corresponding illuminations are listed in Table 2. Considering the characteristics of low contrast and anisotropy of natural grains of shipwreck, the initial value of exposure time is set to 200ms, and pbias∗ is set to the initial value of 1.3e-2. According to the exposure adjustment scheme (Figure 5) mentioned above, four images have been recorded by the CMOS cameras with different exposures. The exposure time of each appropriate reference natural pattern is shown in the final column of Table 2. The four natural patterns shown in Figure 7 are utilized for further analysis. Figure 8 illustrates their corresponding power spectrums.Table 2
Illumination and exposure for natural speckle patterns.
Natural pattern Illumination (lux) Exposure time (ms) A 40.4 800 B 59.5 400 C 80.6 400 D 100.3 800Figure 6
Four regions of interests extracted by the deformation monitoring system.Figure 7
Four natural patterns with different illuminations and exposure time.
(a) (b) (c) (d)Figure 8
Power spectrums of the four natural speckle patterns.
(a) (b) (c) (d)Firstly, the interpolation bias of DIC was evaluated. Because it is difficult to perform regular translation experiments for shipwreck, 20 deformed images were also produced by shifting in Fourier space, which is suggested to be used since it will not introduce extra bias. For each pattern, quintic B-spline was employed, respectively, to estimate the gradients of the reference image and to interpolate the subpixel registration of the deformed images. Then, the interpolation bias was estimated by (26). The value of pbias for each pattern is calculated and given in Table 3.Table 3
Value ofpbias∗ for natural speckle patterns.
Natural pattern p b i a s ∗ A 1.2952e-2 B 1.2956e-2 C 1.2960e-2 D 1.2961e-2The interpolation biases by DIC and by (26) are illustrated in Figure 9. Figure 9 indicates that (1) the theoretical estimations show good agreement with the DIC results; (2) the interpolation biases of the four natural patterns almost coincide with each other, which satisfy the requirements of the deformation monitoring system to the consistency of bias errors. Standard deviation errors of measured displacement for the four natural speckle patterns were shown in Figure 10.Figure 9
Interpolation biases by DIC (marks and solid lines) and by the theoretical equation (dashed lines) for four natural patterns.Figure 10
Standard deviation error of measured displacement for the four natural speckle patterns.


## 4. Discussion

Actual natural patterns obtained from CMOS cameras are corrupted by noise unavoidably. The interpolation bias estimation is derived in the absence of noise. Thus further study needs to verify whether the adopted criterion is still valid in the presence of noise. To address this issue, numerical experiment based on natural patterns was conducted. In most cases, both the reference image and the target images contain noise. Natural pattern C shown in Figure7 was utilized in this numerical experiment. A series of deformed patterns is generated by shifting in Fourier space along the-axis byunits, whereranges from 0 to 1, with an increase of 0.05 pixels. The noise is assumed to be additive Gaussian white noise with zero-mean and standard deviation, where standard deviations are 0 (noise-free), 0.01, 0.02, and 0.04, respectively. Both the interpolation biases by DIC and by theoretical analysis when natural patterns are noisy are shown in Figure 11. Figure 11 indicates that (1) the interpolation bias increases as noise level increases; (2) the numerical experiment results show good agreement with the theoretical estimations; (3) the theoretical criterion adopted can successfully estimate the interpolation biases in the presence of image noise.Figure 11
Interpolation biases by DIC (marks and solid lines) and by the theoretical equation (dashed lines) for four natural patterns when both the reference and the deformed images are noisy.During the superimposition of the noise, the gray values of the speckle images are rescaled to prevent the truncation of the intensity at some pixels. This rescaling process produces changes in the strength and distribution of the noise signal as well as the original speckle image. Noise can change gray distribution of image, so the power spectrums of image contained noise are altered. As mentioned above, the variations of power spectrum caused the changes of interpolation bias.


## 5. Conclusion

A deformation monitoring system based on 2D-DIC for cultural relics protection is established in this work. Compared with regular DIC measurement system, the deformation monitoring system consists of four or more industrial cameras for simultaneously monitoring multiple locations of the cultural relics. In order to synthetically evaluate the results of deformation monitoring, the same bias errors are required for different monitoring locations. Considering the fact that only natural patterns can be used for correlation calculation and the particular illumination requirements in museum, it requires that the deformation monitoring system must have the capability to quantify bias errors and to adjust the quality of natural patterns in order to meet the consistency requirements of bias errors. To tackle this issue, a sinusoidal approximation for the interpolation bias of the IC-GN based DIC method is derived, and a speckle pattern assessment criterion is presented. Based on these theoretical analyses, exposure adjustment scheme of the deformation monitoring system is proposed. The correctness of these approaches is verified by numerical experiments using actual natural patterns. Furthermore, the theoretical criterion remains valid in the presence of noise. Thus, it is believed that the exposure adjustment scheme is promising for developing flexible and robust in situ deformation monitoring systems for use in museum environment, making 2D-DIC technique a really useful tool for in situ long-term monitoring of cultural relics. In addition to the monitoring of shipwreck in the museum environment in our study, the proposed monitoring system also is applicable to measure and monitor sample in other environment with low and varying illuminance.



---

*Source: 5240219-2018-08-29.xml*