# An Experimental Validation of Phase-Based Motion Magnification for Structures with Developing Cracks and Time-Varying Configurations

Authors: M. Civera; L. Zanotti Fragonara; P. Antonaci; G. Anglani; C. Surace
Journal: Shock and Vibration
 (2021)

---

## Abstract

In this study, Computer Vision and Phase-Based Motion Magnification (PBMM) are validated for continuous Structural Health Monitoring (SHM) purposes. The aim is to identify the exact instant of occurrence for damage or abrupt structural changes from video-extracted, very low amplitude (barely visible) vibrations. The study presents three experimental datasets: a box beam with multiple saw cuts of different lengths and angles, a beam with a full rectangular cross section and a mass added at the tip, and the spar of a prototype High-Aspect-Ratio wing. Both mode-shape- and frequency-based approaches are considered, showing the potential to identify the severity and position of the damage as well A high-definition, high-speed camera and a low-cost commercial alternative have been successfully utilised for these video acquisitions. Finally, the technique is also preliminarily tested for outdoor applications with smartphone cameras.


---

## Body

## 1. Introduction

Vibration-based Inspection (VBI) represents, to the present day, the standard framework for the assessment of civil structures and infrastructures and mechanical systems. Indeed, vibration signals originating from normal operating or imposed excitation conditions can be easily and cost-efficiently measured [1]. After being properly processed, they provide insight into the mechanical parameters which govern the local and global dynamic response of the target system [2, 3]. In particular, Structural Health Monitoring (SHM) resorts to several features to detect the presence of damage in a target structure [4]. This allows classifying the techniques mainly as Frequency-Based Damage Detection (FBDD) and Mode-shape-Based Damage Detection (MBDD) [5]. The resonant frequencies and the mode shapes have their own advantages and limitations, yet both are modal parameters, which are widely considered among the best choices for a Damage Sensitive Feature (DSF) [6]. However, the acquisition of the necessary vibration measurements is not free from practical issues. Attached sensors like accelerometers or strain gauges vary locally the mass and the bending stiffness of the investigated system. This has been proved to significantly alter the dynamic response of slender and flexible structures since these pointwise changes produce effects that can be mislabeled as damage [7]. Therefore, there is an urge for noninvasive acquisition techniques; this issue has gained more and more interest in recent years, with a flourishing of noncontact measurement techniques [8]. Even more, many SHM algorithms, while theoretically valid, require many output channels to record the structural response, to ensure the robustness of the results, and/or to provide a high spatial resolution (which is essential for damage localisation and severity assessment [9–13]).Computer Vision-based techniques solve both these issues, guaranteeing a high density of spatial information in a single acquisition, without altering the system of interest. Some relevant examples include the Lukas-Kanade Optical Flow algorithm, applied for video-based SHM in [14] and more recently in [15], and Target Tracking (see, e.g., [16]). This latter approach was also applied with Speeded-Up Robust Features (SURFs) for investigating the nonlinear dynamics of large oscillations in [17]; this and other brightness-based approaches are described in [18]. Nontarget approaches were proposed, for example, in [19]. Convolutional Neural Networks (CNNs) and Faster Region-based Convolutional Neural Networks (Faster R-CNNs) have been proposed as structural visual inspection methods to (fully or partially) replace human-conducted on-site inspections. Two examples can be found (in the same order) in [20, 21].Two recent reviews of optical techniques can be found in [22], for the broader field of structural dynamics, and in [23], specifically for SHM applications.The digital image correlation (DIC) technique deserves a special mention, due to its extensive use for noncontact strain measurements, especially for laboratory testing but also for damage detection (see, e.g., [24, 25]). A complete review of DIC applications can be found in [26].However, standard video acquisition algorithms are still restrained by their low accuracy for small and very small (barely visible) vibrations. In this sense, several magnifying techniques have been proposed in recent years (e.g., [27]). Nevertheless, these approaches were all hampered by the inherent limitations of brightness-based and Lagrangian approaches. An enhanced video processing technique, named Phase-Based Motion Magnification (PBMM [28]), has been proposed in recent years.The first known use of PBMM for the extraction of displacement time history and deflection shapes was documented in [29]. This algorithm has been then proved to be a valid video processing tool for modal identification and SHM tasks such as damage detection, localisation, and severity assessment [30, 31]. In particular, it has been applied in combination with blind source separation [32], 3D digital image correlation (3D-DIC) [33], unscented Kalman filter [34, 35], and particle tracking velocimetry [36]. It has been also extended to stereo-vision for the characterisation of 2-dimensional plates [37] and tested for the monitoring of ancient constructions of archaeological relevance in [38].Indeed, the resulting amplified motions can be used in several ways. Focussing on a single, (almost) pointwise pixel region will return a displacement time history which can be used as a single “virtual” output channel. In this sense, the PBMM algorithm can be utilised, e.g., as virtual motion microscopy [39] and, for SHM purposes, to perform FBDD techniques that require single output acquisitions, e.g., tracking damage-induced frequency shifts [31]. However, considering the large quantity of spatial information captured by any single frame, it makes much more practical sense to perform these operations on many pixel regions, mimicking a multioutput acquisition procedure. This high spatial density of information grants several advantages. For instance, it can be exploited for robust measurements, comparing the response at several locations at the same time. In this sense, video acquisitions are well-known to allow for robust Finite Element Model Updating (FEMU) [40]. Furthermore, multiple output positions allow tracking spatial differences as well, enabling damage localisation [31]. This is of paramount importance as many damage localisation algorithms, while perfectly valid on a theoretical basis, are hampered by the limitation of spatial resolution [11, 12, 41]. This is particularly valid for MBDD approaches.The exploitation of the frame spatial information can be brought to its maximum extent by producing full-field amplified video montages. In this case, the PBMM can be utilised to extract high-definition Operational Deflection Shapes (ODSs). Depending on the experimental setup and the intended use, both full-field and (single or multiple) pointwise PBMM applications have been tested for the researchers reported here. The PBMM was applied to one or multiple small pixel areas to extract amplified displacement time histories (for frequency-related damage indices) or to the whole frame to visualise and capture the global magnified motion of the structure (for mode-shape-related uses).However, to date, the PBMM technique has been validated only for pristine or already-damaged systems. Its potentialities for instantaneous structural assessment seem to have been not fully investigated. The studies reported here are dedicated to highlight the effectiveness of PBMM in this sense. For the easiest task of damage detection, this corresponds to pinpointing the exact instant when damage, or, more generally, a relevant structural change, happens. This can be then extended to the instantaneous damage localisation and for tracking the damage growth over time. Furthermore, since its first applications, the PBMM algorithm has been extensively utilised with high-speed video cameras [29, 34] and complicated experimental setups in a controlled laboratory environment. The case studies reported here show how this method can be extended to outdoor applications, also resorting to commercial-grade reflex and smartphone cameras.Based on the above premise, this paper is organised as follows. In Section2.1, the main theoretical aspects behind the PBMM procedure are recalled. In Section 2.2, the experimental setups are briefly described. Sections 3 and 4 report the results, with related discussion, and the conclusions, respectively.


## 2. Materials and Methods

### 2.1. The Theoretical Background of the PBMM

A video acquisition can be considered as an ordered sequence ofN 2-dimensional frames, each one made up of w×h pixels (width × height). Therefore, any point is defined at any instant t (expressed in terms of frame number) by two spatial coordinates, x=x,y, such that x∈1,w and y∈1,h. In turn, each frame can be decomposed (via Fourier transform) into a set of spatial harmonics, defined by an amplitude (i.e., a pixel brightness) and a phase.Considering the first frame (t=0) as the reference for the whole video, one has the initial brightness profile defined as Ix. For any t>0, this 2-dimensional profile changes and becomes Ix+δx,t, where δx,t is a time-varying, locally defined displacement function. The rationale is thus to directly link the local motions to the corresponding local phase difference.In this sense, Fleet and Jepson [42] and Gautama and Van Hulle [43] investigated the relationship between local phase differences and motion, proving that tracking the contours of constant phase in subsequent frames returns a good approximation of the motion field if the phase is properly spatiotemporally bandpassed in advance. This is doable as the motion δx,t can be assumed to be pixelwise linear and therefore representable as a linear combination of harmonics. However, it is necessary to decompose the 2D Fourier Transform of I into frequency bands that are localised in space x, scale ω, and orientation θ. The Complex Steerable Pyramid (CSP [44]) is utilised to this aim as a set of orientable, spatially multiscale, and spatially localised transform filters.Therefore, the whole brightness profile can be well-approximated by a 2-dimensional Fourier series, i.e.,(1)Ix+δx,t=∑ω=−∞+∞∑θ=1kAω,θeiφθx,t,where Aω,θx,t is the spatial amplitude at x,y and at the instant t, evaluated at the spatial scale ω and orientation θ. φω,θx,t=ωx+δx,t is the corresponding spatial phase.Then, the bandpassed phase can be obtained by simply temporally filtering the spatial phase with a DC balanced filter, thus removing the temporal mean and obtaining:(2)Bωx,t=ωδx,t.At this step, the vibrationδx,t has been fully derived from the image phase. If needed, the temporally filtering step can be used as well for isolating a specific temporal frequency band of interest. As it will be seen later, this can be used to amplify a narrow frequency band around the natural frequency of interest.Given the bandpassed phase from equation (2), the motion magnification procedure aims to postprocess the whole video recording, frame by frame, by modifying the spatial variations of its brightness profile. That means to shift some arbitrary phase contours, defined as φθx,y,t≡c for an arbitrary c.The magnified phase shift is therefore(3)S^ωx,t=Aω,θeiωx+δx,teiαBω=Aω,θeiωx+1+αδx,t,which corresponds to a proportionately amplified motion (exactly 1+α times the original). Thus, for a magnification factor α, the brightness profile becomes Ix+1+αδx,t.This procedure can be performed pointwise, to obtain one or more “virtual sensors” at specific locations, or globally, for a full-field amplification. By amplifying only the frequency range of interest, both applications were utilised in this research. The first case was used to obtain displacement time histories at specific target points, while the second one to extract Operational Deflection Shapes from the structure’s free oscillations.The complete procedure is recapitulated in Figure1, considering the most general case of full-field motion amplification. The approach for “virtual sensors” can be conceived as the same but limited to a (small) area inside the larger frame.Figure 1
The conceptual scheme of the PBMM algorithm, plus the edge detection routine for the extraction of ODSs. The numerical modelling of the OTS.In the first stage, the 2D Fourier Transform is computed over the whole frame. Then, the steered and scaled real-valued filters of the CST are applied to the resulting matrix of complex numbers. This is a convolution process, which results in complex-valued coefficients that define, at any point and along any direction, both the local phase and the local amplitude of the windowed basis functions. As explained before, the temporal bandpass filtering (performed over all frames) allows isolating the temporal frequency of interest, such that it can be amplified by a factorα while the other components of motion are left untouched (or even deamplified). An additional amplitude-weighted spatial smoothing can be performed at this point over σ adjacent pixels, to avoid unrealistic discontinuities due to excessive manipulation and/or noisy measurements. The procedure so far was also proved capable to visualise the otherwise invisible movements of a slot’s edge [31]. Finally, by running an edge detection algorithm on the video recordings phase-shifted around the natural frequency of interest (for this study, the Canny algorithm [45] was applied), the corresponding full-field Operational Deflection Shape (ODS) can be extracted as pixelwise edges. These can be obtained at both the extrema of their vibrational mode; moreover, several ODSs (one per each extremum and per each vibration period) can be collected from a single video acquisition, again granting robust results.The codes available in the toolboxes provided by Wadhwa et al. [28] and Wadhwa et al. [39] have been utilised with technical modifications throughout this whole study [46].


### 2.2. Experimental Setups

Two distinct acquisition systems have been utilised for the laboratory experiments of this interuniversity study. The Space Dynamics Laboratory at Cranfield University provided the two setups described in Figures2 and 3, for the analysis of the spar of a prototype High-Aspect-Ratio (HAR) wing [47] and of the cantilevered box beam, respectively. In both cases, the instrumentation included the following [48]:(i)
A Data Physics® Signal Force™ shaker(ii)
Its DP760 close-loop control system(iii)
A tungsten open-faced light source(iv)
An Olympus® I-speed 3 camera, with its levelled tripod and control interfaceFigure 2
The first experimental setup (prototype wing spar, Cranfield University). (1) Point of view from the high-speed video camera; (2) upper view of the spar without sensors attached; (3)-(4) the IMU sensors.Figure 3
The second experimental setup (cantilevered box beam, Cranfield University).This setup was also utilised in previous studies to investigate the large deflections of the HAR wing spar [17].The camera pixel density was set to1280×1024. Due to the limited internal memory capacity of the HD camera, only 4,897 frames could be recorded in a single acquisition. Thus, different frame rates were tested, reducing the total duration of the recordings. Specifically, 500 frames per second (fps) were utilised for the wing spar and 1000 fps for the cantilevered box beam. For all the acquisitions in both setups, a video stabilisation routine (available on the MatLab® Computer Vision System Toolbox) was applied before the PBMM procedure, to reduce the potentially deleterious effects of imperceptible environmental vibrations.Two Raspberry PI Inertial Measurement Units (IMUs) and one single-point Polytec® OFV-505 Laser Doppler Vibrometer (LDV) were utilised as benchmarks for the video-extracted time histories. The technical specifications of all devices are reported in Table1.Table 1
Details of the measurement equipment (wing spar and box beam).
Sensor typologyMeasureUnitAttached accelerometers:Raspberry PI® IMUs (MPU-6050 InvenSense™ sensor)Sampling frequency100HzAccelerometer full scale range±16gAccelerometer sensitivity2048LSB/gVideo camera:Olympus® I-speed 3™ (CMOS sensor)Shutter typeGlobal exposure, electronicShutter time1μsDisplay resolution1280× 1024PixelsFocusAutomaticLaser Doppler Vibrometer:Polytec® OFV-505 sensor Head™Sampling frequency2000HzLaser typeHelium neonFocusAutomaticLaser Wavelength633nmThe three laboratory experiments will be described in the following subsections. In all cases, the aim was to maintain the overall structural layout, a cantilevered beam-like structure, while introducing different typologies of time-dependent structural changes, i.e., an incremental reduction in stiffness and a sudden reduction in mass.


#### 2.2.1. The HAR Wing Prototype

The geometry of the aluminium spar (depicted in Figure2) is described in Table 2. No damage was inserted in this first case study, which was intended to preliminary benchmark video- and accelerometer-extracted displacement time histories. The camera was pointed to the spar trailing edge; the measurements from the two IMU sensors were utilised for comparison. The acceleration time histories were numerically integrated twice and then superimposed to the video acquisitions at the same cross sections (indicated in Figure 2).Table 2
The geometry of the wing spar.
ParameterMeasureUnitFree length (clamp to tip)ltip706.00mmThicknesst2.00mmMax width (at the clamped section)bmax180.00mmMid-length width (at the section of changing tampering,l=258 mm) b25856.10mmMin width (at the tip)bmin17.04mm


#### 2.2.2. The Multidamaged Box Beam

The second experimental case study (portrayed in Figure3) aimed to simulate a progressive increase of damage (in the sense of the crack depth) in the target structure. However, for practical reasons, it was not possible to have the crack length increase significantly during the recording (especially due to the maximum capacity of the camera internal memory). The practical solution was therefore to concatenate several damage steps. This is not too different from the procedure utilised by [49] to test the potentialities of vibration-based algorithms for instantaneous identification and damage detection.The damage was emulated by means of saw cuts applied to a cantilevered box beam. The geometry of this case study is reported in Table3. The cantilevered box beam can be seen as a first approximation of many structural components widely utilised in aerospace and mechanical engineering, such as turbine blades [50–52].Table 3
The geometry of the box beam.
ParameterMeasureUnitFree lengthL716.00mmWidthW25.00mmHeightH25.00mmThickness of the hollow sectionT2.00mmThe man-made slots were intended to mimic open surface cracks and were inserted in subsequent steps as described in Table4 and portrayed in Figure 4 (case #0 corresponds to the pristine box beam). The first notch was cut at x1=205 mm from the clamped section, perpendicularly to the box beam main axis. The second notch was cut at x2=319 mm (the distance between the two cuts was chosen to be enough to not influence the stress field of one another). This second slot was cut with an inclination of 5 : 1 (i.e., an angle of 11.3° with respect to the perpendicular) to simulate a slanted crack.Table 4
Damage levels inserted in the box beam.
Damage cased1 (mm)d2 (mm)∗#0——#15—#210—#315—#4155#51510#61515∗Intended as the component of the length perpendicular to the box beam main axis.Figure 4
The damage scenarios considered. (a) Crack locations and angles. (b) The severity levels considered (the first case, withd1 = 1 mm, is omitted).
(a)(b)For the definition of the ODSs, 66 points were considered, equally spaced betweenx0=55 mm and the tip (xtip=716 mm). The LDV, pointed at the box beam mid-length, was utilised to benchmark the frequency captured by the video measurements.


#### 2.2.3. The Beam with Falling Mass

The Laboratory of Nondestructive Testing at Politecnico di Torino provided the materials for the third set of experiments. The aim was to identify instantaneous changes of the structure mass from its frequency response. To this aim, a simple cantilever beam was considered, with a removable mass of 8.0 grams added at its tip (Figure5). In this case, a series of output channels were considered as virtual sensors, to investigate the influence of an instantaneous structural change, due to the mass variation, on the beam response at different locations along the beam length. The red dot in Figure 5 (2) corresponds to the half-length of the beam (250 mm from the clamped cross section). Three other pixel regions (marked by the yellow squares) were considered, corresponding to four strain gauges attached to the beam (three at the extrados and one at the intrados, as indicated by the numbers #1 to #4). Specifically, HBM half-bridge linear strain gauges were utilised, with one measuring grid (length 10 mm), resistance 120 Ω, and gauge factor 2.07±1%. These were connected to a laptop via an HBM QuantumX MX440B DAQ and amplifier system. The Catman Easy v3.5.1 software was used for the acquisition and postprocessing of the data. The sampling frequency was set to 100 Hz.Figure 5
The third experimental setup (Politecnico di Torino). (1) The complete setup. (2) Point of view of the camera; the pixels selected for the virtual accelerometer are marked by the red dot (mid-length output channel) and the yellow squares (corresponding to the positions of the four strain gauges; #3 and #4 were applied, respectively, on the upper and lower side of the beam at the same distance from the clamped cross section). (3) Close up on the added mass (which falls during the video acquisition).A commercial-grade reflex camera (specifically, a Canon EOS 1100 D, mounted on a tripod) was utilised for the acquisition, with1280×720 pixels per frame, i.e., circa 30% fewer pixels than in the high-speed, high-definition camera utilised for the first two sets of experiments. The frame rate was also noticeably lower at only 25 fps, i.e., 20 times lower than the lowest sampling frequency considered for the high-speed camera. The camera was positioned on a rigid surface, without any tripod or specific support but with its image stabiliser turned on. The room illumination (neon lights) was utilised. Further technical details can be found in Table 5, while the geometries of the cantilevered beam and added mass are reported in Table 6.Table 5
Details of the reflex camera.
Canon® EOS 1100D™ (CMOS sensor)PropertyShutter typeFocal plane, electronicShutter timeAutomatically set (min 0.25–max 16.7μs)Display resolution1280× 720FocusManually setTable 6
Geometry of the beam and the added mass.
ParameterMeasureUnitBeam lengthLb495.00mmBeam widthWb25.00mmBeam thicknessHb2.00mmTotal length of the added massLw20.00mmLength of the added mass laying over the beamLw′12.00mmWidth of the added massWw14.00mmThickness of the added massHw3.50mm


## 2.1. The Theoretical Background of the PBMM

A video acquisition can be considered as an ordered sequence ofN 2-dimensional frames, each one made up of w×h pixels (width × height). Therefore, any point is defined at any instant t (expressed in terms of frame number) by two spatial coordinates, x=x,y, such that x∈1,w and y∈1,h. In turn, each frame can be decomposed (via Fourier transform) into a set of spatial harmonics, defined by an amplitude (i.e., a pixel brightness) and a phase.Considering the first frame (t=0) as the reference for the whole video, one has the initial brightness profile defined as Ix. For any t>0, this 2-dimensional profile changes and becomes Ix+δx,t, where δx,t is a time-varying, locally defined displacement function. The rationale is thus to directly link the local motions to the corresponding local phase difference.In this sense, Fleet and Jepson [42] and Gautama and Van Hulle [43] investigated the relationship between local phase differences and motion, proving that tracking the contours of constant phase in subsequent frames returns a good approximation of the motion field if the phase is properly spatiotemporally bandpassed in advance. This is doable as the motion δx,t can be assumed to be pixelwise linear and therefore representable as a linear combination of harmonics. However, it is necessary to decompose the 2D Fourier Transform of I into frequency bands that are localised in space x, scale ω, and orientation θ. The Complex Steerable Pyramid (CSP [44]) is utilised to this aim as a set of orientable, spatially multiscale, and spatially localised transform filters.Therefore, the whole brightness profile can be well-approximated by a 2-dimensional Fourier series, i.e.,(1)Ix+δx,t=∑ω=−∞+∞∑θ=1kAω,θeiφθx,t,where Aω,θx,t is the spatial amplitude at x,y and at the instant t, evaluated at the spatial scale ω and orientation θ. φω,θx,t=ωx+δx,t is the corresponding spatial phase.Then, the bandpassed phase can be obtained by simply temporally filtering the spatial phase with a DC balanced filter, thus removing the temporal mean and obtaining:(2)Bωx,t=ωδx,t.At this step, the vibrationδx,t has been fully derived from the image phase. If needed, the temporally filtering step can be used as well for isolating a specific temporal frequency band of interest. As it will be seen later, this can be used to amplify a narrow frequency band around the natural frequency of interest.Given the bandpassed phase from equation (2), the motion magnification procedure aims to postprocess the whole video recording, frame by frame, by modifying the spatial variations of its brightness profile. That means to shift some arbitrary phase contours, defined as φθx,y,t≡c for an arbitrary c.The magnified phase shift is therefore(3)S^ωx,t=Aω,θeiωx+δx,teiαBω=Aω,θeiωx+1+αδx,t,which corresponds to a proportionately amplified motion (exactly 1+α times the original). Thus, for a magnification factor α, the brightness profile becomes Ix+1+αδx,t.This procedure can be performed pointwise, to obtain one or more “virtual sensors” at specific locations, or globally, for a full-field amplification. By amplifying only the frequency range of interest, both applications were utilised in this research. The first case was used to obtain displacement time histories at specific target points, while the second one to extract Operational Deflection Shapes from the structure’s free oscillations.The complete procedure is recapitulated in Figure1, considering the most general case of full-field motion amplification. The approach for “virtual sensors” can be conceived as the same but limited to a (small) area inside the larger frame.Figure 1
The conceptual scheme of the PBMM algorithm, plus the edge detection routine for the extraction of ODSs. The numerical modelling of the OTS.In the first stage, the 2D Fourier Transform is computed over the whole frame. Then, the steered and scaled real-valued filters of the CST are applied to the resulting matrix of complex numbers. This is a convolution process, which results in complex-valued coefficients that define, at any point and along any direction, both the local phase and the local amplitude of the windowed basis functions. As explained before, the temporal bandpass filtering (performed over all frames) allows isolating the temporal frequency of interest, such that it can be amplified by a factorα while the other components of motion are left untouched (or even deamplified). An additional amplitude-weighted spatial smoothing can be performed at this point over σ adjacent pixels, to avoid unrealistic discontinuities due to excessive manipulation and/or noisy measurements. The procedure so far was also proved capable to visualise the otherwise invisible movements of a slot’s edge [31]. Finally, by running an edge detection algorithm on the video recordings phase-shifted around the natural frequency of interest (for this study, the Canny algorithm [45] was applied), the corresponding full-field Operational Deflection Shape (ODS) can be extracted as pixelwise edges. These can be obtained at both the extrema of their vibrational mode; moreover, several ODSs (one per each extremum and per each vibration period) can be collected from a single video acquisition, again granting robust results.The codes available in the toolboxes provided by Wadhwa et al. [28] and Wadhwa et al. [39] have been utilised with technical modifications throughout this whole study [46].


## 2.2. Experimental Setups

Two distinct acquisition systems have been utilised for the laboratory experiments of this interuniversity study. The Space Dynamics Laboratory at Cranfield University provided the two setups described in Figures2 and 3, for the analysis of the spar of a prototype High-Aspect-Ratio (HAR) wing [47] and of the cantilevered box beam, respectively. In both cases, the instrumentation included the following [48]:(i)
A Data Physics® Signal Force™ shaker(ii)
Its DP760 close-loop control system(iii)
A tungsten open-faced light source(iv)
An Olympus® I-speed 3 camera, with its levelled tripod and control interfaceFigure 2
The first experimental setup (prototype wing spar, Cranfield University). (1) Point of view from the high-speed video camera; (2) upper view of the spar without sensors attached; (3)-(4) the IMU sensors.Figure 3
The second experimental setup (cantilevered box beam, Cranfield University).This setup was also utilised in previous studies to investigate the large deflections of the HAR wing spar [17].The camera pixel density was set to1280×1024. Due to the limited internal memory capacity of the HD camera, only 4,897 frames could be recorded in a single acquisition. Thus, different frame rates were tested, reducing the total duration of the recordings. Specifically, 500 frames per second (fps) were utilised for the wing spar and 1000 fps for the cantilevered box beam. For all the acquisitions in both setups, a video stabilisation routine (available on the MatLab® Computer Vision System Toolbox) was applied before the PBMM procedure, to reduce the potentially deleterious effects of imperceptible environmental vibrations.Two Raspberry PI Inertial Measurement Units (IMUs) and one single-point Polytec® OFV-505 Laser Doppler Vibrometer (LDV) were utilised as benchmarks for the video-extracted time histories. The technical specifications of all devices are reported in Table1.Table 1
Details of the measurement equipment (wing spar and box beam).
Sensor typologyMeasureUnitAttached accelerometers:Raspberry PI® IMUs (MPU-6050 InvenSense™ sensor)Sampling frequency100HzAccelerometer full scale range±16gAccelerometer sensitivity2048LSB/gVideo camera:Olympus® I-speed 3™ (CMOS sensor)Shutter typeGlobal exposure, electronicShutter time1μsDisplay resolution1280× 1024PixelsFocusAutomaticLaser Doppler Vibrometer:Polytec® OFV-505 sensor Head™Sampling frequency2000HzLaser typeHelium neonFocusAutomaticLaser Wavelength633nmThe three laboratory experiments will be described in the following subsections. In all cases, the aim was to maintain the overall structural layout, a cantilevered beam-like structure, while introducing different typologies of time-dependent structural changes, i.e., an incremental reduction in stiffness and a sudden reduction in mass.


### 2.2.1. The HAR Wing Prototype

The geometry of the aluminium spar (depicted in Figure2) is described in Table 2. No damage was inserted in this first case study, which was intended to preliminary benchmark video- and accelerometer-extracted displacement time histories. The camera was pointed to the spar trailing edge; the measurements from the two IMU sensors were utilised for comparison. The acceleration time histories were numerically integrated twice and then superimposed to the video acquisitions at the same cross sections (indicated in Figure 2).Table 2
The geometry of the wing spar.
ParameterMeasureUnitFree length (clamp to tip)ltip706.00mmThicknesst2.00mmMax width (at the clamped section)bmax180.00mmMid-length width (at the section of changing tampering,l=258 mm) b25856.10mmMin width (at the tip)bmin17.04mm


### 2.2.2. The Multidamaged Box Beam

The second experimental case study (portrayed in Figure3) aimed to simulate a progressive increase of damage (in the sense of the crack depth) in the target structure. However, for practical reasons, it was not possible to have the crack length increase significantly during the recording (especially due to the maximum capacity of the camera internal memory). The practical solution was therefore to concatenate several damage steps. This is not too different from the procedure utilised by [49] to test the potentialities of vibration-based algorithms for instantaneous identification and damage detection.The damage was emulated by means of saw cuts applied to a cantilevered box beam. The geometry of this case study is reported in Table3. The cantilevered box beam can be seen as a first approximation of many structural components widely utilised in aerospace and mechanical engineering, such as turbine blades [50–52].Table 3
The geometry of the box beam.
ParameterMeasureUnitFree lengthL716.00mmWidthW25.00mmHeightH25.00mmThickness of the hollow sectionT2.00mmThe man-made slots were intended to mimic open surface cracks and were inserted in subsequent steps as described in Table4 and portrayed in Figure 4 (case #0 corresponds to the pristine box beam). The first notch was cut at x1=205 mm from the clamped section, perpendicularly to the box beam main axis. The second notch was cut at x2=319 mm (the distance between the two cuts was chosen to be enough to not influence the stress field of one another). This second slot was cut with an inclination of 5 : 1 (i.e., an angle of 11.3° with respect to the perpendicular) to simulate a slanted crack.Table 4
Damage levels inserted in the box beam.
Damage cased1 (mm)d2 (mm)∗#0——#15—#210—#315—#4155#51510#61515∗Intended as the component of the length perpendicular to the box beam main axis.Figure 4
The damage scenarios considered. (a) Crack locations and angles. (b) The severity levels considered (the first case, withd1 = 1 mm, is omitted).
(a)(b)For the definition of the ODSs, 66 points were considered, equally spaced betweenx0=55 mm and the tip (xtip=716 mm). The LDV, pointed at the box beam mid-length, was utilised to benchmark the frequency captured by the video measurements.


### 2.2.3. The Beam with Falling Mass

The Laboratory of Nondestructive Testing at Politecnico di Torino provided the materials for the third set of experiments. The aim was to identify instantaneous changes of the structure mass from its frequency response. To this aim, a simple cantilever beam was considered, with a removable mass of 8.0 grams added at its tip (Figure5). In this case, a series of output channels were considered as virtual sensors, to investigate the influence of an instantaneous structural change, due to the mass variation, on the beam response at different locations along the beam length. The red dot in Figure 5 (2) corresponds to the half-length of the beam (250 mm from the clamped cross section). Three other pixel regions (marked by the yellow squares) were considered, corresponding to four strain gauges attached to the beam (three at the extrados and one at the intrados, as indicated by the numbers #1 to #4). Specifically, HBM half-bridge linear strain gauges were utilised, with one measuring grid (length 10 mm), resistance 120 Ω, and gauge factor 2.07±1%. These were connected to a laptop via an HBM QuantumX MX440B DAQ and amplifier system. The Catman Easy v3.5.1 software was used for the acquisition and postprocessing of the data. The sampling frequency was set to 100 Hz.Figure 5
The third experimental setup (Politecnico di Torino). (1) The complete setup. (2) Point of view of the camera; the pixels selected for the virtual accelerometer are marked by the red dot (mid-length output channel) and the yellow squares (corresponding to the positions of the four strain gauges; #3 and #4 were applied, respectively, on the upper and lower side of the beam at the same distance from the clamped cross section). (3) Close up on the added mass (which falls during the video acquisition).A commercial-grade reflex camera (specifically, a Canon EOS 1100 D, mounted on a tripod) was utilised for the acquisition, with1280×720 pixels per frame, i.e., circa 30% fewer pixels than in the high-speed, high-definition camera utilised for the first two sets of experiments. The frame rate was also noticeably lower at only 25 fps, i.e., 20 times lower than the lowest sampling frequency considered for the high-speed camera. The camera was positioned on a rigid surface, without any tripod or specific support but with its image stabiliser turned on. The room illumination (neon lights) was utilised. Further technical details can be found in Table 5, while the geometries of the cantilevered beam and added mass are reported in Table 6.Table 5
Details of the reflex camera.
Canon® EOS 1100D™ (CMOS sensor)PropertyShutter typeFocal plane, electronicShutter timeAutomatically set (min 0.25–max 16.7μs)Display resolution1280× 720FocusManually setTable 6
Geometry of the beam and the added mass.
ParameterMeasureUnitBeam lengthLb495.00mmBeam widthWb25.00mmBeam thicknessHb2.00mmTotal length of the added massLw20.00mmLength of the added mass laying over the beamLw′12.00mmWidth of the added massWw14.00mmThickness of the added massHw3.50mm


## 2.2.1. The HAR Wing Prototype

The geometry of the aluminium spar (depicted in Figure2) is described in Table 2. No damage was inserted in this first case study, which was intended to preliminary benchmark video- and accelerometer-extracted displacement time histories. The camera was pointed to the spar trailing edge; the measurements from the two IMU sensors were utilised for comparison. The acceleration time histories were numerically integrated twice and then superimposed to the video acquisitions at the same cross sections (indicated in Figure 2).Table 2
The geometry of the wing spar.
ParameterMeasureUnitFree length (clamp to tip)ltip706.00mmThicknesst2.00mmMax width (at the clamped section)bmax180.00mmMid-length width (at the section of changing tampering,l=258 mm) b25856.10mmMin width (at the tip)bmin17.04mm


## 2.2.2. The Multidamaged Box Beam

The second experimental case study (portrayed in Figure3) aimed to simulate a progressive increase of damage (in the sense of the crack depth) in the target structure. However, for practical reasons, it was not possible to have the crack length increase significantly during the recording (especially due to the maximum capacity of the camera internal memory). The practical solution was therefore to concatenate several damage steps. This is not too different from the procedure utilised by [49] to test the potentialities of vibration-based algorithms for instantaneous identification and damage detection.The damage was emulated by means of saw cuts applied to a cantilevered box beam. The geometry of this case study is reported in Table3. The cantilevered box beam can be seen as a first approximation of many structural components widely utilised in aerospace and mechanical engineering, such as turbine blades [50–52].Table 3
The geometry of the box beam.
ParameterMeasureUnitFree lengthL716.00mmWidthW25.00mmHeightH25.00mmThickness of the hollow sectionT2.00mmThe man-made slots were intended to mimic open surface cracks and were inserted in subsequent steps as described in Table4 and portrayed in Figure 4 (case #0 corresponds to the pristine box beam). The first notch was cut at x1=205 mm from the clamped section, perpendicularly to the box beam main axis. The second notch was cut at x2=319 mm (the distance between the two cuts was chosen to be enough to not influence the stress field of one another). This second slot was cut with an inclination of 5 : 1 (i.e., an angle of 11.3° with respect to the perpendicular) to simulate a slanted crack.Table 4
Damage levels inserted in the box beam.
Damage cased1 (mm)d2 (mm)∗#0——#15—#210—#315—#4155#51510#61515∗Intended as the component of the length perpendicular to the box beam main axis.Figure 4
The damage scenarios considered. (a) Crack locations and angles. (b) The severity levels considered (the first case, withd1 = 1 mm, is omitted).
(a)(b)For the definition of the ODSs, 66 points were considered, equally spaced betweenx0=55 mm and the tip (xtip=716 mm). The LDV, pointed at the box beam mid-length, was utilised to benchmark the frequency captured by the video measurements.


## 2.2.3. The Beam with Falling Mass

The Laboratory of Nondestructive Testing at Politecnico di Torino provided the materials for the third set of experiments. The aim was to identify instantaneous changes of the structure mass from its frequency response. To this aim, a simple cantilever beam was considered, with a removable mass of 8.0 grams added at its tip (Figure5). In this case, a series of output channels were considered as virtual sensors, to investigate the influence of an instantaneous structural change, due to the mass variation, on the beam response at different locations along the beam length. The red dot in Figure 5 (2) corresponds to the half-length of the beam (250 mm from the clamped cross section). Three other pixel regions (marked by the yellow squares) were considered, corresponding to four strain gauges attached to the beam (three at the extrados and one at the intrados, as indicated by the numbers #1 to #4). Specifically, HBM half-bridge linear strain gauges were utilised, with one measuring grid (length 10 mm), resistance 120 Ω, and gauge factor 2.07±1%. These were connected to a laptop via an HBM QuantumX MX440B DAQ and amplifier system. The Catman Easy v3.5.1 software was used for the acquisition and postprocessing of the data. The sampling frequency was set to 100 Hz.Figure 5
The third experimental setup (Politecnico di Torino). (1) The complete setup. (2) Point of view of the camera; the pixels selected for the virtual accelerometer are marked by the red dot (mid-length output channel) and the yellow squares (corresponding to the positions of the four strain gauges; #3 and #4 were applied, respectively, on the upper and lower side of the beam at the same distance from the clamped cross section). (3) Close up on the added mass (which falls during the video acquisition).A commercial-grade reflex camera (specifically, a Canon EOS 1100 D, mounted on a tripod) was utilised for the acquisition, with1280×720 pixels per frame, i.e., circa 30% fewer pixels than in the high-speed, high-definition camera utilised for the first two sets of experiments. The frame rate was also noticeably lower at only 25 fps, i.e., 20 times lower than the lowest sampling frequency considered for the high-speed camera. The camera was positioned on a rigid surface, without any tripod or specific support but with its image stabiliser turned on. The room illumination (neon lights) was utilised. Further technical details can be found in Table 5, while the geometries of the cantilevered beam and added mass are reported in Table 6.Table 5
Details of the reflex camera.
Canon® EOS 1100D™ (CMOS sensor)PropertyShutter typeFocal plane, electronicShutter timeAutomatically set (min 0.25–max 16.7μs)Display resolution1280× 720FocusManually setTable 6
Geometry of the beam and the added mass.
ParameterMeasureUnitBeam lengthLb495.00mmBeam widthWb25.00mmBeam thicknessHb2.00mmTotal length of the added massLw20.00mmLength of the added mass laying over the beamLw′12.00mmWidth of the added massWw14.00mmThickness of the added massHw3.50mm


## 3. Results and Discussion

### 3.1. The HAR Wing Prototype

Regarding the spar of the prototype HAR wing, all the frames available in a single acquisition (4897, due to the internal memory limitation of the HD camera) were utilised. For a sampling frequency of 500 fps, this resulted in slightly more than nine seconds of recording. The video captured a random noise test with 0.0015 g2/Hz (defined in the range 2–60 Hz). A low magnification factor α=3, uniformly applied in the 2–20 Hz range, was sufficient to fully identify the pixelwise displacement time history. Through calibration, a conversion factor of 2.46 pixels per mm was found on the focus plane (circa 130 cm from the camera lens). No spatial smoothing was deemed necessary, so σ=1 was imposed here as well as in all the following experiments. Figure 6 shows the very good match achieved with the data captured by the two IMUs in the time domain (the superimposition of the two signals was made by eye from a human user). For the IMU sensors, the acceleration time series were numerically integrated twice and detrended to remove the integration drift. This first case study served as a calibration test for the algorithm. Indeed, its reliability for time-domain applications is needed for the identification of instantaneous changes in the natural frequencies and mode shapes, as it will be shown when applied to the other video acquisitions of the following experiments.Figure 6
Displacement time histories (wing spar): IMU sensors vs. video PBMM acquisitions (α=3).


### 3.2. The Multidamaged Box Beam

For the second setup, 11μs long triangular impulses were applied at the clamped base, with a peak value of 5 g. The focal length was about 160 cm, resulting in circa 2.93 pixels per mm. The free decay was recorded immediately after the impulse. The video camera frame rate was set to 1000 Hz, for a total duration of less than 4.9 s. The video-captured displacements time histories were benchmarked with the LDV acquisitions, as specified in [32] and here omitted for brevity.The seven video recordings were all amplified by a magnification factorα=16, defined in the same frequency range (30.0–50.0 Hz) for comparability. This was selected to cover the damage-induced frequency shift (estimated from the video recording and summarised in Table 7).Table 7
First natural frequency by damage scenario.
Damage caseFirst natural frequency (Hz)#0 (no damage)46.3#145.7#240.7#334.8#434.5#533.8#632.2For what concerns the ODSs, only 200 frames (i.e., a 0.2 s long tract) were selected for their extraction. This corresponds to a set of nine oscillations for the undamaged case and slightly less for the damaged scenarios (down to six for case #6). This was done since after relatively few oscillations, the (magnified) amplitude of the vibrations decayed noticeably (please note that the ODSs, unlike the mode shapes, are here directly captured from the structure displacements and are therefore not normalised). Instead, the difference among the amplitudes of the first five oscillations was found to be negligible for each recording.Thus, the minima of each oscillation period have been selected and stored, to obtain a time series of comparable deflection shapes. These have been then used to define a continuous envelope, made up by the first five subsequent minima of the deflection, for comparability. As mentioned before, the envelopes of the ODS coming from subsequent scenarios with increasing damage severity have been then concatenated to simulate time-changing structural conditions, to mimic damage growth as suggested in [49].The surface reported in Figure7(a) was then defined over time and space. This refers to the evolution over time of the first natural frequency, which gradually reduced from circa 46.3 Hz to 34.4 Hz due to the increasing local reduction in stiffness.Figure 7
(a) The concatenated minima of the magnified ODSs corresponding to the first natural frequency, ordered for increasing damage severity (0: undamaged; 1–6: damaged as described in Figure4). (b) ODS-based damage detection, comparing the tip deflection of the damaged scenarios to the pristine baseline. (c) The different displacements at three cross sections: x=215 (red), x=325 (blue), and x=L=716 mm (black) (cantilevered box beam, five periods per damage case, α=16).
(a)(b)(c)For damage detection purposes, it is possible to compare the tip deflection of the damaged cases to the pristine baseline. Figure7(b) reports data from the five ODS extracted from the video recording of the undamaged and damaged cases, for the purpose of statistical analysis. The experimental data for the undamaged case were used to estimate the parameters of a Gaussian distribution. The threshold for damage detection was set to three times its standard deviation, corresponding to a 99.73% confidence interval. Any value exceeding this threshold can be confidently linked to a statistically relevant increase of flexibility and thus (for unchanged external conditions) to damage.Furthermore, by comparing cases #1–#6 in increasing order, the progressive growth of the system flexibility is quite evident. This allows for tracking the increasing damage severity. By considering different slicing sections at arbitraryx-coordinates, it is further possible to compare the differential increases (Figure 7(c)). This may help in pinpointing the pointwise source of the additional flexibility, i.e., the damage location. Specifically, three points have been considered: the tip, the first point after the 2nd notch location, and the first point after the 1st notch location. Since the effects of the added flexibility increase when moving further away from the clamped cross section, the tip section is the most affected one by all damage steps. On the other hand, the effects of the first saw cut are almost invisible in the first section (these can be highlighted by normalising the ODSs over their maximum displacement, as performed in [31]), while the effects of the second saw cut can be appreciated even in the second section.


### 3.3. The Beam with Falling Mass

In the last set of experiments, the aim was to use PBMM-extracted displacement time histories to track a sudden change in the mass distribution of the target system. To this aim, 20 video acquisitions were performed: 5 with the beam mass-loaded at the tip (as portrayed in Figure5 without the added mass, i.e., with the beam statically unloaded) and 10 with the beam initially loaded and then with the mass falling mid-recording. In this latter case, this was achieved by slightly pushing the mass, which was purposely placed in a precarious balance. It was double-checked that this operation did not noticeably affect the transverse vibrations of the beam. The input was provided by impacting the beam clamped section. The lower pixel density and frame rate of the reflex camera allowed for much longer recordings, avoiding any problem regarding the internal memory limitation. All acquisitions lasted 40 s (1000 frames); only the relevant frames were retained, namely, each video recording was depurated of the first seconds after the impact, to focus on the free vibrations alone, and of the last portion, after the vibration faded out. This changed case by case, since the settling time for the mass-loaded and unloaded beam varied noticeably (from more than 20 s to less than 10 s). The calibration returned a conversion factor of 2.23 pixels per mm for a focal length of circa 115 cm. All amplifications were performed in the 6.0–10.0 Hz range, with an amplification factor α=4. For illustrative purposes only, the output channel highlighted in red in Figure 5 (2) is first discussed on its own in Figures 8 and 9.Figure 8
Results from the cantilevered beam (3rd experimental setup, mid-length virtual sensor,α=4). (a) the extracted time history; (b) the Fourier Transform of the same; (c) the corresponding time-frequency plot and the instantaneous frequency (IF).
(a)(b)(c)Figure 9
Results from the cantilevered beam (3rd experimental setup, mid-length virtual sensor). (a) An example of IF for the unloaded beam; (b) an example of IF for the mass-loaded beam; (c) boxplots of all acquisitions (5 mass-loaded, 5 unloaded, and 10 with falling mass) and the respective mean values and standard deviations. Please notice that the first seconds (corresponding to the initial impact) were removed from all signals.
(a)(b)(c)Figure8 reports an example of acquisition with the falling mass. In Figure 8(a), the displacement time history (in SI measures) is reported. Its Fourier Transform (Figure 8(b)) suggests how two separate frequency peaks coexist along the time interval considered. These results were double-checked with further measurements made with the four strain gauges indicated in Figure 5 (2), with a sampling frequency of 100 Hz. These were left attached for the duration of all experiments to maintain the mass and stiffness distribution unaltered by their presence.By analysing the video-extracted signal in the time-frequency domain (Figure8(c)), the nonstationarity of the response becomes evident, with a clear instant (at around 19 seconds after the input application) where the peak (corresponding to the first natural frequency of the system) increases by circa 1.16 Hz. This is directly linked to the decrease in the system’s overall mass.Figures9(a) and 9(b) report, respectively, two examples of the time-frequency analysis of the mass-loaded and unloaded configurations, in this order. The measurements refer to the same mid-length point considered above. Apart from confirming the difference in the frequency content of the response, it is clear as well how the beam loaded with the additional mass has a much longer settling time, as expected due to the decreased natural frequency [18]. Figure 9(c) reports the mean and the variance of the three typologies of experiments. The frequency of the loaded beam before the mass fall compares well with the results from the always loaded configuration. The frequency of the unloaded beam after the mass fall is slightly more scattered and higher than the results from the always unloaded configuration (the difference is nevertheless not larger than the second decimal digit). This is most probably due to the unavoidable transient effects which accompany the sudden structural change. These results, nevertheless, unmistakably prove that the PBMM procedure can discern the exact instant of these sudden changes, as well as the dynamic properties of the system under investigation before and after them, from vibrations barely or not at all visible with the naked eye. This was also proved achievable with commercial-grade, relatively low-cost equipment.As seen, the global effects of the mass reduction localised at the free tip can be detected and tracked over time at a distance at least equal to half of the beam length. By using the other output channels indicated in Figure5, it is furthermore possible to both define the influence length of this structural change and compare it to the measurements from the strain gauges.Figure10 reports this comparison. The dashed lines represent the IF as identified from the attached sensors, while the solid lines correspond to the PBMM measurements at the pixel regions corresponding to the same cross section. One can see that the values are consistent, even if the decay of the signal amplitude makes the PBMM less robust over time. The strain measurements, obtained from the physically attached sensors, returned more stable identifications. This can be however potentially solved by increasing the amplification factor.Figure 10
Identified Instantaneous Frequency time histories: PBMM (solid lines) vs. strain gauges (dashed lines), numbered as indicated in Figure5.


### 3.4. An Application for In Situ Surveys

To conclude this discussion, an example of an application for the in situ survey of real-life buildings is briefly described here.This last case study was performed outdoor on a partly cloudy day near Cranfield University, more specifically, in central Milton Keynes, United Kingdom. The exact location is specified in Figure11(a). The target was the two apartment buildings highlighted in red and green in Figure 11(b). The video was recorded from a nearby position (indicated by the black dot in Figure 11(b)) with a commercial smartphone camera, capturing 1920 × 1080 pixels per frame at 30 fps, placed on a flat and rigid surface. The point of view was angled with respect to the façades of both buildings, which makes the application more compelling but also more realistic for standard survey conditions. 15 points per building (marked in Figure 11(c)) were considered. The calibration returned a conversion rate of about 20.40 mm per pixel.Figure 11
(a) Location of the in situ survey. (b) Satellite view of the surveyed area. (c) The first frame of the recorded video, with the position of the 30 virtual sensors highlighted.
(a)(b)(c)The processed video lasted circa 6.67 seconds (200 frames) and was successively framed into four sections (reported in Figure12). The focus was on the horizontal oscillations of the two buildings on the focal plane. The four sections were then analysed separately for the robustness of the results. The frequency range between 0.5 and 2.0 Hz was amplified by a factor α=4, aiming at the lowest frequencies of the two structures.Figure 12
Four sections of the extracted displacement time histories (α=4; red: 15 output channels corresponding to the left building; green: 15 output channels corresponding to the right building).The vibrational behaviour of the two high-rise buildings can be most probably due to ambient vibrations, wind loads, and/or other natural excitation sources, plus human-made vibrations (such as road traffic from nearby roads). For both buildings, the displacements at all output channels (red and green time histories) were in phase, suggesting a dynamic response mainly governed by a fundamental flexural mode.The building on the left showed a higher frequency content at circa 1.1 Hz, while the one on the right at circa 0.7 Hz. This is most probably due to the different orientation of the two buildings, with the former one being stiffer in the direction of interest (which corresponds roughly to the axis of its main moment of inertia). This simple, preliminary example highlights the potential uses for in situ surveys. However, further study will be needed to verify the best uses for PBMM in outdoor environments with natural illumination, aiming at output-only system identification.


## 3.1. The HAR Wing Prototype

Regarding the spar of the prototype HAR wing, all the frames available in a single acquisition (4897, due to the internal memory limitation of the HD camera) were utilised. For a sampling frequency of 500 fps, this resulted in slightly more than nine seconds of recording. The video captured a random noise test with 0.0015 g2/Hz (defined in the range 2–60 Hz). A low magnification factor α=3, uniformly applied in the 2–20 Hz range, was sufficient to fully identify the pixelwise displacement time history. Through calibration, a conversion factor of 2.46 pixels per mm was found on the focus plane (circa 130 cm from the camera lens). No spatial smoothing was deemed necessary, so σ=1 was imposed here as well as in all the following experiments. Figure 6 shows the very good match achieved with the data captured by the two IMUs in the time domain (the superimposition of the two signals was made by eye from a human user). For the IMU sensors, the acceleration time series were numerically integrated twice and detrended to remove the integration drift. This first case study served as a calibration test for the algorithm. Indeed, its reliability for time-domain applications is needed for the identification of instantaneous changes in the natural frequencies and mode shapes, as it will be shown when applied to the other video acquisitions of the following experiments.Figure 6
Displacement time histories (wing spar): IMU sensors vs. video PBMM acquisitions (α=3).


## 3.2. The Multidamaged Box Beam

For the second setup, 11μs long triangular impulses were applied at the clamped base, with a peak value of 5 g. The focal length was about 160 cm, resulting in circa 2.93 pixels per mm. The free decay was recorded immediately after the impulse. The video camera frame rate was set to 1000 Hz, for a total duration of less than 4.9 s. The video-captured displacements time histories were benchmarked with the LDV acquisitions, as specified in [32] and here omitted for brevity.The seven video recordings were all amplified by a magnification factorα=16, defined in the same frequency range (30.0–50.0 Hz) for comparability. This was selected to cover the damage-induced frequency shift (estimated from the video recording and summarised in Table 7).Table 7
First natural frequency by damage scenario.
Damage caseFirst natural frequency (Hz)#0 (no damage)46.3#145.7#240.7#334.8#434.5#533.8#632.2For what concerns the ODSs, only 200 frames (i.e., a 0.2 s long tract) were selected for their extraction. This corresponds to a set of nine oscillations for the undamaged case and slightly less for the damaged scenarios (down to six for case #6). This was done since after relatively few oscillations, the (magnified) amplitude of the vibrations decayed noticeably (please note that the ODSs, unlike the mode shapes, are here directly captured from the structure displacements and are therefore not normalised). Instead, the difference among the amplitudes of the first five oscillations was found to be negligible for each recording.Thus, the minima of each oscillation period have been selected and stored, to obtain a time series of comparable deflection shapes. These have been then used to define a continuous envelope, made up by the first five subsequent minima of the deflection, for comparability. As mentioned before, the envelopes of the ODS coming from subsequent scenarios with increasing damage severity have been then concatenated to simulate time-changing structural conditions, to mimic damage growth as suggested in [49].The surface reported in Figure7(a) was then defined over time and space. This refers to the evolution over time of the first natural frequency, which gradually reduced from circa 46.3 Hz to 34.4 Hz due to the increasing local reduction in stiffness.Figure 7
(a) The concatenated minima of the magnified ODSs corresponding to the first natural frequency, ordered for increasing damage severity (0: undamaged; 1–6: damaged as described in Figure4). (b) ODS-based damage detection, comparing the tip deflection of the damaged scenarios to the pristine baseline. (c) The different displacements at three cross sections: x=215 (red), x=325 (blue), and x=L=716 mm (black) (cantilevered box beam, five periods per damage case, α=16).
(a)(b)(c)For damage detection purposes, it is possible to compare the tip deflection of the damaged cases to the pristine baseline. Figure7(b) reports data from the five ODS extracted from the video recording of the undamaged and damaged cases, for the purpose of statistical analysis. The experimental data for the undamaged case were used to estimate the parameters of a Gaussian distribution. The threshold for damage detection was set to three times its standard deviation, corresponding to a 99.73% confidence interval. Any value exceeding this threshold can be confidently linked to a statistically relevant increase of flexibility and thus (for unchanged external conditions) to damage.Furthermore, by comparing cases #1–#6 in increasing order, the progressive growth of the system flexibility is quite evident. This allows for tracking the increasing damage severity. By considering different slicing sections at arbitraryx-coordinates, it is further possible to compare the differential increases (Figure 7(c)). This may help in pinpointing the pointwise source of the additional flexibility, i.e., the damage location. Specifically, three points have been considered: the tip, the first point after the 2nd notch location, and the first point after the 1st notch location. Since the effects of the added flexibility increase when moving further away from the clamped cross section, the tip section is the most affected one by all damage steps. On the other hand, the effects of the first saw cut are almost invisible in the first section (these can be highlighted by normalising the ODSs over their maximum displacement, as performed in [31]), while the effects of the second saw cut can be appreciated even in the second section.


## 3.3. The Beam with Falling Mass

In the last set of experiments, the aim was to use PBMM-extracted displacement time histories to track a sudden change in the mass distribution of the target system. To this aim, 20 video acquisitions were performed: 5 with the beam mass-loaded at the tip (as portrayed in Figure5 without the added mass, i.e., with the beam statically unloaded) and 10 with the beam initially loaded and then with the mass falling mid-recording. In this latter case, this was achieved by slightly pushing the mass, which was purposely placed in a precarious balance. It was double-checked that this operation did not noticeably affect the transverse vibrations of the beam. The input was provided by impacting the beam clamped section. The lower pixel density and frame rate of the reflex camera allowed for much longer recordings, avoiding any problem regarding the internal memory limitation. All acquisitions lasted 40 s (1000 frames); only the relevant frames were retained, namely, each video recording was depurated of the first seconds after the impact, to focus on the free vibrations alone, and of the last portion, after the vibration faded out. This changed case by case, since the settling time for the mass-loaded and unloaded beam varied noticeably (from more than 20 s to less than 10 s). The calibration returned a conversion factor of 2.23 pixels per mm for a focal length of circa 115 cm. All amplifications were performed in the 6.0–10.0 Hz range, with an amplification factor α=4. For illustrative purposes only, the output channel highlighted in red in Figure 5 (2) is first discussed on its own in Figures 8 and 9.Figure 8
Results from the cantilevered beam (3rd experimental setup, mid-length virtual sensor,α=4). (a) the extracted time history; (b) the Fourier Transform of the same; (c) the corresponding time-frequency plot and the instantaneous frequency (IF).
(a)(b)(c)Figure 9
Results from the cantilevered beam (3rd experimental setup, mid-length virtual sensor). (a) An example of IF for the unloaded beam; (b) an example of IF for the mass-loaded beam; (c) boxplots of all acquisitions (5 mass-loaded, 5 unloaded, and 10 with falling mass) and the respective mean values and standard deviations. Please notice that the first seconds (corresponding to the initial impact) were removed from all signals.
(a)(b)(c)Figure8 reports an example of acquisition with the falling mass. In Figure 8(a), the displacement time history (in SI measures) is reported. Its Fourier Transform (Figure 8(b)) suggests how two separate frequency peaks coexist along the time interval considered. These results were double-checked with further measurements made with the four strain gauges indicated in Figure 5 (2), with a sampling frequency of 100 Hz. These were left attached for the duration of all experiments to maintain the mass and stiffness distribution unaltered by their presence.By analysing the video-extracted signal in the time-frequency domain (Figure8(c)), the nonstationarity of the response becomes evident, with a clear instant (at around 19 seconds after the input application) where the peak (corresponding to the first natural frequency of the system) increases by circa 1.16 Hz. This is directly linked to the decrease in the system’s overall mass.Figures9(a) and 9(b) report, respectively, two examples of the time-frequency analysis of the mass-loaded and unloaded configurations, in this order. The measurements refer to the same mid-length point considered above. Apart from confirming the difference in the frequency content of the response, it is clear as well how the beam loaded with the additional mass has a much longer settling time, as expected due to the decreased natural frequency [18]. Figure 9(c) reports the mean and the variance of the three typologies of experiments. The frequency of the loaded beam before the mass fall compares well with the results from the always loaded configuration. The frequency of the unloaded beam after the mass fall is slightly more scattered and higher than the results from the always unloaded configuration (the difference is nevertheless not larger than the second decimal digit). This is most probably due to the unavoidable transient effects which accompany the sudden structural change. These results, nevertheless, unmistakably prove that the PBMM procedure can discern the exact instant of these sudden changes, as well as the dynamic properties of the system under investigation before and after them, from vibrations barely or not at all visible with the naked eye. This was also proved achievable with commercial-grade, relatively low-cost equipment.As seen, the global effects of the mass reduction localised at the free tip can be detected and tracked over time at a distance at least equal to half of the beam length. By using the other output channels indicated in Figure5, it is furthermore possible to both define the influence length of this structural change and compare it to the measurements from the strain gauges.Figure10 reports this comparison. The dashed lines represent the IF as identified from the attached sensors, while the solid lines correspond to the PBMM measurements at the pixel regions corresponding to the same cross section. One can see that the values are consistent, even if the decay of the signal amplitude makes the PBMM less robust over time. The strain measurements, obtained from the physically attached sensors, returned more stable identifications. This can be however potentially solved by increasing the amplification factor.Figure 10
Identified Instantaneous Frequency time histories: PBMM (solid lines) vs. strain gauges (dashed lines), numbered as indicated in Figure5.


## 3.4. An Application for In Situ Surveys

To conclude this discussion, an example of an application for the in situ survey of real-life buildings is briefly described here.This last case study was performed outdoor on a partly cloudy day near Cranfield University, more specifically, in central Milton Keynes, United Kingdom. The exact location is specified in Figure11(a). The target was the two apartment buildings highlighted in red and green in Figure 11(b). The video was recorded from a nearby position (indicated by the black dot in Figure 11(b)) with a commercial smartphone camera, capturing 1920 × 1080 pixels per frame at 30 fps, placed on a flat and rigid surface. The point of view was angled with respect to the façades of both buildings, which makes the application more compelling but also more realistic for standard survey conditions. 15 points per building (marked in Figure 11(c)) were considered. The calibration returned a conversion rate of about 20.40 mm per pixel.Figure 11
(a) Location of the in situ survey. (b) Satellite view of the surveyed area. (c) The first frame of the recorded video, with the position of the 30 virtual sensors highlighted.
(a)(b)(c)The processed video lasted circa 6.67 seconds (200 frames) and was successively framed into four sections (reported in Figure12). The focus was on the horizontal oscillations of the two buildings on the focal plane. The four sections were then analysed separately for the robustness of the results. The frequency range between 0.5 and 2.0 Hz was amplified by a factor α=4, aiming at the lowest frequencies of the two structures.Figure 12
Four sections of the extracted displacement time histories (α=4; red: 15 output channels corresponding to the left building; green: 15 output channels corresponding to the right building).The vibrational behaviour of the two high-rise buildings can be most probably due to ambient vibrations, wind loads, and/or other natural excitation sources, plus human-made vibrations (such as road traffic from nearby roads). For both buildings, the displacements at all output channels (red and green time histories) were in phase, suggesting a dynamic response mainly governed by a fundamental flexural mode.The building on the left showed a higher frequency content at circa 1.1 Hz, while the one on the right at circa 0.7 Hz. This is most probably due to the different orientation of the two buildings, with the former one being stiffer in the direction of interest (which corresponds roughly to the axis of its main moment of inertia). This simple, preliminary example highlights the potential uses for in situ surveys. However, further study will be needed to verify the best uses for PBMM in outdoor environments with natural illumination, aiming at output-only system identification.


## 4. Conclusions

The study reported here focused on the detection of instantaneous changes in the vibrational response of a target structure, which can be directly linked to damage or a changing structural configuration. The potentialities of video processing in general, and Phase-Based Motion Magnification (PBMM) in particular, have been tested. Three different experimental setups were performed in two distinct laboratories, with two different acquisition systems–specifically, a high-speed, high-definition camera and a low-price alternative. Finally, the PBMM technique was also tested for in situ surveying outdoor and under natural illumination, with a commercial smartphone. The results (in order of appearance) proved the following:(1)
Under appropriate conditions, the PBMM can return displacement time histories as accurate as physically attached IMU sensors.(2)
By magnifying the whole frame rather than single-pixel areas, the structure ODSs can be extracted at any period of its oscillations and tracked over time to follow damage-induced local increases in the structure flexibility, which causes the structure’s oscillations to gradually increase in amplitude (for a constant driving force).(3)
The high pixel density can be further utilised to locate these areas of reduced stiffness.(4)
The time-frequency analysis of the PBMM-amplified displacement time histories can reliably and accurately highlight the exact instant of sudden structural changes at different locations on the structure itself.(5)
Commercial-grade reflex and smartphone cameras can provide reliable results, even at low frame rates and for outdoor uses with natural (thus, uncontrolled and nonoptimal) illumination. This makes them a viable alternative to high-speed cameras for damage assessment, even if at the cost of a lower folding frequency and overall diminished accuracy.These findings confirm the validity of Computer Vision and PBMM for instantaneous SHM applications, intended to monitor structural changes at the exact moment of their occurrence. This has been verified as well for more complex tasks such as damage localisation and severity assessment, which requires several output channels for both large spatial resolution and robustness. Depending on the intended use, both full-field and pointwise PBMM (over one or more small pixel areas) can be conveniently applied. Future studies will focus on applications for field surveys with low-cost instrumentation.



---

*Source: 5518163-2021-10-13.xml*