SUGISAKAMASANORI

Journal of Robotics, Networking and Artificial Life
Vol. 9(2); September (2022), pp. 192–201
ISSN (Online):2352-6386; ISSN (Print):2405-9021
https://alife-robotics.org/jrnal.html

Research Article

Feature Extraction of Mold Defects on Fine Arts Painting using
Derivative Oriented Thresholding
Hilman Nordin1, Bushroa Abdul Razak1,2, Norrima Mokhtar3, Mohd Fadzil Jamaludin2

Department of Mechanical Engineering, Faculty of Engineering, Universiti Malaya, 50603, Kuala Lumpur Malaysia
Centre of Advanced Manufacturing and Material Processing (AMMP Centre), Faculty of Engineering, Universiti Malaya, 50603, Kuala Lumpur
Malaysia3
Department of Electrical Engineering, Faculty of Engineering, Universiti Malaya, 50603, Kuala Lumpur Malaysia
1
2

ARTIC LE INFO
Article History
Received 04 December 2021
Accepted 19 July 2022

Keywords
Image processing
Defect detection
Derivative oriented thresholding
Fine art

ABSTRACTPaintings can be damaged by natural causes or accidents. One of the crucial natural damages was
frequently caused by mold defects. The mold discovery is an important step in the restoration ofdamaged paintings. The procedure is usually tedious and depends heavily on the qualitative visual
judgement of an expert restorer. The aim of this work is to assist the restoration process via an
automatic mold defect detection technique based on derivative and image analysis. This new
method, designated as Derivative Level Thresholding (DLT), combines binarization and detection
algorithms to detect mold rapidly and accurately from scanned high-resolution images of apainting. This work also benchmarks the performance of the proposed method to existingbinarization techniques of Otsu’s Thresholding Method, Minimum Error Thresholding (MET)and Contrast Adjusted Thresholding Method. Experimental results from the analysis of 20
samples from high-resolution scans of 2 mold-stained painting have shown that the DLT method
is the most robust with the highest sensitivity rate of 84.73% and 68.40% accuracy.
© 2022 The Author. Published by Sugisaka Masanori at ALife Robotics Corporation Ltd
This is an open access article distributed under the CC BY-NC 4.0 license(http://creativecommons.org/licenses/by-nc/4.0/)

suitable method of repair. Traditionally, restoration
processes are conducted manually, relying heavily on the
judgment and skills of the restorers. Using the traditional
method, assessment and monitoring before restoration iscostly, time consuming and depending too much on theskills of the restorers.

1. Introduction
The physical damage on the artwork caused by several
factors such as mishandling, high humidity, rapid
fluctuations in temperatures and interactions with
pollution-dust particles in the air. These factors cause
growth of molds, craquelures and other types of wear on
the materials especially for improperly stored artworks.
If not detected and treated early, it will cause significant
irreversible damage to the original painting. However,these defects can be evaluated and subsequently repairedby expert restorers. The flow of the restoration work
consists of assessing the current conditions of thepaintings, identifying defects and determining the

To counter this issue, with the advent of technology,
computer-assisted scanning and image processing
techniques are available to assist in the restoration
process. Many researchers focus on nondestructive
detection and assessment via vision-based technology as
tool to improve productivity, cost and time management.

Corresponding author’s E-mail: hilmannordin@siswa.um.edu.my, norrimamokhtar@um.edu.my, bushroa@um.edu.my, ibnjamaludin@um.edu.my

192

These techniques use hi-tech detection methods andcomputer algorithms to objectively emulate the visualassessment and judgment of the restorers. Imaging
technologies for the scanning of 2D artworks includes
mass spectrometry[1], photoluminescence spectroscopy[2], x-ray fluorescence analysis[3], and shearography[4].
Imaging data obtained from the scanning of the 2D
artworks can then be analysed to extract the required
information using techniques such as heuristic graphsearching[5],hyperspectral
crack
detection[6],colorimetry and watershed segmentation[7], and userintervention-based detection methods[8], [9].

Non-destructive or non-invasive analytical imaging
technique is preferable for painting monitoring andassessment. In this context, mesoscopy, enables therecording of details at very high resolutions. Images
scanned at high resolution contains accurate colour
information that may be discernible to the naked eye.
Subsequent processing of the scanned data, such asimage segmentations and thresholding, enables
identification of features such as defects andinconsistencies. The combined technique of mesoscopy
and image analysis have been demonstrated by Win et al[10] in the automatic detection of defects in coated metal
specimens for the manufacturing industry. However, thetechnique has yet to be utilized in the restoration
processes of artworks such as drawings and paintings.
This paper proposes a derivative oriented thresholding
method for the automated detection of mold on paintings.
This method combines two processes of binarization anddetection. In the binarization process, the original image
is converted into a binary image containing the defects
while the detection process identifies the defects using afiltration approach. This newly proposed binarization
process is benchmarked with existing thresholding based
binarization
methodsofContrast
AdjustedThresholding[10], Otsu’s Method[11] and Minimum
Error Thresholding (MET)[12]. The resulting binary
images from these different methods are then filtered in
the detection process, where the mold detection results
are then compared to a ground truth image. The groundtruth image is produced by manually labeling each pixel
as either defects or non-defects, based on the feedback
from an expert restorer. The accuracy and sensitivity ofthe detection results are then rated by comparison to thisground truth image.

Fig. 1 Painting 1 – Ink on Paper

2. Methodology and Testing
2.1. Image Acquisition and Sampling
Two paintings comprising of ink sketches on paper by awell-known Malaysian artist, the late Ibrahim Hussein(1936-2009), were selected as the subject of this study.
A Niji-X High-Resolution Scanner (Kyoto University,
Kyoto, Japan) was used to scan the paintings at aresolution of 600 dpi. Figure 1 and Figure 2 show the twopaintings, designated as Painting 1 and Painting 2. These

Fig.2 Painting 2 – Ink on Paper

193

two paintings were chosen as they contained molddefects.
For mold detection process, the painting images are
divided into smaller image samples. A 200 x 200 pixels
capture area was assigned for the sample size, whichcorresponded to a scanned area of 8.4 mm x 8.4 mm on
the actual artwork. Thus, each recorded pixel wasapproximately 42 µm x 42 µm in size. The sampling
process has generated a total of 494 sample images from
Painting 1 and 391 sample images from Painting 2.
However, for the purpose of this study, only 20 image
samples containing mold defects were selected, 14 from
Painting 1 and 6 from Painting 2.

Fig. 3 Samples Selected from Painting 1
2.3. Derivative Level Thresholding

2.2. Image Acquisition and Preparation

A new binarization method which does not produce athreshold level is proposed in this study. Instead, themethod derives the final binary image from a
combination of binary images that was binarized at
different threshold levels. For simplicity, the maximum
number of threshold images to be considered is set at 20,
which would produce a set of 20 binary images 𝐼𝐼𝑗𝑗 . The
binary images are collected by setting the threshold value
𝑡𝑡𝑗𝑗 = j/20, where j = 1, 2, …, 20.

The captured images were then converted into grayscale
images for pre-processing. The grayscale image can be
expressed in L gray levels [1,2, … , 𝐿𝐿] . Each levelconsists of m points and the total number of points, M, isthe sum of 𝑚𝑚𝑗𝑗 where j are the individual levels by: 𝑀𝑀 =𝑚𝑚1 + 𝑚𝑚2 +. . . +𝑚𝑚𝐿𝐿 . In the grayscale images, the molddefects are expected to have gray levels value, s, between
0 (black) and L (white). Figure 3 and Figure 4 shows theselected 20 samples of the originally scanned specimen
images containing the mold defects.

The resultant image consists of the background andforeground, and the mold defects would be identified in
one or more images as black pixels against a whitebackground. The percentage values of black pixels foreach 20 images are calculated, and then plotted against
the value of j = 1, 2, …, 20. The gradient of the curve, y,is calculated by taking its first order derivative, dy/dx,

Defects extraction from the grayscale images was
performed by transforming them into their corresponding
binary images. Binarization is carried out by determining
the threshold value, t, which is a gray level that divides
the images into two sets: 𝐶𝐶0 (foreground) and𝐶𝐶1 (background). The set 𝐶𝐶0 consists of points with graylevels of [1,2, … , 𝑡𝑡] while 𝐶𝐶1 (have gray levels of [𝑡𝑡 +1, 𝑡𝑡 + 2, … , 𝐿𝐿].

As the mold defects are postulated to have gray levels
values between 1 to L, determining the correct thresholdvalue is essential for mold defect detection. In this paper,
three established image thresholding methods wereselected to determine the threshold gray value, t, and forbinarization of the images. The image thresholding
methods selected are Otsu’s Method for Thresholding[11], Minimum Error Thresholding (MET) Method andContrast Adjusted Otsu Thresholding. The three methods
will be compared to the proposed Derivative Level
Thresholding method.

Fig. 4 Samples Selected from Painting 2
194

yielding a bimodal histogram. Both the curves y and its
derivative, dy/dx, is shown in Figure 5 and Figure 6
respectively.

BinaryImage 𝑰𝑰𝒏𝒏 ;SubtractedBinaryImage (m) ;NumberOfPoints R; //From Equation (1) and (2)ModifiedFirstPoint 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 1 ∗;ModifiedSecondPoint 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 2 *;From Figure 5, we can see that the black pixel percentage
values increase with the increase in the value of jincreases. In the grayscale samples, the molds havegrayscale values that are lighter than the artwork strokes
but are darker than the background values. This results inthe bimodal curve having two peaks that contains theartwork strokes and the background grayscale values, as
indicated in Figure 6. The mold defects can be extracted
by processing the images in between the two peaks.

for 𝑚𝑚 = 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 1 ∗∶ 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 2 ∗for 𝑛𝑛 = 1: 𝑹𝑹
𝑆𝑆(𝑚𝑚) = 𝐼𝐼𝑛𝑛+1 − 𝐼𝐼𝑛𝑛
end
end
…

Subsequently, the locations of the two peaks, 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 1 and𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 2 , where 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 2 > 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 1 are then located, fromwhich the number of points, r, is determined. The
number of points, r, is calculated by:
𝑟𝑟 = 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 2 − 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 1 for 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 2 − 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 1 > 4(1)

𝑟𝑟 = �𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 2 − 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 1 � + 4 for 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 2 − 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 1 ≤ 4 (2)For the second case in (2), the value of 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 1 and𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 2 is modified to value of points 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 1 * and𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 2 *:𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 1 ∗ = 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 1 − 2
𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 2 ∗ = 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 2 + 2

Fig. 5 Percentage of black pixel value, yThe percentage of black pixels of the resulting images

(3)

(4)

where 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 2 ∗> 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 1 ∗ to supply two additional
points for the case so that 𝑟𝑟 > 4, while for the case in (1),the value 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 1 ∗= 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 1 and 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 2 ∗= 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 2 .

The next step is to produce subtracted binary image, I_s,
from every combination point pairs possible, from𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 1 ∗ to 𝑗𝑗𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 2 ∗. The total number of point pairs, R,
is calculated by:
𝑅𝑅 =

𝑟𝑟!

2!(𝑟𝑟−2)!

(5)

Fig. 6 The derivative of curve y, dy/dx
𝑆𝑆(𝑚𝑚) , where 𝑚𝑚 = 1, 2, . . . , 𝑅𝑅 , is then plotted andanalyzed. The resulting curve, designated as 𝑦𝑦2 , has
multiple peaks and valleys culminated from different sets

The process can be explained in a pseudocode form as
follows:
ALGORITHM 1: PSEUDOCODE
195

Fig. 7 Curve y2 showing the peaks and valleys as aresults of the image subtraction process.
of pairing between points. Curve 𝑦𝑦2 is then plotted, as
shown in Figure 7.
Curve 𝑦𝑦2 is a 6 results series (indicated with arrows) canbe identified where each series begins with a maximum
value of black pixel to its minimum, and is separated witha straight line connecting the current series to the next.
Ideally, some of the resulting images, 𝑆𝑆(𝑚𝑚), are adequate
to be used in identifying the molds. The image that fit thiscriterion is called the optimized subtracted image, 𝐼𝐼𝑠𝑠 .
To obtain the optimized subtracted image, 𝐼𝐼𝑠𝑠 , with visible
mold defects image, the corresponding m values are
determined. The m values for obtaining the optimized
subtracted image, S, is designated as 𝑚𝑚𝑠𝑠 . The averagevalues for all the peaks, 𝑞𝑞𝑎𝑎𝑎𝑎𝑎𝑎 , and valleys, 𝑣𝑣𝑎𝑎𝑎𝑎𝑎𝑎 , in the 𝑦𝑦2
curve are calculated as follow:
𝑞𝑞𝑎𝑎𝑎𝑎𝑎𝑎 =𝑣𝑣𝑎𝑎𝑎𝑎𝑎𝑎 =𝑞𝑞1 +𝑞𝑞2 +⋯+𝑞𝑞𝑁𝑁(6)

𝑣𝑣1 +𝑣𝑣2 +⋯+𝑣𝑣𝑁𝑁(7)

𝑁𝑁𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝

𝑁𝑁𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣

Fig. 8 Flow Diagram of the Derivative Level
Methodwhere the lower limit, 𝑚𝑚𝐿𝐿𝐿𝐿 , is determined by finding thelowest m value that corresponds to the valleys, v, suchthat 𝑦𝑦2 (𝑚𝑚𝐿𝐿𝐿𝐿 ) > 𝑣𝑣𝑎𝑎𝑎𝑎𝑎𝑎 and the upper limit, 𝑚𝑚𝑈𝑈𝑈𝑈 , isdetermined by finding the highest m value that
corresponds to the peaks, q, such that 𝑦𝑦2 (𝑚𝑚𝑈𝑈𝑈𝑈 ) < 𝑞𝑞𝑎𝑎𝑎𝑎𝑎𝑎 .
The obtained value is then rounded to the nearest integer
to get 𝑚𝑚𝑆𝑆 .

Finally, the binary subtracted image used for defectdetection can be obtained from the labeled image 𝐼𝐼𝑆𝑆 (𝑚𝑚𝑆𝑆 ).
The flow diagram shown in Figure 8 summarizes theoverall process of the Derivative Level Thresholding.

The two values derived from both equations cannot be
used to determine the value of m directly, as the averagevalue may not be positioned on the 𝑦𝑦2 curve. The valuesof 𝑞𝑞𝑎𝑎𝑎𝑎𝑎𝑎 and 𝑣𝑣𝑎𝑎𝑎𝑎𝑎𝑎 would be used to calculate the closest
value for 𝑚𝑚𝑠𝑠 which is:
𝑚𝑚𝑠𝑠 = 𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟 �

𝑚𝑚𝑈𝑈𝑈𝑈 +𝑚𝑚𝐿𝐿𝐿𝐿
2

�

(8)

196

2.4. Defect Detection and Analysis
Filtering and defect analysis can also be conducted on thebinary images to locate and characterize defects. In a
black and white image, the defects are represented by thewhite pixels.
2.4.1. Connected Component Filtering for Noise
Figure 9 shows the overall image processing for thedefect detection which comprises of the process after the
Derivative Level Thresholding has been carried out, as
well as after the other selected thresholding methods.
Once the binarization is complete, the connectedcomponent filtering then be conducted. Prior to thefiltering process, a connected component analysis using
flood-fill algorithm is performed to locate groups ofpixels. In this process, an unlabeled pixel is first locatedand a flood-fill algorithm is used to label adjacent pixels
to be in the same group. In this study, 4-connectedneighborhood component is used to determine pixel
grouping. Each pixel group will be represented with asize, 𝐴𝐴𝑘𝑘 , measured in unit pixel. The connectedcomponents are first filtered according to size, and thefirst filter will only store pixel groups that are larger than𝑎𝑎𝑚𝑚𝑚𝑚𝑚𝑚 and smaller than 𝑎𝑎𝑚𝑚𝑚𝑚𝑚𝑚 :Fig. 9 Flow Diagram of the overall image processing forthe defect detection.
The hole filter works by filtering group of pixels whichhave a difference of filled image area, 𝐴𝐴𝑓𝑓𝑓𝑓𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙 to image
area, 𝐴𝐴𝑘𝑘 , that is larger than a preset scale threshold value.
A filled image area is the total number of pixels in thesame group of pixels with holes filled. By using this holefilter, group of pixels with holes will be filtered out. In
this study, the hole filter threshold value is set at 0.1. Thismeans that the group pixels with difference in size of(9)

𝑎𝑎𝑚𝑚𝑚𝑚𝑚𝑚 < 𝐴𝐴𝑘𝑘 < 𝑎𝑎𝑚𝑚𝑚𝑚𝑚𝑚

Single pixels will not be stored, as well as group of pixels
that are connected in diagonals. The value of 𝑎𝑎𝑚𝑚𝑚𝑚𝑚𝑚 is set
to 20 while 𝑎𝑎𝑚𝑚𝑚𝑚𝑚𝑚 is set to 5000 pixels. Next, the selectedgroup of pixels will be refined using a hole filter.

Table 1 Defect Detection Performance for Sample 5No.

Binarization Method

1

Otsu’s Method
Minimum Error
Thresholding (MET)
MethodContrast Adjusted
Thresholding Method(CA)
Derivative Level
Thresholding (DLT)
Ground truth*

2
3
4
5

Number of
Defects
Detected
9

Number of
Defects Correctly
Detected1

Total DefectSurface Area
(pixels)
13662

7

3

17099

42.32

12

1

9614

23.80

10

3

16614

41.12

3

3

891

2.21

197

Percentage ofDefects (%)
33.82

10% and more between 𝐴𝐴𝑘𝑘 and 𝐴𝐴𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓 will be filtered out,as they are confirmed to have holes.
The filter is calculated by:(𝐴𝐴𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓 −𝐴𝐴𝑘𝑘 )𝐴𝐴𝑘𝑘

Fig. 10 Binary Images of Sample 1E from left: (a)Ground Truth Binary, and results of (b) Contrast
Adjusted Thresholding, (c) MET, (d) Otsu and the (e)proposed Derivative Level Method.

(10)

Subsequently, a line filter is also applied on the images.
In this process, the pixel group is accepted when it has anarea to perimeter ratio of more than 0.6. This filter will
eliminate lines - which will usually have a low area toperimeter ratio, ranging from 0.1 to 0.5, especially forstraight lines.

compare the performance of each thresholding anddetection method. The results will be presented in two
sections – the first section will focus on Sample 1E which
originated from Painting 1. The second section is an
overview of the results obtained from all 20 samples thatconsists of samples from Painting 1 and Painting 2.

2.4.2. Connect Component Analysis for Mold
Characterization

3.1. Sample 1E Results

The selected group of pixels are then remapped into thefinal binary image and are subjected to another connectedcomponent analysis using a similar 4-connectedneighborhood connected component analysis approach.
Data on defect properties such as the area size (in pixels),
centroid (location) and morphology are collected forcomparison. The defect detection is carried out for the
newly proposed Derivative Level Thresholding method
and the three existing thresholding methods. The results
obtained are then compared to determine their relative
performance.

As the images are captured at a resolution of 650 dpi,
the sizes of each pixel are approximately 1.6𝑛𝑛𝑚𝑚2 . The
variation in the size of defects of on Sample 1E ranges
from 4.8 𝑛𝑛𝑚𝑚2 to 705.6 𝑛𝑛𝑚𝑚2 . Figure 10 shows thecomparison of the binarization results from the four
methods. The results images visibly suggested that theresults for the proposed method has included the defects
in the resulting binary image, along with false positives.
The results are compared to a ground truth image
indicated in Figure 10(a) to confirm the accuracy andsensitivity of the defect detection performance of thevarious binarization methods. The ground truth isrepresented as an image that is manually produced fromthe feedback of expertise on the actual amount of molddefects. This is used as the benchmark to compare theperformance of each method using their resultant defectimages. The sensitivity and accuracy of the binarization
methods can be calculated by comparing the detection
results with the data from the ground truth image. The
sensitivity or true positive rate is determined as:

3. Results and Discussions
In this study, the proposed Derivative Level
Thresholding is compared with three existingbinarization by thresholding methods. These methods
are implemented in MATLAB R2016a and computed onan Intel(R) Core™ i7-4500U 1.80GHz processor with
8GB RAM on a Windows 10 Pro platform. The images
captured using mesoscopic technique are processed using
the selected binarization methods in accordance to theflow diagram in Figure 8.
The comparative performance of all four methods were
evaluated experimentally for detecting mold defects on
the selected artworks. 20 samples were selected fromdifferent locations on the scanned image of the artworkwith known mold locations from both the original
painting image and the restored painting image. The
respective ground truth images are also obtained to

𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 =𝑇𝑇𝑇𝑇

𝑇𝑇𝑇𝑇+𝐹𝐹𝐹𝐹(11)

and the accuracy is determined as:
𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴 =198

𝑇𝑇𝑇𝑇+𝑇𝑇𝑇𝑇

𝑇𝑇𝑇𝑇+𝐹𝐹𝐹𝐹+𝐹𝐹𝐹𝐹+𝑇𝑇𝑇𝑇(12)

where TP is True Positive, FP is False Positive, FN is

indicator on the general performance of all four methods.
The robustness of each method used in mold defectdetection can be determined from the calculated valuesof the accuracy and the sensitivity.

False Negative and TN is True Negative.
As the binary images were processed with the Connected
Component Filtering it can be said that while the
Derivative Level Thresholding is robust with higher
sensitivity (96.75%) and accuracy (60.94%) to MET
method in the case of defect detection in Sample 1E.

The results showed that the newly proposed Derivative
Level Thresholding method has performed better than theFig. 12 Overall Defect Detection Performance for 20
SamplesFig. 11 Sensitivity and Accuracy level of Methods for
Sample 1E

three existing binarization methods. The DLT method
has the highest average sensitivity at 84.73% ascompared to the other methods having average sensitivity
values ranging from 17 – 61%. DLT also has a highaverage accuracy 68.40%, second only to the Contrast
Adjusted Thresholding Method with the latter having thelowest average sensitivity value. The average accuracy
for DLT is also higher than the accuracy value discussed
for Sample 1E. The study has also found that the value of
accuracy for the DLT method is generally higher insamples that has less thick strokes.

Table 1 summarizes the defect detection results from
Sample 1E with Figure 11 presenting the comparison ingraph.
Table 1 shows the detect detection performance of the
newly proposed method with comparisons to Otsu’s,
MET and Contrast Adjusted Thresholding methods forthe analysis of Sample 1E. The ground truth values,
showing the actual size and number of defects, are also
included for comparison. The results show that the
Derivative Level Thresholding has managed to correctly
identify all 3 defects with lesser false positive ascompared to the MET method. This can be seen in Figure10(e) when compared with the ground truth image in
Figure 10(a).

From the results shown in Figure 12, it can be deduced
that the Derivative Level Thresholding Method isefficient in detecting mold defects. The high sensitivity
value means that the binary image produced by the DLT
method was able to correctly detect the defects, while thelower accuracy value indicate it has successfully done so
at a cost. The method has considered more pixels in thesamples as defects, resulting in higher false positive
values, thus lowering the accuracy.

3.2. Thresholding Method Overall Performance
To measure the overall performance of the methods, 20
samples were selected from Painting 1 and Painting 2. 14
samples from Painting 1, and 6 from Painting 2 wereselected in this overall evaluation. As shown in Figure 3
and Figure 4, the samples selected depict different typesof artwork details comprising of different stroke
thicknesses and colors. This is purposely chosen as a fairThis problem can be addressed by having a better
filtering after the binary image has been produced,
sufficient to reduce false positives while at the same timeensuring that actual defects do not get filtered out. In the199

future, a better way of classifying the binary images
resulting from the methods is required, as it can beconcluded that the rudimentary filtering suggested in thisstudy is not fully capable of avoiding false positives in
the mold defect detections. The accuracy can beincreased by considering all the defects detected
separately, rather than using a blanket filtering approach.

3.
4.
5.

In addition, instead of treating the results from themethods suggested in this study as the final mold defects,
they can be treated as features extracted that can be usedin machine learning methods to correctly classify themold defects and non-molds. This will improve the final
mold detection results and reduce the error generated
from the restored image samples.

6.

7.

8.

4. Conclusions
An automatic mold defects detection method, theDerivative Level Thresholding (DLT) Method has beendeveloped to locate mold-type defects on high resolution
scanned images of artwork paintings. The performanceof this newly proposed method was compared to three
existing common binarization methods for the evaluationof the 20 samples. It was found that the DLT method isbetter in terms of average accuracy (up to 68%) and has
the highest average sensitivity of 84.73%. In general, theDLT method is shown to be robust and effective indistinguish molds from various types of painting samples.
The methods suggested in the study can be further
developed with machine learning methods to optimize
their performance.

9.

10.

11.
12.

Authors Introduction

Acknowledgement

Mr. Hilman Nordin

Hilman Nordin received the B.Eng
degree in computer-aided design andmanufacturing engineering and theM.Eng.Sc. degree in mechanicalengineering (mechanical systems)
from Universiti Malaya, Kuala
Lumpur, Malaysia in 2009 and 2013,
respectively. He is currently a Ph.D.
candidate at Universiti Malaya. His
research interest include image processing and analysis,
high-resolution scanning and its industrial applications andmechanical systems.

This study is partially sponsored by Impact Oriented
Interdisciplinary Research Grant, Universiti Malaya
with Project No.: IIRG034B-2019.

References
1.

2.

Kokkinaki, O., et al., Comparative study of laser
induced breakdown spectroscopy and mass
spectrometry for the analysis of cultural heritage
materials. Journal of Molecular Structure, 2013.
1044: p. 160-166.
Feng, Y.Q., N.N. Wang, and H.X. Ju, Highly
EfficientElectrochemiluminescenceof
Cyanovinylene-Contained Polymer Dots in Aqueous

Medium and Its Application in Imaging Analysis.
Analytical Chemistry, 2018. 90(2): p. 1202-1208.
Galli, A. and L. Bonizzoni, True versus forged in thecultural heritage materials: the role of PXRF analysis.
X-Ray Spectrometry, 2014. 43(1): p. 22-28.
Buchta, D., et al., Artwork Inspection by
Shearography with Adapted Loading. Experimental
Mechanics, 2015. 55(9): p. 1691-1704.
Luo, Z.H., et al., Extraction of microcracks in rock
images based on heuristic graph searching andapplication. Computers & Geosciences, 2015. 85: p.
22-35.
Deborah, H., N. Richard, and J.Y. Hardeberg,
Hyperspectral Crack Detection in Paintings. 2015
Colour and Visual Computing Symposium (Cvcs),
2015.
Guarneri, M., et al., 3D remote colorimetry andwatershed segmentation techniques for fresco andartwork decay monitoring and preservation. Journalof Archaeological Science, 2014. 46: p. 182-190.
Desai, S.D., et al., User Intervention Based Detection& Removal of Cracks from Digitized Paintings. 2014
Fifth International Conference on Signal and Image
Processing (Icsip 2014), 2014: p. 15-19.
Desai, S.D., et al., Detection & Removal of Cracks
from Digitized Paintings and Images by User
Intervention. 2013 Second International Conference
on Advanced Computing, Networking and Security(Adcons 2013), 2013: p. 51-55.
Win, M., et al., A Contrast Adjustment Thresholding
Method for Surface Defect Detection Based on
Mesoscopy. IEEE Transactions on Industrial
Informatics, 2015. 11(3): p. 642-649.
Otsu, N., A threshold selection method from graylevel histograms. Automatica, 1975. 11(285-296): p.
23-27.
Kittler, J. and J. Illingworth, Minimum Error
Thresholding. Pattern Recognition, 1986. 19(1): p.
41-47.

200

Prof. Dr. Bushroa Abd Razak
Prof. Datin Ir. Dr. Bushroa Abd
Razak received her B.Eng. inElectrical and Electronic Engineeringfrom University of Manchester
Institute Science and Technology,
UK in 1997, which was merged toManchester
University.
Consequently, she received her
Master in Engineering Science and
PhD from University of Malaya, Malaysia in 2004 and2012, respectively. She started her career in 2004 at the
Department of Engineering Design and Manufacture, UM,and later in 2013 at the Department of MechanicalEngineering in the same university when both departments
were merged. Apart from teaching, she is actively
involved in research, management and consultation works.
In the late of 2020, she was appointed as a Head of
Department of Mechanical Engineering, University
Malaya. During her tenure, she has contributed impactfulin securing the Engineering Accreditation Council (EAC)
as well as revising new curriculum review for courses in
Mechanical Engineering. She holds a few memberships
including a board member of Advanced Manufacturingand Material Processing Centre, UM. She is awarded a
Chartered Engineer of IET, UK. In July 2016, she waspromoted to Associate Professor. In 2015-2016, she went
to Univ of California, LA, USA for her post-doctoral
research program sponsored by the Ministry of Higher
Education, Malaysia. She was also awarded as a
Professional Engineer by the Board of Engineers (BEM) in
March 2017. Till now, she is the author and co-author of
more than 100 publications in international journals andproceedings with H-index reaches above 25. Her researchinterest includes fundamental, application, technology andcommercialization in Surface Engineering and its related
field of study. She is passionate in research that contributes
to society.

Dr. Norrima Mokhtar
Norrima Mokhtar received the
Bachelor of Engineering (B. Eng)
degree in Electrical Engineeringfrom University of Malaya in 2000.
After working two years with
International
Telecommunication
Industry with attachment at Echo
Broadband GmbH, she managed tosecure Panasonic Scholarship which
required intensive screening at the national level in 2002.
She finished her Master of Engineering in Oita University,
Japan under financial support received from Panasonic
Scholarship in 2006. Consequently, she was appointed asa lecturer to serve the Department of ElectricalEngineering, University of Malaya immediately after
graduating with her Master of Engineering. As part of her
career development, she received SLAB/SLAI scholarship
to attain her Ph.D. in 2012. She held several important
positions such as Coordinator for Industrial Training,
Coordinator for Research Laboratory, Coordinator for
EBA-Consortium(Keio University), Coordinator for
Nagaoka University of Technology and Coordinator for
Manchester Metropolitan University(UK). Dr. Norrima
was also the principal investigator for High Impact
Research Grant (MOHE) worth of RM 1 million alongwith Science Fund and CREST grant. She is the author andco-author of more than 50 publications in international
journals and proceedings in Sensors, Automation, Image
Processing, Human-Computer Interface, Brain-Computer
Interface, UAV and Robotics. To date, she had
successfully supervised 3 PhD and 3 Master Engineering
Science students (by research). She is also active as
reviewer for many reputable journals and several
international conferences.
Mr. Fadzil Jamaludin
Mohd Fadzil Jamaludin is a SeniorResearch Officer at the Centre ofAdvanced
Manufacturing
andMaterial
Processing(AMMP
Centre), Universiti Malaya. He hasreceived the Bachelor of Engineering(B. Eng) degree in Computer Aided
Designand
Manufacturing(CAD/CAM) and Masters ofEngineering Science (M.Eng.Sc) in 2003 and 2010,
respectively, from Universiti Malaya, Kuala Lumpur,
Malaysia. His research interest includes precision joining
technologies, additive manufacturing and mixed reality
development for research and education.

201