# Image Stitching Based Measurements of Medical Screws

Zoran Cenev, Timo Prusi, Reijo Tuokko

► To cite this version:

Zoran Cenev, Timo Prusi, Reijo Tuokko. Image Stitching Based Measurements of Medical Screws. 7th International Precision Assembly Seminar (IPAS), Feb 2014, Chamonix, France. pp.69-78, 10.1007/978-3-662-45586-9_10. hal-01260734

HAL Id: hal-01260734

https://inria.hal.science/hal-01260734

Submitted on 22 Jan 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Image stitching based measurements of medical screws

Zoran Cenev, Timo Prusi and Reijo Tuokko

Department of Mechanical Engineering and Industrial Systems,
Tampere University of Technology, Finland

zoran.cenev@tut.fi; timo.prusi@tut.fi;
reijo.tuokko@tut.fi

**Abstract.** Image stitching is a method that forms a bigger image out of two or more smaller images with a certain overlap in their field-of-view. Usage of this technique is present in photography, medical imaging and other fields as well. The potential for using image stitching for measurement purposes in industrial applications has not been investigated as thoroughly, but it is receiving more attention. This paper examines the possibility for creating a machine vision-based measurement system that employs commercially available image-stitching platforms for measuring the length of long medical screws. Three methodologies were tested for performing medical screw measurements and this paper summarizes the findings of the image-stitching approach.

**Keywords:** image stitching, measurements, machine vision

# 1 Introduction

It is well known that screws are manufactured in lot sizes. The common inspection of these mechanical fasteners is by trial and error i.e. pushing them through thread gauges. Often, if a screw fails the inspection process, they are disposed of and recycled for another use. Unlike mechanical screws, medical screws (see Fig. 1 below) are built from expensive materials [1] which cannot be recycled if they are faultily produced. Therefore, the inspection process has to be accurate enough in order to support the fine-tuning of the screw's manufacture. One very important aspect of medical screws that necessitates their high-quality inspection is the fact that they enter, stay and degrade within the human body. As a result, the quality regulations in the United States of America (stipulated by the Food and Drug Association, the FDA) and in Europe (overseen by the European Medicines Agency, EMEA) are extremely rigorous.

This paper introduces three machine vision approaches to the inspection of a medical screw as shown in Fig. 1. The inspections process is based on measuring certain screw parameters, such as the thread inner and outer diameter, the screw's total length, and the diameter of the neck (the part between the screw head and the beginning of the thread), and verifying that they are within given tolerances.
<figure><img src="image_2.png" /><figcaption>Fig. 1. Medical screw. White rectangle indicates field-of-view of 90 mm x 120 mm</figcaption></figure>

Currently, these measurements are done using a manual caliper. However, this way of measuring is very inefficient when measuring a batch of screws. For example, this approach is prone to human errors and features low reliability, paper-based data and inadequate opportunities for automation. Therefore building a machine vision-based measurement system is regarded as a very valuable approach to the problem and has been researched in great detail.

Dimensions such as the thread's inner and outer diameter, and the diameter of the neck can be inspected with any machine vision set-up that contains an ordinary industrial camera equipped with a high quality lens that captures a field-of-view of a few dozen millimeters. However, the real challenge arises when measuring the screw length had to be realized. Hence, the focus of this paper is on reporting the results and analysis of an image-stitching based metrology approach for measuring the screw length.

Applications of the image-stitching technique in metrology were reported in the articles [2][3][4][5]. According to the findings reported in [2] and [3], image stitching was successfully implemented in performing AFM (Atomic Force Microscope) line-width measurements through the manual stitching of two images. The article in [4] reports achieving significant similarities in accuracy of the linear measurements of stitched compared with non-stitched imaging in cone beam computed tomography. Commercial use of image stitching metrology in generating high-resolution nano-scaled topography maps is reported in [5].

## 2 Screw Measurement Approaches

A medical screw 80 mm in length is manufactured with a length tolerance of 0.5mm. All the cross-sectional measurements, such as the thread outer/inner diameter and the neck diameter are manufactured with a tolerance of 0.05mm. Nielsen [6, page 3] suggests that the measurement resolution should be at least one fourth of the smallest manufacturing tolerance, i.e. 0.0125 mm in this case.

A field-of-view of about 90 mm x 12 mm was desired in order to look at all the component dimensions and allow some margins for part positioning (see Fig. 1).

The first approach was a solution in which all measurements could be done from one single image. Therefore, the overall measurement resolution in millimeters was the smallest required resolution from all the previously mentioned ones, i.e. 0.0125 mm. This is the smallest feature that needed to be detected from the single image (at least 1 pixel in size) leading to a theoretical spatial resolution of 1 pix/0.0125 mm or,
other way around, 80 pix / mm. However, according to the Nyquist-Shannon theorem [7], [8], the spatial resolution has to be at least two times greater than the calculated ratio, i.e. 160 pix / mm. Moreover, a rule of thumb in machine vision systems is that the spatial resolution should be 4 to 10 times greater than the calculated ratio. Therefore a spatial resolution of 320 pix / mm was desired.

In order to cover the above mentioned FOV with the calculated spatial resolution, we needed an image of size:

$$ width_{pixels} = 90 \text{ mm} * 320 \frac{\text{pix}}{\text{mm}} = 28800 \text{ pix} \quad (1) $$

$$ height_{pixels} = 12 \text{ mm} * 320 \frac{\text{pix}}{\text{mm}} = 3840 \text{ pix} \quad (2) $$

In reality, camera sensors with this kind of aspect ratio (width/height) are not available. Common aspect ratios are in the range of 4:3. Assuming an image height of 3840 pix and an aspect ratio of 4:3, the image width would be only 5120 pix covering only 16 mm of the length of the screw. Or, to put it another way, if we want a 28 800 pixel wide image, the image height would be (with 4:3 aspect ratio) 21 600 pix, requiring about 622 megapixel camera covering a FOV of 90 mm x 66.5 mm. This approach demands an extremely high resolution camera which make it very expensive, and therefore impractical for this case.

Another approach was to apply the Coordinate Measuring Machine CMM method. This means that the screw can be positioned on a linear axis and the camera can capture images from both ends of the screw. The length can be derived from the distance measured from the movement of the linear axis combined with the differences between the screw ends of the images. This approach is not the focus of this paper, and will not be examined in further detail.

Third approach was image-stitching-based metrology, and this approach is described in the text below.

Image stitching is a process that takes multiple images with overlapping field-of-views and outputs a panoramic image. As illustrated in Fig. 2, the panoramic image on the far right is obtained by stitching the two input images on the left. It is evident from Fig. 2 that the input images have a certain FOV overlap [9].

<figure><img src="image_3.png" /><figcaption>Fig. 2. The Image-Stitching Process</figcaption></figure>

The idea behind this approach is to capture certain amount of images along the medical screw, stitch them together and then perform measurements from the obtained panoramic image.
# 3 Machine Vision Setup

## 3.1 Physical Components

On the left of Fig. 3 one can see the machine vision setup. This consists of a monochromatic industrial camera (2560 x 1920 pixel resolution) equipped with a telecentric lens giving a field-of-view of 18 mm x 12 mm. This gives a spatial resolution of about 160 pix / mm, which theoretically satisfies the requirement for measuring the cross-sectional dimensions with the specified measurement resolution calculated in Chapter 2. However, this spatial resolution is only half of the desired one.

<figure><img src="image_4.png" /><figcaption>Fig. 3. Machine Vision System for Image Capturing (left) panoramic stitched image (right)</figcaption></figure>

The fixture on which the screw resided was attached to a two-dimensional displacement table and was moving in the direction along the screw axis. Red ring light was used as a front illumination source, and it was positioned between the camera and the target.

A black background was placed between the screw and the fixture, Fig. 3 (right). The role of the background was to enable high contrast and to introduce more features in order to ease the image stitching process. The remaining components (not visible on the image above) are the computer to which the camera was connected and a controller that was used to control 2D moving table.

## 3.2 Stitching Platforms

This work sees the image stitching process as a black box. This means that a certain input goes into the box and a certain output comes out of the box, whereas the examiner did not know what was happening inside. This meant that ready-made im-
age stitching software platforms such as the "Photomerge" tool in Photoshop and Microsoft's Image Composite Editor ICE were used.

There are two reasons why these platforms were chosen over any others. First, the role of the user was just to input the images that he or she wanted to stitch, and the stitching process was performed without any further user involvement. Second, these platforms were capable of working with many image formats.

A planar reposition function supported by both platforms was used in order to avoid any rotational and spherical effects onto the stitching process.

### 3.3 Image Stitching Concept Validation Experiment

The concept validation was done by stitching images in order to obtain a panoramic image (Fig. 3, right) and then test the accuracy and the repeatability of the length measurements. The testing procedure started by realizing the image acquisition. Fig. 4 shows how the images were acquired and stored.

<figure><img src="image_5.png" /><figcaption>Fig. 4. Data buffering layout</figcaption></figure>

The data from the image acquisition was divided into two sets, each having 10 clusters of images. Each cluster contained 11 images in the first set and 22 images in the second set; see Fig. 4. The amount of overlap required for the stitching process in the first and second set was ~60% and ~80% respectively.

The experiment continued by making panoramic images from each cluster using the two commercially available stitching platforms, Microsoft ICE and Adobe Photoshop. Additionally, in order to verify the repeatability of the stitching process, clusters 1.1 and 2.1 were both stitched 10 times. As a result, each of the four groups of stitched images contains 10 panoramic images.

After the stitching phase of the experiment, the length of the screw in 40 stitched images was measured with a Clamp (Rake) tool in the National Instrument Vision
Assistant software platform. This tool finds the edges from rectangular region of interest (visible in Fig. 5) and calculates the distance between first and last edges found. Because of tool's operating principle, it is important to align the tool to the target being measured. When measuring screw lengths, alignment was assured by measuring screw orientation (deviation from horizontal) and then rotating the image so that the screw was horizontal before measuring the length with Clamp (Rake) tool. From the individual measurements, the average dimension, standard deviation and error range (max value - min value) were calculated. The repeatability of the Clamp (Rake) tool was estimated by recording 100 images of a known length metal piece (length 16 mm) that was small enough to fit in the field-of-view of a single image. Fig. 5 shows the metal piece and the Clamp (Rake) tool. The results and analysis of these calculations are provided below.

<figure><img src="image_6.png" /><figcaption>Fig. 5. Screen capture of Vision Assistant showing the metal piece</figcaption></figure>

## 4 Results

The error range and standard deviation of measuring the length of the metal piece from the 100 individual images were 0.03 % and 0.01 % respectively.

As Table 1 shows, the error range and standard deviation in measuring the screw length ranged from 0.09 to 1.78 % and from 0.03 to 0.47 % of the screw length, respectively.

<table><caption>Table 1. Error range and standard deviation of the screw length obtained from the panoramic images</caption><thead><tr><th>Stitching Platform</th><th>Clusters of 11 images (Group 1)</th><th>Clusters of 22 images (Group 3)</th></tr></thead><tbody><tr><td><strong>Microsoft ICE</strong></td><td></td><td></td></tr><tr><td>- Average Length [pix]</td><td>11 903</td><td>11 881</td></tr><tr><td>- Error Range [%]</td><td>0.10</td><td>0.09</td></tr><tr><td>- Standard Deviation [%]</td><td>0.04</td><td>0.03</td></tr><tr><td><strong>Photoshop's photomerge</strong></td><td></td><td></td></tr><tr><td>- Average Length [pix]</td><td>11 930</td><td>11 935</td></tr><tr><td>- Error Range [%]</td><td>0.24</td><td>1.78</td></tr><tr><td>- Standard Deviation [%]</td><td>0.07</td><td>0.47</td></tr></tbody></table>
The best case was when the stitching process was performed with Microsoft ICE platform on a cluster from 22 images with a maximum error range of 0.09 % and a standard deviation of 0.03 %. Converting pixels to millimeters was done with calibrating the machine vision setup with National Instrument Vision Assistant calibration tools and laser marked calibration grid. In addition, the actual length of the screw was measured with a CMM. The obtained maximum error range was 0.072 mm and maximum standard deviation 0.024 mm.

## 4.1 Image stitching process behavior

Since the image-stitching process was seen as a black box and since there was no possibility to have control over the stitching process, its behavior was examined. Four phenomena were identified: (1) Different image size for each panorama, (2) obtaining convex panoramic images, (3) unexpected stitching and (4) unrepeatable stitching.

### Different image size

Table 2 shows the pixel size range (i.e. maximum width – minimum width and maximum height – minimum height) of each group of stitched images from both sets, Microsoft ICE and Photoshop.

<table><caption>Table 2. Image size range of the stitched images obtained from both sets</caption><thead><tr><th rowspan="2">Image sets and<br>stitching platforms</th><th colspan="2">Image size range [pix]</th></tr><tr><th>Width (~ 13 000 pix)</th><th>Height (~ 2 000 pix)</th></tr></thead><tbody><tr><td colspan="3"><strong>Set 1 - Microsoft ICE</strong></td></tr><tr><td>Group 1</td><td>10</td><td>12</td></tr><tr><td>Group 2</td><td>1</td><td>5</td></tr><tr><td colspan="3"><strong>Set 1 - Photoshop's photomerge</strong></td></tr><tr><td>Group 1</td><td>31</td><td>13</td></tr><tr><td>Group 2</td><td>55</td><td>6</td></tr><tr><td colspan="3"><strong>Set 2 - Microsoft ICE</strong></td></tr><tr><td>Group 3</td><td>14</td><td>14</td></tr><tr><td>Group 4</td><td>2</td><td>0</td></tr><tr><td colspan="3"><strong>Set 2 - Photoshop's photomerge</strong></td></tr><tr><td>Group 3</td><td>27</td><td>39</td></tr><tr><td>Group 4</td><td>30</td><td>18</td></tr></tbody></table>

### Convex Panoramic Images

Another phenomenon that appeared was the bending effect, or convex panoramic image. This image is shown in Fig. 6 and a similar image was obtained with both stitching platforms. The reason for this is unclear, because how the stitching platforms are “black boxes”, i.e. the authors do not know how the stitching algorithm works. However, this effect was reduced by adding more features to the background, see Fig. 3 (right). Therefore we can propose a hypothesis that the reason for this distortion is
lack of distinctive features along the screw, and especially in the upper and lower sections of the image (quite constant grey in Fig. 6). However, this was not tested.

<figure><img src="image_7.png" /><figcaption>Fig. 6. Convex panoramic image</figcaption></figure>

## Unexpected stitching

This phenomenon refers to undefined behavior of both stitching platforms and their abnormal outcome see Fig. 7.

<figure><img src="image_8.png" /><figcaption>Fig. 7. Abnormal panoramic image</figcaption></figure>

However, these kinds of images can be identified and thus deleted with a previously set image size threshold value.

## Unrepeatable stitching

Another undesired effect that was discovered from the stitching image process is the instability of the stitching platform itself. For example, the sizes of images obtained by stitching the same set of images differ by up to 55 pixels or 0.42 %, as shown in Table 2 (see results for Groups 2 and 4). Naturally this also affects also the length measurements. Table 3 shows the repeatability of the screw length measurement results of image groups 2 and 4 stitched with both platforms.

<table><caption>Table 3. Screw length deviations in panoramic images stitched from the same cluster of images</caption><thead><tr><th>Stitching platform</th><th>Cluster of 11 images<br>(Group 2)</th><th>Cluster of 22 images<br>(Group 4)</th></tr></thead><tbody><tr><td><strong>Microsoft ICE</strong></td><td></td><td></td></tr><tr><td>- Error Range [%]</td><td>0.01</td><td>0.01</td></tr><tr><td>- Standard Deviation [%]</td><td>0.00</td><td>0.01</td></tr><tr><td><strong>Photoshop's photomerge</strong></td><td></td><td></td></tr><tr><td>- Error Range [%]</td><td>0.56</td><td>1.09</td></tr><tr><td>- Standard Deviation [%]</td><td>0.18</td><td>0.27</td></tr></tbody></table>
# 5 Analysis and Discussion

The following findings can be identified from the results presented above:

* Table 3 shows that Microsoft ICE is more repeatable than Photoshop. This is supported by Table 2, especially when looking at the results of groups 2 and 4.

* Table 1 shows that also the length measurements have a smaller error range and standard deviation with Microsoft ICE than with Photoshop.

* Major concern is the fact that the average length varies (see Table 1) even though the images were captured from the same target with the same hardware setup, and the input images were the same for both stitching platforms.

* The software platforms' error range and standard deviation in Table 2 are only affected by the repeatability of the image stitching process. In contrast, the results in Table 1 are not only affected by image stitching process, but also by hardware and environment-related noise and unrepeatability. Therefore the best case in Table 1 (Microsoft ICE and Set 2) has about three times greater error range and standard deviation than the measurements made from the metal piece which fitted into a single image FOV.

Since the smallest manufacturing tolerance was 0.05 mm, the resolution of the measurement system was expected to be a fourth of that, i.e. 0.0125 mm. Hence the obtained maximum error of 0.072 mm and the maximum standard deviation of 0.024 mm do not satisfy the desired criteria for length measurement with the desired cross-sectional measurement resolution. However, since the length manufacturing tolerance is exactly one order of magnitude higher than the one for cross-sectional dimensions, the measurement resolution can be increased for the same factor, resulting in a new length measurement resolution of 0.125 mm. Taking this into consideration, measuring the length can be achieved with the presented approach.

# 6 Conclusion

This paper asserts that measuring the length of long components such as a medical screw with a low-cost machine vision system with commercially available image stitching platforms is possible up to a certain level of measurement resolution.

Firstly, it should be remembered that the number of images used to create a panoramic image depends from the stitching environment. Microsoft ICE stitches marginally better with more input images; on the other hand Photoshop stitches much better with less.

Secondly, it is very likely that the stitching process can exhibit unbound and inconsistent behavior, such as the ones depicted in section 4.1.
Thirdly, the uncertainty from the stitching process is not the only source of uncer-
tainties and errors in the actual measurements. The image acquisition and measure-
ment algorithms also cause uncertainties in the measurements made from stitched
images.

Fourth aspect is the interfacing and automation aspect. When building a measure-
ment application utilizing a ready-made stitching platform, it should be as easy as
possible to just input an image sequence and get a stitched image out. Therefore, the
stitching platform should have an easy-to-use application protocol interface (API).
Photoshop has this, but Microsoft ICE does not. Of course, building your own stitch-
ing platform is an option, but meeting the performance of Microsoft ICE would be
challenging.

7 Acknowledgements

This work was done in a project funded by Finnish Funding Agency for Technolo-
gy and Innovation (TEKES). Authors would like to thank TEKES and participating
companies. In addition, authors are very grateful for the support from their colleagues
Riku Heikkilä and Niko Siltala from the Microfactory research group at the Depart-
ment of Mechanical Engineering and Industrial Systems.

8 References

[1] Törmälä, P., Vainioapää, S., & Rokkanen, P. (1992, January 28). U.S. Patent No. 5,084,051. Washington DC.

[2] Fu, J., Dixson, R., Orji, G., Vorburger, T., & Nguyen, C. V. (2005). Linewidth measurement from a stitched AFM image. *International Conference on Characterization and Metrology for ULSI Technology*. Dallas: American Institute of Physics.

[3] Chu, W., Fu, J., & Vorburger, T. V. (2010). Subpixel image stitching for linewidth measurement based on digital image correlation. *Measurement Science and Technology*, 105104 (5pp).

[4] Srimawong, P., Krisanachinda, A., & Chindasombatjaroen, J. (2012). Accuracy of Linear Measurements in Stitched Versus Non-Stitched Cone Beam Computed Tomography Images. *Challenges of Quality Assurance in Radiation Medicine*, (p. 40). Phitsanulok, Thailand.

[5] Nanosurf Application Note. (n.d.). Analyzing Large Surfaces using AFM Stitching. Liestal, Switzerland.

[6] Nielsen, H. S. (1999). ISO 14253-1 Decision Rules-Good or Bad. *NCSL-I Workshop and Symposium*, (p. 7). Charlotte, NC, USA.

[7] Olshausen, B. A. (2000). Aliasing. *PSC 129-Sensory Processes*, 1-6.

[8] Candès, E. J., & Wakin, M. B. (2008). An Introduction To Compressive Sampling. *IEEE SIGNAL PROCESSING MAGAZINE*, 21-30.

[9] Levin, A., Zomet, A., Peleg, S., & Weiss, Y. (2004). Seamless Image Stitching in the Gradient. *Computer Vision-ECCV*, 377-389.