# Thermal camera calibration with cooled down chessboard

T Herrmann, C Migniot, O Aubreton

<figure><img src="image_1.png" /></figure>

To cite this version:

T Herrmann, C Migniot, O Aubreton. Thermal camera calibration with cooled down chessboard. Quantitative InfraRed Thermography Conference, May 2019, mulhouse, France. hal-03275440

HAL Id: hal-03275440

https://hal.science/hal-03275440

Submitted on 1 Jul 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Thermal camera calibration with cooled down chessboard

by T. Herrmann*, C. Migniot** and O. Aubreton*

* University of Burgundy, ImViA Laboratory CNRS ERL 6000, 12 rue de la Fonderie, 71200, Le Creusot, France;

**ImViA EA 7535, Univ. Bourgogne Franche-Comté, Dijon, France

## Abstract

Calibration of a camera have various applications in computer vision. This process is particularly challenging with a thermal camera instead of a standard color one. A calibration target with two layers of different thermal emissivity is commonly used but reflects appear and degrade the corner extraction of the chessboard. While traditional and commonly used methods heat the target chessboard, we cool it down that inverts the distribution of emission of the two layers. The reflects on the reflective layer enhance the contrast instead of decreasing it. Experiments on various configurations and orientations demonstrate the superiority of the proposed method.

## 1. Introduction

In computer vision, calibration is a primary problem of acquisition and a required step for scene interpretation. Camera calibration consists of estimating two types of parameters: the intrinsic parameters are linked to the sensor of the used camera (optical center and focal length) and the extrinsic parameters represent the position of the camera in space. The intrinsic parameters, related to the components of the camera, is generally provided by the manufacturer. The extrinsic parameters depend on the scene and must be re-estimated each time the camera is positioned again. For the acquisition of 3D points from a depth camera, the calibration matches the points of the scene in the frame of reference of the camera but also a point in the 3D space with a pixel of the depth image. The extrinsic calibration is carried out using a target with significant controlled points. To estimative them, a target with a particular pattern is acquired with various orientations. The most common targets are chessboards; black and white in the case of a color camera. The objective is to detect the intersections of the tiles on the images and to match them to the coordinates of the target in the real 3D space.

A thermal camera does not capture the same wavelengths as color cameras. A conventional target with color pattern will therefore be difficult to interpret by a thermal camera. However, the goal of multiple works was to magnify the low contrast obtained and thus computing calibration in particular to allow joint calibration with a thermal camera and a color camera. Prakask [7] heated a checker-board pattern with a flood lamp because of the emissivity difference between black and white regions. Saporano [9] used a heat lamp to heat the target and a glazed finish ceramic tile backing to retain heat longer. Vidas [14] used a target consisting of a grid of regularly sized squares cut out of a thin material.

To enhance the contrast in various wavebands, the recent works used a target where the white and black part of the standard target are made by two different materials having very distant emission values (generally plastic or wood and polished metal). Peric [6] compared the results obtained by first target built from polished aluminum with black opaque insulating tape and a second target from cardboard with aluminum foil. Saint-Laurent [10] digitally printed a checkerboard pattern applying matt varnish to maximize contrast and lifetime. A sandblast finish minimizes specular reflections and the plain aluminum core extends contrast persistence in time. Ursamentiaga [13] printed calibration targets on aluminum composite material with a continuous industrial flatbed printer used for advertising boards. To jointly calibrate color and thermal cameras, Ursine [12] build target whose the background material is a copper plate, with emissivity low emissivity and the squares were painted with high emissivity spray ink. For the same purpose, Takashi [11] used black base board made of resin and white planner plates made of aluminum. A tone mapping post process enhance the contrast.

The technical characteristics of a camera vary and impose specific constraints on the calibration techniques. For example while modern color cameras mostly have a good resolution, often greater than 1920 by 1080 pixels, the most expensive thermal cameras very rarely exceed 1024 by 720 pixels. Most thermal imaging cameras do not produce images larger than 640 by 480 pixels. The targets used are then quite large and placed at a short distance from the camera. To counter the noise and the low resolution of its acquisition, Zoetgnande [17] increased significantly the number of acquisitions with different poses from the target required for the calibration. Luhman [5] presented an overview of the effects on the acquisition by 5 different thermal cameras on the standard classification methods.

More complex and more difficult to integrate devices have also been devised for better target analysis. Grid of ligh miniature light-bulbs has to be used to be the feature points of the target [17]. Ellmauthaler [1] modeled the radiation pattern of each light bulb as an ellipse and considered the center of mass of the extracted ellipsoidal region as the starting calibration point, which is refined iteratively using alternating mappings to and from an undistorted grid model. Gschwandtner [3] installed a pattern of electrical elements emitting IR radiation (a set of resistors mounted in the centroid of each square). Separate calibration boards for the intrinsic calibration of the IR and visible-light camera for

<figure><img src="image_3.png" /></figure>
extrinsic calibration between the cameras are used. Laguela [4] build a grid based on burning lamps five delrin spheres and seven cubes of different sizes. Yu [15] used small circular thermostatic heaters as sources of thermal radiations using a high-resolution camera. In Gonzalez [2], scale bars with four reflective targets situated at the corners of the structure are provided geometric and thermal information about building facades. The calibration field consisted on a wood plank with 64 light bulbs.

Even if a plastic and metal-based thermal target is more easily detected using a thermal camera, the contrast is often insufficient. To alleviate this problem the target temperature is often changed by heating using various devices such as lamps [13] or heat guns [15]. The increase in temperature intensifies the contrasts in the image by changing the emissivity values of the materials which will improve the detection. A material with a high thermal inertia such as ceramic can be pressed against the target in order to keep it at the right temperature for the duration of the acquisition [9].

However, copper behaves on the wavelengths as a mirror, except that it does not return light but heat. From some angles it therefore returns wrong temperature. However, the camera being an electric heating device during its operation, the reflection of the heat of the camera, of a possible operator or even of various hot objects, appears on polish metal. When a reflection occurs, the part that generated it returns the same value as the reflected object, often warmer than the background temperature. This phenomenon is amplified when the target is heated: the plastic part, having a high reflectivity value returns a very light shade while the metal part having a low reflectivity value will returns a lower value. However, if one or more pieces of metal have reflections produced by hot objects, these will returns comparable values as plastic part. The estimation of the intersections of the target then becomes more complex.

Rather than minimizing the number of hot reflections in the case of a hot target, we propose in this paper to cool the target. By doing it the emission temperature of the metal remains constant due to the low emissivity of the material, however, the one of the plastic should decreases drastically. However if the camera perceives plastic as colder than metal then metal parts return low values and plastic parts return high values.

## 2. Methodology

Every objects emit a certain amount of energy in the form of electromagnetic waves when their temperature exceed 0°K. Their propensity to emit their internal energy can be measure by comparing the amount of heat they emit with the amount of heat emitted by a black body at the same temperature. A high emissivity leads to an larger emission of the internal heat in the form of infrared radiation or visible light in case of very high temperature. Two materials with different emissivity will have various answer to heating. The emissivity reflects the amount of energy emitted by an object. It largely causes the rendering of the acquisition from a thermal camera. When heating two materials at the same temperature, the one with the higher emissivity appears hotter on the thermal image. In the case where the emissivity of two objects remain proportional to the temperature, further increasing their internal temperature induces increasing the temperature variation perceived by the camera. The emitting energy depends not only on the heat but also on the wavelengths of the heat sources and its own absorption capacity.

<figure><img src="image_4.png" /><figcaption>Fig. 1. Histograms of acquisitions of a target in a front view: a) with a heated target, b) with a heated target and in presence of a reflect, c) with a cooled target and d) with a cooled target and in presence of a reflect.</figcaption></figure>
When heating, the different materials of the target will present two distinct behaviors. The plastic part (with high emissivity) disperses a lot of heat and therefore appears much clearer on the thermal image. The polished copper (with emissivity value close to 0 ga) disperses less energy and therefore appears darker. In this paper, we used a target made of polished copper pattern printed on an epoxy plate. Figure 1-a shows the histogram obtained from the acquisition of a heated target. The histogram presents two distinct peaks: the first one corresponding to the copper parts and the second one corresponding to epoxy parts.

Cooling the target produces a better contrast. While intensities are similar than the ones obtained from heating, the correspondence with the materials are reversed: copper with low emissivity appears in high intensity, on the contrary epoxy with high emissivity appears in low intensity. Figure 1-c shows the histogram produced with this configuration. The first peak corresponds to the epoxy parts and the second one corresponds to copper parts.

Despite copper trends to emit only few heat, it has a high reflection capacity due to its own weak absorption capacity. It is therefore able to reflect the various sources of heat surrounding him. Reflections are also created that are artifacts for further processing. Figure 1-b shows the histogram provided after heating the target and in presence of hot reflections. Figure 1-d shows the histogram with the same reflections but after cooling the target. Reflections increase the intensity but only occurs on the reflective material (copper). Thus part of the copper peak in the histograms is shifted to higher values. In case of heating the target, the copper corresponds to the lower intensities. The shifted values are also merged with the peak corresponding to the epoxy. On the contrary in case of cooling the target, the copper corresponds to the higher intensities. The shifted vales are also further away from the peak corresponding to the epoxy. Therefore it is much more convenient to separate the two materials after cooling than after heating.

Cooling the target does not prevent the reflections from appearing, but the intensity shift affects no more the materials segmentation. The edges between epoxy and copper in presence of reflexions is disambiguated. To highlight this phenomenon, we designed an experimental protocol of target acquisition under various temperature conditions and in the presence or not of reflections.

## 3. Results

To evaluate our process, we have realized acquisitions of the target under three conditions of temperature: with a heated target, with a cooled target and with a target at the room temperature (ambient). Then we have tested the robustness of the detection by varying the angle of the target according to the camera.

Acquisitions were realized from a FLIR One Pro camera with a resolution of 160x120 pixels, capable of capturing wavelengths between 8 and 12 µm with an accuracy of 150 mK. The target is a classic thermal test pattern consisting of an one millimeter thick epoxy plate on which a thin layer of copper has been deposited. For each situation, a first acquisition is done with only an empty wall behind the camera to prevent from reflections. Then a heat source was placed behind the camera to produce a hot reflection. The Figure 2 shows a reflection on the same target in the same position but in various temperatures. It appears in all the configuration but with various effects : in the ambient target the contrast is low even where the reflection does not appear (Fig. 2 (a)); in the heated target the contrast is correct but the reflection degrades the localization of the cross points (Fig. 2 (b)); in the cooled target the reflection do not degrade the contrast.

<figure><img src="image_5.png" /><figcaption>Fig. 2: Influence of a hot reflection on a target at ambient temperature (a), a heated target (b) and a cooled target (c).</figcaption></figure>

The front view is the most common configuration and the easiest to process. Indeed the crossings are regularly spaced and there is no perspective effect. However, the orientation of the target must varies for a correct calibration. The detection of the feature points has to be robust to orientation change of the target. In the following we have used a particular experimental setup to characterize the relative position of the target and the camera (Figure 3). The camera is placed at 42 cm to the target with an angle $Θ_c$. The target is oriented by an angle relative to the axis movement of the camera $Θ_t$. The Figure 4 shows images acquired the 4 configurations that we considered.
<figure><img src="image_6.png" /><figcaption>Fig. 3: Representation of the experimental setup</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 4: Examples of image acquired on each configuration and temperature situation: first line with an ambient target, second line with a heated target, third line with a cooled target.</figcaption></figure>

In order to assess the calibration capacity from our acquisitions, we applied on them a target detection algorithm [8] based on Harris corner detection (Fig. 5). For cooled target, the gray level of the acquired images are inversed to match the black and white squares with the template of the algorithm. With the ambient target, the contrasts are not sufficient to obtain an estimation. With the heated target, the corners are properly estimated on the parts where the reflection do not appear but some intersections are hidden by the reflexion. With the cooled target, all the intersections stand out well and the great contrast allows a suitable estimation.

We have realized 14 acquisitions of each configurations and temperatures with various reflections and conditions of illumination. The points detection algorithm of [8] was applied on each image. This algorithm detects a target if at least 90% of the desired points are recognized. Otherwise we have try to detect a smaller target to obtain detected points. The Table 1 shows the average rate of correctly detected points in an image for each configuration. The proposed protocol provides performances suitable for an efficient calibration of thermal cameras. A heated target enhances the contrast but reflections induce significant artifacts. A cooled target reduces this effect and provides a robust calibration. A target in ambient condition provides very low detection rate. The reflections strongly impact the detection when the target is taken from a front view. That's why the difference of the detection rate between heated and cooled
targets is high in configurations 1 and 2 (front view) and low in the other configurations. With the cooled target, the results are robust and we observe few errors in front view. On the other hand the performances with the heated target are particularly degraded by reflexions in the front views.

We can notice that we have considered only the most common hot reflections. Nevertheless, in particular conditions, cold reflections could appear which will have the same effect on cooled targets as the hot reflections on the heated targets. This is a limitation of our protocol but this particular case is perfectly treated by the classic protocol of target heating. An other issue is the effect over time. Once heated or cooled the target will gradually return to ambient temperature. However the time tolerance of a target cooled by a simple loaf of ice easily exceeds ten minutes without degrading the quality of the calibration which is sufficient for a common industrial use. If the application requires it, materials with high thermal inertia such as ceramic for hot targets or ice block for cold targets could be used or a heating/cooling device can be added to the system.

<figure><img src="image_8.png" /><figcaption>Fig. 5: Detection points obtained by [8] on images acquired in each configurations.</figcaption></figure>

<table><caption>Table 1: Rate of correctly detected points on each configurations.</caption><thead><tr><th></th><th>Configuration 1</th><th>Configuration 2</th><th>Configuration 3</th><th>Configuration 4</th></tr></thead><tbody><tr><th>Ambient</th><td>8.9%</td><td>12.1%</td><td>11.4%</td><td>3.2%</td></tr><tr><th>Heated</th><td>73.6%</td><td>77.5%</td><td>85.4%</td><td>87.1%</td></tr><tr><th>Cooled</th><td>100%</td><td>97.9%</td><td>93.9%</td><td>86.1%</td></tr></tbody></table>

## 4. Conclusion

We propose a new procedure aiming to optimize the calibration of thermal camera from a copper and epoxy target. Unlike the conventional treatment which consists in heating the target we cool it. While on heated targets the hot reflections deteriorate the contrast between the two materials, they increase that on cooled targets. A more robust calibration is then obtained, in particular with pronounced orientation angles of the target.

The effect is only obtained on hot reflections and over ten minutes. But it meets the most common practical applications. The process is very simple to set up and effective on practical cases. Our future work will consist in developing a target of which we will have control of the temperature, for example from an electric current, to obtain a more accurate evaluation of the observed effect.
REFERENCES

[1] Ellmauthaler A., et al., "A novel iterative calibration approach for thermal infrared cameras," ICIP, pp. 2182-2186, 2013.

[2] González-Jorge H. Et al., "Single image rectification of thermal images for geometric studies in façade inspections," Infrared Physique Technology pp.421-426 (5), 2012.

[3] Gschwandtner M., Kwitt R., Uhl A., and Pree W., "Infrared camera calibration for dense depth map construction," Intelligent Vehicles Symposium, pp. 857-862(IV), 2011.

[4] Lagüela S., González-Jorge H., Armesto J. and Arias P., "Calibration and verification of thermographic cameras for geometric measurements", Infrared Physics & Technology, pp, 92-99 (54) 2011.

[5] Luhman T., Piechel J. and Roelfs T., "Geometric Calibration of Thermographic Cameras", Thermal Infrared Remote Sensing pp. 27-42 2013.

[6] Peric D., Likic V., Spanovic M., Sekulic R., Kocic J., "Geometric calibration of multi-sensor image fusion system with thermal infrared and low light camera", Proc. of SPIE, (9250), 2014.

[7] Prakash S., Lee P.Y., Caelli T. and Raupach T., "Robust thermal camera calibration and 3D mapping of object surface temperatures", Thermosense (28), 2006.

[8] Rufli M., Scaramuzza D., Siegwart R., "Automatic detection of checkerboards on blurred and distorted images", International Conference on Intelligent Robots and Systems, 2008.

[9] Saponaro P., Sorensen S., Rhein S. and Kambhamettu C., "Improving calibration of thermal stereo cameras using heated calibration board", ICIP pp. 4718-4722 2015.

[10] St-Laurent L., Mikhnevich M., Bubel A. and D. Prévost, "Passive calibration board for alignment of VIS-NIR, SWIR and LWIR images", QIRT 2016.

[11] Takashi S., Masayuki T. and Masatoshi O., "Accurate Joint Geometric Camera Calibration of Visible and Far-Infrared Cameras", Image Sensors and Imaging Systems pp, 7-13(7) 2017.

[12] Ursine W., et al., "Thermal / visible autonomous stereo vision system calibration methodology for non-controlled environments", QIRT, 2012.

[13] Usamentiaga R., Garcia D.F., Ibarra-Castanedo C. and Maldague X., "Highly accurate geometric calibration for infrared cameras using inexpensive calibration targets", Measurement, pp. 105-116 2017.

[14] Vidas S. Et al., "A mask-based approach for the geometric calibration of thermal-infrared cameras" Transactions on Instrumentation and Measurement, pp, 1625-1635 (61) 2012.

[15] Yu-Ming et al., "Acquisition of 3d surface temperature distribution of a car body", International Conference in Information Acquisition, 2005.

[16] Yu Z. Et al., "Camera Calibration of Thermal-Infrared Stereo Vision System", Conference on Intelligent Systems Design and Engineering Applications, pp. 197-201, 2013.

[17] Zoetgnande Y., Fougères A.J., Cormier G. and Dillenseger J.L., "Robust low resolution thermal stereo camera calibration", ICMV 2018.