My custom Thermal + Visible Image Blending Pipeline - Page 1
Forum section: electronics / general

author: matohak
author_rank: Contributor
timestamp: November 06, 2018, 12:55:57 pm
content: I'm new to thermal imaging and got a Flir C3 off ebay.Realised that there are so much limitations on the FLIR software. It's hard to batch process files; can't really align the visible image with the thermal one; limited blending options etc.So I've decided to make my own pipeline using Python and Blender (Actually, a special build of it that allows me to add my own Python script to the compositor:https://github.com/bitsawer/blender-custom-nodes):With Blender I can create my own image processing pipeline. With 100s of images I can keyframe how much displacement there is between the thermal and visible image, colour curves, and adjust alpha value between the visible image, edges detection, and thermal image.Given the poor thermal resolution of the camera I've made it such that edges are own shown at the warm bits of the images.In the example above, you can just about see the edges of people carrying a rowing boat on their shoulders into a boathouse.PS. I'm using a modified version of this script to batch export temperatures as BW images:https://github.com/Nervengift/read_thermal.py/blob/master/read-thermal.py

author: Vipitis
author_rank: Frequent Contributor
timestamp: November 06, 2018, 02:33:39 pm
content: Definitely looks interesting, do you work with the embedded visual image of the radiometric jpg?

author: cdev
author_rank: Super Contributor
timestamp: November 06, 2018, 02:44:13 pm
content: I have a cheap ($20) conventional IR thermometer. It can be used to laboriously look for cold or hot spots but its resolution leaves much to be desired, also the data would need to be extracted to be useful. However using something much like what you are doing, the data from it could be made a lot more useful, I think. At least for any still subject.

author: matohak
author_rank: Contributor
timestamp: November 06, 2018, 08:16:39 pm
content: I just used read-thermal.py above. It reads the exif within the jpg which has the sensor's raw data and processes the data into temperature readings.The edges and the image were extracted from the visual image, of course.

author: matohak
author_rank: Contributor
timestamp: November 06, 2018, 08:17:50 pm
content: i don't see how would that work. Are you going to make 500 point measurements and enter the values into a spreadsheet to make a thermal image?

author: cdev
author_rank: Super Contributor
timestamp: November 06, 2018, 08:26:35 pm
content: Quote from: matohak on November 06, 2018, 08:17:50 pmLOL, basically, I was thinking about automating something like that with a tilt-swivel mount, but frankly, its such a PITA that buying a cheap thermal imager would likely be much smarter.How much did you pay for yours? Is there any risk of getting a bad unit?

author: Vipitis
author_rank: Frequent Contributor
timestamp: November 06, 2018, 09:57:35 pm
content: There have been crazier projects:https://www.eevblog.com/forum/thermal-imaging/diy-pinhole-thermal-camera/Or more like what you were thinking off:https://www.niklasroy.com/project/195/DIY_thermal_imagingBut back ontopic: what makes your process special? From what I understood it's basically the same as the flir Tools App can do on my phone, but optimized for badge processing? Do you just discard the lower end of the readings and make them and blend the rest?

author: cdev
author_rank: Super Contributor
timestamp: November 06, 2018, 10:35:45 pm
content: I don't have a process. I was just speculating on what would be the most economical means of 'thermal imaging' mostly so I could better weatherproof, find air leaks etc.Your technique is really quite good, not just from a technical point of view also artistically and also its good data vis, in that the date comes alive better than most of the thermal images I see. (by combining the fairly low-res thermal data with much higher res standard photography, your images are very easy to understand and its clear that you get a lot of mileage out of the data you have).I'm basically rediscovering Blender right now (because I finally have a video card that can work with Blender as it is supposed to be used) and would love to do something like that.Quote from: Vipitis on November 06, 2018, 09:57:35 pmhave to look into low cost thermal imaging. I have waited up until now because I keep thinking a price breakthrough is right around the corner, but I think it would be a good thing to have to save money on heating and cooling, sooner rather than later.

author: matohak
author_rank: Contributor
timestamp: November 07, 2018, 11:51:56 am
content: Not sure who this reply is for. The links are for cdev but the questions are for me?Yes basically I just made a pipeline that is more open to customisation1) Easier for batch processing. Let's say you take 200 images you can just keyframe the same processing settings on a continuous sets of images.2) In FLIR tools you have to choose either i) MSX blending, in which you loose all the colours of the original image or ii) thermal overlay in which the alpha values of the thermal image is constant so you can't see the colours very well. This pipeline would allow you to:a) Choose and tune your own colour mapb) Choose to blend in a mixture of edges + visible image + thermal image. Each of the three layers could have their own alpha values. For example, I find that it's good to decrease the alpha value of the thermal image at the hotspots and show the edges instead.3) I can't seem to find a tool for parallax adjustment on FLIR tools so nearby images always seem to be misaligned? Again on my pipeline allows batch parallax adjustment to align the visible + thermal images.In the future, I would like to implement a neural network (a U-net) for foreground/background separation and automatically adjust the colour scales based on only the foreground =)

author: cdev
author_rank: Super Contributor
timestamp: November 07, 2018, 01:37:07 pm
content: What image sensor are you using, and what is its cost/vendor? Do you think its the best choice in its price range?

author: Vipitis
author_rank: Frequent Contributor
timestamp: November 07, 2018, 02:53:25 pm
content: OP is using a FLIR C3 camera ->https://www.flir.com/products/c3/a tiny thermal imager with an integrated visual camera and LED torch. It's really expensive compared to other products, but it shines in terms of size and ruggedness. FLIR uses a process they call MSX to superimpose the visual image edges so your final image has some contrast as he thermal image is very low resolution, blurryly upscaled and fixed focus. There are only a few products that compare with the C3, you can get handheld imagers with a screen and better resolution for less: Seek Reveal(pro/XR) or the Leopold which is basically a Seek reskin. But you lack the touchscreen and the visible light camera. Seek sensor also have their own issues in terms of image quality compared to flir, but yield a vastly greater resolution. You can find comparisons and discussions in other threads in this forum. For less money you can get phone dongles with higher resolution but without a visual camera(TE-M1, Seek Compact XR/Pro); a single with the very same sensor and visible camera (FLIR ONE 1st Gen/3rd Gen) and for the same price you can get higher resolution phone dongles (FLIR ONE 2nd Gen/3rd Gen PRO, TE-Q1, Therm App). But you lose he ruggedness and also have no standalone device. Sometimes the C2/C3 ist exactly what you need (although the CAT S60/S61 have all the features and much more in a single device for about the same price). Of course there are more thermal cameras with visible and all, but they have a different formfactor and price.As far as I understand it now, your pipeline allows for even more blend modes and the combination of MSX/Alpha Blend; more control over the palette and batch processing. But you still have to set those values manually, don't you? I just feel like there are a lot of programs already that allow you to combine the thermal and visual in different ways. Be it Flir Tools, joe-c software from he sticky, Palette Generator,...Do you think it would be possible to build an executable that allows you to input files and adjust the key values?

author: matohak
author_rank: Contributor
timestamp: November 07, 2018, 11:03:03 pm
content: Thank you Vipitis for giving such detailed answers for me. The C3 was indeed quite expensive for the thermal resolution - I got it for around £400 (that's already a lot cheaper than the retail price) off ebay possibly as a demo product.But the camera is really convenient - it looks just like a smartphone so I can snap around without being too conspicuous. I take it out for dog walks and may bring it outdoors and would just fit into my pocket easier than my phone would fit.I had considered the phone dongles before but got worried about them snapping off (after reading the negative reviews on Amazon). I did not regret getting the c3 over the phone dongles, but I wished I had saved some money and got the c2 instead as they have the same sensor - Wifi was completely useless for me. For other devices like the therm-app, it was really hard to find them cheaply on eBay. Hence the price difference would be quite large if I decide to get one.However, I saw this a few days ago:https://www.thermal.com/seekshot-series.htmlit looks like the device is going to beat the c3 at every aspect at a similar sell price and can do videos. Any thoughts?-------------Unfortunately I don't think I'll have the spare time to build an exe... that'll require writing up a GUI from scratch.