Thermal Camera+ App by Georg Friedrich Superresolution Comparison - Page 1
Forum section: electronics / general

author: Vipitis
author_rank: Frequent Contributor
timestamp: October 26, 2017, 03:44:51 pm
content: I wanted to post a comparison to the different options, to get some feedback and also serve as advice for people thinking about a purchase.First the setup. Camera is the Lepton sensor in my CAT S60 phone on a tripod, in portrait(so what you see is 90° to the side) target is some pipes in our basement. Emissivity is set to 95% and Reflected Apparent Temperature 300K. Palette use is Iron. Images were taken once the superresolution indicator was green.Issues with this comparison: The scale of the palette and therefore the histogram is not locked! locking the palette and changing anything in settings messes up the shutter and the app needs to be quit and restarted. You can see that the min/max temperature are not consistent.There are a few settings to chose from. Manual for reference:http://thermcam.binaryblogger.de/manualA/#superres1. Superresolution: on / off2. Superresolution sharpening mode: Medium / Harder3. Upscaling: Bilinear / Nearest Neighbor***Attachments:1. No Superrsolution, Nearest Neighbor2. Superrsolution withMediumsharpening, Nearest Neighbor3. Superrsolution withHardersharpening, Nearest Neighbor4. No Superrsolution, Bilinear5. Superrsolution withMediumsharpening, Bilinear6. Superrsolution withHardersharpening, Bilinear

author: Ultrapurple
author_rank: Super Contributor
timestamp: October 26, 2017, 03:54:13 pm
content: True superresolution relies on small movements (hand shake) - it simplydoes not workif the imaging device is fixed and stationary on a tripod and looking at a stationary subject.There's an explanation herethat describes how superresolution works to use sub-pixel data from several consecutive but slightly different frames - the differences it uses are either camera shake or other movements such as panning.Noise reduction, on the other hand,mayneed a stationary camera (as described in the manual link you give).

author: Vipitis
author_rank: Frequent Contributor
timestamp: October 26, 2017, 04:21:55 pm
content: Given the tripod used, the minimal force applied to touch the capture button, will provide a slight shake that should be sufficient.Stacking multiple captures graphically with same temperature scale could be done to reduce noise. But they inconsistency of the Lepton over time and FFC will be an issue.

author: tomas123
author_rank: Frequent Contributor
timestamp: October 26, 2017, 10:31:17 pm
content: @VipitisNice workI wrote in this forum something about the SuperResolution of thermal images.SR ist not a magic tool with large improvements. Nothing is better than real pixelsSuperresolution is sharpening of the imagewithoutfalse temperature values on contrast edges.This a bad side effect of sharpening with Gaussian blur.As sample look at the "cooler" temperature (light yellow rings) around the pins of the last oversharpen images (Gaussian blur) in this post:Your CAT S60 lepton sensor with (low) 80x60 pixel has no big potential for more details with SuperResolution.With a lepton 160x120 pixels you have noticeable more potential for more sharpness on fine (!) thermal structures (see video below in full screen, it's the same algorithm/code as in Thermal Camera+ App).The greatest SR effect by thermal images is thedenoiseof (live view = video) images with a small (!!) temperature spans.In your images with a large span of 21 Kelvin by a 2^8 LUT grayscale (=80mK steps) you have by design only little noise.If you interested in this topic, make 10 normal thermal images and stack the images with photoshop. There you can play about with the technical possibilities.a sample with lepton 160x120Quote from: tomas123 on April 28, 2016, 09:40:35 pma sample of90 (!!) framesof Flir Exx:Quote from: tomas123 on January 07, 2014, 09:15:05 pmQuote from: tomas123 on January 08, 2014, 01:07:33 pm

author: Vipitis
author_rank: Frequent Contributor
timestamp: October 26, 2017, 11:07:07 pm
content: I emailed Bullit and asked, if a upgrade is possible. They denied this because the Lepton3 is larger then what they used and they would have to redesign the whole PCB etc.I only got 80x60 pixels and 2 died already. I want to find out what I can do with it. I am currently exploring many different options in creating more then the standard flir processed images.the Thermal Camera+ app gives me real pixels and a functional way to lock the palette. I am currently attempting photogrammetry, but need to find a way to mask the min/max and scale from the image so they get ignored by VisualSFM.The app sadly on captures single images, no sequences, not timers.

author: tomas123
author_rank: Frequent Contributor
timestamp: October 26, 2017, 11:18:24 pm
content: look at my footerThere you find many old hints for post processing of the Flir radiometric jpgs (saved by Flir app).You can do this with simple scripts (look palette, photogrammetry eg)

author: frenky
author_rank: Unknown
timestamp: October 27, 2017, 06:47:32 am
content: Just to make things clear... Super-resolution and image stacking are two different things.While image stacking can greatly reduce noise it does not increase resolution.for super resolution you need to take multiple images that are slightly offset. Then upscale each pixel and align all images.This way you get extra information about values that are between neighbour pixels of each frame.

author: Ultrapurple
author_rank: Super Contributor
timestamp: October 27, 2017, 07:16:38 am
content: Quote from: frenky on October 27, 2017, 06:47:32 amI completely agree with all of what you say here. You said it far better than I managed!As we've seen on the Zotac video by Thomas123, true superresolution is capable of really impressive results.Combining superresolution, stacking and sharpening can give further remarkable improvements. I haven't properly been able to do all three on moving images yet but I'm sure someone clever out there will be able to do so.

author: frenky
author_rank: Unknown
timestamp: October 27, 2017, 07:36:17 am
content: I did some super-resolution experiments while trying to improve original SeekThermal.I mounted Seek module on stepper motor and took photos while motor was rotating:Result were quite better than original Seek images:Original:Many images aligned and stacked to achieve super-resolution:If you look at IC pins. I would never be able to get such an improvement only with stacking.I did my aligning by hand & eye in Gimp.But there are out the box solutions made for astrophotography and could be used for thermal imaging:AutoStakkertAutoStakkert! is all about alignment and stacking of image sequences, minimizing the influence of atmospheric distortions (seeing). Its goal is to create high quality images of the Planets, the Sun, and the Moon, without too much hassle.https://www.autostakkert.com/wp/download/RegiStaxRegistax 6 software for alignment/stacking/processing of imageshttp://www.astronomie.be/registax/download.html

author: Vipitis
author_rank: Frequent Contributor
timestamp: October 27, 2017, 10:07:21 am
content: The last example even has stiching with it.Panoramic stiching with rotating motion to further, get a larger image.In the end a solution that does all 3 things would be the goal, and it works in the capturing app.

author: tomas123
author_rank: Frequent Contributor
timestamp: October 27, 2017, 12:16:16 pm
content: Quote from: frenky on October 27, 2017, 07:36:17 am... as you can see in my postimg above, it works great ;-)Quote from: tomas123 on January 07, 2014, 09:15:05 pm

author: frenky
author_rank: Unknown
timestamp: October 28, 2017, 07:20:45 am
content: Quote from: tomas123 on October 27, 2017, 12:16:16 pmI didn't notice that but it's good to hear that we can use it for thermal.Sent from my LG-D855 using Tapatalk

author: in.Vitro
author_rank: Regular Contributor
timestamp: December 02, 2017, 08:00:51 am
content: Sorry for OT, but is there a chance for a Video Mode?

author: Ultrapurple
author_rank: Super Contributor
timestamp: December 02, 2017, 09:38:30 am
content: ThermViewer driving a Therm-App camera has superresolution on video (and still images) - it works remarkably well.When comparing my regular Therm-App (384 x 288) with ThermViewer superresolution against a Therm-App Pro (640 x 480) without superresolution, the lower resolution imager gave at least as good an image, subjectively, as its higher resolution sibling.Superresolution like thisdoesrequire a small amount of hand shake: it simply does not work on locked-off tripod images (you get double-size images, but no rxtra actualresolution).For an example of offline superresolution video processing,see this video of the Spain-Gibraltar border, which was filmed at 640 x 480 and superresolution processed to 1280 x 960 usingVideo Enhancer 2.

author: Vipitis
author_rank: Frequent Contributor
timestamp: December 02, 2017, 07:31:40 pm
content: As far as I know - video superresolution is doing continues multiframe superresolution.so it looks at the past 4 frames. and when your imager does 9hz - you can easily get ghosting artifacts on moving targets like cars or people.Thermal Camera+ does not support video capture. It also lacks any timer or intervall settings which could subsititude video. My main concern with the app right now is that it burns in the min and max temperature as well as the palette gradient and to archive that - images will get upscaled to 508x640 - either using the Nearest Neighbors setting - basically making every pixel become 4 pixels - or the bilinear upscale by mixing the additional pixels form either a 120x160 supperresolution picture or the 60x80 native picture.Supperresolution has two sharpening modes - while "Medium" is giving more accurate readings, as well as looking better for NN and bilinear upscale the "Harder" sharpening mode will make fine details and edges more visible by inducing more contrast to some edges and more noise to the overall image (I would love to compare it toFlir DDEwhich they developped for survaliance applications - using same sensor and same target).The data burn in basically voids data of a few pixels and therefore should have a setting to be turned off or be moved off the image and into the 28 wide column of pixels for the gradient.I did not succeed to make a imagemagick script to convert a image out of the Thermal Camera+ app into a .jpg without burned in gradient or data to use for photogrammetry or stitching.I know that it is possible to somehow extract a raw thermal image from FLIR radiometric .jpeg with some sript found on the forum but I failed to do so as well. and the FLIR firmware upsclaes the images from the CAT S60 lepton - or Flir ONE - to 240x320 "IR resolution" in flir tools which is 24x (= 16x) the effective resolution. As well as to a .jpg resolution of 480x640(which could be due to the to bad resolution and still downscaled and cropped visula image being overlayed in MSX). Flir also applies histogramm equalization and some kind of streching to apply the whole color range of the palette on only the range of temperature shown in the image. This has the result of images capture with the myFLIR app, of the same target but with different background highs and lows will look different even if I set the range of temperature to the same min and max the whole image will look different because factors like gain and gamma can't be changed(It might be that some of those factors are calculated through the parameters in Flir tools: Emissivity, Refl. temp., Distance, Atmosphereic temp., Ext. optics temp., Ext. optics trans. and Relative humidity).I am still looking for a solution to capture multiple images - by hand, as interval or as 9hz video - using the very same fake color - or a radiometric fashion where I can adjust the fake color palette in post. A solution would be to turn your all images into a raw array of 80x60 pixels and generate a 14bit greyscale that covers the whole sensor output range in 50mK steps. Flir Tools can generate a 320x240 .cvs with temperature values... but this means they read 16 temperatures out of a single Lepton pixel to 3 decimal places in °C and I know how much of that is bullshit in the end.Right now Thermal Camera+ with Superresolution on "Harder" (or maybe I should do "Medium" instead) and NN zooming with locked palette(which locks the range of the temperatures as well as gamma) and shooting from multiple angles for photogrammetry or getting closer and shooting an array of a few rows with good overlap for stiching in ICE - this is only effective at a large FoV when the target is flat as perspective becomes an issue with planar motion.Videos and Timelapses made in myFLIR app are bad because locking the temp range of the is broken due to the Shutter going crazy and unless your burn in a spot you can't get any data. not even a min/max and gradient. Flir radiometric video is only part of their high end research cameras.

author: tomas123
author_rank: Frequent Contributor
timestamp: December 03, 2017, 05:37:52 pm
content: Don't forget, that "continues multiframe superresolution" of Thermal Camera+ also reduce the noise in thermal images with low temperature span.I can confirm the "histogramm equalization" in Flir images. With the 16bit RAW images (inside your thermal jpgs) you can bypass the equalization.It's simple, see my tutorials.IMHO a 80x60 Lepton (CAT S60 , Flir One G1 for IOS) is not a good basis for image enhancements.see my sample images:https://www.eevblog.com/forum/thermal-imaging/flir-e4-thermal-imaging-camera-teardown/msg802245/#msg802245Quote from: tomas123 on November 17, 2015, 02:50:47 pm

author: Vipitis
author_rank: Frequent Contributor
timestamp: December 03, 2017, 08:57:37 pm
content: I think if you shoot a still target with a few superresolution(handheld) shots and then stack the images you get rid of more noise.I have to try and understand all your work with exiftools and learn to use it - but that is a project for next weekend.

author: Ultrapurple
author_rank: Super Contributor
timestamp: December 04, 2017, 10:01:59 am
content: Quote from: Vipitis on December 03, 2017, 08:57:37 pmYes, you're right. This image was basically done that way - a 10 second video (rather than individual photos) totalling about 250 frames, processed in Registax 6 to produce a superresolution, noise-reduced version. Click the image for more info.The vertical line in the centre of the windscreen is down to my stupidity in not turning off the cross-hairs in ThermViewer before making the images.