New image enhancement techniques coming to thermal imaging - discussion - Page 1
Forum section: electronics / general

author: Fraser
author_rank: Super Contributor
timestamp: March 07, 2025, 01:54:14 pm
content: As many will know, we are seeing far more Artificial Intelligence (AI) software included in our daily lives. Thermal imaging is no exception to this technological step. Whether AI can offer us true benefits in thermal imaging I have yet to decide.As an example of where AI is already present in the thermal imaging marketplace, HikMicro use it to enhance the imagery of their Eco series of cameras. These cameras use a relatively low resolution 96 x 96 pixel microbolometer that has AI algorithms applied to its thermal scene output to create a 240 x 240 pixel image for the user. This is done without (apparently) impacting the 25fps frame rate as it does not use the older “Super Resolution” image stacking technique. When looking at the sales blurb for HikMicro ECO series cameras they state that the AI has been taught about many scenes and it pulls upon that data (knowledge ?) to enhance the scene data that is presented to it. An interesting approach. The problem I have with AI is that it can only be as good as it’s programming and reference data sets unless it is a learning AI. The old “Garbage in, Garbage out” saying applies.https://www.hikmicrotech.com/pt/explore/industrial-blog/HIKMICRO-SuperIR-An-Advanced-Algorithm/I recently tested the Thermal Master “P2” dongle camera and this includes an “X3” image enhancement mode that claims to double the resolution through algorithms. I seem to recall mention of AI algorithms being used for the “X3” mode in some sales blurb from Thermal Master. So this could be another example of a thermal imaging system producing enhanced imagery beyond the basic capabilities of the microbolometer, through the use of AI. I tested the “P2” over many weeks and noted that the “X3” mode definitely improved the appearance of some thermal scenes, whilst making others appear “weird” and just “not right”. I have to wonder whether I was seeing the AI’s limitations in terms of it “understanding” the scene data that it was receiving and not being able to correctly enhance the scene. Thermal Master advised that they are still developing the image enhancement technique and agreed that some thermal scenes are not as well enhanced as others at this stage of development. They are working to improve their “X3” mode. Thankfully Thermal Master allow the “X3” image enhancement to be switched off.So these are just two cases where it would appear that AI is a significant part of the thermal scene processing and enhancement process within the camera. I have to wonder whether we will see more AI creeping into thermal camera designs in the near future. We already see extensive use of DSP technology in our cameras so AI may be the next logical step ? Whether this is a change for good is a matter for discussion as the user has to trust the AI to not trash the scene data set or mislead with incorrect modifications to a thermal scenes data. WYSIWYG is important in Thermography and image enhancement techniques need to be used with intelligence and care.It is all too easy to become drawn into the “Wow” factor of recent advances in AI capabilities but OEM’s will need to keep a cool head if our future thermal imaging cameras are to be what we actually need for tasks. AI is very much a buzz word loved by marketing teams as it sounds “modern and cool”. Marketing teams can really stuff up a product if they unduly influence the design teams decisions.We live in an era of AI enhancements in technology so it will be interesting to see how the thermal imaging market develops.Fraser

author: Fraser
author_rank: Super Contributor
timestamp: March 07, 2025, 02:00:22 pm
content: From a Forbes commentary on the Thermal Master P2 camera…….“ Despite its compact dimensions, the P2 has some powerful features and an infrared resolution of 256 x 192, with the option to upscale to 512 x 384 in HD mode, which uses millions of data points and is AI-powered with deep-learning algorithms to enhance the clarity of the images for enhancing fine details even when its used in challenging outdoor environments, something that’s not always possible with larger and more expensive devices.”(https://www.forbes.com/sites/marksparrow/2025/02/01/update-to-thermal-masters-tiny-p2-smartphone-camera/)So AI does indeed appear to be used in the image enhancement process.Fraser

author: Fire Doger
author_rank: Regular Contributor
timestamp: March 07, 2025, 02:05:59 pm
content: Better idea: Slap an AI on scopes, screw Niquist, we have AI!Is x2 really AI? you can sprinkle some noise and make it believable

author: Fraser
author_rank: Super Contributor
timestamp: March 07, 2025, 02:20:37 pm
content: I actually wonder what is meant by “AI” and “Deep learning AI” these days when applied to relatively low processor power consumer goods. My understanding of true AI was that powerful computers with access to huge data repositories were required. We are starting to see “AI” associated with all manner of domestic goods that have no connectivity to a more powerful computing system or a large data repository. I am assuming that there is a “Full fat AI” and an “AI Lite” with the latter having a very focussed data set, and a far more restricted AI capability, that fits into small embedded computer systems and SoC type applications ?As I have already stated, maybe we are witnessing “AI” as the latest “Buzz word” being used by marketing teams wanting to make their product sound “clever” and “cool” ?Fraser

author: Fire Doger
author_rank: Regular Contributor
timestamp: March 07, 2025, 04:31:51 pm
content: AI, deep learning, neural networks are very clearly defined. (for data scientists)For marketing people this is AI:

author: Fraser
author_rank: Super Contributor
timestamp: March 07, 2025, 04:54:24 pm
content: Yep, marketing BSThis is why I an trying to dig a little deeper into what "AI" actually means in the thermal imaging camera realm. Interpolation, and even "smart" interpolation techniques have been around for a very long time but we are now seeing claims of "AI Image enhancement" that is not just simple interpolation based. Are we seeing common DSP based image processing or is there truly an AI engine of some sort at work here ? I am still leaning towards your suggestion that much of this is marketing BS because of the emergence of "AI" as a current and cool topic of conversation.Fraser

author: Fraser
author_rank: Super Contributor
timestamp: March 07, 2025, 05:02:52 pm
content: I am reminded of the early IRISYS IRI1011 thermal imaging camera that used a 16x16 pixel FPA and a Compaq PDA host to produce a 128x128 pixel thermal image. When I first saw the pictures of this feat I though "what kind of sorcery is this" but, in truth, it was just the clever use of interpolation to upscale the very low resolution image and "blur" the pixels edges. The approach worked for the intended market and IRISYS were open about the true optical resolution of the thermal sensor FPA.Fraser

author: Vipitis
author_rank: Frequent Contributor
timestamp: March 11, 2025, 09:16:23 pm
content: As a post on this forum has got me interested in information science and contributed to my direction to doing my bachelors in a related field... I feel like I can comment on it with some authority.First of all "AI" is a horrible term and I will not use it. To be more accurate and not confuse people with marketing BS, it's model. Could be language models, could be image decoders, could be radiance fields. all very different. It's important to define a simple system as in what is the input what is the output.For a language model(GPT = decoder only transformer) the input is text (tokenized into IDs) and the output is a probability distribution of the next token.For an image decoder (diffusion/image generation) the input is a text prompt (tokenized and embedded into a dense representation), a timestep and some image. The output is an image where a small amount of noise has been attempted to be removed with the goal of moving the resulting image output closer to the input prompt (in kind of a shared embedding space/CLIP).Radiance fields (NERF) are an idea to do structure from motion, but by learning the scene. So input is a set of coordinates and direction (a camera view). And it outputs the pixels (an image) via density estimations per ray.The important part here is that models are learned systems. Using either a labeled (supervised) training set, unlabeled (unsupervised/semi supervised) or a set of environment rules, actions and rewards (reinforcement learning). Learning can be done via backpropagation (loss and gradient descent) or some genetic algorithm on the RL side.Systems might have access to multiple models and datasets (think RAG). But models do not. They learn a dense representation of the training data and then use their parameters to predict the outputs.So for the task of super resolution/upscaling it's an interpolation task. A few pixels in, more pixels out.This can be done with various computer vision models like various CNN, UNEt or even ViT (vision transformer). Notably successful are these models in computer games (DLSS, FSR4, XeSS, AutoSR) where these systems improve latency by doing upscaling and also anti aliasing (and denoising). For these systems the input is multiple past frames (with jiggle offset, depth buffer information and sometimes more). These models are extremely fast <2ms and would also work for a camera system, even in a pov context. They are a temporal solution so include artifacts like trailing and smearing, but much less than TAA.But here is the big issue. To train such a model, you need labeled data. Labeled meaning you have input and output pairs. So for video games you simple render the game at a really high resolution and highest settings. And then the low resolution no AA inputs. This can be done offline and doesn't require real time. A similar approach also works for phones with limiting results.Some might remember Topaz ai, as it featured denoising, upscaling, sharpening and even coloration. They trained their early models with synthetic data. Meaning they start with a high quality, low noise, high resolution image and then synthetically degrade it with fake noise, down sampling, blurring etc to try and reconstruction the original image. This gives great results for photography, but also has issues and limits.Such an approach could work for thermal cameras. But not beyond the quality of the best cameras available right now.We know that small training dataset for thermal images exist publicly. Such include annotate objects and even parallel faces (vis, nir, lwir).There is a chance train on much larger images from visible and nir cameras and hope for models to generalize well into the domain of thermal images... Which I haven't seen done yet. But I also haven't looked into the literature.So if any company says "AI", inquire what models they use and more importantly, what training data they have.I see some potential in increasing resolution, and reducing temporal artifacts from existing superresolution features. Nosie reduction is also worth a try. However it should be noted that learning based models, are never reliable and heavily biased. The new datapoints shouldn't be trusted for measurements or identification.

author: EE-digger
author_rank: Frequent Contributor
timestamp: March 12, 2025, 05:32:35 pm
content: Many of the "products" of AI we see today have little or no intelligence behind them. I'd rather call them AG for Artificial Genesis, since they have artificial origins. I realize that artificial may be a bit harsh but when an image, sound or decision is created from billions of bits from real things, I would no longer call the end result "real".AI in imaging is fine and will probably yield impressive results. As long as you know the limitations of that AI in terms of what is real and what is an educated guess based on estimates, predictions, etc. you're probably ok.