parallax - Page 1
Forum section: electronics / general

author: artag
author_rank: Super Contributor
timestamp: July 12, 2024, 12:50:24 am
content: I like the effect of an edge-sharpened visual camera superimposed on a thermal image to make it easier to recognise the view (though increasing resolution is making that less important) but actual implementations suffer from parallax and need a usually manual correction to line up the images accurately. Adding an infra-red macro lens usually makes the visible image unusable - either the supplementary lens obscures the other sensor or the changed FOV makes the images unmatched.It seems as though there's a few ways to fix this. My knowledge of optics isn't great so perhaps someone could tell me why these wouldn't work :1. Perform autofocus on one or both images and use that to measure the distance (like an old-fashioned rangefinder). Then use the measured distance to correct the image overlap in software.2. Use a semi-clear lens such as zinc selenide to cover both sensors. This will result in a yellow-tinted image and a different focal length. Image colour is not a huge problem (it's often just monochrome for edge sharpening and maybe some colour correction is possible) and an approximation to focal length correction could be made by placing the sensors in different planes or even moving one of them.3. Place a half-silvered, front-silvered mirror in front of the visible light lens at an angle. Reflect the IR sideways into the thermal sensor. It won't pass through any glass so will be subject only to the thermal lens. Part of the light will pass through the mirror and into the visible-light lens system. The two images will then have separated optical paths.4. In an inversion of 3, have a flat plate mirror of germanium or chalcogenide glass which would reflect the visible image sideways and pass the IR image through. Maybe even have a plano-convex lens where the plano side acts as a mirror and the convex side focusses the IR.

author: Bill W
author_rank: Super Contributor
timestamp: July 13, 2024, 10:40:31 am
content: Quote from: artag on July 12, 2024, 12:50:24 am1 would work - but only at the  focal plane2 won't work because you still have two sensors on separate optical axes.3 & 4 would work.There is also 5 - where manual focus of the thermal lens is used to converge the optical sensor axis mechanically at the desired distanceWith relatively wide angle lenses, and being at the 'cheap' end of things, these are not going to be worthwhile commercially. The old rule that spending too much on fixing a 'cheap' system means any buyer could simply buy the more expensive system that works properly in the first place.You would also still get the phantom images issue of looking through glass - where 'MSX' tries to show a tree through the window in thermal.Bill

author: artag
author_rank: Super Contributor
timestamp: July 13, 2024, 01:08:57 pm
content: Thanks for the feedback.I guess you're right that they wouldn't be cheap any more, though might be fun DIY. The cheapest would be the zinc selenide lens, as you say it wouldn't be ideal but maybe puts the sensors closer together than the common solutions.Is there any one doing a 'proper' system, albeit expensive ? Maybe using some improved version of one of the above.

author: CatalinaWOW
author_rank: Super Contributor
timestamp: July 13, 2024, 05:30:56 pm
content: You can only correct parallax if the subject has no depth. You can sorta-kinda correct but there will always be pieces of the scene invisible to each camera when the scene has depth.The only fix for this is to have a common optical path from the scene. It can be achieved by a partially reflecting mirror. If the mirror is an interference filter the split can be nearly lossless with each sensor receiving nearly all of their in band photons. Chief drawbacks are bulk and fiddly alignment + cost of the filter.

author: Vipitis
author_rank: Frequent Contributor
timestamp: July 15, 2024, 02:39:36 pm
content: using a beam splitter and common optical path does kinda work. I have tried to do that by using a Germanium window. It did require both systems to focus individually, and was somewhat limited to a narrow fov.

author: Phil1977
author_rank: Frequent Contributor
timestamp: July 15, 2024, 02:53:25 pm
content: No 6. Place 3 VIS-cams around the thermal imager, combine their pictures to a synthetic aperture picture which has the IR-Imager as a virtual optical axis.Would probably mainly need a potent video processing unit.

author: dietert1
author_rank: Super Contributor
timestamp: July 15, 2024, 03:28:09 pm
content: Our FLIR camera supports manual parallax adjustment, but only for distances above 0.3 m, where the parallax is already small as the distance between the two imagers is about 2 cm.I think a beam splitter should work pretty well as one doesn't need a lot of light for the RGB imager.Regards, Dieter

author: Phil1977
author_rank: Frequent Contributor
timestamp: July 15, 2024, 03:51:41 pm
content: The problem with the beam splitter is that it works best if it is transparent for thermal IR - that means in best case you use Germanium which is quite expensive for larger pieces.If you do it vice versa, if you e.g. use a gold plated glass that is highly reflective for thermal IR but transmits a small part of VIS, then you get a quite affordable system but the thermal IR reflector is always reducing your contrasts and makes it more difficult to get absolute temperature measurements than it anyhow is.

author: artag
author_rank: Super Contributor
timestamp: July 15, 2024, 03:58:27 pm
content: What about dichroic glass ? That's used in halogen bulbs to allow IR to pass out the back whilst reflecting visible as a parabolic mirror. I don't know what wavelength it passes though, nor how much of the IR is absorbed by the glass.I like Phil's virtual camera idea. I think something like this is done for wrap-around car reversing mirrors so probably doesn't need an insane amount of processing. Especially for something like PCB views where you've likely got a PC and maybe a GPU to do it. Maybe it could also provide depth imaging. However it might make focussing more difficult - 3 instead of 1.

author: Phil1977
author_rank: Frequent Contributor
timestamp: July 15, 2024, 04:15:39 pm
content: The problem with all dichroic filters is that it´s very difficult to get it highly efficient and broadband. For the halogen bulbs it was for sure better than nothing, but for (precise) thermal imaging you have to keep the SNR for low temperatures really good. A few percent of self-emission would probably spoil the thermal image quality - the real thermal camera experts here can tell more about this for sure.

author: CatalinaWOW
author_rank: Super Contributor
timestamp: July 15, 2024, 04:23:50 pm
content: Quote from: artag on July 15, 2024, 03:58:27 pmThere might also be possibilities with a pellicle, a thin film stretched over a frame to make it flat. The minimal thickness minimizes absorbtion losses making materials like polyethylene feasible. The downside is that I know no way to lay an interference filter down on polyethylene so you are limited to partially reflecting mirrors.Another option to explore to improve the contrast is to cool elements visible to the sensor. A little bit helps a lot, but staying above the dew point, and calibration for thermometry is more challenging.What you do depends on whether geometric accuracy or temperature accuracy is more important to you. For my uses of thermal cameras looking at circuit cards even several degrees of error isn't too important, but there are applications that need (or at least really want) fractional degree accuracy.

author: Phil1977
author_rank: Frequent Contributor
timestamp: July 15, 2024, 07:06:35 pm
content: Quote from: artag on July 15, 2024, 03:58:27 pmThe idea to look for the car-surround-view algorithms seems promising. The real "synthetic aperture" approach is mathematically quite demanding, but the cars proof that you can do this on quite a low level GPU. But I don't know if the car systems really can reconstruct an arbitrary 3-dimensional environment or if they are limited to a standard road topology.The older systems I know of own driving experience just merge together two or more pictures after perspective correction - that´s really easy to do, but wouldn't help for the IR-cam.I´ve seen in Niu and Tesla the systems where you can turn around the view around the car however you like, these seem to be more advanced - but how advanced is more advanced?

author: artag
author_rank: Super Contributor
timestamp: July 15, 2024, 07:48:58 pm
content: I was imagining something like the software for stitching stills together into a panorama. But tbh it's probably driven by the user and just does the detailed work given a key match. That would be OK for initial alignment but not really for continuous use.My needs aren't really very great. I'm just extrapolating from the slightly underwhelming cheap offerings and wondering what's possible.I do have an idea in mind.I use an optical stereoscopic microscope for a lot of work. It has a long arm mount and produces a superb image but sometimes the need to use eyepieces is annoying. A Vision Engineering unit is very expensive and takes a lot of space. A USB microscope tends to be laggy even though I have a conveniently placed monitor.I can imagine a digital microscope setup (probably streaming over HDMI rather than USB) with a suitably floating bench support and the possibility of overlaying thermal data. It could be quite big as most of the optics would be behind the LCD screen. I'm not sure if the lack of stereo vision would be a big handicap - the cheap digital microscopes I've tried have mostly suffered from small screen sizes and lag.As CatalinaWOW says, the thermal image alone is pretty usable - the optical overlay helps locate the image but the thermal image has enough detail that it's pretty usable on its own. I've just doubled resolution by going from the Seek camera to a 256x192 TS001 which helps as well.

author: Geoff-AU
author_rank: Frequent Contributor
timestamp: July 16, 2024, 12:43:38 am
content: Quote from: dietert1 on July 15, 2024, 03:28:09 pmMine too. I emailed them asking for the slider to have more range, so that my $400 entry-level IR phone dongle could have better close-in performance for working on PCBs.They replied, telling me to spend $4000 on a unit that has better specs.Needless to say this will be my first and last FLIR device.Zinc Selenide lenses do work for addressing the parallax adjustment limits in my case (and way cheaper than the $4k FLIR). But I haven't been sufficiently motivated to design a holder for it.

author: Vipitis
author_rank: Frequent Contributor
timestamp: July 16, 2024, 11:21:12 pm
content: how about putting a mirror on some kind of actuator or wheel, sync it with the framerate of both cameras so it never reduces your thermal signal? and you then simply grab a short vis exposure of the diverted path for the outlines (maybe discard all other frames).

author: CatalinaWOW
author_rank: Super Contributor
timestamp: July 17, 2024, 12:34:54 am
content: Quote from: Vipitis on July 16, 2024, 11:21:12 pmClever idea. Shouldn't be expensive to implement, though quite a bit of brain juice will be expended.

author: Nogtail
author_rank: Regular Contributor
timestamp: July 17, 2024, 04:21:22 am
content: I tried this a few years ago with a glass pane taken out of a picture frame that was somewhat reflective to IR. I placed it at a 45 degree angle as a beam splitter. I ended up having a couple of issues: Firstly, the glass I used wasn't anywhere near flat which was causing distortions to the reflected thermal image. Another problem was the IR image needed lens distortion correction to align with the visible image. Probably not too difficult, but I didn't have the time to spend on it. Using a mirror with the reflective coating removed might be an easy way of getting a fairly flat sheet of glass. Perhaps a large microscope slide would also work.

author: artag
author_rank: Super Contributor
timestamp: July 17, 2024, 07:12:36 am
content: Quote from: CatalinaWOW on July 17, 2024, 12:34:54 amMaybe combine it with the calibration shutter.Didn't early cameras have a continuously rotating shutter rather than today's intermittent ones ?Seems like a step backwards but in many ways a spinning shutter is better than a flip in/out one

author: Phil1977
author_rank: Frequent Contributor
timestamp: July 17, 2024, 08:27:12 am
content: Maybe one more pragmatic idea: Put a small VIS-cam in front of the IR-cam. Take the image, freeze it and swivel the VIS-cam away for thermography.Or do the same with a removable VIS-mirror and fixed cams.

author: Bill W
author_rank: Super Contributor
timestamp: July 17, 2024, 11:40:32 am
content: Quote from: artag on July 17, 2024, 07:12:36 amYes they had continuous rotating shutters / choppers, but that was because they used sensors that only responded to changes.I never tried a bolometer and chopper as a 'shutterless' system - you'd get constant imagery if only 30 or 15 fps. The required front area would always be a problem as a commercial proposition.

author: artag
author_rank: Super Contributor
timestamp: July 17, 2024, 03:07:07 pm
content: Another possibility is to use the parallax as a feature, rather than a bug, in the manner of rangefinder cameras. It's quite hard to focus using a thermal image alone as heat bleed smudges the edges and the real features you see are likely due to local cooling or emissivity differences.This is somewhat like the cross-coupling between display offset and focus controls, but with a different priority so that minimising parallax (manually or by cross-correlation of edges) optimises focus.

author: Vipitis
author_rank: Frequent Contributor
timestamp: September 26, 2024, 03:04:17 pm
content: this video has a demonstration of using a mirror ball and projection to get a nearly parallax free overlapping image -- even with the cameras completely apart. Maybe works for non sphere mirrors two if you solve a lot of math.https://youtu.be/rJPKTCdk-WI?t=1587

author: CatalinaWOW
author_rank: Super Contributor
timestamp: September 26, 2024, 03:14:37 pm
content: Early in the thread it was mentioned that the cost of doing it right is why it isn't often done. Truly right is pan spectral optics and a common focal plane for all imaged spectral bands. Pan spectral isn't easy but can be achieved with reflective optics and windows either vanishingly thin or of near unobtanium materials. Sandwiches focal planes can do the detection since in general the semi conductor materials are transparent to wavelength longer than their detection range, but there are serious complexities in alignment and connection.