TEQ to CSV conversion - Page 1
Forum section: electronics / general

author: NHFranco
author_rank: Newbie
timestamp: February 12, 2018, 03:04:11 pm
content: Dear all,Noob here.I'm Nuno Franco, a biologist from Porto, Portugal, and I've started working on thermal imaging applied to animal welfare science.Since I couldn't get proper funding, I had to make do with a TE - Q1, and I have to say that until now I've been getting interesting results. It saves in proprietary .TEQ files, in PNG and JPEG, the latter coupled with CSV filesTwo colleagues from the Bioengineering department managed to write a MATLAB script that reads the CSV (now compiled to a small app) that takes each image or a batch of images, removes background temperature (i.e. whatever is not an animal in the picture) and gives mean and median surface temperature, with some nice histograms to go with it. We expect to write an open-access paper on it until the end of March.It's a painstaking job to use the proprietary software to contour every animal in every picture, and then make an average, and this is why I asked my colleagues to develop the MATLAB script.Problem is we have some data collected from before I knew I'd be using this app, but now we can't use it, because it's all in TEQ format.So I'd like to know if there's any way of taking the data in TEQ files to build SCV files? (Joe-C)? .Thanks

author: rhb
author_rank: Super Contributor
timestamp: February 12, 2018, 03:50:04 pm
content: If no one offers up a conversion program, post the information you have on the TEQ format and an example input file. No promises, but likely it's a pretty simple task. It would help if the TEQ example is something simple like a hot stripe against a uniform background.

author: NHFranco
author_rank: Newbie
timestamp: February 12, 2018, 04:20:28 pm
content: Thank you.TEQ files are relatively large (2 MB), so I've put one up on this link:https://app.box.com/s/pxjw8j7qisuk4vxp6e4k66t1alcqq8cyI attach a screen grab of how it looks like in the TE Q1 Analyst software. There's not much on the TEQ file extension, the users' manual states that:teq is customized format for saving Thermal Expertimage information in Thermal Expert Q1 program. It savescurrent image information which contains temperature data,colormap, visible image (if exist), number and position ofmarkers, and emissivity. When teq file is loaded, thoseinformation are loaded and applied automatically. Since thisformat is customized format, it cannot be used in ordinaryimage viewer

author: rhb
author_rank: Super Contributor
timestamp: February 12, 2018, 08:21:40 pm
content: A simple image is important. The example image is too complex to attack at the byte level. A warm or cold bar on a constant background is what's needed along with all of the data you can extract using their software (e.g. value ranges, compression type, colormap name, etc)Have you asked Thermal Expert for information about the format? OEMs will sometimes provide information to academics that they won't supply to regular customers.

author: mahony
author_rank: Regular Contributor
timestamp: February 12, 2018, 10:43:14 pm
content: Hi,I had a quick look at the image format.There is the raw data (16bit) after 498 header bytes. I tried to attach a .png image file but that does not seem to work...Not sure about the rest but might be some calibrated image in there too.RegardsToniEdit: .png was above 1MB -> jpg did the jobEdit2: using int32 you can see the same image again after the first image block... interesting

author: mahony
author_rank: Regular Contributor
timestamp: February 12, 2018, 11:17:06 pm
content: Even easier than thought... just use the second block as float32 - this directly gives you temperatures I assume.The attached image was read in a simple sequence as float32 values and scaled to display values from 22 (blue limit) to 38 (red limit). In the case of this image the offset is 221680 bytes, might be different in other images - just check it. See the final attached temperature image.What I find really interesting is the fact that you have got the same 'line-shift' that I have. That is your leftmost line seems to be belonging in the rightmost place. I have got the same behaviour on my cam.Edit: just checked with your posted image above. The value seem to be spot on. Very easy file reverse engineering

author: NHFranco
author_rank: Newbie
timestamp: February 14, 2018, 10:09:22 am
content: Now I'm sure I'm way out of my league, here (hence why I asked coleagues from the bioengineering department to help me with this).Is there a way I could convert the TEQ files to CSV? I can try to get a simpler image, as rhb suggested, if it helps.BTW, I'm pretty sure my colleagues wouldn't mind sharing the program we are testing for CSV analysis. It's still at the Beta stage, but we're talking about making it freely available, once the paper it's published.

author: rhb
author_rank: Super Contributor
timestamp: February 14, 2018, 02:16:08 pm
content: I was waiting to hear if someone had a translator before starting to write something. Mahony provided what should be enough information. My request for a simpler image was concern about things like the scan line being on the wrong side. Mahony has the advantage of experience with thermal imaging. I'm too cheap to buy one, though it would be nice to have.I'll try to work in time to write a small C program today. Normally I use awk for format conversions, but awk doesn't handle binary. I've just never gotten enthused about python or perl for the simple reason I was very good with parsing very complex formats with awk. This is an instance where python or perl would be a more portable choice.

author: rhb
author_rank: Super Contributor
timestamp: February 14, 2018, 04:15:13 pm
content: I'm having problems with the format. Could you post an image both in TEQ and CSV format?There appears to be a header, a pair of image blocks and then more data. I'm concerned about the data after the images.

author: rhb
author_rank: Super Contributor
timestamp: February 14, 2018, 05:36:58 pm
content: OK Here's what I know:496 byte header384x288 16 bit int image384x288 32 bit float image600x600 32 bit imageThe first two images appear to be thermal. The 3rd image is the regular photo image stored 4 bytes per pixel.

author: mahony
author_rank: Regular Contributor
timestamp: February 14, 2018, 06:48:21 pm
content: Nice catch! I assumed there might be some more Image data but was not aware of the visual photo.I am currently not at Home, otherwise I might able to write a quick C# app to extract the floating point temperature data and dump it to a csv. Maybe someone else can help out?On the other hand, it should be only a couple on lines in matlab to read the temperature data from teq.

author: rhb
author_rank: Super Contributor
timestamp: February 15, 2018, 12:34:32 am
content: Your header length got it close enough to take the time. I don't have a TIR camera and at the current prices, not likely to. I just happen to really despise proprietary formats. I've made a lot of money translating them, but I should very much prefer to have spent that time doing something more interesting.Once I have the CSV version so I can parse out the header I'll write a program to do the conversion to produce identical CSV files.

author: NHFranco
author_rank: Newbie
timestamp: February 15, 2018, 10:30:47 am
content: It can't save simultaneously in two different formats, but I can go downstairs to the lab and take a few simple consecutive images for each format.

author: NHFranco
author_rank: Newbie
timestamp: February 15, 2018, 10:34:46 am
content: Hi Mahony.We chose CSV because many thermal cameras can save in this format, or have software tools available for converting to it. But if can be done by adding a few lines on MATLAB, with your guidance I'm sure my colleagues could sort that out (as for me, I'm a Biology geek, not a tech geek).

author: rhb
author_rank: Super Contributor
timestamp: February 15, 2018, 01:02:49 pm
content: Quote from: NHFranco on February 15, 2018, 10:30:47 amPick a simple static image. A picture of a contact thermometer would be ideal if you have one handy, but not necessary. I want to examine the header for data not included in the CSV file.