CSQ file format - Page 1
Forum section: electronics / general

author: gtatters
author_rank: Contributor
timestamp: October 23, 2017, 01:58:21 am
content: I was wondering if anyone has experience with extracting radiometric information from FLIR's csq file format (CSQ=compressed sequence file)?I am looking to import the data into ImageJ for further analysis, and using FLIR tools or ResearchIR to convert the file format is slow when you have thousands of video files.Here is a sample csq in case this file type is not familiar.https://www.dropbox.com/s/ec6i18egpzkv7e6/IR_2017-05-19_0222.csq?dl=0CheersGlenn

author: tomas123
author_rank: Frequent Contributor
timestamp: October 24, 2017, 07:56:39 pm
content: looks like a regular flir sequence fileCode:$ exiftool IR_2017-05-19_0222.csqExifTool Version Number : 10.40File Name : IR_2017-05-19_0222.csqDirectory : .File Size : 27 MBFile Modification Date/Time : 2017:10:24 21:52:16+02:00File Access Date/Time : 2017:10:24 21:52:11+02:00File Inode Change Date/Time : 2017:10:24 21:52:16+02:00File Permissions : rw-r--r--File Type : FLIRFile Type Extension : fffMIME Type : application/unknownCreator Software : RTPEmissivity : 0.95Object Distance : 1.00 mReflected Apparent Temperature : 20.0 CAtmospheric Temperature : 20.0 CIR Window Temperature : 20.0 CIR Window Transmission : 1.00Relative Humidity : 50.0 %Planck R1 : 21546.203Planck B : 1507.2Planck F : 1Atmospheric Trans Alpha 1 : 0.006569Atmospheric Trans Alpha 2 : 0.012620Atmospheric Trans Beta 1 : -0.002276Atmospheric Trans Beta 2 : -0.006670Atmospheric Trans X : 1.900000Camera Temperature Range Max : 150.0 CCamera Temperature Range Min : -40.0 CCamera Temperature Max Clip : 160.0 CCamera Temperature Min Clip : -60.0 CCamera Temperature Max Warn : 150.0 CCamera Temperature Min Warn : -40.0 CCamera Temperature Max Saturated: 180.0 CCamera Temperature Min Saturated: -60.0 CCamera Model : FLIR T1030sc 12__Camera Part Number : 72501-0201Camera Serial Number : 72501090Camera Software : 33.0.0Lens Model : FOL83Lens Part Number : T198942Lens Serial Number : 74200341Field Of View : 11.9 degFilter Model :Filter Part Number :Filter Serial Number :Planck O : -6331Planck R2 : 0.016229488Raw Value Median : 14976Raw Value Range : 2384Date/Time Original : 2017:05:19 12:45:33.583-07:00Focus Step Count : 65535Focus Distance : 0.7 mRaw Thermal Image Width : 1024Raw Thermal Image Height : 768Raw Thermal Image Type : JPGRaw Thermal Image : (Binary data 209360 bytes, use -b option to extract)Meas 1 Type : SpotMeas 1 Params : 512 384Meas 1 Label : 1Palette Colors : 224Above Color : 170 128 128Below Color : 50 128 128Overflow Color : 67 216 98Underflow Color : 41 110 240Isotherm 1 Color : 100 128 128Isotherm 2 Color : 100 110 240Palette Method : 0Palette Stretch : 2Palette File Name : /FLIR/usr/etc/iron.palPalette Name : IronPalette : (Binary data 672 bytes, use -b option to extract)GPS Map Datum : WGS84Peak Spectral Sensitivity : 9.5 umlook at your magic byte sequence and split the video in single frameshttp://u88.n24.queensu.ca/exiftool/forum/index.php?topic=5279.msg25567#msg25567

author: gtatters
author_rank: Contributor
timestamp: October 24, 2017, 09:09:29 pm
content: It's the binary raw data I was hoping to share. It is not in a format I recognise and wondered if you could take a look if you have time.Thank you!

author: tomas123
author_rank: Frequent Contributor
timestamp: October 24, 2017, 09:58:15 pm
content: now I understand your questionCode:$ cat split.pl#!/usr/bin/perlundef $/;$_ = <>;$n = 100;# FLIR camera E40$pat="\x46\x46\x46\x00\x52\x54";# Flir Tools (comment out)#$pat = "\x46\x46\x46\x00\x43\x41\x50";for $content (split(/(?=$pat)/)) {open(OUT, ">seq" . ++$n . ".fff");binmode OUT;print OUT $content;close(OUT);}now split your videoCode:$ perl -f split.pl IR_2017-05-19_0222.csqwe got 167 framesCode:$ ls *.fffseq101.fffseq117.fffseq133.fffseq149.fffseq165.fffseq181.fffseq197.fffseq213.fffseq229.fffseq245.fffseq261.fffseq102.fffseq118.fffseq134.fffseq150.fffseq166.fffseq182.fffseq198.fffseq214.fffseq230.fffseq246.fffseq262.fffseq103.fffseq119.fffseq135.fffseq151.fffseq167.fffseq183.fffseq199.fffseq215.fffseq231.fffseq247.fffseq263.fffseq104.fffseq120.fffseq136.fffseq152.fffseq168.fffseq184.fffseq200.fffseq216.fffseq232.fffseq248.fffseq264.fffseq105.fffseq121.fffseq137.fffseq153.fffseq169.fffseq185.fffseq201.fffseq217.fffseq233.fffseq249.fffseq265.fffseq106.fffseq122.fffseq138.fffseq154.fffseq170.fffseq186.fffseq202.fffseq218.fffseq234.fffseq250.fffseq266.fffseq107.fffseq123.fffseq139.fffseq155.fffseq171.fffseq187.fffseq203.fffseq219.fffseq235.fffseq251.fffseq267.fffseq108.fffseq124.fffseq140.fffseq156.fffseq172.fffseq188.fffseq204.fffseq220.fffseq236.fffseq252.fffseq268.fffseq109.fffseq125.fffseq141.fffseq157.fffseq173.fffseq189.fffseq205.fffseq221.fffseq237.fffseq253.fffseq110.fffseq126.fffseq142.fffseq158.fffseq174.fffseq190.fffseq206.fffseq222.fffseq238.fffseq254.fffseq111.fffseq127.fffseq143.fffseq159.fffseq175.fffseq191.fffseq207.fffseq223.fffseq239.fffseq255.fffseq112.fffseq128.fffseq144.fffseq160.fffseq176.fffseq192.fffseq208.fffseq224.fffseq240.fffseq256.fffseq113.fffseq129.fffseq145.fffseq161.fffseq177.fffseq193.fffseq209.fffseq225.fffseq241.fffseq257.fffseq114.fffseq130.fffseq146.fffseq162.fffseq178.fffseq194.fffseq210.fffseq226.fffseq242.fffseq258.fffseq115.fffseq131.fffseq147.fffseq163.fffseq179.fffseq195.fffseq211.fffseq227.fffseq243.fffseq259.fffseq116.fffseq132.fffseq148.fffseq164.fffseq180.fffseq196.fffseq212.fffseq228.fffseq244.fffseq260.fffsee your frame rateCode:$ exiftool -DateTimeOriginal *.fff======== seq101.fffDate/Time Original : 2017:05:19 12:45:33.583-07:00======== seq102.fffDate/Time Original : 2017:05:19 12:45:33.617-07:00======== seq103.fffDate/Time Original : 2017:05:19 12:45:33.650-07:00======== seq104.fffDate/Time Original : 2017:05:19 12:45:33.683-07:00======== seq105.fffDate/Time Original : 2017:05:19 12:45:33.717-07:00...======== seq262.fffDate/Time Original : 2017:05:19 12:45:38.951-07:00======== seq263.fffDate/Time Original : 2017:05:19 12:45:38.984-07:00======== seq264.fffDate/Time Original : 2017:05:19 12:45:39.018-07:00======== seq265.fffDate/Time Original : 2017:05:19 12:45:39.051-07:00======== seq266.fffDate/Time Original : 2017:05:19 12:45:39.085-07:00======== seq267.fffDate/Time Original : 2017:05:19 12:45:39.118-07:00======== seq268.fffDate/Time Original : 2017:05:19 12:45:39.151-07:00167/(39,151-33,583)=30 fpscheck the last frameCode:$ exiftool seq267.fffExifTool Version Number : 10.40File Name : seq267.fffDirectory : .File Size : 157 kBFile Modification Date/Time : 2017:10:24 23:47:13+02:00File Access Date/Time : 2017:10:24 23:49:57+02:00File Inode Change Date/Time : 2017:10:24 23:47:13+02:00File Permissions : rw-r--r--File Type : FLIRFile Type Extension : fffMIME Type : application/unknownCreator Software : RTPEmissivity : 0.95Object Distance : 1.00 mReflected Apparent Temperature : 20.0 CAtmospheric Temperature : 20.0 CIR Window Temperature : 20.0 CIR Window Transmission : 1.00Relative Humidity : 50.0 %Planck R1 : 21546.203Planck B : 1507.2Planck F : 1Atmospheric Trans Alpha 1 : 0.006569Atmospheric Trans Alpha 2 : 0.012620Atmospheric Trans Beta 1 : -0.002276Atmospheric Trans Beta 2 : -0.006670Atmospheric Trans X : 1.900000Camera Temperature Range Max : 150.0 CCamera Temperature Range Min : -40.0 CCamera Temperature Max Clip : 160.0 CCamera Temperature Min Clip : -60.0 CCamera Temperature Max Warn : 150.0 CCamera Temperature Min Warn : -40.0 CCamera Temperature Max Saturated: 180.0 CCamera Temperature Min Saturated: -60.0 CCamera Model : FLIR T1030sc 12__Camera Part Number : 72501-0201Camera Serial Number : 72501090Camera Software : 33.0.0Lens Model : FOL83Lens Part Number : T198942Lens Serial Number : 74200341Field Of View : 11.9 degFilter Model :Filter Part Number :Filter Serial Number :Planck O : -6331Planck R2 : 0.016229488Raw Value Median : 13782Raw Value Range : 516Date/Time Original : 2017:05:19 12:45:39.118-07:00Focus Step Count : 65535Focus Distance : 0.7 mRaw Thermal Image Width : 1024Raw Thermal Image Height : 768Raw Thermal Image Type : JPGRaw Thermal Image : (Binary data 156552 bytes, use -b option to extract)Meas 1 Type : SpotMeas 1 Params : 512 384Meas 1 Label : 1Palette Colors : 224Above Color : 170 128 128Below Color : 50 128 128Overflow Color : 67 216 98Underflow Color : 41 110 240Isotherm 1 Color : 100 128 128Isotherm 2 Color : 100 110 240Palette Method : 0Palette Stretch : 2Palette File Name : /FLIR/usr/etc/iron.palPalette Name : IronPalette : (Binary data 672 bytes, use -b option to extract)GPS Map Datum : WGS84Peak Spectral Sensitivity : 9.5 umtricky is the embedded thermal image, because "ff d8" are the magic bytes of compressed JPGCode:$ exiftool -b -RawThermalImage seq267.fff > 267.rawCode:$ hexdump -C -n 64 267.raw00000000 ff d8 ff f7 00 0b 10 03 00 04 00 01 01 11 00 ff |................|00000010 f8 00 0d 01 ff ff 00 21 00 5c 01 37 00 40 ff da |.......!.\.7.@..|00000020 00 08 01 01 00 06 00 00 00 00 7a c2 c6 86 89 e5 |..........z.....|00000030 4a d2 51 94 22 6a e6 a2 aa ba ee 73 6a 62 08 74 |J.Q."j.....sjb.t|Imagemagick can't identify this jpgCode:identify: Unsupported marker type 0xf7I think it's a Flir proprietary 16 bit compressed formattry to open the *.fff files with Flir Tools

author: tomas123
author_rank: Frequent Contributor
timestamp: October 24, 2017, 10:20:06 pm
content: Code:ff d8 ff f7This is the lossless jpg format JPG-LShttp://www.digitalpreservation.gov/formats/fdd/fdd000151.shtmlCode:$ ffmpeg -codecs | grep jpeglsffmpeg version 3.4 Copyright (c) 2000-2017 the FFmpeg developers...DEVILS jpegls JPEG-LS$ ffmpeg -f image2 -vcodec jpegls -i 267.raw -f image2 -vcodec png 267.pngffmpeg version 3.4 Copyright (c) 2000-2017 the FFmpeg developersInput #0, image2, from '267.raw':Duration: 00:00:00.04, start: 0.000000, bitrate: N/AStream #0:0: Video: jpegls, gray16le(bt470bg/unknown/unknown), 1024x768, lossless, 25 tbr, 25 tbn, 25 tbcStream mapping:Stream #0:0 -> #0:0 (jpegls (native) -> png (native))Press  to stop, [?] for helpOutput #0, image2, to '267.png':Metadata:encoder : Lavf57.83.100Stream #0:0: Video: png, gray16be, 1024x768, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbcMetadata:encoder : Lavc57.107.100 pngframe= 1 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A speed=0.21xvideo:629kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown$ identify 267.png267.png PNG 1024x768 1024x768+0+0 16-bit Grayscale Gray 644587B 0.000u 0:00.009now stretch to max/minCode:$ convert -auto-level 267.png 267.jpg

author: gtatters
author_rank: Contributor
timestamp: October 24, 2017, 11:09:02 pm
content: Thank you Tomas! Yes, this is what I couldn't figure out. I was not aware of a lossless jpg format. This was really helpful!

author: tomas123
author_rank: Frequent Contributor
timestamp: October 25, 2017, 07:40:58 am
content: I thought, that splitting is your first problem.How did you split the stream into individual images?PS: I wrote 2 years ago in the exiftool forum about the Flir jpg format in JPG-LS used in Flir Ultramax Imageshttp://u88.n24.queensu.ca/exiftool/forum/index.php/topic,6763.msg33943.html#msg33943

author: gtatters
author_rank: Contributor
timestamp: October 25, 2017, 08:26:43 am
content: Well, since I don't write Perl, I didn't split the images, I was just using a Hex editor and could follow some of your original messages back and forth with Phil Harvey to decipher some things. I program in R and work in ImageJ which are not ideal for this purpose, but work for a non-programmer like myself.I hadn't got to the stage of splitting frames up, since I didn't know what the compressed format was anyway. I could see the FFF header, but honestly wasn't sure where to go from there.If you are ever interested in writing up an ImageJ plugin (mostly Java, I think), that could import FLIR SEQ, CSQ, and JPG, you would be doing the research community a huge favour! Flir's research grade software is expensive (I have licenses for ResearchIR max) but with every new version they remove useful features from older versions.

author: tomas123
author_rank: Frequent Contributor
timestamp: October 25, 2017, 10:41:45 am
content: Quote from: gtatters on October 25, 2017, 08:26:43 amyou don't need some perl skills and it works also great with the portable perl version for windowssee my post herehttp://u88.n24.queensu.ca/exiftool/forum/index.php/topic,5279.msg25567.html#msg25567Code:using:Linux: ./split.pl 320x1.seqportabe Windows: perl-5.18.0.1-32bit-portable\perl\bin\perl.exe -f split.pl SEQ1000.seq

author: gtatters
author_rank: Contributor
timestamp: October 25, 2017, 11:17:01 am
content: Oh, I can run the scripts on my mac. It's developing an easy to employ tool for non-computer savvy colleagues I am ultimately hoping to figure out. ImageJ is the open source software for image analysis, and it would be ideal to have a direct importer.

author: Bud
author_rank: Super Contributor
timestamp: October 25, 2017, 02:19:07 pm
content: The other useful plugin for ImageJ would be a SuperResolution one that takes a stack of images and combines in a super resolution image. I have a feeling it may be fairly simple but I do not have any Java skills at all.

author: gtatters
author_rank: Contributor
timestamp: October 26, 2017, 12:19:51 am
content: Sorry to trouble you tomas, but what would be the syntax with ffmpeg for converting all the frames contained within one .raw file. Your nice example above shows how to convert one image (frame 267), but when I try to expand this to a .raw file with all the fff data, I cannot seem to get ffmpeg to work.I am trying this:Code:ffmpeg -f image2 -vcodec jpegls -i tempfile.raw -f image2 -vcodec png burst%02d.pngbut get the following output with error:Code:ffmpeg version 3.3.2 Copyright (c) 2000-2017 the FFmpeg developersbuilt with Apple LLVM version 8.1.0 (clang-802.0.38)configuration: --prefix=/usr/local/Cellar/ffmpeg/3.3.2 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-libmp3lame --enable-libvpx --enable-libx264 --enable-libxvid --enable-opencl --disable-lzma --enable-vdalibavutil 55. 58.100 / 55. 58.100libavcodec 57. 89.100 / 57. 89.100libavformat 57. 71.100 / 57. 71.100libavdevice 57. 6.100 / 57. 6.100libavfilter 6. 82.100 / 6. 82.100libavresample 3. 5. 0 / 3. 5. 0libswscale 4. 6.100 / 4. 6.100libswresample 2. 7.100 / 2. 7.100libpostproc 54. 5.100 / 54. 5.100Input #0, image2, from 'tempfile.raw':Duration: 00:00:00.04, start: 0.000000, bitrate: N/AStream #0:0: Video: jpegls, gray16le(bt470bg/unknown/unknown), 1024x768, lossless, 25 tbr, 25 tbn, 25 tbcStream mapping:Stream #0:0 -> #0:0 (jpegls (native) -> png (native))Press  to stop, [?] for helpOutput #0, image2, to 'burst%02d.png':Metadata:encoder : Lavf57.71.100Stream #0:0: Video: png, gray16be, 1024x768, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbcMetadata:encoder : Lavc57.89.100 pngframe= 1 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A speed=0.232xvideo:878kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

author: -jeffB
author_rank: Regular Contributor
timestamp: October 26, 2017, 01:08:01 am
content: Quote from: gtatters on October 25, 2017, 08:26:43 amI don't get by this forum as much as I used to, and it's been ten years or so since I contributed to ImageJ, but this is interesting enough for me to drop a bookmark.I'm going to be tied up for the next week or so, but I'm intrigued at the notion of dusting off my ImageJ plug-in skills...

author: Bud
author_rank: Super Contributor
timestamp: October 26, 2017, 04:33:34 am
content: This is how I import .SEQ into ImageJFile->Import->Rawand use these settings (for 320x240 but I guess offsets should be same for other resolutions). Specify the required number of frames to import.Here is someone describing his superresolution composition method using photoshop (scroll down to "Processing")https://petapixel.com/2015/02/21/a-practical-guide-to-creating-superresolution-photos-with-photoshop/

author: gtatters
author_rank: Contributor
timestamp: October 26, 2017, 05:00:47 am
content: I don't know how to write plugins for imagej, but I did write two macros to import SEQ files (converged on the same solution to Bud).The first macro imports the SEQ into ImageJ (using defaults that work for me) and the second macro converts the raw into temperature, according to equations developed by tomas123 and Phil Harvey. The first macro calls Exiftool and extracts the calibration constants.But if someone can put this together as an ImageJ plugin (ie. faster performance than macro), I would be grateful. Now I just need to get this to work for FLIR JPG and FLIR CSQ and I'm set.BTW, this code is still a work in progress, I only share it in response to jeffB who might dust off his plug-in skills!First macro:Code:// offsetbyte is 1372 for SEQ files recorded to computer (works for at least two diff cameras)// offsetbyte is 1540528 for SEQ files recorded to a FLIR SC660// offsetbyte is 1542956 or 1540480 for SEQ files recorded to a FLIR SC640// gapbytes is 3020 for direct SEQ recorded files// gapbytes is 1424 for thermacam researcher pro captured seq filesvar offsetbyte = 1372;var gapbytes = 1424;var nframes = 10000;var imagewidth = 640;var imageheight = 480;var converttotemperature = 1;var usevirtual = 1;var minpix=1;var maxpix=65535;macro "Import FLIR SEQ" {//Create Dialog BoxDialog.create("Information for Thermal Video Import");Dialog.addNumber("Offset Bytes", offsetbyte, 0, 8, "bytes");Dialog.addNumber("Gaps Between Frames:", gapbytes, 0, 8, "bytes");Dialog.addNumber("Number of Frames: ", nframes, 0, 8, "frames");Dialog.addNumber("Image Width:", imagewidth, 0, 6, "pixels");Dialog.addNumber("Image Height:", imageheight, 0, 6, "pixels");Dialog.addCheckbox("Convert to Temperature on Import", converttotemperature);Dialog.addCheckbox("Use Virtual Stack", usevirtual);Dialog.show();//Define Variable for Importoffsetbyte = Dialog.getNumber();gapbytes = Dialog.getNumber();nframes = Dialog.getNumber();imagewidth = Dialog.getNumber();imageheight = Dialog.getNumber();converttotemperature = Dialog.getCheckbox();usevirtual = Dialog.getCheckbox();filepath=File.openDialog("Select a File");file=File.openAsString(filepath);print("Loading: ", filepath);print("\n");run("Raw...", "open=filepath image=[16-bit Unsigned] width=imagewidth height=imageheight offset=offsetbyte number=nframes gap=gapbytes little-endian use=usevirtual");OS=getInfo("os.name");if(OS=="Mac OS X"){flirvals=exec("/usr/local/bin/exiftool", "-Planck*", "-*Emissivity", "-*Distance", "-*Temperature", "-*Transmission", "-*Humidity", "-*Height", "-*Width", "-*Original", "-*Date", filepath);}if(substring(OS, 0, 7)=="Windows"){flirvals=exec("c:/Windows/exiftool.exe", "-Planck*", "-*Emissivity", "-*Distance", "-*Temperature", "-*Transmission", "-*Humidity", "-*Height", "-*Width", "-*Original", "-*Date", filepath);print(flirvals);}var PR1 = parseFloat(substring(flirvals, indexOf(flirvals, ":", indexOf(flirvals, "Planck R1"))+1, indexOf(flirvals, "\n", indexOf(flirvals, "Planck R1")) ));var PB = parseFloat(substring(flirvals, indexOf(flirvals, ":", indexOf(flirvals, "Planck B"))+1, indexOf(flirvals, "\n", indexOf(flirvals, "Planck B")) ));var PF = parseFloat(substring(flirvals, indexOf(flirvals, ":", indexOf(flirvals, "Planck F"))+1, indexOf(flirvals, "\n", indexOf(flirvals, "Planck F")) ));var PO = parseFloat(substring(flirvals, indexOf(flirvals, ":", indexOf(flirvals, "Planck O"))+1, indexOf(flirvals, "\n", indexOf(flirvals, "Planck O")) ));var PR2 = parseFloat(substring(flirvals, indexOf(flirvals, ":", indexOf(flirvals, "Planck R2"))+1, indexOf(flirvals, "\n", indexOf(flirvals, "Planck R2")) ));var E = parseFloat(substring(flirvals, indexOf(flirvals, ":", indexOf(flirvals, "Emissivity"))+1, indexOf(flirvals, "\n", indexOf(flirvals, "Emissivity")) ));var OD = parseFloat(substring(flirvals, indexOf(flirvals, ":", indexOf(flirvals, "Object Distance"))+1, indexOf(flirvals, "\n", indexOf(flirvals, "Object Distance")) ));var RTemp = parseFloat(substring(flirvals, indexOf(flirvals, ":", indexOf(flirvals, "Reflected Apparent Temperature"))+1, indexOf(flirvals, "C\n", indexOf(flirvals, "Reflected Apparent Temperature")) ));var ATemp = parseFloat(substring(flirvals, indexOf(flirvals, ":", indexOf(flirvals, "Atmospheric Temperature"))+1, indexOf(flirvals, "C\n", indexOf(flirvals, "Atmospheric Temperature")) ));var IRWTemp = parseFloat(substring(flirvals, indexOf(flirvals, ":", indexOf(flirvals, "IR Window Temperature"))+1, indexOf(flirvals, "C\n", indexOf(flirvals, "IR Window Temperature")) ));var IRT = parseFloat(substring(flirvals, indexOf(flirvals, ":", indexOf(flirvals, "IR Window Transmission"))+1, indexOf(flirvals, "\n", indexOf(flirvals, "IR Window Transmission")) ));var RH = parseFloat(substring(flirvals, indexOf(flirvals, ":", indexOf(flirvals, "Relative Humidity"))+1, indexOf(flirvals, "%\n", indexOf(flirvals, "Relative Humidity")) ));var imagewidth = parseFloat(substring(flirvals, indexOf(flirvals, ":", indexOf(flirvals, "Width"))+1, indexOf(flirvals, "\n", indexOf(flirvals, "Width")) ));var imageheight = parseFloat(substring(flirvals, indexOf(flirvals, ":", indexOf(flirvals, "Height"))+1, indexOf(flirvals, "\n", indexOf(flirvals, "Height")) ));print("Camera Calibration Constants:");setFont("SansSerif", 12);print("Planck R1: ", d2s(PR1,9));print("Planck B: ", d2s(PB,9));print("Planck F: ", d2s(PF,0));print("Planck O: ", d2s(PO,0));print("Planck R2: ", d2s(PR2,9));print("Thermal Image Width: ", imagewidth);print("Thermal Image Height: ", imageheight);print("\n");print("Default Object Parameters:");print("Emissivity: ", d2s(E,2));print("Object Distance: ", d2s(OD,2));print("Reflected Apparent Temperature: ", d2s(RTemp,2));print("Atmospheric Temperature: ", d2s(ATemp,2));print("IR Window Temperature: ", d2s(IRWTemp,2));print("IR Window Transmission: ", d2s(IRT,3));print("Relative Humidity: ", d2s(RH,2));print("\n");Stack.getStatistics(count, mean, min, max, std);var minpix=min;var maxpix=max;setMinAndMax(minpix, maxpix);if(converttotemperature) {run("Raw2Temp");}run("Calibration Bar...", "location=[Upper Right] fill=White label=Black number=5 decimal=1 font=12 zoom=1 bold overlay");}Second macro:Code:var PR1=21106.77;var PR2=0.012545258;var PB=1501;var PF=1;var PO=-7340;var E = 0.95;var OD = 1.0;var RTemp = 20.0;var ATemp = 20.0;var IRWTemp = 20.0;var IRT = 1.0;var RH = 50.0;macro "Raw2Temp" {Dialog.create("Object Parameters to Calculate Temperature");Dialog.addMessage("Camera Calibration Constants:");Dialog.addNumber("Planck R1:", PR1, 2, 12, "unitless"); //21106.77 //21546.203Dialog.addNumber("Planck R2:", PR2, 8, 12, "unitless"); //0.012545258 //0.016229488Dialog.addNumber("Planck B:", PB, 0, 5, "unitless"); //1501 //1507.2Dialog.addNumber("Planck F:", PF, 0, 2, "unitless");//1Dialog.addNumber("Planck O:", PO, 0, 5, "unitless"); //-7340 //-6331Dialog.addMessage("Object Parameters:");Dialog.addNumber("Object Emissivity:", E, 3, 6, "unitless");Dialog.addNumber("Object Distance:", OD, 1, 6, "m");Dialog.addNumber("Reflected Temperature (C):", RTemp, 2, 6, "C");Dialog.addNumber("Atmospheric Temperature (C):", ATemp, 2, 6, "C");Dialog.addNumber("Window Temperature (C):", IRWTemp, 2, 6, "C");Dialog.addNumber("Window Transmittance:", IRT, 3, 6, "unitless");Dialog.addNumber("Relative Humidity:", RH, 2, 6, "%");Dialog.show();var PR1 = Dialog.getNumber();var PR2 = Dialog.getNumber();var PB = Dialog.getNumber();var PF = Dialog.getNumber();var PO = Dialog.getNumber();var E = Dialog.getNumber();var OD = Dialog.getNumber();var RTemp = Dialog.getNumber();var ATemp = Dialog.getNumber();var IRWTemp = Dialog.getNumber();var IRT = Dialog.getNumber();var RH = Dialog.getNumber();ATA1 = 0.006569; //Atmospheric Trans Alpha 1ATA2 = 0.012620; //Atmospheric Trans Alpha 2ATB1 = -0.002276; //Atmospheric Trans Beta 1ATB2 = -0.006670; //Atmospheric Trans Beta 2ATX = 1.900000; //Atmospheric Trans Xemisswind = 1- IRT;reflwind = 0 // anti-reflective coating on windowh2o = (RH/100)*exp(1.5587+0.06939*(ATemp)-0.00027816*(ATemp)^2+0.00000068455*(ATemp)^3); // converts relative humidity into water vapour pressure (I think in units mmHg)tau1 = ATX*exp(-sqrt(OD/2)*(ATA1+ATB1*sqrt(h2o)))+(1-ATX)*exp(-sqrt(OD/2)*(ATA2+ATB2*sqrt(h2o))); // atmos transmittance from object to windowtau2 = ATX*exp(-sqrt(OD/2)*(ATA1+ATB1*sqrt(h2o)))+(1-ATX)*exp(-sqrt(OD/2)*(ATA2+ATB2*sqrt(h2o))); // atmos transmittance from window to camerarawrefl1= PR1/(PR2*(exp(PB/(RTemp+273.15))-PF))-PO // radiance reflecting off the object before the windowrawrefl1attn = (1-E)/E*rawrefl1 // attn = the attenuated radiance (in raw units)rawatm1 = PR1/(PR2*(exp(PB/(ATemp+273.15))-PF))-PO // radiance from the atmosphere (before the window)rawatm1attn = (1-tau1)/E/tau1*rawatm1 // attn = the attenuated radiance (in raw units)rawwind = PR1/(PR2*(exp(PB/(IRWTemp+273.15))-PF))-POrawwindattn = emisswind/E/tau1/IRT*rawwindrawrefl2 = PR1/(PR2*(exp(PB/(RTemp+273.15))-PF))-POrawrefl2attn = reflwind/E/tau1/IRT*rawrefl2rawatm2 = PR1/(PR2*(exp(PB/(ATemp+273.15))-PF))-POrawatm2attn = (1-tau2)/E/tau1/IRT/tau2*rawatm2rawsubtract = (rawatm1attn + rawatm2attn + rawwindattn + rawrefl1attn + rawrefl2attn)rawdivisor = E*tau1*IRT*tau2run("32-bit", "stack");run("Macro...", "code=v=" +PB+ "/log(" +PR1+ "/(" +PR2+ "*(v/" +rawdivisor+ "-" +rawsubtract+ "+" +PO+ "))+" +PF+ ")-273.15 stack");mintemp=PB/log(PR1/(PR2*(minpix/rawdivisor-rawsubtract+PO))+PF)-273.15;maxtemp=PB/log(PR1/(PR2*(maxpix/rawdivisor-rawsubtract+PO))+PF)-273.15;setMinAndMax(mintemp, maxtemp);}

author: Bud
author_rank: Super Contributor
timestamp: October 26, 2017, 05:59:15 am
content: Should the two macros be in two individual files?

author: gtatters
author_rank: Contributor
timestamp: October 26, 2017, 06:14:07 am
content: Yes, sorry about that. I just placed them into my Startup Macro so that they are available when you boot up imageJ.If you goto Plugins-->Macros-->Startup Macros, you can paste those at the bottom, hit save, and restart ImageJ. Then they will both show up in the menu item Plugins-->Macros.You may need to customise a few things, I just shared after seeing your post with the import-->raw option (which I used within the macro).

author: gtatters
author_rank: Contributor
timestamp: November 03, 2017, 05:39:12 am
content: Any chance you know how to extract (using Exiftool) all frame times from a .seq file without having to split first into .fff files?For example:Code:exiftool -DateTimeOriginal sample.seqwill only yield one value,Code:Date/Time Original : 2016:06:06 15:41:09.884+01:00but if I split into multiple .fff files, using tomas123's codeCode:perl split.pl sample.seqandCode:exiftool -DateTimeOriginal sample.seqproduces:Code:======== frame00001.fffDate/Time Original : 2016:06:06 15:41:09.884+01:00======== frame00002.fffDate/Time Original : 2016:06:06 15:41:10.912+01:00======== frame00003.fffDate/Time Original : 2016:06:06 15:41:11.944+01:00======== frame00004.fffDate/Time Original : 2016:06:06 15:41:12.968+01:00======== frame00005.fffDate/Time Original : 2016:06:06 15:41:13.996+01:00======== frame00006.fffDate/Time Original : 2016:06:06 15:41:14.924+01:00======== frame00007.fffDate/Time Original : 2016:06:06 15:41:15.952+01:00======== frame00008.fffDate/Time Original : 2016:06:06 15:41:16.980+01:008 image files readI have some large .seq and .csq files that take ages to extract .fff files.Any suggestions?

author: Bud
author_rank: Super Contributor
timestamp: November 03, 2017, 06:35:17 am
content: Custom programming I guess... not through scripting.

author: gtatters
author_rank: Contributor
timestamp: November 03, 2017, 08:07:57 am
content: Quote from: gtatters on October 26, 2017, 12:19:51 amNevermind! I figured it out, by exporting a series of extracted tiff files to an avi:Code:ffmpeg -f image2 -vcodec tiff -r 30 -i 'frame%05d.tiff' -pix_fmt gray16be -vcodec png file.avi

author: gtatters
author_rank: Contributor
timestamp: February 08, 2018, 02:03:10 pm
content: I had a follow up question/problem with the split function in perl. I have been running this successfully for months on a Mac/Unix system/environment. I am able to split out all frames to .fff files.Recently I tried this on a Windows 10 machine. Installed Exiftool (had to turn off windows defender!). Installed Activestate perl and ffmpeg all successfully.But when I run the split script on a .csq file, the .fff files created in windows are different size from that created in unix. See attachments created under the two different environments. Usually it creates the correct number of files on both operating systems, suggesting the search pattern is fine.The culprit appears to be how \x0A is interpreted in Unix vs. Windows. Windows inserts \x0D\x0A when writing the .fff files, creating a mess. Then when i pass the .fff files to exiftool, they are not seen as FLIR format files."binmode out" is set in the perl script and I've read that this is important for working in windows environments.Does anyone have any suggestions? I am trying to help out a friend who only uses windows.Here is the perl script:Code:#!/usr/bin/perlundef $/;$_ = <>;$n = 0;$pat="\x46\x46\x46\x00";my $directory = "temp";unless(-e $directory or mkdir $directory) {die "Unable to create $directory\n";}# FLIR camera E40# Flir Tools (comment out)#$pat = "\x46\x46\x46\x00";for $content (split(/(?=$pat)/)) {open(OUT, ">temp/frame" . sprintf("%05d",++$n) . ".fff");binmode OUT;print OUT $content;close(OUT);}

author: tomas123
author_rank: Frequent Contributor
timestamp: February 11, 2018, 04:55:01 pm
content: I see in my notes:in 2013 I used successful the code above under windows with the (old) Version 5.18 32 Bit ofportablestrawberry Perl for Windowshttp://strawberryperl.com/releases.html

author: gtatters
author_rank: Contributor
timestamp: February 12, 2018, 01:35:53 am
content: Thanks for the suggestion. I tried it out, and there is some improvement, but still some issues, and these are enough to cause exiftool to fail, so I am stuck again.Using strawberry-perl-portable 32 bit on windows 10 (the only OS I can try this on).In this example (see image), 3 instances of x0A are replaced with x0D. In the non-portable perl installed on windows, the number of these errors is a few hundred in one file.Any suggestions?

author: tomas123
author_rank: Frequent Contributor
timestamp: February 12, 2018, 01:46:16 pm
content: sorry for your trouble, but I haven't yet enough time, to code another solution for you.The perl-code is very simple, so you can it fast convert to another script language (java, php etc.)I fast entry for decoding binaries with php is my Flir panorama script :Code:php splitjpg.php -i IR_0556.jpg -r stitch-ps16.png -o pano1pslink:https://www.eevblog.com/forum/thermal-imaging/flir-e4-thermal-imaging-camera-teardown/msg348715/#msg348715from an old post:Quote from: tomas123 on December 04, 2013, 02:23:30 pmplease post your solution here

author: gtatters
author_rank: Contributor
timestamp: February 12, 2018, 02:05:38 pm
content: No worries, Thomas, I wasn't expecting you to create a new solution! I was perhaps hoping that someone might know why the perl code is so OS dependent, since it works fine on mac and linux.As for other coding languages, I'm a biologist, not a computer scientist, so progress will be slow. But I'll have a look at the php approach to see if I can work it out.Thank you for pointing me toward that. I'm keeping track of my working solutions at my github site also:https://github.com/gtatters/Thermimage

author: gtatters
author_rank: Contributor
timestamp: February 13, 2018, 01:53:56 am
content: So far, I found a temporary solution, using a python script fromhttps://github.com/mypalmike/csplitbinstead of perl.Install python (v2.7 is fine), then install pip (https://github.com/BurntSushi/nfldb/wiki/Python-&-pip-Windows-installation), thenCode:pip install csplitbthen break up the .csq using:Code:python.exe csplitb.py --prefix frame --suffix .fff --number 5 46464600525450 thermalvid.csq"46464600525450" is the string to search for each fff start.Anyhow, I managed to get it to work in windows! If I figure out the perl approach, I'll post a solution.

author: tomas123
author_rank: Frequent Contributor
timestamp: February 13, 2018, 11:17:03 am
content: Hi gtaters,I took a look inside the perl docs and here is the solution (windows 10):the perl script.plCode:#!/usr/bin/perl$n = 100;$pat="\x46\x46\x46\x00\x52\x54";binmode F;open F, '<:raw', 'IR_2017-05-19_0222.csq'or die "Could not open file !";$file = do { local $/; <F> };close F;# Flir Tools (comment out)#$pat = "\x46\x46\x46\x00\x43\x41\x50";for my $content (split(/(?=$pat)/, $file)) {open(OUT, ">seq" . ++$n . ".fff");binmode(OUT, ":raw");print OUT $content;close(OUT);}used with strawberry-perl-5.18.1.1-32bit-portableCode:>strawberry-perl-5.18.1.1-32bit-portable\perl\bin\perl.exe -f split.pl IR_2017-05-19_0222.csq>exiftool -b -RawThermalImage seq*.fff -w _%f.jpgls168 image files read168 output files created>ffmpeg -f image2 -vcodec jpegls -start_number 101 -i _seq%3d.jpgls output.mp4ffmpeg version N-50911-g9efcfbe Copyright (c) 2000-2013 the FFmpeg developersbuilt on Mar 13 2013 21:26:48 with gcc 4.7.2 (GCC)[libx264 @ 04223000] kb/s:130.01please unzip the attachment for viewing the video

author: gtatters
author_rank: Contributor
timestamp: February 14, 2018, 04:53:02 pm
content: Hi Tomas,I have made progress and had success! Thanks to your help and in combination with a friend who works in perl, I've managed to create my own perl split.pl file I can run with arguments. See code below. This can split based on some of the known FLIR headers, jpgels, or tiff.I verified that this works on CSQ or SEQ files running perl on my Mac and running strawberry perl (portable) on a Windows 10 system.Thank you for all your help. Code below might be clumsy, but it works for me.Code:#!/usr/bin/perl#use strict;#use warnings;use Getopt::Std;our($opt_i, $opt_o, $opt_b, $opt_p, $opt_x);my $n=0;my $folder = "temp";my $infilename = "filename.csq";my $outfilebase = "frame";my $outfilename = "";my $pattype = "fff";my $patfff = "\x46\x46\x46\x00"; # split a generic fff header (some seq files recorded on camera have simple headers like this)my $patfcf = "\x46\x46\x46\x00\x43\x41\x50"; # split the fcf file type header (older video type)my $patseq = "\x46\x46\x46\x00\x43\x41\x4D"; # split the seq file type header (based on camera controlled by computer)my $patcsq = "\x46\x46\x46\x00\x52\x54\x50"; # split the csq file type header (compressed video format)my $patjpegls = "\xff\xd8\xff\xf7"; # split out jpegls headers from .raw file generated from exiftoolmy $pattiff = "II\\*\0";my $outext = "fff";my $file = "";my (%opt)=();getopts("h:i:o:b:p:x:",\%opt);if ($opt{h}){print qq(Usage: perl split -i filename -o outputfoldername -b basename -p splitpattern -x outputfileextensionoptions:-i input filename-o output folder-b base output name-p split pattern (fff, fcf, seq, csq, jpegls, tiff)-x output extension (fff, jpegls, tiff)\n.);exit}if (!defined $opt{i} & !defined $opt{o} & !defined $opt{b} & !defined $opt{p} & !defined $opt{x}){die "Error: Please specify input file, output folder, the output filename base, pattern to split, and output file extension.\n"}if (!defined $opt{i}){die "Error: Please specify the input file. \n"}if (!defined $opt{o}){die "Error: Please specify the output folder.\n"}if (!defined $opt{b}){die "Error: Please specify the output filename base.\n"}if (!defined $opt{p}){die "Error: Please specify the split pattern.\n"}if (!defined $opt{x}){die "Error: Please specify the output file extension.\n"}my $pattype = "$opt_p";$folder = $opt{o};$infilename = $opt{i};$outfilebase = $opt{b};$outext = ".$opt{x}";if ($opt{p} eq "fff"){$pat = $patfff;} elsif ($opt{p} eq "fcf"){$pat= $patfcf;} elsif ($opt{p} eq "seq"){$pat = $patseq;} elsif ($opt{p} eq "jpegls"){$pat = $patjpegls;} elsif ($opt{p} eq "tiff"){$pat = $pattiff;}else {die "Error: Please specify the split pattern.\n"}# Create output folder to store split filesunless(-e $folder or mkdir $folder) {die "Unable to create $directory\n";}print "Processing $infilename\n";# Use binary mode for portability between operating systemsbinmode F;open F, '<:raw', $infilename;$file = do { local $/; <F> };close F;# Split infilename based on $patfor my $content (split(/(?=$pat)/, $file)) {$outfilename = ">$folder/$outfilebase" . sprintf("%05d",++$n) . $outext;open(OUT, $outfilename);binmode(OUT, ":raw");print OUT $content;close(OUT);}A worked example and further details are elaborated here:https://github.com/gtatters/Thermimage/blob/master/BashConvertFLIR.md

author: tomas123
author_rank: Frequent Contributor
timestamp: February 14, 2018, 09:06:28 pm
content: thanks for sharing your results

author: lukaszgryglicki
author_rank: Newbie
timestamp: March 19, 2019, 06:13:56 pm
content: Hi, thanks for the help.I've bought FLIR E75 thermal camera and FLIR tools cannot be installed on Mac.But I've written a Go program that takes *.csq file, splits into JPEGLS files then uses a shell script (which uses convert and FFmpeg) to generate a final image.I've registered only to put a link to my program, hopefully somebody find it useful:https://github.com/lukaszgryglicki/csqconv