Flir Ex: Realtime raw radiometric data streaming via UVC - Page 1
Forum section: electronics / general

author: tmbinc
author_rank: Frequent Contributor
timestamp: March 05, 2015, 11:20:56 pm
content: [Apologize if this is common knowledge and I just missed the memo. As everyone knows, the main thread is _huge_ and there was some talk about this, but I didn't find the final verdict of whether it's possible or not.]I was trying to catch a fox that ate our pet rabbits from our garden on camera (people say they return to the scene of the crime). I wasn't successful (yet!), but I learned to hate the FLIR UVC implementation since it's randomly stops working during the night.In an attempt to rectify the situation, I looked at the UVC implementation on the camera, and figured out that there's a dormant function to stream 9Hz raw radiometric data instead of just capturing the screen. With Windows and the regular UVC drivers, the best I could capture was the screen with 3.3Hz (for whatever reason; 15Hz should be possible) at (effectively) 8-bit over a pre-selected temp range.The catch is that the UVC driver on the Flir explicitely disables radiometric streaming for the Z3 (=Flir Ex). It enables it for other cameras, though - i think the Flir Ex0 suports it out-of-the-box. The UVC stack is implemented in usbfnvideo.dll, which unfortunately is part of nk.bin, so it can't be easily patched (at least, I can't with my limited WinCE knowledge and my unwillingness to brick the E4).The great thing is that FLIR sucks at implementing device restrictions; they have a dedicated grabber thread for the Flir Ex-series, and it _does_ indeed implement grabbing the raw data (via FVD1: instead of capturing the screen), and they _let_ you select capturing that (via setting bFormatIndex in the stream control), but they don't announce the ability for that in the UVC descriptors, so any "well-behaving" UVC client will not allow you to select it.I've hacked libuvc (which is based on libusb) and - tada, it just worked. I patched libuvc pretty crudely so I'm sure there's a better way to do it. I've attached the patch and the crude sample (which just dumps out raw data into a file).Let me know if you find this useful.

author: OrBy
author_rank: Regular Contributor
timestamp: March 06, 2015, 01:14:13 am
content: First time I have seen any info on this myself - nice work!Would love to see what the output can turn out like.

author: tmbinc
author_rank: Frequent Contributor
timestamp: March 10, 2015, 07:56:12 pm
content: A few hundred lines of code later (simply record all frames with a max temp of higher than some threshold, re-mapping to 8-bit, then encoding as mp4), I got a good picture of the beast:Mission accomplished.

author: Bud
author_rank: Super Contributor
timestamp: March 11, 2015, 01:55:43 am
content: Hi tmbinc, if you could post a compiled executable and a patched dll that would help us try.

author: tomas123
author_rank: Frequent Contributor
timestamp: August 12, 2015, 06:31:02 pm
content: edit 08.09.2015----------------------link list for this thread: ----------------------You need a E4 firmware 2.3.0!! FW 1.19.8 doesn't work (see post below)source codefor above video fuchsjagdhttps://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg736344/#msg736344calculate temperaturefrom 16 bit RAW valueshttps://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg748218/#msg748218https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg744673/#msg744673rebuild a radiometric flir jpegfrom a single RAW framehttps://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg732177/#msg732177useImageJfor reading a RAW video filehttps://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg732349/#msg732349stream RAW thermal 16-bit video and display a rescaled8-bit video in a OpenCV window:https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg749464/#msg749464source code:https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg748654/#msg748654stream a8-bit video to V4L2 loopback devices ( /dev/video0 ):https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg756530/#msg756530an older post for streaming the E4 screen video to a raspberry pi (UVC, no RAW video)https://www.eevblog.com/forum/testgear/capturing-video-from-flir-e4-to-portable-device/msg368083/#msg368083---------------------- end link list ----------------------only for information:I'm played around with the uvc patch, but it doesn't work with my E4 (RAW Video).E4Modell: E4 1.1Software: 1.19.8first compile patched UVC libraryCode:$ uname -aLinux ubu64 3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux$ apt-get install CMake$ apt --installed list| grep -i usblibusb-0.1-4/trusty,now 2:0.1.12-23.3ubuntu1 amd64 libusb-1.0-0/trusty,now 2:1.0.17-1ubuntu2 amd64 $ sudo apt-get update && sudo apt-get install build-essential$ sudo apt-get install libjpeg-dev$ sudo apt-get install libusb-1.0-0-dev$ git clone https://github.com/ktossell/libuvc// don't forget to apply the patch from tmbinc in /src/$ patch < libuvc.patchpatching file stream.c$ cd libuvc$ mkdir build$ cd build$ cmake ..-- The C compiler identification is GNU 4.8.4-- The CXX compiler identification is GNU 4.8.4-- Check for working C compiler: /usr/bin/cc-- Check for working C compiler: /usr/bin/cc -- works-- Detecting C compiler ABI info-- Detecting C compiler ABI info - done-- Check for working CXX compiler: /usr/bin/c++-- Check for working CXX compiler: /usr/bin/c++ -- works-- Detecting CXX compiler ABI info-- Detecting CXX compiler ABI info - done-- No build type selected, default to Release-- No target type selected, default to shared library-- Building libuvc with JPEG support.-- Configuring done-- Generating done-- Build files have been written to: ../flir/libuvc/build$ makeScanning dependencies of target uvc[ 11%] Building C object CMakeFiles/uvc.dir/src/ctrl.c.o[ 22%] Building C object CMakeFiles/uvc.dir/src/ctrl-gen.c.o[ 33%] Building C object CMakeFiles/uvc.dir/src/device.c.o[ 44%] Building C object CMakeFiles/uvc.dir/src/diag.c.o[ 55%] Building C object CMakeFiles/uvc.dir/src/frame.c.o[ 66%] Building C object CMakeFiles/uvc.dir/src/init.c.o[ 77%] Building C object CMakeFiles/uvc.dir/src/stream.c.o[ 88%] Building C object CMakeFiles/uvc.dir/src/misc.c.o[100%] Building C object CMakeFiles/uvc.dir/src/frame-mjpeg.c.oLinking C shared library libuvc.so[100%] Built target uvc$ sudo make install[100%] Built target uvcInstall the project...-- Install configuration: "Release"-- Installing: /usr/local/lib/libuvc.so-- Installing: /usr/local/include/libuvc/libuvc.h-- Installing: /usr/local/include/libuvc/libuvc_config.h-- Installing: /usr/local/lib/cmake/libuvc/libuvcConfig.cmake-- Installing: /usr/local/lib/cmake/libuvc/libuvcConfigVersion.cmake-- Installing: /usr/local/lib/cmake/libuvc/libuvcTargets.cmake-- Installing: /usr/local/lib/cmake/libuvc/libuvcTargets-release.cmakerenew library cache and compile flir.c tool from user tmbincCode:$ sudo ldconfig$ /usr/bin/cc -o flir ../src/flir.c -luvccheck Flir E4 USB announcementsCode:$ lsusb -vBus 001 Device 006: ID 09cb:1007Device Descriptor:bLength 18bDescriptorType 1bcdUSB 2.00bDeviceClass 239 Miscellaneous DevicebDeviceSubClass 2 ?bDeviceProtocol 1 Interface AssociationbMaxPacketSize0 64idVendor 0x09cbidProduct 0x1007bcdDevice 0.00iManufacturer 7 FLIR SystemsiProduct 8 FLIR Ex-SeriesiSerial 0bNumConfigurations 1Configuration Descriptor:bLength 9bDescriptorType 2wTotalLength 241bNumInterfaces 3bConfigurationValue 1iConfiguration 0bmAttributes 0x80(Bus Powered)MaxPower 500mAInterface Association:bLength 8bDescriptorType 11bFirstInterface 0bInterfaceCount 2bFunctionClass 14 VideobFunctionSubClass 3 Video Interface CollectionbFunctionProtocol 0iFunction 10 FLIR USB VideoInterface Descriptor:bLength 9bDescriptorType 4bInterfaceNumber 0bAlternateSetting 0bNumEndpoints 1bInterfaceClass 14 VideobInterfaceSubClass 1 Video ControlbInterfaceProtocol 0iInterface 2 FLIR Ex-SeriesVideoControl Interface Descriptor:bLength 13bDescriptorType 36bDescriptorSubtype 1 (HEADER)bcdUVC 1.00wTotalLength 85dwClockFrequency 0.001000MHzbInCollection 1baInterfaceNr( 0) 1VideoControl Interface Descriptor:bLength 18bDescriptorType 36bDescriptorSubtype 2 (INPUT_TERMINAL)bTerminalID 1wTerminalType 0x0201 Camera SensorbAssocTerminal 0iTerminal 0wObjectiveFocalLengthMin 0wObjectiveFocalLengthMax 0wOcularFocalLength 0bControlSize 3bmControls 0x00020260Focus (Absolute)Focus (Relative)Zoom (Absolute)Focus, AutoVideoControl Interface Descriptor:bLength 9bDescriptorType 36bDescriptorSubtype 3 (OUTPUT_TERMINAL)bTerminalID 3wTerminalType 0x0101 USB StreamingbAssocTerminal 0bSourceID 6iTerminal 0VideoControl Interface Descriptor:bLength 7bDescriptorType 36bDescriptorSubtype 4 (SELECTOR_UNIT)bUnitID 4bNrInPins 1baSource( 0) 1iSelector 0VideoControl Interface Descriptor:bLength 11bDescriptorType 36bDescriptorSubtype 5 (PROCESSING_UNIT)Warning: Descriptor too shortbUnitID 5bSourceID 4wMaxMultiplier 0bControlSize 2bmControls 0x00000217BrightnessContrastHueSharpnessGainiProcessing 0bmVideoStandards 0x1bNoneNTSC - 525/60SECAM - 625/50NTSC - 625/50VideoControl Interface Descriptor:bLength 27bDescriptorType 36bDescriptorSubtype 6 (EXTENSION_UNIT)bUnitID 6guidExtensionCode {d41f59fa-5094-463a-b3bb-e7858a831fa3}bNumControl 4bNrPins 1baSourceID( 0) 5bControlSize 2bmControls( 0) 0x0fbmControls( 1) 0x00iExtension 0Endpoint Descriptor:bLength 7bDescriptorType 5bEndpointAddress 0x81 EP 1 INbmAttributes 3Transfer Type InterruptSynch Type NoneUsage Type DatawMaxPacketSize 0x0008 1x 8 bytesbInterval 16Interface Descriptor:bLength 9bDescriptorType 4bInterfaceNumber 1bAlternateSetting 0bNumEndpoints 1bInterfaceClass 14 VideobInterfaceSubClass 2 Video StreamingbInterfaceProtocol 0iInterface 0VideoStreaming Interface Descriptor:bLength 14bDescriptorType 36bDescriptorSubtype 1 (INPUT_HEADER)bNumFormats 1wTotalLength 79bEndPointAddress 130bmInfo 0bTerminalLink 3bStillCaptureMethod 0bTriggerSupport 0bTriggerUsage 0bControlSize 1bmaControls( 0) 27VideoStreaming Interface Descriptor:bLength 27bDescriptorType 36bDescriptorSubtype 4 (FORMAT_UNCOMPRESSED)bFormatIndex 1bNumFrameDescriptors 1guidFormat {59555932-0000-1000-8000-00aa00389b71}bBitsPerPixel 16bDefaultFrameIndex 1bAspectRatioX 0bAspectRatioY 0bmInterlaceFlags 0x00Interlaced stream or variable: NoFields per frame: 2 fieldsField 1 first: NoField pattern: Field 1 onlybCopyProtect 0VideoStreaming Interface Descriptor:bLength 38bDescriptorType 36bDescriptorSubtype 5 (FRAME_UNCOMPRESSED)bFrameIndex 1bmCapabilities 0x03Still image supportedFixed frame-ratewWidth 320wHeight 240dwMinBitRate 912384dwMaxBitRate 912384dwMaxVideoFrameBufferSize 153600dwDefaultFrameInterval 2666664bFrameIntervalType 3dwFrameInterval( 0) 666666dwFrameInterval( 1) 1333332dwFrameInterval( 2) 2666664Endpoint Descriptor:bLength 7bDescriptorType 5bEndpointAddress 0x82 EP 2 INbmAttributes 2Transfer Type BulkSynch Type NoneUsage Type DatawMaxPacketSize 0x0200 1x 512 bytesbInterval 0Interface Descriptor:bLength 9bDescriptorType 4bInterfaceNumber 2bAlternateSetting 0bNumEndpoints 2bInterfaceClass 8 Mass StoragebInterfaceSubClass 6 SCSIbInterfaceProtocol 80 Bulk-OnlyiInterface 0Endpoint Descriptor:bLength 7bDescriptorType 5bEndpointAddress 0x83 EP 3 INbmAttributes 2Transfer Type BulkSynch Type NoneUsage Type DatawMaxPacketSize 0x0200 1x 512 bytesbInterval 0Endpoint Descriptor:bLength 7bDescriptorType 5bEndpointAddress 0x04 EP 4 OUTbmAttributes 2Transfer Type BulkSynch Type NoneUsage Type DatawMaxPacketSize 0x0200 1x 512 bytesbInterval 0Device Qualifier (for other device speed):bLength 10bDescriptorType 6bcdUSB 2.00bDeviceClass 239 Miscellaneous DevicebDeviceSubClass 2 ?bDeviceProtocol 1 Interface AssociationbMaxPacketSize0 64bNumConfigurations 1Device Status: 0x0000(Bus Powered)now grep a streamCode:$ sudo libuvc/flir e4-1.rawUVC initializedDevice foundDevice openedDEVICE CONFIGURATION (09cb:1007/) ---Status: idleVideoControl:bcdUVC: 0x0100VideoStreaming(1):bEndpointAddress: 130Formats:UncompressedFormat(1)bits per pixel: 16GUID: 5955593200001000800000aa00389b71 (YUY2)default frame: 1aspect ratio: 0x0interlace flags: 00copy protect: 00FrameDescriptor(1)capabilities: 03size: 320x240bit rate: 912384-912384max frame size: 153600default interval: 1/3interval: 1/15interval: 1/7interval: 1/3END DEVICE CONFIGURATIONbmHint: 0000bFormatIndex: 1bFrameIndex: 1dwFrameInterval: 666666wKeyFrameRate: 0wPFrameRate: 0wCompQuality: 0wCompWindowSize: 0wDelay: 1dwMaxVideoFrameSize: 153600dwMaxPayloadTransferSize: 157696bInterfaceNumber: 1Streaming...got frame 1439397994:439760, 153600 bytesgot frame 1439397994:533467, 153600 bytesgot frame 1439397994:629556, 153600 bytesgot frame 1439397994:726339, 153600 bytesgot frame 1439397994:820088, 153600 bytesgot frame 1439397994:916235, 153600 bytesgot frame 1439397995:22432, 153600 bytesgot frame 1439397995:115699, 153600 bytesgot frame 1439397995:209889, 153600 bytesgot frame 1439397995:304196, 153600 bytessplit the stream and convert a single frameCode:$ split -b 153600 e4-1.raw$ convert -size 320x240 yuv:xaa -separate 4.pngthe result is the known screen image and not a RAW image:(

author: tomas123
author_rank: Frequent Contributor
timestamp: August 12, 2015, 08:07:38 pm
content: the same process with an E40 works fineE40Modell E40 0.10Software: 2.23.14USB annoucementCode:$ lsusb -vBus 001 Device 008: ID 09cb:1005Device Descriptor:bLength 18bDescriptorType 1bcdUSB 2.00bDeviceClass 239 Miscellaneous DevicebDeviceSubClass 2 ?bDeviceProtocol 1 Interface AssociationbMaxPacketSize0 64idVendor 0x09cbidProduct 0x1005bcdDevice 0.00iManufacturer 6 FLIR SystemsiProduct 7 FLIR Exx SeriesiSerial 0bNumConfigurations 1Configuration Descriptor:bLength 9bDescriptorType 2wTotalLength 345bNumInterfaces 4bConfigurationValue 1iConfiguration 0bmAttributes 0xc0Self PoweredMaxPower 0mAInterface Association:bLength 8bDescriptorType 11bFirstInterface 0bInterfaceCount 2bFunctionClass 2 CommunicationsbFunctionSubClass 2 Abstract (modem)bFunctionProtocol 255 Vendor Specific (MSFT RNDIS?)iFunction 9 FLIR Network CameraInterface Descriptor:bLength 9bDescriptorType 4bInterfaceNumber 0bAlternateSetting 0bNumEndpoints 1bInterfaceClass 2 CommunicationsbInterfaceSubClass 2 Abstract (modem)bInterfaceProtocol 255 Vendor Specific (MSFT RNDIS?)iInterface 0CDC Header:bcdCDC 1.10CDC Call Management:bmCapabilities 0x00bDataInterface 1CDC ACM:bmCapabilities 0x00Endpoint Descriptor:bLength 7bDescriptorType 5bEndpointAddress 0x81 EP 1 INbmAttributes 3Transfer Type InterruptSynch Type NoneUsage Type DatawMaxPacketSize 0x0008 1x 8 bytesbInterval 4Interface Descriptor:bLength 9bDescriptorType 4bInterfaceNumber 1bAlternateSetting 0bNumEndpoints 2bInterfaceClass 10 CDC DatabInterfaceSubClass 0 UnusedbInterfaceProtocol 0iInterface 0Endpoint Descriptor:bLength 7bDescriptorType 5bEndpointAddress 0x82 EP 2 INbmAttributes 2Transfer Type BulkSynch Type NoneUsage Type DatawMaxPacketSize 0x0200 1x 512 bytesbInterval 0Endpoint Descriptor:bLength 7bDescriptorType 5bEndpointAddress 0x03 EP 3 OUTbmAttributes 2Transfer Type BulkSynch Type NoneUsage Type DatawMaxPacketSize 0x0200 1x 512 bytesbInterval 0Interface Association:bLength 8bDescriptorType 11bFirstInterface 2bInterfaceCount 2bFunctionClass 14 VideobFunctionSubClass 3 Video Interface CollectionbFunctionProtocol 0iFunction 10 FLIR USB VideoInterface Descriptor:bLength 9bDescriptorType 4bInterfaceNumber 2bAlternateSetting 0bNumEndpoints 1bInterfaceClass 14 VideobInterfaceSubClass 1 Video ControlbInterfaceProtocol 0iInterface 4 FLIR Exx SeriesVideoControl Interface Descriptor:bLength 13bDescriptorType 36bDescriptorSubtype 1 (HEADER)bcdUVC 1.00wTotalLength 85dwClockFrequency 0.001000MHzbInCollection 1baInterfaceNr( 0) 3VideoControl Interface Descriptor:bLength 18bDescriptorType 36bDescriptorSubtype 2 (INPUT_TERMINAL)bTerminalID 1wTerminalType 0x0201 Camera SensorbAssocTerminal 0iTerminal 0wObjectiveFocalLengthMin 0wObjectiveFocalLengthMax 0wOcularFocalLength 0bControlSize 3bmControls 0x00020260Focus (Absolute)Focus (Relative)Zoom (Absolute)Focus, AutoVideoControl Interface Descriptor:bLength 9bDescriptorType 36bDescriptorSubtype 3 (OUTPUT_TERMINAL)bTerminalID 3wTerminalType 0x0101 USB StreamingbAssocTerminal 0bSourceID 6iTerminal 0VideoControl Interface Descriptor:bLength 7bDescriptorType 36bDescriptorSubtype 4 (SELECTOR_UNIT)bUnitID 4bNrInPins 1baSource( 0) 1iSelector 0VideoControl Interface Descriptor:bLength 11bDescriptorType 36bDescriptorSubtype 5 (PROCESSING_UNIT)Warning: Descriptor too shortbUnitID 5bSourceID 4wMaxMultiplier 0bControlSize 2bmControls 0x00000217BrightnessContrastHueSharpnessGainiProcessing 0bmVideoStandards 0x1bNoneNTSC - 525/60SECAM - 625/50NTSC - 625/50VideoControl Interface Descriptor:bLength 27bDescriptorType 36bDescriptorSubtype 6 (EXTENSION_UNIT)bUnitID 6guidExtensionCode {d41f59fa-5094-463a-b3bb-e7858a831fa3}bNumControl 4bNrPins 1baSourceID( 0) 5bControlSize 2bmControls( 0) 0x0fbmControls( 1) 0x00iExtension 0Endpoint Descriptor:bLength 7bDescriptorType 5bEndpointAddress 0x84 EP 4 INbmAttributes 3Transfer Type InterruptSynch Type NoneUsage Type DatawMaxPacketSize 0x0008 1x 8 bytesbInterval 16Interface Descriptor:bLength 9bDescriptorType 4bInterfaceNumber 3bAlternateSetting 0bNumEndpoints 1bInterfaceClass 14 VideobInterfaceSubClass 2 Video StreamingbInterfaceProtocol 0iInterface 0VideoStreaming Interface Descriptor:bLength 15bDescriptorType 36bDescriptorSubtype 1 (INPUT_HEADER)bNumFormats 2wTotalLength 145bEndPointAddress 133bmInfo 0bTerminalLink 3bStillCaptureMethod 0bTriggerSupport 0bTriggerUsage 0bControlSize 1bmaControls( 0) 27bmaControls( 1) 27VideoStreaming Interface Descriptor:bLength 27bDescriptorType 36bDescriptorSubtype 4 (FORMAT_UNCOMPRESSED)bFormatIndex 1bNumFrameDescriptors 1guidFormat {59555932-0000-1000-8000-00aa00389b71}bBitsPerPixel 16bDefaultFrameIndex 1bAspectRatioX 0bAspectRatioY 0bmInterlaceFlags 0x00Interlaced stream or variable: NoFields per frame: 2 fieldsField 1 first: NoField pattern: Field 1 onlybCopyProtect 0VideoStreaming Interface Descriptor:bLength 38bDescriptorType 36bDescriptorSubtype 5 (FRAME_UNCOMPRESSED)bFrameIndex 1bmCapabilities 0x03Still image supportedFixed frame-ratewWidth 320wHeight 240dwMinBitRate 912384dwMaxBitRate 912384dwMaxVideoFrameBufferSize 153600dwDefaultFrameInterval 1333332bFrameIntervalType 3dwFrameInterval( 0) 333333dwFrameInterval( 1) 666666dwFrameInterval( 2) 1333332VideoStreaming Interface Descriptor:bLength 27bDescriptorType 36bDescriptorSubtype 4 (FORMAT_UNCOMPRESSED)bFormatIndex 2bNumFrameDescriptors 1guidFormat {46374d30-0000-1000-8000-00aa00389b71}bBitsPerPixel 16bDefaultFrameIndex 1bAspectRatioX 0bAspectRatioY 0bmInterlaceFlags 0x00Interlaced stream or variable: NoFields per frame: 2 fieldsField 1 first: NoField pattern: Field 1 onlybCopyProtect 0VideoStreaming Interface Descriptor:bLength 38bDescriptorType 36bDescriptorSubtype 5 (FRAME_UNCOMPRESSED)bFrameIndex 1bmCapabilities 0x03Still image supportedFixed frame-ratewWidth 320wHeight 246dwMinBitRate 912384dwMaxBitRate 912384dwMaxVideoFrameBufferSize 157440dwDefaultFrameInterval 1333332bFrameIntervalType 3dwFrameInterval( 0) 333333dwFrameInterval( 1) 666666dwFrameInterval( 2) 1333332Endpoint Descriptor:bLength 7bDescriptorType 5bEndpointAddress 0x85 EP 5 INbmAttributes 2Transfer Type BulkSynch Type NoneUsage Type DatawMaxPacketSize 0x0200 1x 512 bytesbInterval 0Device Qualifier (for other device speed):bLength 10bDescriptorType 6bcdUSB 2.00bDeviceClass 239 Miscellaneous DevicebDeviceSubClass 2 ?bDeviceProtocol 1 Interface AssociationbMaxPacketSize0 64bNumConfigurations 1Device Status: 0x0001Self Poweredgrep a streamCode:$ sudo libuvc/flir e40.rawUVC initializedDevice foundDevice openedDEVICE CONFIGURATION (09cb:1004/0a7f4000-c632-0000-0000-000000000000) ---Status: idleVideoControl:bcdUVC: 0x0100VideoStreaming(1):bEndpointAddress: 130Formats:UncompressedFormat(1)bits per pixel: 16GUID: 5955593200001000800000aa00389b71 (YUY2)default frame: 1aspect ratio: 0x0interlace flags: 00copy protect: 00FrameDescriptor(1)capabilities: 03size: 320x240bit rate: 912384-912384max frame size: 153600default interval: 1/7interval: 1/30interval: 1/15interval: 1/7UncompressedFormat(2)bits per pixel: 16GUID: 46374d3000001000800000aa00389b71 (F7M0)default frame: 1aspect ratio: 0x0interlace flags: 00copy protect: 00FrameDescriptor(1)capabilities: 03size: 320x246bit rate: 912384-912384max frame size: 157440default interval: 1/7interval: 1/30interval: 1/15interval: 1/7END DEVICE CONFIGURATIONbmHint: 0000bFormatIndex: 1bFrameIndex: 1dwFrameInterval: 666666wKeyFrameRate: 0wPFrameRate: 0wCompQuality: 0wCompWindowSize: 0wDelay: 1dwMaxVideoFrameSize: 157440dwMaxPayloadTransferSize: 157696bInterfaceNumber: 1Streaming...got frame 1439394751:346415, 157440 bytesgot frame 1439394751:411283, 157440 bytesgot frame 1439394751:479331, 157440 bytesgot frame 1439394751:546262, 157440 bytesgot frame 1439394751:610415, 157440 bytesgot frame 1439394751:679598, 157440 bytesgot frame 1439394751:741424, 157440 bytesgot frame 1439394751:810494, 157440 bytesgot frame 1439394751:874681, 157440 bytesgot frame 1439394751:938867, 157440 bytes....the streams contains RAW FFF framesCode:$ hexdump -C e40.raw00000000 f4 48 f6 48 f7 48 03 49 04 49 02 49 01 49 00 49 |.H.H.H.I.I.I.I.I|00000010 14 49 fc 48 ff 48 04 49 fe 48 0a 49 fd 48 0d 49 |.I.H.H.I.H.I.H.I|...000257f0 09 49 19 49 18 49 0e 49 0e 49 0c 49 13 49 32 49 |.I.I.I.I.I.I.I2I|00025800 46 46 46 00 55 56 43 00 00 00 00 00 00 00 00 00 |FFF.UVC.........|00025810 00 00 00 00 64 00 00 00 40 00 00 00 02 00 00 00 |....d...@.......|split the stream and convert a frameCode:$ split -b 157440 e40.raw$ convert -depth 16 -size 320x240 -auto-level gray:xaa 8.pngit's aRAWimagecheck frame rate with exiftool (15 fps)Code:$ dd if=e40.raw of=e40allframe.ff bs=153600 skip=1$ exiftool -Date* x*======== xaaDate/Time Original : 2015:08:12 17:39:48.413+01:00======== xabDate/Time Original : 2015:08:12 17:39:48.479+01:00======== xacDate/Time Original : 2015:08:12 17:39:48.545+01:00======== xadDate/Time Original : 2015:08:12 17:39:48.611+01:00======== xaeDate/Time Original : 2015:08:12 17:39:48.677+01:00======== xafDate/Time Original : 2015:08:12 17:39:48.743+01:00======== xagDate/Time Original : 2015:08:12 17:39:48.809+01:00======== xahDate/Time Original : 2015:08:12 17:39:48.875+01:00======== xaiDate/Time Original : 2015:08:12 17:39:48.941+01:00======== xajDate/Time Original : 2015:08:12 17:39:49.007+01:00======== xakDate/Time Original : 2015:08:12 17:39:49.073+01:00======== xalDate/Time Original : 2015:08:12 17:39:49.139+01:00======== xamDate/Time Original : 2015:08:12 17:39:49.205+01:00======== xanDate/Time Original : 2015:08:12 17:39:49.271+01:00======== xaoDate/Time Original : 2015:08:12 17:39:49.337+01:00======== xapDate/Time Original : 2015:08:12 17:39:49.403+01:00or all values:Code:$ exiftool xaaExifTool Version Number : 9.54File Name : xaaDirectory : .File Size : 154 kBFile Modification Date/Time : 2015:08:12 18:16:45+02:00File Access Date/Time : 2015:08:12 18:27:35+02:00File Inode Change Date/Time : 2015:08:12 18:16:45+02:00File Permissions : rw-r--r--File Type : FLIRMIME Type : application/unknownEmissivity : 0.95Object Distance : 1.00 mReflected Apparent Temperature : 20.0 CAtmospheric Temperature : -4.0 CIR Window Temperature : 20.0 CIR Window Transmission : 1.00Relative Humidity : 50.0 %Planck R1 : 14345.714Planck B : 1347.7Planck F : 1Atmospheric Trans Alpha 1 : 0.006569Atmospheric Trans Alpha 2 : 0.012620Atmospheric Trans Beta 1 : -0.002276Atmospheric Trans Beta 2 : -0.006670Atmospheric Trans X : 1.900000Camera Temperature Range Max : 120.0 CCamera Temperature Range Min : -20.0 CCamera Model : FLIR E40Camera Part Number : 49001-2001Camera Serial Number : 490337633Camera Software : 20.0.0Lens Model : FOL18Lens Part Number :Lens Serial Number :Field Of View : 25.0 degFilter Model :Filter Part Number :Filter Serial Number :Planck O : -5711Planck R2 : 0.01234Raw Value Median : 18457Raw Value Range : 780Date/Time Original : 2015:08:12 17:39:48.413+01:00Focus Step Count : 1526Focus Distance : 5.3 m

author: tmbinc
author_rank: Frequent Contributor
timestamp: August 12, 2015, 09:21:31 pm
content: Meh. Okay.I'm on 2.3, so maybe it was added later. Could someone with a Flir Ex with 2.3 firmware give it a try? I wouldn't know what else it could be.

author: tomas123
author_rank: Frequent Contributor
timestamp: August 12, 2015, 10:54:06 pm
content: Do you see differences in the usb announcements "VideoControl Interface Descriptor" between my firmware 1.19.8 and your firmware 2.3?Code:$ lsusb -vBus 001 Device 006: ID 09cb:1007Device Descriptor:bLength 18bDescriptorType 1bcdUSB 2.00bDeviceClass 239 Miscellaneous DevicebDeviceSubClass 2 ?bDeviceProtocol 1 Interface AssociationbMaxPacketSize0 64idVendor 0x09cbidProduct 0x1007bcdDevice 0.00iManufacturer 7 FLIR SystemsiProduct 8 FLIR Ex-SeriesiSerial 0bNumConfigurations 1Configuration Descriptor:bLength 9bDescriptorType 2wTotalLength 241bNumInterfaces 3bConfigurationValue 1iConfiguration 0bmAttributes 0x80(Bus Powered)MaxPower 500mAInterface Association:bLength 8bDescriptorType 11bFirstInterface 0bInterfaceCount 2bFunctionClass 14 VideobFunctionSubClass 3 Video Interface CollectionbFunctionProtocol 0iFunction 10 FLIR USB VideoInterface Descriptor:bLength 9bDescriptorType 4bInterfaceNumber 0bAlternateSetting 0bNumEndpoints 1bInterfaceClass 14 VideobInterfaceSubClass 1 Video ControlbInterfaceProtocol 0iInterface 2 FLIR Ex-SeriesVideoControl Interface Descriptor:bLength 13bDescriptorType 36bDescriptorSubtype 1 (HEADER)bcdUVC 1.00wTotalLength 85dwClockFrequency 0.001000MHzbInCollection 1baInterfaceNr( 0) 1VideoControl Interface Descriptor:bLength 18bDescriptorType 36bDescriptorSubtype 2 (INPUT_TERMINAL)bTerminalID 1wTerminalType 0x0201 Camera SensorbAssocTerminal 0iTerminal 0wObjectiveFocalLengthMin 0wObjectiveFocalLengthMax 0wOcularFocalLength 0bControlSize 3bmControls 0x00020260Focus (Absolute)Focus (Relative)Zoom (Absolute)Focus, AutoVideoControl Interface Descriptor:bLength 9bDescriptorType 36bDescriptorSubtype 3 (OUTPUT_TERMINAL)bTerminalID 3wTerminalType 0x0101 USB StreamingbAssocTerminal 0bSourceID 6iTerminal 0VideoControl Interface Descriptor:bLength 7bDescriptorType 36bDescriptorSubtype 4 (SELECTOR_UNIT)bUnitID 4bNrInPins 1baSource( 0) 1iSelector 0VideoControl Interface Descriptor:bLength 11bDescriptorType 36bDescriptorSubtype 5 (PROCESSING_UNIT)Warning: Descriptor too shortbUnitID 5bSourceID 4wMaxMultiplier 0bControlSize 2bmControls 0x00000217BrightnessContrastHueSharpnessGainiProcessing 0bmVideoStandards 0x1bNoneNTSC - 525/60SECAM - 625/50NTSC - 625/50VideoControl Interface Descriptor:bLength 27bDescriptorType 36bDescriptorSubtype 6 (EXTENSION_UNIT)bUnitID 6guidExtensionCode {d41f59fa-5094-463a-b3bb-e7858a831fa3}bNumControl 4bNrPins 1baSourceID( 0) 5bControlSize 2bmControls( 0) 0x0fbmControls( 1) 0x00iExtension 0Endpoint Descriptor:bLength 7bDescriptorType 5bEndpointAddress 0x81 EP 1 INbmAttributes 3Transfer Type InterruptSynch Type NoneUsage Type DatawMaxPacketSize 0x0008 1x 8 bytesbInterval 16Interface Descriptor:bLength 9bDescriptorType 4bInterfaceNumber 1bAlternateSetting 0bNumEndpoints 1bInterfaceClass 14 VideobInterfaceSubClass 2 Video StreamingbInterfaceProtocol 0iInterface 0VideoStreaming Interface Descriptor:bLength 14bDescriptorType 36bDescriptorSubtype 1 (INPUT_HEADER)bNumFormats 1wTotalLength 79bEndPointAddress 130bmInfo 0bTerminalLink 3bStillCaptureMethod 0bTriggerSupport 0bTriggerUsage 0bControlSize 1bmaControls( 0) 27VideoStreaming Interface Descriptor:bLength 27bDescriptorType 36bDescriptorSubtype 4 (FORMAT_UNCOMPRESSED)bFormatIndex 1bNumFrameDescriptors 1guidFormat {59555932-0000-1000-8000-00aa00389b71}bBitsPerPixel 16bDefaultFrameIndex 1bAspectRatioX 0bAspectRatioY 0bmInterlaceFlags 0x00Interlaced stream or variable: NoFields per frame: 2 fieldsField 1 first: NoField pattern: Field 1 onlybCopyProtect 0VideoStreaming Interface Descriptor:bLength 38bDescriptorType 36bDescriptorSubtype 5 (FRAME_UNCOMPRESSED)bFrameIndex 1bmCapabilities 0x03Still image supportedFixed frame-ratewWidth 320wHeight 240dwMinBitRate 912384dwMaxBitRate 912384dwMaxVideoFrameBufferSize 153600dwDefaultFrameInterval 2666664bFrameIntervalType 3dwFrameInterval( 0) 666666dwFrameInterval( 1) 1333332dwFrameInterval( 2) 2666664Endpoint Descriptor:bLength 7bDescriptorType 5bEndpointAddress 0x82 EP 2 INbmAttributes 2Transfer Type BulkSynch Type NoneUsage Type DatawMaxPacketSize 0x0200 1x 512 bytesbInterval 0Interface Descriptor:bLength 9bDescriptorType 4bInterfaceNumber 2bAlternateSetting 0bNumEndpoints 2bInterfaceClass 8 Mass StoragebInterfaceSubClass 6 SCSIbInterfaceProtocol 80 Bulk-OnlyiInterface 0Endpoint Descriptor:bLength 7bDescriptorType 5bEndpointAddress 0x83 EP 3 INbmAttributes 2Transfer Type BulkSynch Type NoneUsage Type DatawMaxPacketSize 0x0200 1x 512 bytesbInterval 0Endpoint Descriptor:bLength 7bDescriptorType 5bEndpointAddress 0x04 EP 4 OUTbmAttributes 2Transfer Type BulkSynch Type NoneUsage Type DatawMaxPacketSize 0x0200 1x 512 bytesbInterval 0Device Qualifier (for other device speed):bLength 10bDescriptorType 6bcdUSB 2.00bDeviceClass 239 Miscellaneous DevicebDeviceSubClass 2 ?bDeviceProtocol 1 Interface AssociationbMaxPacketSize0 64bNumConfigurations 1Device Status: 0x0000(Bus Powered)Quote from: tmbinc on March 10, 2015, 07:56:12 pmI'm very interested in your Fuchsjagd code snippets

author: OrBy
author_rank: Regular Contributor
timestamp: August 13, 2015, 03:27:23 am
content: Since mine is 2.3.0 I spun up a Ubuntu 14.04.3 VM (First time every trying my hand at compiling anything) and got as far at the line listed above before I for the following error:Code:orby@ubuntu:~/libuvc/build$ /usr/bin/cc -o flir ../src/flir.c -luvc../src/flir.c: In function ‘cb’:../src/flir.c:30:3: warning: format ‘%d’ expects argument of type ‘int’, but argument 2 has type ‘__time_t’ [-Wformat=]printf("got frame %d:%d, %d bytes\n", tv.tv_sec, tv.tv_usec, frame->data_bytes);^../src/flir.c:30:3: warning: format ‘%d’ expects argument of type ‘int’, but argument 3 has type ‘__suseconds_t’ [-Wformat=]../src/flir.c:30:3: warning: format ‘%d’ expects argument of type ‘int’, but argument 4 has type ‘size_t’ [-Wformat=]Everything seemed to go word for word as outlined in tomas123 detailed instructions till that point.I have no issues saying that I am in way over my head so unless anyone has any suggestions I don't think I can test further.I would love to get a libuvc with this patch with this working on windows but my limited looking into it seems to indicate that the USB driver wrangling situation to accomplish this would be even more complex.

author: Bud
author_rank: Super Contributor
timestamp: August 13, 2015, 05:27:44 am
content: Quote from: tomas123 on August 12, 2015, 06:31:02 pmSame here, with FW 2.3.0Code:root@Ubuntu32:/home/some/Temp/libuvc# ./myflir test6.rawUVC initializedDevice foundDevice openedDEVICE CONFIGURATION (09cb:1007/) ---Status: idleVideoControl:bcdUVC: 0x0100VideoStreaming(1):bEndpointAddress: 130Formats:UncompressedFormat(1)bits per pixel: 16GUID: 5955593200001000800000aa00389b71 (YUY2)default frame: 1aspect ratio: 0x0interlace flags: 00copy protect: 00FrameDescriptor(1)capabilities: 03size: 320x240bit rate: 912384-912384max frame size: 153600default interval: 1/3interval: 1/15interval: 1/7interval: 1/3END DEVICE CONFIGURATIONbmHint: 0000bFormatIndex: 1bFrameIndex: 1dwFrameInterval: 666666wKeyFrameRate: 0wPFrameRate: 0wCompQuality: 0wCompWindowSize: 0wDelay: 1dwMaxVideoFrameSize: 153600dwMaxPayloadTransferSize: 157696bInterfaceNumber: 1Streaming...got frame 1439442632:196459, 153600 bytesgot frame 1439442632:290426, 153600 bytesgot frame 1439442632:384196, 153600 bytesgot frame 1439442632:478789, 153600 bytesgot frame 1439442632:573349, 153600 bytesDone streaming.Device closedUVC exitedAn extracted frame then is just a screen capture.

author: tomas123
author_rank: Frequent Contributor
timestamp: August 13, 2015, 09:24:48 am
content: Quote from: OrBy on August 13, 2015, 03:27:23 amThis is only a compiler warning, not an error. I got the excact same warning.Simple use the successful compiled flir binary.

author: tmbinc
author_rank: Frequent Contributor
timestamp: August 13, 2015, 09:39:12 am
content: I don't have my E4 here right now. I'll investigate. Sorry for wasting everyone's time. (It definitely worked for me, as you can see in the youtube video)

author: tomas123
author_rank: Frequent Contributor
timestamp: August 13, 2015, 09:56:34 am
content: Sorry, but I'm don't understand the "bFormatIndex- 1" hack.Can you give us some more informations about the patch?Which stream from the lsusb tree (see above) we are grabbing?Thanks for help!Code:diff --git a/src/stream.c b/src/stream.cindex 571be16..aabb6ff 100644--- a/src/stream.c+++ b/src/stream.c@@ -183,6 +183,7 @@ uvc_error_t uvc_query_stream_ctrl(/* prepare for a SET transfer */if (req == UVC_SET_CUR) {SHORT_TO_SW(ctrl->bmHint, buf);+ ctrl->bFormatIndex = 2;buf = ctrl->bFormatIndex;buf = ctrl->bFrameIndex;INT_TO_DW(ctrl->dwFrameInterval, buf + 4);@@ -222,6 +223,7 @@ uvc_error_t uvc_query_stream_ctrl(if (req != UVC_SET_CUR) {ctrl->bmHint = SW_TO_SHORT(buf);ctrl->bFormatIndex = buf;+ ctrl->bFormatIndex = 1;ctrl->bFrameIndex = buf;ctrl->dwFrameInterval = DW_TO_INT(buf + 4);ctrl->wKeyFrameRate = SW_TO_SHORT(buf + 8);@@ -828,7 +830,7 @@ uvc_error_t uvc_stream_start(strmh->pts = 0;strmh->last_scr = 0;- frame_desc = uvc_find_frame_desc_stream(strmh, ctrl->bFormatIndex, ctrl->bFrameIndex);+ frame_desc = uvc_find_frame_desc_stream(strmh, ctrl->bFormatIndex - 1, ctrl->bFrameIndex);if (!frame_desc) {ret = UVC_ERROR_INVALID_PARAM;goto fail;@@ -1037,7 +1039,7 @@ void _uvc_populate_frame(uvc_stream_handle_t *strmh) {* is going to be reopen_on_change anyway*/- frame_desc = uvc_find_frame_desc(strmh->devh, strmh->cur_ctrl.bFormatIndex,+ frame_desc = uvc_find_frame_desc(strmh->devh, strmh->cur_ctrl.bFormatIndex - 1,strmh->cur_ctrl.bFrameIndex);frame->frame_format = strmh->frame_format;

author: tomas123
author_rank: Frequent Contributor
timestamp: August 13, 2015, 10:09:17 am
content: I think, the RAW Video of E40 isCode:bFormatIndex= 2dwMaxVideoFrameBufferSize = 157440but the flir.c log:Code:bFormatIndex: 1bFrameIndex: 1dwMaxVideoFrameSize: 157440...very magiccompress output fromlsusb -vE40VideoStreaming Interface Descriptor:bFormatIndex1bBitsPerPixel 16wWidth 320wHeight 240dwMaxVideoFrameBufferSize 153600 // = 320x240x2 Byte ( Screen Shot Video)VideoStreaming Interface Descriptor:bFormatIndex2bBitsPerPixel 16wWidth 320wHeight 246 // 6 extra lines for the FFF Header (see exiftool output above)dwMaxVideoFrameBufferSize157440// 157440 is same size as the output from the RAW Video grab toolE4VideoStreaming Interface Descriptor:bFormatIndex1bBitsPerPixel 16wWidth 320wHeight 240dwMaxVideoFrameBufferSize 153600 // = 320x240x2 Byte ( Screen Shot Video)

author: OrBy
author_rank: Regular Contributor
timestamp: August 13, 2015, 01:19:35 pm
content: Quote from: tomas123 on August 13, 2015, 09:24:48 amI will spin that VM back up and test it out when I get off work later today then.Quote from: tmbinc on August 13, 2015, 09:39:12 amI don't think anyone thinks you have wasted their time. (I don't) There could be different configurations or variables that could be at play here.What USB mode is your camera in when capturing? (just UVC or one of the mixed modes?)

author: tomas123
author_rank: Frequent Contributor
timestamp: August 13, 2015, 09:48:57 pm
content: Quote from: OrBy on August 13, 2015, 01:19:35 pmI checked all UVC variants without success.

author: OrBy
author_rank: Regular Contributor
timestamp: August 13, 2015, 11:23:49 pm
content: Just took it for a little spin but getting some odd results:Code:orby@ubuntu:~$ sudo libuvc/build/flir e4-1.rawUVC initializedDevice foundDevice openedDEVICE CONFIGURATION (09cb:1007/) ---Status: idleVideoControl:bcdUVC: 0x0100VideoStreaming(1):bEndpointAddress: 130Formats:UncompressedFormat(1)bits per pixel: 16GUID: 5955593200001000800000aa00389b71 (YUY2)default frame: 1aspect ratio: 0x0interlace flags: 00copy protect: 00FrameDescriptor(1)capabilities: 03size: 320x240bit rate: 912384-912384max frame size: 153600default interval: 1/3interval: 1/15interval: 1/7interval: 1/3END DEVICE CONFIGURATIONbmHint: 0000bFormatIndex: 1bFrameIndex: 1dwFrameInterval: 666666wKeyFrameRate: 0wPFrameRate: 0wCompQuality: 0wCompWindowSize: 0wDelay: 1dwMaxVideoFrameSize: 153600dwMaxPayloadTransferSize: 157696bInterfaceNumber: 1Streaming...got frame 1439507727:121939, 153600 bytesgot frame 1439507727:122047, 153600 bytesorby@ubuntu:~$ split -b 153600 e4-1.raworby@ubuntu:~$ convert -size 320x240 yuv:xaa -separate 4.pngconvert.im6: unexpected end-of-file `xaa': No such file or directory @ error/yuv.c/ReadYUVImage/428.It seems like it may be the raw feed at least for me.Attached are the 2 png's it spat out and the zip has the raw and a xaa file it in it.

author: Bud
author_rank: Super Contributor
timestamp: August 14, 2015, 03:20:37 am
content: Quote from: tmbinc on August 12, 2015, 09:21:31 pmCan you share your patched stream.c ?

author: Bud
author_rank: Super Contributor
timestamp: August 14, 2015, 03:24:34 am
content: @OrByI am getting same error and same result (Ubuntu 11.10 on a VM). But the png produced is not radiometric. Yours does not seem to be it either, it does not contain radiometric metadata.

author: tomas123
author_rank: Frequent Contributor
timestamp: August 14, 2015, 08:03:25 am
content: @Orbyyour png-image is radiometric with swapped bytessee my tutorial here:https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg348398/#msg348398Code:>identify 4-0.png4-0.png PNG 320x240 320x240+0+0 16-bit sRGB 67.3KB 0.000u 0:00.001>convert -define png:swap-bytes=on 4-0.png -auto-level raw.pngbetter use the splitted partCode:convert -depth 16 -size 320x240 -auto-level gray:xaa raw.png@Orby + @BudYou have both the firmware 2.3.0.Find the difference between the configurationcheck "lsusb -v"our logs from flir.c are 100% identicalCode:UVC initializedDevice foundDevice openedDEVICE CONFIGURATION (09cb:1007/) ---Status: idleVideoControl:bcdUVC: 0x0100VideoStreaming(1):bEndpointAddress: 130Formats:UncompressedFormat(1)bits per pixel: 16GUID: 5955593200001000800000aa00389b71 (YUY2)default frame: 1aspect ratio: 0x0interlace flags: 00copy protect: 00FrameDescriptor(1)capabilities: 03size: 320x240bit rate: 912384-912384max frame size: 153600default interval: 1/3interval: 1/15interval: 1/7interval: 1/3END DEVICE CONFIGURATIONbmHint: 0000bFormatIndex: 1bFrameIndex: 1dwFrameInterval: 666666wKeyFrameRate: 0wPFrameRate: 0wCompQuality: 0wCompWindowSize: 0wDelay: 1dwMaxVideoFrameSize: 153600dwMaxPayloadTransferSize: 157696bInterfaceNumber: 1Streaming...A interesting difference to my E40 stream:The E4 RAW stream contains no FFF Header:Image Size: 320x240x2 Byte = frame size =153600 ByteThe E40 RAW Stream contains full FFF Imgagesframe size157440 bytesee sample in my posthttps://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729688/#msg729688

author: OrBy
author_rank: Regular Contributor
timestamp: August 14, 2015, 01:16:22 pm
content: Quote from: tomas123 on August 14, 2015, 08:03:25 amYes I kinda thought that it had the look like it was backwards and that it needed just a bit more processing to look right. I ran out of time to play last night but I was going to try messing with it a little more when time permitted.It's interesting how the E4 is missing the header info - guess they cant make it too easy!I will post up my lsusb -v log when I get some time back on my PC.

author: tomas123
author_rank: Frequent Contributor
timestamp: August 14, 2015, 01:26:23 pm
content: The only missing header infos are the time stamps.All other Flir values to calculate the real temperature of a pixel are (calibrated) constants.You can get them from every radiometric jpg of the same (!) camera.I hope tmbinc give us some explanations to the bFormatIndex-hackand some code snippets for our own Fuchsjagd

author: Bud
author_rank: Super Contributor
timestamp: August 14, 2015, 01:31:51 pm
content: By non-radiometric i mean no temperature scaling information is in the file. I do not have access to my computer at the moment but if you load the image in Flir Tools i think temperature measurements will be incorrect and all the way between -40 +300 or something. To get it right i think the image levels need to be scaled against a known reference, same as you do when creating panoramas after stitching step.Anyway, try loading it in Flir Tools to see if it can be correctly measured.

author: tomas123
author_rank: Frequent Contributor
timestamp: August 14, 2015, 01:35:34 pm
content: @BudCurrently you have not radiometric raw values in your stream. Therefore Flir Tools can't read anything.You need a RAW stream, like Orby E4.Then you can inject a extracted 16 bit raw png in a radiometric jpg (from the same E4!) and the temperature measurement of Flir Tools works finesimple use my (panorama) script splitjpg.php for injectinghttps://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg348715/#msg348715// no -auto-levelCode:$ convert -depth 16 -size 320x240 gray:xaa raw.png$ php splitjpg.php -i IR_0xxx.jpg -r raw.png -o sampleLoad the sample.jpg in Flir Tools

author: Bud
author_rank: Super Contributor
timestamp: August 14, 2015, 03:32:50 pm
content: Yes i have already tried that with my non- radiometric streamed images and if i use your splitjpg script and a reference radiometric jpg, i get a proper thermal image in Flir Tools but temperature readings are still somewhat off, probably because the reference jpg was not from the same scene. I will try tonight without the scaling step.Having said that, i begin to believe that may not be difference between the E4 hacked radiometric streamed images vs screen capture streamed ones, providing i turn measurements and everything else off on the camera, and also turn MSX off so the screen only displays the thermal image. I can do this on my doctored E4 2.3.0.I will still try to get the camera stream same way as OrBy's so i can post processes images received using the two methods and compare the end result.

author: OrBy
author_rank: Regular Contributor
timestamp: August 14, 2015, 04:10:54 pm
content: Quote from: Bud on August 14, 2015, 03:32:50 pmI can grab some more out of each stream and put them up for comparison later today if needed/wanted.

author: Bud
author_rank: Super Contributor
timestamp: August 14, 2015, 04:53:07 pm
content: Quote from: Bud on August 14, 2015, 03:32:50 pm...except perhaps swapped bytes in radiometric stream.

author: tomas123
author_rank: Frequent Contributor
timestamp: August 14, 2015, 11:14:35 pm
content: Quote from: Bud on August 14, 2015, 03:32:50 pmStream of the monitor view is nice but a raw stream is really cool (surveillance camera, measurements)See the great video from tmbinc.I hope that tmbinc give us the code for converting a raw stream. I see a nice application for my raspberry pi.Don't forget that Flir Tools can grab a raw stream from E40 (switch from video to signal)https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg377140/#msg377140the video stream of E4 grab with the free FLIR IR Camera Playerhttps://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg378926/#msg378926

author: tomas123
author_rank: Frequent Contributor
timestamp: August 14, 2015, 11:37:10 pm
content: @OrByYour upgraded the E4 from 1.18.7 to 2.3.0.Interesting!Maybe the (hidden) raw stream of your E4 is activated by a switch in your config file.Quote from: OrBy on February 12, 2015, 04:23:13 am

author: OrBy
author_rank: Regular Contributor
timestamp: August 15, 2015, 03:15:39 am
content: Yes I did upgrade mine to 2.3.0Here is my lsusb -v dump:(I hope it's the right section required)Code:Bus 001 Device 002: ID 09cb:1007Couldn't open device, some information will be missingDevice Descriptor:bLength 18bDescriptorType 1bcdUSB 2.00bDeviceClass 239 Miscellaneous DevicebDeviceSubClass 2 ?bDeviceProtocol 1 Interface AssociationbMaxPacketSize0 64idVendor 0x09cbidProduct 0x1007bcdDevice 0.00iManufacturer 7iProduct 8iSerial 0bNumConfigurations 1Configuration Descriptor:bLength 9bDescriptorType 2wTotalLength 241bNumInterfaces 3bConfigurationValue 1iConfiguration 0bmAttributes 0x80(Bus Powered)MaxPower 500mAInterface Association:bLength 8bDescriptorType 11bFirstInterface 0bInterfaceCount 2bFunctionClass 14 VideobFunctionSubClass 3 Video Interface CollectionbFunctionProtocol 0iFunction 10Interface Descriptor:bLength 9bDescriptorType 4bInterfaceNumber 0bAlternateSetting 0bNumEndpoints 1bInterfaceClass 14 VideobInterfaceSubClass 1 Video ControlbInterfaceProtocol 0iInterface 2VideoControl Interface Descriptor:bLength 13bDescriptorType 36bDescriptorSubtype 1 (HEADER)bcdUVC 1.00wTotalLength 85dwClockFrequency 0.001000MHzbInCollection 1baInterfaceNr( 0) 1VideoControl Interface Descriptor:bLength 18bDescriptorType 36bDescriptorSubtype 2 (INPUT_TERMINAL)bTerminalID 1wTerminalType 0x0201 Camera SensorbAssocTerminal 0iTerminal 0wObjectiveFocalLengthMin 0wObjectiveFocalLengthMax 0wOcularFocalLength 0bControlSize 3bmControls 0x00020260Focus (Absolute)Focus (Relative)Zoom (Absolute)Focus, AutoVideoControl Interface Descriptor:bLength 9bDescriptorType 36bDescriptorSubtype 3 (OUTPUT_TERMINAL)bTerminalID 3wTerminalType 0x0101 USB StreamingbAssocTerminal 0bSourceID 6iTerminal 0VideoControl Interface Descriptor:bLength 7bDescriptorType 36bDescriptorSubtype 4 (SELECTOR_UNIT)bUnitID 4bNrInPins 1baSource( 0) 1iSelector 0VideoControl Interface Descriptor:bLength 11bDescriptorType 36bDescriptorSubtype 5 (PROCESSING_UNIT)Warning: Descriptor too shortbUnitID 5bSourceID 4wMaxMultiplier 0bControlSize 2bmControls 0x00000217BrightnessContrastHueSharpnessGainiProcessing 0bmVideoStandards 0x1bNoneNTSC - 525/60SECAM - 625/50NTSC - 625/50VideoControl Interface Descriptor:bLength 27bDescriptorType 36bDescriptorSubtype 6 (EXTENSION_UNIT)bUnitID 6guidExtensionCode {d41f59fa-5094-463a-b3bb-e7858a831fa3}bNumControl 4bNrPins 1baSourceID( 0) 5bControlSize 2bmControls( 0) 0x0fbmControls( 1) 0x00iExtension 0Endpoint Descriptor:bLength 7bDescriptorType 5bEndpointAddress 0x81 EP 1 INbmAttributes 3Transfer Type InterruptSynch Type NoneUsage Type DatawMaxPacketSize 0x0008 1x 8 bytesbInterval 16Interface Descriptor:bLength 9bDescriptorType 4bInterfaceNumber 1bAlternateSetting 0bNumEndpoints 1bInterfaceClass 14 VideobInterfaceSubClass 2 Video StreamingbInterfaceProtocol 0iInterface 0VideoStreaming Interface Descriptor:bLength 14bDescriptorType 36bDescriptorSubtype 1 (INPUT_HEADER)bNumFormats 1wTotalLength 79bEndPointAddress 130bmInfo 0bTerminalLink 3bStillCaptureMethod 0bTriggerSupport 0bTriggerUsage 0bControlSize 1bmaControls( 0) 27VideoStreaming Interface Descriptor:bLength 27bDescriptorType 36bDescriptorSubtype 4 (FORMAT_UNCOMPRESSED)bFormatIndex 1bNumFrameDescriptors 1guidFormat {59555932-0000-1000-8000-00aa00389b71}bBitsPerPixel 16bDefaultFrameIndex 1bAspectRatioX 0bAspectRatioY 0bmInterlaceFlags 0x00Interlaced stream or variable: NoFields per frame: 2 fieldsField 1 first: NoField pattern: Field 1 onlybCopyProtect 0VideoStreaming Interface Descriptor:bLength 38bDescriptorType 36bDescriptorSubtype 5 (FRAME_UNCOMPRESSED)bFrameIndex 1bmCapabilities 0x03Still image supportedFixed frame-ratewWidth 320wHeight 240dwMinBitRate 912384dwMaxBitRate 912384dwMaxVideoFrameBufferSize 153600dwDefaultFrameInterval 2666664bFrameIntervalType 3dwFrameInterval( 0) 666666dwFrameInterval( 1) 1333332dwFrameInterval( 2) 2666664Endpoint Descriptor:bLength 7bDescriptorType 5bEndpointAddress 0x82 EP 2 INbmAttributes 2Transfer Type BulkSynch Type NoneUsage Type DatawMaxPacketSize 0x0200 1x 512 bytesbInterval 0Interface Descriptor:bLength 9bDescriptorType 4bInterfaceNumber 2bAlternateSetting 0bNumEndpoints 2bInterfaceClass 8 Mass StoragebInterfaceSubClass 6 SCSIbInterfaceProtocol 80 Bulk-OnlyiInterface 0Endpoint Descriptor:bLength 7bDescriptorType 5bEndpointAddress 0x83 EP 3 INbmAttributes 2Transfer Type BulkSynch Type NoneUsage Type DatawMaxPacketSize 0x0200 1x 512 bytesbInterval 0Endpoint Descriptor:bLength 7bDescriptorType 5bEndpointAddress 0x04 EP 4 OUTbmAttributes 2Transfer Type BulkSynch Type NoneUsage Type DatawMaxPacketSize 0x0200 1x 512 bytesbInterval 0And here are the contents of my e8.cfg:Code:.caps entry.caps.config entry.caps.config.name text "app E8".caps.config.revision text "1.0".caps.config.image entry.caps.config.image.framegrab entry.caps.config.image.framegrab.fusion entry.caps.config.image.framegrab.fusion.enabled bool true.caps.config.image.framegrab.fusion.pip entry.caps.config.image.framegrab.fusion.pip.enabled bool true.caps.config.image.framegrab.fusion.hcf entry.caps.config.image.framegrab.fusion.hcf.enabled bool true.caps.config.image.services entry.caps.config.image.services.store entry.caps.config.image.services.store.enabled bool true.caps.config.image.services.store.radiometric entry.caps.config.image.services.store.radiometric.enabled bool true.caps.config.image.services.store.incompatible entry.caps.config.image.services.store.incompatible.enabled bool false.caps.config.image.services.store.incompatible.level int32 0.caps.config.image.settings entry.caps.config.image.settings.enabled bool true.caps.config.image.settings.IRwidth int32 320.caps.config.image.settings.IRheight int32 240.caps.config.image.sysimg entry.caps.config.image.sysimg.alarms entry.caps.config.image.sysimg.alarms.enabled bool true.caps.config.image.sysimg.alarms.measfunc entry.caps.config.image.sysimg.alarms.measfunc.enabled bool true.caps.config.image.sysimg.alarms.measfunc.maxCount int32 3.caps.config.image.sysimg.alarms.humidity entry.caps.config.image.sysimg.alarms.humidity.enabled bool true.caps.config.image.sysimg.alarms.humidity.maxCount int32 1.caps.config.image.sysimg.alarms.insulation entry.caps.config.image.sysimg.alarms.insulation.enabled bool true.caps.config.image.sysimg.alarms.insulation.maxCount int32 1.caps.config.image.sysimg.irMarkers entry.caps.config.image.sysimg.irMarkers.enabled bool true.caps.config.image.sysimg.irMarkers.spot entry.caps.config.image.sysimg.irMarkers.spot.enabled bool false.caps.config.image.sysimg.irMarkers.spot.maxCount int32 0.caps.config.image.sysimg.irMarkers.arrow entry.caps.config.image.sysimg.irMarkers.arrow.enabled bool true.caps.config.image.sysimg.irMarkers.arrow.maxCount int32 4.caps.config.image.sysimg.irMarkers.box entry.caps.config.image.sysimg.irMarkers.box.enabled bool false.caps.config.image.sysimg.irMarkers.box.maxCount int32 0.caps.config.image.sysimg.measureFuncs entry.caps.config.image.sysimg.measureFuncs.enabled bool true.caps.config.image.sysimg.measureFuncs.diff entry.caps.config.image.sysimg.measureFuncs.diff.enabled bool true.caps.config.image.sysimg.measureFuncs.diff.maxCount int32 1.caps.config.image.sysimg.measureFuncs.diff.calcMask int32 65526.caps.config.image.sysimg.measureFuncs.isotherm entry.caps.config.image.sysimg.measureFuncs.isotherm.enabled bool true.caps.config.image.sysimg.measureFuncs.isotherm.calcMask int32 20.caps.config.image.sysimg.measureFuncs.isotherm.dual bool false.caps.config.image.sysimg.measureFuncs.isotherm.fixScale bool false.caps.config.image.sysimg.measureFuncs.isotherm.interval bool true.caps.config.image.sysimg.measureFuncs.isotherm.invInterval bool false.caps.config.image.sysimg.measureFuncs.isotherm.maxCount int32 1.caps.config.image.sysimg.measureFuncs.mbox entry.caps.config.image.sysimg.measureFuncs.mbox.enabled bool true.caps.config.image.sysimg.measureFuncs.mbox.calcMask int32 1924.caps.config.image.sysimg.measureFuncs.mbox.maxCount int32 5.caps.config.image.sysimg.measureFuncs.mcircle entry.caps.config.image.sysimg.measureFuncs.mcircle.enabled bool false.caps.config.image.sysimg.measureFuncs.mcircle.calcMask int32 1924.caps.config.image.sysimg.measureFuncs.mcircle.maxCount int32 0.caps.config.image.sysimg.measureFuncs.mline entry.caps.config.image.sysimg.measureFuncs.mline.enabled bool false.caps.config.image.sysimg.measureFuncs.mline.calcMask int32 1924.caps.config.image.sysimg.measureFuncs.mline.maxCount int32 0.caps.config.image.sysimg.measureFuncs.reftemp entry.caps.config.image.sysimg.measureFuncs.reftemp.enabled bool true.caps.config.image.sysimg.measureFuncs.reftemp.calcMask int32 1924.caps.config.image.sysimg.measureFuncs.reftemp.maxCount int32 1.caps.config.image.sysimg.measureFuncs.script entry.caps.config.image.sysimg.measureFuncs.script.enabled false.caps.config.image.sysimg.measureFuncs.script.maxCount int32 0.caps.config.image.sysimg.measureFuncs.spot entry.caps.config.image.sysimg.measureFuncs.spot.enabled bool true.caps.config.image.sysimg.measureFuncs.spot.calcMask int32 514.caps.config.image.sysimg.measureFuncs.spot.maxCount int32 5.caps.config.image.sysimg.visualMarkers entry.caps.config.image.sysimg.visualMarkers.enabled bool true.caps.config.image.sysimg.visualMarkers.spot entry.caps.config.image.sysimg.visualMarkers.spot.enabled bool false.caps.config.image.sysimg.visualMarkers.spot.maxCount int32 0.caps.config.image.sysimg.visualMarkers.arrow entry.caps.config.image.sysimg.visualMarkers.arrow.enabled bool true.caps.config.image.sysimg.visualMarkers.arrow.maxCount int32 4.caps.config.image.sysimg.visualMarkers.box entry.caps.config.image.sysimg.visualMarkers.box.enabled bool false.caps.config.image.sysimg.visualMarkers.box.maxCount int32 0.caps.config.image.contadj entry.caps.config.image.contadj.minSpanFactor entry.caps.config.image.contadj.minSpanFactor.enabled bool true.caps.config.image.contadj.minSpanFactor.factorAuto double 2.0.caps.config.image.contadj.minSpanFactor.factorManual double 2.0.caps.config.image.targetNoise entry.caps.config.image.targetNoise.enabled bool false.caps.config.image.targetNoise.targetNoiseMk int32 0.caps.config.image.zoom entry.caps.config.image.zoom.enabled bool true.caps.config.image.zoom.maxFactor double 8.caps.config.system entry.caps.config.system.focus entry.caps.config.system.focus.laser entry.caps.config.system.focus.laser.updateFocus entry.caps.config.system.focus.laser.updateFocus.enabled bool true.caps.config.ui.image entry.caps.config.ui.image.adjust entry.caps.config.ui.image.adjust.enabled bool true.caps.config.ui.image.adjust.manual bool true.caps.config.ui entry.caps.config.ui.fusion entry.caps.config.ui.fusion.PIP entry.caps.config.ui.fusion.PIP.enabled bool true.caps.hw entry.caps.hw.sdcard entry.caps.hw.sdcard.enabled bool false

author: Bud
author_rank: Super Contributor
timestamp: August 15, 2015, 04:01:25 am
content: I followed tomas123's steps to re-compile libuvc library and now it works for me too (thanks tomas123 for posting the compilation log, I think I can learn Chinese faster than Linux ).Attached is a frame from the video stream, loaded in FLIR Tools and some measurements enabled. All makes sense and matches what is measured by the camera. For the record, the camera is a native E4 fw 2.3.0 doctored for resolution and enhanced menu.So yes the patch works, it is possible to stream radiometric video from E4 (oh well, for now just in a file and using Linux), thanks user tmbinc

author: tomas123
author_rank: Frequent Contributor
timestamp: August 15, 2015, 08:29:30 am
content: Great!Please describe all steps, how you can load a frame in Flir Tools and attach a splitted single raw frame (as .zip)I think, I must make a firmware update from 1.19 to 2.3.0don't touch a running system

author: Bud
author_rank: Super Contributor
timestamp: August 15, 2015, 11:40:56 pm
content: All as previously described:Pre-requisits for image processing software:- imagemagick- php- splitjpg.php scriptsource:https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg348715/#msg348715- a sample image from the same E4 camera- FLIR Toolsworkflow:- capture a RAW video- split to individual framesCode:split -b 153600 test1.raw- convert the required frame (in this example the first frame) to a RAW pngCode:convert -depth 16 -size 320x240 gray:xaa raw1.pngconvert the RAW png to radiometric jpg (for a template use a sample jpg image, e.g. FLIR0123.jpg, from the same camera:Code:php splitjpg.php -i FLIR0123.jpg -r raw1.png -o test1.jpgnote: after this step the resulting test1.jpg preview will appear the same as the template image, do not worry: the actual RAW data is hidden behind the image.Now load this JPG into FLIR Tools to rebuild the preview image, place temperature measurements, apply a desired palette and resave.I do video capture, split and convert on a Ubuntu VM, everything else on a Windows 7.

author: Bud
author_rank: Super Contributor
timestamp: August 16, 2015, 06:12:02 am
content: Found a great little utility called ImageJhttp://imagej.nih.gov/ij/Holy sh!t, have not seen it before, it is a jewel. For the purpose of this thread it can directly open raw files created by tmbinc's software and resave in a bunch of different formats including individual split frames. Also has a pile of built-in pallettes (Look Up Table they call it) and can colorize the video and even save the video in AVI format ready to be played !A sample is attached - a radiometric stream capture from the E4, colorized using ImageJ.test13.avi<- click hereWith this sort of post processing software radiometric streaming from E4 all of a sudden makes sense.

author: tomas123
author_rank: Frequent Contributor
timestamp: August 16, 2015, 12:31:54 pm
content: Great!!!ImageJ read my E40 raw stream with little endian byte order and 3840 Byte Flir-FFF-Header between frames@BudI linked your last two posts in my E4 link list:https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg342072/#msg342072see topic 15Mike linked my post to the the head of the infinite threadhttps://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/"Tomas123's links on post- processing and measurements using E4 images"Please add in your second last post a link to my splitjpg.php sourcehttps://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg348715/#msg348715@tmbincI hope you can give us your Fuchsjagd code snippets

author: Bud
author_rank: Super Contributor
timestamp: August 16, 2015, 06:45:13 pm
content: Quote from: tomas123 on August 16, 2015, 12:31:54 pmDone.I wish ImageJ could combine multiple frames in a single image for superresolution, but have not figured out yet if it can.A newer version is here:http://fiji.sc/DownloadsBetter palettes, +tons of image manipulation menus, a holy grail of image processing

author: tomas123
author_rank: Frequent Contributor
timestamp: August 17, 2015, 08:44:01 am
content: a little bit off topic: superresolution with RAW thermal videoI think, you know my posts to superresolution.(see my link list #17 ->https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg363688/#msg363688)I have had good experiences with:- Registax (sharp with wavelets)- AviStack (sharp with wavelets)- Photoshop CS3 -> automate -> merge to HDR (automat. align images)-> quality like Registax/AviStax (CS3 works better as CS5)... do not expect too much and resize your images/video before stacking

author: Bud
author_rank: Super Contributor
timestamp: August 20, 2015, 05:58:26 am
content: Quote from: Bud on August 16, 2015, 06:45:13 pmOK I think there is a few ways to experiment with superresolution in ImageJ. For the purpose of this topic the workflow to create a combined image from a sequence of video frames may look as follows:- capture a few seconds of a raw video as already described above- open the video in ImageJ (File->Import->RAW)- if necessary, delete bad or unwanted frames (use scroll bar to select a frame, then "Image->Stacks->Delete Slice"- combine the frames (Image->Stacks->Z Project)

author: tmbinc
author_rank: Frequent Contributor
timestamp: August 20, 2015, 09:43:45 pm
content: Sorry for not keeping up with this thread.The idea is that even though the E4 doesn't announce the bFormatIndex=2 RAW stream via the USB descriptor, it still allows to select it. The code that restricts the raw mode based on the camera type does cripple the descriptor, but it doesn't prevent you from still selecting it.The libuvc patch is what I've posted (i just double-checked against what's still on my raspi2), I applied it on top ofhttps://github.com/ktossell/libuvc/commits/6ff39f17e3f3a26f437bc6b3c09cc6c289db7891. It does this in a very lame and stupid way by switching it back and forth, to make the library take the original stream descriptor, but send the right UVC command.I attached the "fuchsjags" source, but it's really not great - it calculates min and max, scales that to 0..255, inserts the texts and the pointer, converts it to ARGB for the framebuffer, and also dumps the raw and converted values into files. I have a different script that used avconv to encode it to MP4 ("avconv -f rawvideo -pix_fmt gray -s:v 320x240 -r 9 -i /mnt/fox/r1_proc.raw -c:v libx264 /mnt/fox/output.mp4").

author: tomas123
author_rank: Frequent Contributor
timestamp: August 20, 2015, 10:45:54 pm
content: Thank YouMy next project name is Marderjagd

author: tomas123
author_rank: Frequent Contributor
timestamp: August 23, 2015, 08:41:14 pm
content: Quote from: tomas123 on August 15, 2015, 08:29:30 amsuccessful upgraded the E4 from 1.19.8 to 2.3.0 with Flir ToolsThe resolution hack survived the update (nothing else to do).the RAW stream grabbing now works fine

author: Bud
author_rank: Super Contributor
timestamp: August 23, 2015, 08:59:32 pm
content: Great, so RAW streaming did not work in 1.19.8 ...

author: Bud
author_rank: Super Contributor
timestamp: August 23, 2015, 09:38:55 pm
content: @tomas123I wonder if you then could take a challenge and compile the other sample from libuvc which supposedly creates a window and displays the streamed video (source modified for 320x240x15 is attached). It requires a library called opencv to be installed. I tried but as I said my Chinese is much better than my Linux, no matter how hard I tried I got the dreaded opencv "Rebuild the library with Windows, GTK+ 2.x or Carbon support" error. I am sure I am missing something obvious but I just do not know what.If this code works and if combined with tmbinc's code this may become a usable program for E4 that displays, saves and processes E4 radiometric video.

author: tmbinc
author_rank: Frequent Contributor
timestamp: August 24, 2015, 06:53:45 am
content: By the way, my test tool already shows the output on a framebuffer. It's not great and was just meant for debugging... Should be easy though to add a LUT and some color scheme, as well as "calibrated" temperature. I'm not sure though that the radiometric metadata is available.I also tried doing a 2D-FFT to assist in fine-tuning focus. But the results haven't been that useful. Does someone have a better idea how to do this? (I.e. how to evaluate focus based on image processing)

author: Zucca
author_rank: Unknown
timestamp: August 24, 2015, 04:05:37 pm
content: Quote from: tomas123 on August 23, 2015, 08:41:14 pmTemped to do the same a long time ago, didn´t have time yet:https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg607838/#msg607838and do thejanekivimenu mod. Hope the raw capturing will work regardless the menu mod.

author: Bud
author_rank: Super Contributor
timestamp: August 28, 2015, 04:09:26 am
content: Quote from: tmbinc on August 24, 2015, 06:53:45 amThe opencv library is a tool for computer vision, so I'd think it may have something in it for what you need.

author: tomas123
author_rank: Frequent Contributor
timestamp: August 29, 2015, 03:34:38 pm
content: Quote from: Bud on August 23, 2015, 09:38:55 pmHi Bud,your code snippet works fine, no changes are necessaryedit 30.08.15: found the sourcehttps://github.com/ktossell/libuvc/blob/master/src/test.cthe steps:(1) check versionCode:$ lsb_release -aDescription:Ubuntu 14.04.1 LTS(2) compile opencvuse this tutorialhttp://www.bogotobogo.com/OpenCV/opencv_3_tutorial_ubuntu14_install_cmake.phpCode:$ mkdir OpenCV$ cd OpenCV$ git clone https://github.com/Itseez/opencv.git$ cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D BUILD_NEW_PYTHON_SUPPORT=ON -D WITH_V4L=ON -D INSTALL_C_EXAMPLES=ON -D INSTALL_PYTHON_EXAMPLES=ON -D BUILD_EXAMPLES=ON -D WITH_QT=ON -D WITH_OPENGL=ON ..$ make$ sudo make install$ sudo ldconfig(3) compile your sample// ignore warnings// notehttp://stackoverflow.com/questions/9614872/opencv-application-not-library-linking-errorsCode:$ /usr/bin/cc -o mytest mytest.c -luvc `pkg-config --cflags --libs opencv`mytest.c: In function ‘cb’:mytest.c:50:66: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]printf("callback! length = %u, ptr = %d\n", frame->data_bytes, (int) ptr);^mytest.c:50:3: warning: format ‘%u’ expects argument of type ‘unsigned int’, but argument 2 has type ‘size_t’ [-Wformat=]printf("callback! length = %u, ptr = %d\n", frame->data_bytes, (int) ptr);(4) run your codeCode:$ sudo ./mytestUVC initializedDevice foundDevice openedDEVICE CONFIGURATION (09cb:1007/) ---Status: idleVideoControl:bcdUVC: 0x0100VideoStreaming(1):bEndpointAddress: 130Formats:UncompressedFormat(1)bits per pixel: 16GUID: 5955593200001000800000aa00389b71 (YUY2)default frame: 1aspect ratio: 0x0interlace flags: 00copy protect: 00FrameDescriptor(1)capabilities: 03size: 320x240bit rate: 912384-912384max frame size: 153600default interval: 1/3interval: 1/15interval: 1/7interval: 1/3END DEVICE CONFIGURATIONbmHint: 0000bFormatIndex: 1bFrameIndex: 1dwFrameInterval: 666666wKeyFrameRate: 0wPFrameRate: 0wCompQuality: 0wCompWindowSize: 0wDelay: 1dwMaxVideoFrameSize: 153600dwMaxPayloadTransferSize: 157696bInterfaceNumber: 1Streaming for 10 seconds...set_ae_mode: Pipe (-9)callback! length = 153600, ptr = -886544208set_exp_abs: Pipe (-9)init doneopengl support availablecallback! length = 157690, ptr = -886544208callback! length = 157690, ptr = -886544208callback! length = 157690, ptr = -886544208callback! length = 157690, ptr = -886544208and you get a window with a nice false color video

author: Ben321
author_rank: Frequent Contributor
timestamp: August 29, 2015, 07:39:49 pm
content: Quote from: tmbinc on March 05, 2015, 11:20:56 pmCool, but could you please attach the compiled file instead of the C source code and patch? I want a binary to use directly. I don't have much experience with C programming, but I really would like to get to use this (I'm considering getting a Flir E4, part of the Flir Ex series, but whether or not I get it will depend on what features I can access, via EASY to use hacks, and having to compile a hack is NOT easy). So can you pleas compile this, and then post the compiled binary file here?

author: tomas123
author_rank: Frequent Contributor
timestamp: August 30, 2015, 10:16:24 am
content: this a generic linux source code and you must compile it for your CPU (ARM / Intel etc) and your Linux versiontmbinc compiled the code on a raspberry pi 2 (quad-core ARM7)you need a firmware 2.3.0read my step by step tutorial for ubuntu x64https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729604/#msg729604[/url

author: tomas123
author_rank: Frequent Contributor
timestamp: August 30, 2015, 08:26:42 pm
content: @Budjust for fun I changed the sample code from libuvc for 16 Bit RAW and appended the auto-level code from tmbinchttps://github.com/ktossell/libuvc/blob/master/src/test.ccompile the code like above describedCode:# /usr/bin/cc -o mytest mytest.c -luvc `pkg-config --cflags --libs opencv`... some warnings# sudo ./mytestUVC initializedDevice foundDevice opened...the source:Code:/********************************************************************** Software License Agreement (BSD License)** Copyright (C) 2010-2012 Ken Tossell* All rights reserved.** Redistribution and use in source and binary forms, with or without* modification, are permitted provided that the following conditions* are met:** * Redistributions of source code must retain the above copyright* notice, this list of conditions and the following disclaimer.* * Redistributions in binary form must reproduce the above* copyright notice, this list of conditions and the following* disclaimer in the documentation and/or other materials provided* with the distribution.* * Neither the name of the author nor other contributors may be* used to endorse or promote products derived from this software* without specific prior written permission.** THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS* FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE* COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,* BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;* LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER* CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN* ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE* POSSIBILITY OF SUCH DAMAGE.*********************************************************************/#include <stdio.h>#include <opencv/highgui.h>#include "libuvc/libuvc.h"struct cb_context {FILE *out;struct timeval tv_start;int frames;};void cb(uvc_frame_t *frame, void *ptr) {uvc_frame_t *rgb;uvc_error_t ret;IplImage* cvImg;printf("callback! length = %u, ptr = %d\n", frame->data_bytes, (int) ptr);rgb = uvc_allocate_frame(frame->width * frame->height * 3);if (!rgb) {printf("unable to allocate rgb frame!");return;}ret = uvc_any2rgb(frame, rgb);if (ret) {uvc_perror(ret, "uvc_any2rgb");uvc_free_frame(rgb);return;}cvImg = cvCreateImageHeader(cvSize(rgb->width, rgb->height),IPL_DEPTH_8U,3);unsigned short *pix = frame->data;unsigned char *fb_proc;fb_proc = malloc(320 * 240);int i, x, y;// calc min, max, rmsint min = 0x10000, max = 0;for (i = 0; i < 320 * 240; ++i) {if (pix < min) min = pix;if (pix > max) { max = pix; }}int scale = 0x10000 / (max - min);// stretch level and shift 16 to 8 Bitfor (y = 0; y < 240; ++y){for (x = 0; x < 320; ++x) {int v = (pix[y * 320 + x] - min) * scale >> 8;fb_proc[y * 320 + x] = v;}}CvSize image_size;image_size.height = 240;image_size.width = 320;int channels = 1;IplImage *image = cvCreateImageHeader(image_size, IPL_DEPTH_8U, channels);cvSetData(image, fb_proc, image->widthStep);cvNamedWindow("Test", CV_WINDOW_AUTOSIZE);cvShowImage("Test", image);cvWaitKey(10);cvReleaseImageHeader(&cvImg);uvc_free_frame(rgb);}int main(int argc, char **argv) {uvc_context_t *ctx;uvc_device_t *dev;uvc_device_handle_t *devh;uvc_stream_ctrl_t ctrl;uvc_error_t res;struct cb_context cb_ctx = {0};/* Initialize a UVC service context. Libuvc will set up its own libusb* context. Replace NULL with a libusb_context pointer to run libuvc* from an existing libusb context. */res = uvc_init(&ctx, NULL);if (res < 0) {uvc_perror(res, "uvc_init");return res;}puts("UVC initialized");res = uvc_find_device(ctx, &dev,0, 0, NULL);if (res < 0) {uvc_perror(res, "uvc_find_device");} else {puts("Device found");res = uvc_open(dev, &devh);if (res < 0) {uvc_perror(res, "uvc_open");} else {puts("Device opened");uvc_print_diag(devh, stderr);/* Try to negotiate a 640x480 30 fps YUYV stream profile */res = uvc_get_stream_ctrl_format_size(devh, &ctrl, UVC_FRAME_FORMAT_YUYV, 320, 240, 15 /* YUV 422, aka YUV 4:2:2. try _COMPRESSED */);/* Print out the result */uvc_print_stream_ctrl(&ctrl, stderr);if (res < 0) {uvc_perror(res, "get_mode");/* device doesn't provide a matching stream */} else {/* Start the video stream. The library will call user function cb:* cb(frame, (void*) 12345)*/res = uvc_start_streaming(devh, &ctrl, cb, &cb_ctx, 0);if (res < 0) {uvc_perror(res, "start_streaming"); /* unable to start stream */} else {puts("Streaming for 10 seconds...");uvc_error_t resAEMODE = uvc_set_ae_mode(devh, 1);uvc_perror(resAEMODE, "set_ae_mode");int i;for (i = 1; i <= 10; i++) {/* uvc_error_t resPT = uvc_set_pantilt_abs(devh, i * 20 * 3600, 0); *//* uvc_perror(resPT, "set_pt_abs"); */uvc_error_t resEXP = uvc_set_exposure_abs(devh, 20 + i * 5);uvc_perror(resEXP, "set_exp_abs");sleep(1);}sleep(20);uvc_stop_streaming(devh);puts("Done streaming.");}}uvc_close(devh);puts("Device closed");}uvc_unref_device(dev);}uvc_exit(ctx);puts("UVC exited");return 0;}RAW live stream:@ tmbincWhat do you mean with this code snippset from fuchsjagd?It's like a 20x20 pixel watermark.You can see it in your video at the bottom on the left side.Code:unsigned short *pix = frame->data;int x, y;for (y = 160; y < 180; ++y) {for (x = 60; x < 80; ++x) {pix[y * 320 + x] = pix[y * 320 + 60];}}Thank you for the fuchsjagd codeI changed it for Ubuntu x64, only the frame buffer part it's a little bit tricky.

author: Bud
author_rank: Super Contributor
timestamp: August 31, 2015, 04:25:50 am
content: Ya baby, now we are talking! Thanks tomas123!Though I had to update to Ubuntu 14.whatever, otherwise the OpenCV library did not compile. That is why I avoid Linux, the developers have no clue as to what flavor of Linux and what version their product works with. Guessing game every time and you better have all Developers packages in the Universe installed to mitigate the pain.Anyway, your latest code worked and I took a liberty to tailor it a little. I made use of the original buffer initialization from the original test file, as there seemed to be some duplication in last version. I also commented out the frame data length message that was flooding the screen - I think we got the code working to the point we do not need to monitor that anymore. And a nice thing: I replaced CV_WINDOW_AUTOSIZE with CV_WINDOW_NORMAL, and now the window is resizable.I also returned to waiting for a key press to end streaming vs a timeout.BTW, why is this code only falls thru if ENTER key is pressed but not any other key ?Code:// stream until key is pressed.char c;read(0, &c, 1);/* End the stream. Blocks until last callback is serviced */uvc_stop_streaming(devh);puts("Done streaming.");So this actually is becoming a useful piece of software. You can make snapshots from the window (a sample attached). I think autoleveling may need to be improved, it did not always work well.EDIT: attached the source

author: Bud
author_rank: Super Contributor
timestamp: September 01, 2015, 03:59:04 am
content: @tomas123OK some more awkward code added to track mouse coordinates and display pixel value under the cursor using a slider control as a gauge meterSo this pretty much completes a Proof of Concept and now the actual pixel temperature can be calculated and displayed, you will find my comments in the code. I recall you were the one who reverse engineered FLIR temperature calculation? This at the moment is beyond my ability so pls see if you can update the code with it. Thks!What else could be added is perhaps a center spot and box measures. I think this can be done with OpenCV.For now you can position mouse cursor at the center of the image to emulate a center spot and then moving the camera will read temperature from it.EDIT: just in case: I compile using the following command:Code:gcc -o flirfox2 flirfox2.c -luvc -lm `pkg-config --cflags --libs opencv`

author: Bud
author_rank: Super Contributor
timestamp: September 01, 2015, 04:15:48 am
content: @tomas123And here is something interesting from me playing with window modes. When I used CV_WINDOW_OPENGL, moved the window on the screen, exploded to full screen then back, noticed something weird (selection is mine):For some reason the camera changed frame data length and it seemed to stayed consistent until I returned from full screen mode. What the heck the camera was transmitting and how the heck the code managed to display a proper image???Did not have time to play more with that, may be after adding streaming data to file will see if can replicate it and look what is in the extra data.

author: tomas123
author_rank: Frequent Contributor
timestamp: September 01, 2015, 09:21:30 am
content: I can confirm this additional bytes.In my Ubuntu virtual machine I often have spontaneous glitches associated with changes of usb frame lenght.With camera connected direct to host machine USB this glitches are rarely.The additional bytes are without informations (most zeros)I solved this glitches in the juchsjagd code by hardcoding the frame size with 320x240x2 Byte (crop the tail).Quote from: Bud on September 01, 2015, 03:59:04 amthis is really hardYou can't grep a Flir temperature value from a 8 Bit grayscale video stream without additional informations (high/low value etc)Therefore the best way is to enlarge the image size and imprint the temperature values (like max/min) outside the visible video.the nice running fuchsjagd code from tmbinc contains all necessary code snippsets (font.h)https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg736344/#msg736344the mathematical equations for calculating RAW to temperature I posted here:- sample with excel sheet:http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,4898.msg27546.html#msg27546- sample with php codehttps://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg342072/#msg342072As attachment I post my excel sheet with all formulas and some unpublishedinfos from FlirThe required Flir parameters are calibrated constants for every camera. You get this values from every radiometric jpg with this exiftool command:Code:>exiftool -Atmos* -Planck* -Object* -Emis* -Reflected* -Relativ* sample.jpgAtmospheric Temperature : 20.0 CAtmospheric Trans Alpha 1 : 0.006569Atmospheric Trans Alpha 2 : 0.012620Atmospheric Trans Beta 1 : -0.002276Atmospheric Trans Beta 2 : -0.006670Atmospheric Trans X : 1.900000Planck R1 : 15314.136Planck B : 1404.6Planck F : 1Planck O : -978Planck R2 : 0.011565135Object Distance : 1.00 mEmissivity : 0.60Reflected Apparent Temperature : 20.0 CRelative Humidity : 50.0 %

author: tmbinc
author_rank: Frequent Contributor
timestamp: September 01, 2015, 09:58:49 am
content: Quote from: tomas123 on August 30, 2015, 08:26:42 pmHaha, I did forget about that code. Sorry. I have some dirt on my lens (or sensor) which I didn't cleanup yet (out of fear to break something). This created a spot that read a slightly larger temperature, which triggered the "Fuchs temperature threshold" ("Ftt").The, erm, solution was to just copy the area next to it over the spot.

author: Pantheron
author_rank: Contributor
timestamp: September 02, 2015, 12:03:18 am
content: Hello guys,i tried this also, but i cant get a stream with the first flir.c release.I also got some other streaming descriptors than you, probably you can help me out?Thank you in advance.It is an FLIR E4 wit FW 2.30 and hacked for Resolution and Menu (Firmware was original at 2.30)Code:root@iJudge:/home/pantheron/libuvc# build/flir e4x.rawUVC initializedDevice foundunsupported descriptor subtype: 3unsupported descriptor subtype: 13unsupported descriptor subtype: 3unsupported descriptor subtype: 13Device openedDEVICE CONFIGURATION (05ac:8510/CC2CAV0DA1DN9KE0) ---Status: idleVideoControl:bcdUVC: 0x0100VideoStreaming(1):bEndpointAddress: 130Formats:UncompressedFormat(1)bits per pixel: 16GUID: 5955593200001000800000aa00389b71 (YUY2)default frame: 5aspect ratio: 0x0interlace flags: 00copy protect: 00FrameDescriptor(1)capabilities: 00size: 160x120bit rate: 24576000-147456000max frame size: 38400default interval: 1/29interval: 1/29interval: 1/25interval: 1/23interval: 1/14FrameDescriptor(2)capabilities: 00size: 176x144bit rate: 24576000-147456000max frame size: 50688default interval: 1/29interval: 1/29interval: 1/25interval: 1/23interval: 1/14FrameDescriptor(3)capabilities: 00size: 320x240bit rate: 24576000-147456000max frame size: 153600default interval: 1/29interval: 1/29interval: 1/25interval: 1/23interval: 1/14FrameDescriptor(4)capabilities: 00size: 352x288bit rate: 24576000-147456000max frame size: 202752default interval: 1/29interval: 1/29interval: 1/25interval: 1/23interval: 1/14FrameDescriptor(5)capabilities: 00size: 640x480bit rate: 24576000-147456000max frame size: 614400default interval: 1/29interval: 1/29interval: 1/25interval: 1/23interval: 1/14FrameDescriptor(6)capabilities: 00size: 480x360bit rate: 24576000-147456000max frame size: 345600default interval: 1/29interval: 1/29interval: 1/25interval: 1/23interval: 1/14FrameDescriptor(7)capabilities: 00size: 1280x720bit rate: 24576000-147456000max frame size: 1843200default interval: 1/10interval: 1/10MJPEGFormat(2)bits per pixel: 0GUID: 4d4a5047000000000000000000000000 (MJPG)default frame: 2aspect ratio: 0x0interlace flags: 00copy protect: 00FrameDescriptor(1)capabilities: 00size: 960x544bit rate: 24576000-147456000max frame size: 1044480default interval: 1/29interval: 1/29interval: 1/25interval: 1/23interval: 1/14FrameDescriptor(2)capabilities: 00size: 1024x576bit rate: 24576000-147456000max frame size: 1179648default interval: 1/29interval: 1/29interval: 1/25interval: 1/23interval: 1/14FrameDescriptor(3)capabilities: 00size: 1280x720bit rate: 24576000-147456000max frame size: 1843200default interval: 1/29interval: 1/29interval: 1/25interval: 1/23interval: 1/14END DEVICE CONFIGURATIONbmHint: af90bFormatIndex: 5bFrameIndex: 157dwFrameInterval: 32766wKeyFrameRate: 0wPFrameRate: 0wCompQuality: 0wCompWindowSize: 0wDelay: 4176dwMaxVideoFrameSize: 0dwMaxPayloadTransferSize: 4197312bInterfaceNumber: 254get_mode: Invalid mode (-51)Device closedUVC exitedroot@iJudge:/home/pantheron/libuvc#

author: Pantheron
author_rank: Contributor
timestamp: September 02, 2015, 12:44:08 am
content: Okay,it tooks the isight camera of the macbook...So i added the vendor and product id directly into the code for the flir e4 and now it works

author: Bud
author_rank: Super Contributor
timestamp: September 02, 2015, 01:04:03 am
content: Good point. Can you post the changed code line , thks.

author: Bud
author_rank: Super Contributor
timestamp: September 02, 2015, 05:36:16 am
content: Here is a version that streams data in a uncompressed 8-bit grayscale AVI file. It can then be opened and colorized in ImageJ.

author: tomas123
author_rank: Frequent Contributor
timestamp: September 02, 2015, 06:47:34 am
content: Quote from: Bud on September 02, 2015, 01:04:03 amCode:res = uvc_find_device(ctx, &dev,0, 0, NULL);change 0,0 to vid,pidhttps://int80k.com/libuvc/doc/group__device.htmluvc_error_tuvc_find_device( uvc_context_t * ctx,uvc_device_t ** dev,int vid,int pid,const char * sn)Finds a camera identified by vendor, product and/or serial number.Parameters ctx UVC context in which to search for the camera dev Reference to the camera, or NULL if not found vid Vendor ID number, optional pid Product ID number, optional sn Serial number or NULL

author: Bud
author_rank: Super Contributor
timestamp: September 02, 2015, 07:13:41 am
content: Thanks tomas123 for your input and information/files in your previous post!

author: Bud
author_rank: Super Contributor
timestamp: September 04, 2015, 06:45:33 am
content: Here is a version that supports both RAW 16-bit and grayscale 8-bit streaming to fileUsage:./flirfox4 :Display video, no streaming to file./flirfox4 -v :same as above./flirfox4 -r filename :stream 16-bit RAW thermal video to file, no header./flirfox4 -g filename :stream 8-bit grayscale auto-leveled video to file with AVI format headerBoth can be opened in ImageJ program, color table applied and can be saved in a variety of formats.EDIT: and it also has E4 PID/VID hardcoded, so it only be looking for E4 camera.EDIT: do not expect well written code from me. First, I am Linux impaired. Second, I heard about C language that there is such thing and that is about it. Third, I am not a programmer.

author: Bud
author_rank: Super Contributor
timestamp: September 05, 2015, 08:58:00 am
content: Temperature calculation added. Try it folks if it works for you.You will need a file called e4data.txt with your camera's calibration data. You create the file using exiftool as tomas123 explained here:https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg744673/#msg744673You also need to make a sample image from your camera, exiftool extracts the data from it (google for exiftool download if you do not have it).Run exiftool command against the sample E4 file:Code:exiftool -Atmos* -Planck* -Object* -Emis* -Reflected* -Relativ* FLIR_sample.jpg >e4data.txtOnce made and inspected, place the e4data.txt file in the folder with the compiled executable. For now the program just displays temperature (at mouse pointer coordinates) in the top left corner. Compare it with readings on your camera by pointing the mouse and camera's crosshair to a same hot spot.Thanks tomas123 for the formulas!

author: tomas123
author_rank: Frequent Contributor
timestamp: September 05, 2015, 02:32:35 pm
content: great workunfortunately now I have not enough time to go deeper in your codea fast check withUbunturesults in a "nan" value for temperature (see screenshot)Code:$ exiftool -Atmos* -Planck* -Object* -Emis* -Reflected* -Relativ* FLIR3344.jpg >e4data.txt$ cat e4data.txtAtmospheric Temperature : 20.0 CPlanck R1 : 13061.76Planck B : 1352.2Planck F : 2.7Planck O : -7334Planck R2 : 0.027282318Object Distance : 1.00 mEmissivity : 0.95Reflected Apparent Temperature : 20.0 CRelative Humidity : 50.0 %$ /usr/bin/cc -o flirfox6 flirfox6.c -luvc `pkg-config --cflags --libs opencv` -lm$ sudo ./flirfox6I will tomorrow debug your nice code.another note:I compiled successful the libuvc patch on macOS X 10.9.my raw note for interested users (use macports):Code://check$ port installed | grep activecmake @3.3.1_0 (active)libusb @1.0.19_0 (active)jpeg @9a_1 (active)$ git clone https://github.com/ktossell/libuvcCloning into 'libuvc'...remote: Counting objects: 526, done.remote: Total 526 (delta 0), reused 0 (delta 0), pack-reused 526Receiving objects: 100% (526/526), 263.90 KiB | 309.00 KiB/s, done.Resolving deltas: 100% (305/305), done.Checking connectivity... done.$ cd libuvc/// now use patch in the src folder$ patch < libuvc.patchpatching file stream.ccd ..$ mkdir build$ cd build$ cmake ..-- The C compiler identification is AppleClang 5.1.0.5030040-- The CXX compiler identification is AppleClang 5.1.0.5030040-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works-- Detecting C compiler ABI info-- Detecting C compiler ABI info - done-- Detecting C compile features-- Detecting C compile features - done-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works-- Detecting CXX compiler ABI info-- Detecting CXX compiler ABI info - done-- Detecting CXX compile features-- Detecting CXX compile features - done-- No build type selected, default to Release-- No target type selected, default to shared library-- Found JPEG: /opt/local/lib/libjpeg.dylib-- Building libuvc with JPEG support.-- Configuring doneCMake Warning (dev):Policy CMP0042 is not set: MACOSX_RPATH is enabled by default. Run "cmake--help-policy CMP0042" for policy details. Use the cmake_policy command toset the policy and suppress this warning.MACOSX_RPATH is not specified for the following targets:uvcThis warning is for project developers. Use -Wno-dev to suppress it.-- Generating done-- Build files have been written to: /Users/jfried/Documents/Flir/uvc/libuvc/build$ makeScanning dependencies of target uvc[ 10%] Building C object CMakeFiles/uvc.dir/src/ctrl.c.o[ 20%] Building C object CMakeFiles/uvc.dir/src/ctrl-gen.c.o[ 30%] Building C object CMakeFiles/uvc.dir/src/device.c.o[ 40%] Building C object CMakeFiles/uvc.dir/src/diag.c.o[ 50%] Building C object CMakeFiles/uvc.dir/src/frame.c.o[ 60%] Building C object CMakeFiles/uvc.dir/src/init.c.o[ 70%] Building C object CMakeFiles/uvc.dir/src/stream.c.o[ 80%] Building C object CMakeFiles/uvc.dir/src/misc.c.o[ 90%] Building C object CMakeFiles/uvc.dir/src/frame-mjpeg.c.o[100%] Linking C shared library libuvc.dylib[100%] Built target uvcthe RAW streaming of Flir E4 works fine under OS X (sample code from tmbinc)Code:// fast check with my full pathes$ cc -o flir flir.c -I/opt/local/include/ -I/Users/tomas/Documents/Flir/uvc/libuvc/build/include -I/Users/tomas/Documents/Flir/uvc/libuvc/include/ -L/Users/tomas/Documents/Flir/uvc/libuvc/build -luvc$ ./flirUVC initializedusage: ./flir out.raw./flir 1UVC initializedDevice foundDevice openedDEVICE CONFIGURATION (09cb:1007/) ---Status: idleVideoControl:bcdUVC: 0x0100VideoStreaming(1):bEndpointAddress: 130Formats:UncompressedFormat(1)bits per pixel: 16GUID: 5955593200001000800000aa00389b71 (YUY2)default frame: 1aspect ratio: 0x0interlace flags: 00copy protect: 00FrameDescriptor(1)capabilities: 03size: 320x240bit rate: 912384-912384max frame size: 153600default interval: 1/3interval: 1/15interval: 1/7interval: 1/3END DEVICE CONFIGURATIONbmHint: 0000bFormatIndex: 1bFrameIndex: 1dwFrameInterval: 666666wKeyFrameRate: 0wPFrameRate: 0wCompQuality: 0wCompWindowSize: 0wDelay: 1dwMaxVideoFrameSize: 153600dwMaxPayloadTransferSize: 157696bInterfaceNumber: 1Streaming...got frame 1441305922:212093, 153600 bytesgot frame 1441305922:308068, 153600 bytesgot frame 1441305922:404825, 153600 bytesgot frame 1441305922:500120, 153600 bytesgot frame 1441305922:594571, 153600 bytes...now I have some trouble with OpenCV under OS X ...

author: Bud
author_rank: Super Contributor
timestamp: September 05, 2015, 09:45:47 pm
content: @tomas123Oh-oh, seems a problem with exiftool outputting less data when run in linux vs windows. I ran exiftool in Windows, it dumps 15 parameters in total. Yours seems to dump only 10, that is what caused the nan error.I will check what version of exiftool executable i have in windows.

author: Bud
author_rank: Super Contributor
timestamp: September 06, 2015, 01:42:38 am
content: So my exiftool Windows version is 9.9, this is what it outputs:Code:Atmospheric Temperature : 20.0 CAtmospheric Trans Alpha 1 : 0.00656Atmospheric Trans Alpha 2 : 0.01262Atmospheric Trans Beta 1 : -0.0022Atmospheric Trans Beta 2 : -0.0066Atmospheric Trans X : 1.90000Planck R1 : 14901.4Planck B : 1400.9Planck F : 2.5Planck O : -7187Planck R2 : 0.02804Object Distance : 1.00 mEmissivity : 0.95Reflected Apparent Temperature : 20.0 CRelative Humidity : 50.0 %@tomas123: seems yours does not output alpha/beta/X, not sure if this is how exiftool works in LinuxIn the program I used the formulas with Distance from your Excel spreadsheet, that required I believe all 15 entries.BTW, the last version is just using atmospheric temperature, object distance, reflected temperature and humidity read from the sample file. I intended later to replace them with sliders added to the video window so it would be possible to adjust for the current scene, since this information is not in the video stream.

author: Bud
author_rank: Super Contributor
timestamp: September 06, 2015, 06:44:23 am
content: Latest version,Temperature label moved to its own overlay so it will not appear in recorded filesA crosshair cursor is added to read temperature, use the mouse to drag the cursor. Left mouse Double click resets position to the image center.The cursor appears in 8-bit grayscale video files, won't appear in 16-bit RAW video files.Added sliders to adjust a few ambient parameters .For now I reverted to a fixed size window, as the sliders eat window space, shrinking the image. Would probably add a command line switch later to allow for window resize.

author: tomas123
author_rank: Frequent Contributor
timestamp: September 06, 2015, 09:58:33 am
content: Quote from: tomas123 on September 05, 2015, 02:32:35 pmsorry, that I'm wasted your timewith this steps on my virtual machineCode:$ lsb_release -aDescription:Ubuntu 14.04.1 LTS$ sudo apt-get install exiftoolNote, selecting 'libimage-exiftool-perl' instead of 'exiftool'$ sudo apt-get install libimage-exiftool-perl$ exiftool -ver9.46I got an old exiftool version from Jan. 11, 2014: "Version 9.46 production release" (without the values for atmosphere transmissivity)... and to make matters worse, I'm a coder of the Flir part of exiftoolI downloaded the current version 10.0.1 from Phils homepageand now all works fineIf you prefer to parse XML in C than you can use the X parameter in exiftool:Code:Image-ExifTool-10.01/exiftool -X -Atmos* -Planck* -Object* -Emis* -Reflected* -Relativ* FLIR3344.jpg<?xml version='1.0' encoding='UTF-8'?><rdf:RDF xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#'><rdf:Description rdf:about='FLIR3344.jpg'xmlns:et='http://ns.exiftool.ca/1.0/' et:toolkit='Image::ExifTool 10.01'xmlns:FLIR='http://ns.exiftool.ca/APP1/FLIR/1.0/'><FLIR:AtmosphericTemperature>20.0 C</FLIR:AtmosphericTemperature><FLIR:AtmosphericTransAlpha1>0.006569</FLIR:AtmosphericTransAlpha1><FLIR:AtmosphericTransAlpha2>0.012620</FLIR:AtmosphericTransAlpha2><FLIR:AtmosphericTransBeta1>-0.002276</FLIR:AtmosphericTransBeta1><FLIR:AtmosphericTransBeta2>-0.006670</FLIR:AtmosphericTransBeta2><FLIR:AtmosphericTransX>1.900000</FLIR:AtmosphericTransX><FLIR:PlanckR1>18453.355</FLIR:PlanckR1><FLIR:PlanckB>1460.6</FLIR:PlanckB><FLIR:PlanckF>1</FLIR:PlanckF><FLIR:PlanckO>-7003</FLIR:PlanckO><FLIR:PlanckR2>0.013993904</FLIR:PlanckR2><FLIR:ObjectDistance>1.00 m</FLIR:ObjectDistance><FLIR:Emissivity>0.95</FLIR:Emissivity><FLIR:Emissivity>0.95</FLIR:Emissivity><FLIR:ReflectedApparentTemperature>20.0 C</FLIR:ReflectedApparentTemperature><FLIR:RelativeHumidity>50.0 %</FLIR:RelativeHumidity></rdf:Description></rdf:RDF>

author: tomas123
author_rank: Frequent Contributor
timestamp: September 06, 2015, 04:28:59 pm
content: Because there are some minor differences between the temperature of your crosshair cursor and the E4 screen I rechecked my excel sheet with a sample image.(1) load image in Flir Tools and export the temperature off all pixels in a csv file(2) with exiftool and imagemagick export the raw values of some pixels(3) with Excel compare calculated temperature with exported values from Flir ToolsThe values in Flir Tools and my Excel sheet are identical with all digits (milliKelvin)The calculations in your code are also right. I checked it with:Code:185: sprintf(szText, "%3.1f C / RAW: %i",t, pix[cpY * 320 + cpX]);The temperature differences under crosshair I can't explain with averaging of 2x2 or 3x3 pixels on the E4 displayall steps in bashCode:// export with Flir Tools the temperature of all Pixels (Flir Tools export as *.csv)$ head -n11 FLIR0080.csvFile: ;C:\Users\admin\Desktop\1\FLIR0080.jpgParameters:;Emissivity:;0,95;Refl. temp.:;20 °C;Distance:;1 m;Atmospheric temp.:;20 °C;Ext. optics temp.:;20 °C;Ext. optics trans.:;1;Relative humidity:;50 %;Frame 1;18,359;18,373;18,373;18,359;18,387;18,429;18,401;18,457;18,457;18,443;18,443;18,429;18,415;18,415;18,401;18,401;18,401;18,387;18,387;18,373;18,359;18,345;18,331// print a readable forms jfried$ grep Frame FLIR0080.csv | tr ";" "\n" | head -n 17Frame 118,35918,37318,37318,35918,38718,42918,40118,45718,45718,44318,44318,42918,41518,41518,40118,401// get Flir parameters for excel sheet$ /Image-ExifTool-10.01/exiftool -Atmos* -Planck* -Object* -Emis* -Reflected* -Relativ* FLIR0080.jpgAtmospheric Temperature : 20.0 CAtmospheric Trans Alpha 1 : 0.006569Atmospheric Trans Alpha 2 : 0.012620Atmospheric Trans Beta 1 : -0.002276Atmospheric Trans Beta 2 : -0.006670Atmospheric Trans X : 1.900000Planck R1 : 15119.786Planck B : 1405.2Planck F : 2.5Planck O : -7652Planck R2 : 0.027600825Object Distance : 1.00 mEmissivity : 0.95Reflected Apparent Temperature : 20.0 CRelative Humidity : 50.0 %// export RAW values of radiometric image$ exiftool -b -RawThermalImage FLIR0080.jpg > FLIR0080.png// swap bytes$ convert -define png:swap-bytes=on FLIR0080.png FLIR0080swap.png// print RAW values of first 16 Pixel (top left)$ for i in $(seq 0 15); do convert FLIR0080swap.png -crop 1x1+$i+0 -format "%[fx:round(65535*u.r)]" info:;echo ; done12167121681216812167121691217212170121741217412173121731217212171121711217012170a pixel around the center of image (the crosshair)$ convert FLIR0080swap.png -crop 1x1+160+120-format "%[fx:round(65535*u.r)]" info:12644=> 24,785 °C (see imprinted spot 24,8°C)see my links in signature for detailsandhttps://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg350250/#msg350250

author: Bud
author_rank: Super Contributor
timestamp: September 06, 2015, 07:26:00 pm
content: Thanks tomas123. Could the difference be because of rounding errors in Linux math, the formulas have power of 2 and 3 as well as square roots, so rounding errors may accumulate. I think what can be done is to run the temp calculation routine with a known input value and check results of each step in the routine against your spreadsheet. This way we can catch where the difference occurs.

author: Bud
author_rank: Super Contributor
timestamp: September 06, 2015, 07:32:25 pm
content: @tomas123That is interesting the doctored libuvc works on Mac OS. Nice. I guess may also work on Win10, i heard it overcomes the limitations that prevented libuvc from working with camera streaming in Win7.

author: tomas123
author_rank: Frequent Contributor
timestamp: September 06, 2015, 09:42:23 pm
content: The differences are only visible in rare situations at higher temperatures and only small (<0,5K).I'm not sure, but it doesn't look like a average of 4x4 pixels.Also this differences are not rounding errors.It's possible, that the E4 raw video stream contains less "magic Flir" post procession filters like the radiometric images (jpg).

author: Bud
author_rank: Super Contributor
timestamp: September 07, 2015, 05:58:06 am
content: Must be some secret sauce in E4, since temperature calculation in the program matches pretty well your Excel spreadsheet, I checked for a pixel level 10000:T_Obj -8.397801 -8.397789Anyway, attached is the version with MJPG compression added ( -m switch, will save 8-bit MJPG compressed file).I will probably stop here for now.EDIT: I think this program with some additions may be useful as a data logger for rapidly changing temperatures, since readings are available for each frame. Even this version can be used for that. Write video to a 8-bit file, open the file in any video viewer and step through the frames to read temperature at the cursor.

author: tomas123
author_rank: Frequent Contributor
timestamp: September 07, 2015, 08:49:28 am
content: Quote from: Bud on September 07, 2015, 05:58:06 amYou've done a great jobI like your solution for temperature measurement under the free movable crosshair.I did not have enough time to work with my raspberry pi (spycam for animals).The OpenVC window is not the best solution for a ssh remote session disconnected with screen.As a next step, I'll try to send the 8 Bit video stream to a v4l2loopback device (a video for linux loopback device like /dev/video1).PS: I wasted many time for running your c code under Mac OS X.The OpenCV library under OS X is a pain. I have unresolved problems with the OpenCV heap inside the function cb(frame,ptr) and the Qt library.I think the best solution is switching to the newer opencv c++ apiBut I give upAs I wrote here, the patched libuvc library works stable under OS X:https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg747569/#msg747569

author: Bud
author_rank: Super Contributor
timestamp: September 08, 2015, 02:55:07 am
content: Yes I did not exactly like OpenCV but was easier for an inexperienced person like me to come up with some code. I run Ubuntu 14 on a Win7 guest using Oracle VirtualBox (32Gb RAM) and OpenCV appears to be glitchy, not sure if this is because it runs on a VM. It also seems to write 9 fps video with a 15fps header which causing the video to play too fast. Not sure how it ends up with 9 fps.Anyway, this is what I call "better than nothing" and I tried to summarize the steps below:The purpose of this topic and this program was to get FLIR E4 camera to stream RAW thermal 16-bit video that can be written in a file as uncompressed 16-bit grayscale video for further processing elsewhere. As a by side feature, the program can display a rescaled 8-bit video in a window and can also write 8-bit grayscale uncompressed and MJPG compressed files that include temperature measurement under a spot cursor that can be moved/set in the program using the mouse. The program also have a few adjustment controls that allow to change several environmental parameters and obtain corrected temperature measurements in real time.The program seems to only work in Linux at this time because the underlying libraries (libuvc and OpenCV) seem to not provide required functionality in Win7 or MacOS.Steps to get it running in Linux (I used Ubuntu 14):Download libuvc source and patch it as in this post from tomas123:https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729604/#msg729604(use the patch file provided by user tmbinc in the zip archive attached to the very first message in this topic)Download and Compile OpenCV library as in this post from tomas123 (step 2)https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg742931/#msg742931Compile the program codegcc -o flirfox9 flirfox9.c -luvc -lm `pkg-config --cflags --libs opencv`Take a sample image from your camera, use exiftool to extract calibration constants for your camera from it (google for exiftool download if you do not have it).Run exiftool command against the sample E4 file:exiftool -Atmos* -Planck* -Object* -Emis* -Reflected* -Relativ* FLIR_sample.jpg >e4data.txtPlace the produced e4data.txt file in the folder with the compiled executable.Connect the camera and run the program./flirfox9 Options:-r filename :write 16-bit RAW data to file-g filename :write 8-bit grayscale data to file-m filename :write 8-bit grayscale MJPG compressed data to file-v :only display 8-bit grayscale videoUse the mouse to click-n-drug the temperature cursor to the desired position. left mouse dbl-click resets the cursor to the image center.For post processing you may want to take a look at a nice piece of software called ImageJhttp://imagej.nih.gov/ij/It has bunch of image processing options plus bunch of color palettes (LUTs) to color the video and resave in various formats.

author: tomas123
author_rank: Frequent Contributor
timestamp: September 08, 2015, 01:56:49 pm
content: a great tutorialI edited my entry post on the first page and added a link list for new readers:https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729604/#msg729604

author: tomas123
author_rank: Frequent Contributor
timestamp: September 16, 2015, 10:20:47 pm
content: some results from playing around with the code from tmbinc:hardware: a raspberry pi 2 with a 5 inch hdmi display from wavesharehttp://www.waveshare.com/wiki/5inch_HDMI_LCDraspberry image: 2015-05-05-raspbian-wheezy.imgfirst installpatched uvclibto get the raw video stream from a Flir Exhttps://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729604/#msg729604https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg623251/#msg623251I changed the source from tmbinc for a 32 bit framebuffer...https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg736344/#msg736344...for my display with 800x480 in16 Bitdepth (RGB565) and added some comments for using the framebuffer.For better viewing I rescaled the video to 640x480.source: flirfox2x.ccompile withCode:/usr/bin/cc -o flirfox2x ./flirfox2x.c -luvc -lmusingCode:$ sudo ./flirfox2x out.raw 20000UVC initializedDevice foundDevice openedDEVICE CONFIGURATION (09cb:1007/) ---Status: idleVideoControl:bcdUVC: 0x0100VideoStreaming(1):bEndpointAddress: 130Formats:UncompressedFormat(1)bits per pixel: 16GUID: 5955593200001000800000aa00389b71 (YUY2)default frame: 1aspect ratio: 0x0interlace flags: 00copy protect: 00FrameDescriptor(1)capabilities: 03size: 320x240bit rate: 912384-912384max frame size: 153600default interval: 1/3interval: 1/15interval: 1/7interval: 1/3END DEVICE CONFIGURATIONbmHint: 0000bFormatIndex: 1bFrameIndex: 1dwFrameInterval: 666666wKeyFrameRate: 0wPFrameRate: 0wCompQuality: 0wCompWindowSize: 0wDelay: 1dwMaxVideoFrameSize: 153600dwMaxPayloadTransferSize: 157696bInterfaceNumber: 1Streaming... got frame 1442439612:475769, 153600 bytes, [12419..14340 rms 12865.158203 rmax 143] got frame 1442439612:564221, 153600 bytes, [12415..14340 rms 12864.794922 rmax 284] got frame 1442439612:666238, 153600 bytes, [12416..14339 rms 12864.424805 rmax 424] got frame 1442439612:764188, 153600 bytes, [12417..14342 rms 12864.861328 rmax 563] got frame 1442439612:866825, 153600 bytes, [12417..14339 rms 12864.305664 rmax 700]you can clean your screen from frame buffer output withCode:xrefresh -d :0.0next step is writing the video to aV4L2 loopback devices ( /dev/video0 )https://github.com/umlaeute/v4l2loopbackThe installation of v4l2loopback on a raspberry 2 with 2015-05-05-raspbian-wheezy.img was tricky, because there are no sources included for building kernel modules.(no /lib/modules/$(uname -r)/build )All downloads to get the kernel sources for 3.18.7-v7+ failsI found here a working solution with roll back from 3.18.7-v7+ to the older kernel 3.18.0https://github.com/umlaeute/v4l2loopback/issues/84Code:sudo aptitude install -y linux-image-rpi2-rpfv linux-headers-rpi2-rpfvecho -e "kernel=vmlinuz-3.18.0-trunk-rpi2\ninitramfs=initrd.img-3.18.0-trunk-rpi followkernel" | sudo tee -a /boot/config.txtsudo rebootthe next steps for installing v4l2loopback are simpleCode:$ uname -aLinux raspberryLCD 3.18.0-trunk-rpi2 #1 SMP PREEMPT Debian 3.18.5-1~exp1+rpi19 (2015-08-08) armv7l GNU/Linux$ git clone https://github.com/umlaeute/v4l2loopback$ cd v4l2loopback$ makeBuilding v4l2-loopback driver...make -C /lib/modules/`uname -r`/build M=/home/pi/v4l2loopback/v4l2loopback modulesmake: Entering directory '/usr/src/linux-headers-3.18.0-trunk-rpi2'Makefile:10: *** mixed implicit and normal rules: deprecated syntaxCC  /home/pi/v4l2loopback/v4l2loopback/v4l2loopback.oBuilding modules, stage 2.MODPOST 1 modulesCC /home/pi/v4l2loopback/v4l2loopback/v4l2loopback.mod.oLD  /home/pi/v4l2loopback/v4l2loopback/v4l2loopback.komake: Leaving directory '/usr/src/linux-headers-3.18.0-trunk-rpi2'$ sudo su# make installmake -C /lib/modules/`uname -r`/build M=/home/pi/v4l2loopback/v4l2loopback modules_installmake: Entering directory '/usr/src/linux-headers-3.18.0-trunk-rpi2'Makefile:10: *** mixed implicit and normal rules: deprecated syntaxINSTALL /home/pi/v4l2loopback/v4l2loopback/v4l2loopback.koDEPMOD 3.18.0-trunk-rpi2make: Leaving directory '/usr/src/linux-headers-3.18.0-trunk-rpi2'depmod -a `uname -r`$ sudo modprobe v4l2loopback$ lsmodModule Size Used byv4l2loopback 23372 0...source: flirfox-loop.ccompileCode:/usr/bin/cc -o flirfox-loop ./flirfox-loop.c -luvc -lmI simple mixed tmbinc source with the v4l2loopback test.c filesource:https://github.com/umlaeute/v4l2loopback/blob/master/examples/test.cusingCode:$ sudo ./flirfox-loop out.raw 20000UVC initializedDevice foundDevice openedDEVICE CONFIGURATION (09cb:1007/) ---Status: idleVideoControl:bcdUVC: 0x0100VideoStreaming(1):bEndpointAddress: 130Formats:UncompressedFormat(1)bits per pixel: 16GUID: 5955593200001000800000aa00389b71 (YUY2)default frame: 1aspect ratio: 0x0interlace flags: 00copy protect: 00FrameDescriptor(1)capabilities: 03size: 320x240bit rate: 912384-912384max frame size: 153600default interval: 1/3interval: 1/15interval: 1/7interval: 1/3END DEVICE CONFIGURATIONbmHint: 0000bFormatIndex: 1bFrameIndex: 1dwFrameInterval: 666666wKeyFrameRate: 0wPFrameRate: 0wCompQuality: 0wCompWindowSize: 0wDelay: 1dwMaxVideoFrameSize: 153600dwMaxPayloadTransferSize: 157696bInterfaceNumber: 1using output device: /dev/video0vid_format->type =2vid_format->fmt.pix.width =320vid_format->fmt.pix.height =240vid_format->fmt.pix.pixelformat =1497715271vid_format->fmt.pix.sizeimage =76800vid_format->fmt.pix.field =1vid_format->fmt.pix.bytesperline=320vid_format->fmt.pix.colorspace =8Streaming... got frame 1442439057:790436, 153600 bytes, [12283..12491 rms 12423.203125 rmax 124] got frame 1442439057:887940, 153600 bytes, [12284..12485 rms 12422.550781 rmax 247] got frame 1442439057:988181, 153600 bytes, [12280..12489 rms 12420.328125 rmax 369] got frame 1442439058:88068, 153600 bytes, [12279..12486 rms 12418.430664 rmax 490] got frame 1442439058:187958, 153600 bytes, [12281..12484 rms 12418.650391 rmax 609] got frame 1442439058:297959, 153600 bytes, [12281..12481 rms 12419.558594 rmax 727]You can start flirfox-loop in the background (as sample a screen session) and log out.The program writes the screen video to /dev/video0 device.From this point you can use all video players orwebcam programsto pass the video from /dev/video0 to a html page etc...As sample screenshot from assh session with x11 forwarding the mplayerCode:$ ssh -X pi@192.168.1.70$ mplayer tv:// -tv driver=v4l2:device=/dev/video0MPlayer svn r34540 (Raspbian), built with gcc-4.6 (C) 2000-2012 MPlayer Teammplayer: could not connect to socketmplayer: No such file or directoryFailed to open LIRC support. You will not be able to use your remote control.Playing tv://.TV file format detected.Selected driver: v4l2name: Video 4 Linux 2 inputauthor: Martin Olschewski <olschewski@zpr.uni-koeln.de>comment: first try, more to come ;-)v4l2: your device driver does not support VIDIOC_G_STD ioctl, VIDIOC_G_PARM was used instead.Selected device: Dummy video device (0x0000)Capabilities: video capture read/write streamingsupported norms:inputs: 0 = loopback;Current input: 0Current format: GREY

author: tomas123
author_rank: Frequent Contributor
timestamp: September 16, 2015, 10:35:15 pm
content: unfortunately I have some trouble with charging the Ex with a raspberry 2the max_usb_current switch doesn't work:Code:$ sudo nano /boot/config.txtsafe_mode_gpio=4max_usb_current=1the only solution for me was using a USB hub with a power supply

author: Bud
author_rank: Super Contributor
timestamp: September 17, 2015, 03:15:36 am
content: Wow, very nice!

author: tomas123
author_rank: Frequent Contributor
timestamp: September 17, 2015, 07:55:13 am
content: try the V4L2 loopback devices underUbuntu( /dev/video0 )it's simpleCode:$ sudo apt-get install v4l2loopback-dkmsSetting up v4l2loopback-dkms (0.8.0-1) ...$ sudo modprobe v4l2loopback// check your video devices video0 / video1$ ls /dev/video*- edit flirfox-loop.c for your loopback device video0 / video1- compile flirfox-loop.c withCode:/usr/bin/cc -o flirfox-loop ./flirfox-loop.c -luvc -lmread the v4l2loopback wiki for nice sampleshttps://github.com/umlaeute/v4l2loopback/wikiGStreamerFFmpegMPlayerSkypeffserver is a nice tool for a webcam servergoogle for:ffserver /dev/video0

author: Bud
author_rank: Super Contributor
timestamp: September 18, 2015, 02:09:06 am
content: Quote from: tomas123 on September 16, 2015, 10:35:15 pmIt is not always working even with a conventional computer. I think FLIR did a poor job with implementation of the E4 USB charging circuit.

author: tomas123
author_rank: Frequent Contributor
timestamp: January 10, 2016, 10:53:44 pm
content: @adminPlease move this thread with essential informations about Flir E4 RAW streaming to the new subforum "Thermal Imaging"Thank You

author: eg14
author_rank: Contributor
timestamp: October 27, 2017, 02:34:25 am
content: Can someone please help me? I'm on Mac. Trying to stream raw data from my E30bx. the frame size is 153600 and I'm just getting a stream of what's on the display, not the raw video. Any ideas?

author: Bud
author_rank: Super Contributor
timestamp: October 27, 2017, 02:29:00 pm
content: There is a setting in the config file to enable radiometric video. If you know how to do that you can check if it is present and add it if it is not there. Not sure if it also can be set manually using rset command in Telnet, but you can try.

author: eg14
author_rank: Contributor
timestamp: October 27, 2017, 04:42:40 pm
content: I do have that enabled. I noticed in Tomas's log from his E40 he had thisCode:Streaming...got frame 1439394751:346415, 157440 bytesgot frame 1439394751:411283, 157440 bytesgot frame 1439394751:479331, 157440 byteswhereas I have thisCode:Streaming...got frame 1509121947:109894, 153600 bytesgot frame 1509121947:176200, 153600 bytesgot frame 1509121947:241827, 153600 bytesgot frame 1509121947:307965, 153600 bytesThe frames are too small to be radiometric.When it's setting up the stream I see that the radiometric option is there...Code:UncompressedFormat(2)bits per pixel: 16GUID: 46374d3000001000800000aa00389b71 (F7M0)default frame: 1aspect ratio: 0x0interlace flags: 00copy protect: 00FrameDescriptor(1)capabilities: 03size: 320x246bit rate: 912384-912384max frame size: 157440default interval: 1/7interval: 1/30interval: 1/15interval: 1/7but it doesn't seem to grab that stream.

author: eg14
author_rank: Contributor
timestamp: October 27, 2017, 06:16:37 pm
content: Making some progress. I had to checkout version 6ff39f17e3f3a26f437bc6b3c09cc6c289db7891 of libuvc. Now my frames are the correct size, but when I split them neither Imagemagick nor exiftool recognize them as being images.

author: eg14
author_rank: Contributor
timestamp: October 27, 2017, 06:43:21 pm
content: More progress. I used Tomas's split.pl script. I had to modify the magic bytes line to match mineCode:#!/usr/bin/perlundef $/;$_ = <>;$n = 100;# FLIR camera E40$pat="\x46\x46\x46\x00\x55\x56";for $content (split(/(?=$pat)/)) {open(OUT, ">seq" . ++$n . ".fff");binmode OUT;print OUT $content;close(OUT);}From my raw data file:Code:00025800 46 46 46 00 55 56 43 00 00 00 00 00 00 00 00 00 |FFF.UVC.........|00025810 00 00 00 00 64 00 00 00 40 00 00 00 02 00 00 00 |....d...@.......|00025820 03 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 |................|00025830 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|00025840 20 00 00 00 71 00 00 00 01 00 00 00 80 00 00 00 | ...q...........|now exiftool "sort of" recognizes that the frames are thermal imagesCode:ExifTool Version Number : 10.55File Name : seq102.fffDirectory : .File Size : 154 kBFile Modification Date/Time : 2017:10:27 13:32:26-05:00File Access Date/Time : 2017:10:27 13:40:12-05:00File Inode Change Date/Time : 2017:10:27 13:32:26-05:00File Permissions : rw-r--r--File Type : FLIRFile Type Extension : fffMIME Type : application/unknownCreator Software : UVCEmissivity : 0.95Object Distance : 1.00 mReflected Apparent Temperature : 20.0 CAtmospheric Temperature : 20.0 CIR Window Temperature : 20.0 CIR Window Transmission : 1.00Relative Humidity : 50.0 %Planck R1 : 14659.802Planck B : 1391.5Planck F : 1Atmospheric Trans Alpha 1 : 0.006569Atmospheric Trans Alpha 2 : 0.012620Atmospheric Trans Beta 1 : -0.002276Atmospheric Trans Beta 2 : -0.006670Atmospheric Trans X : 1.900000Camera Temperature Range Max : 120.0 CCamera Temperature Range Min : -20.0 CCamera Temperature Max Clip : 670.0 CCamera Temperature Min Clip : -40.0 CCamera Temperature Max Warn : 720.0 CCamera Temperature Min Warn : -20.0 CCamera Temperature Max Saturated: 150.0 CCamera Temperature Min Saturated: -60.0 CCamera Model : FLIR E30bxCamera Part Number : 49001-1901Camera Serial Number : 49023364Camera Software : 20.0.0Lens Model : FOL18Lens Part Number :Lens Serial Number :Field Of View : 25.0 degFilter Model :Filter Part Number :Filter Serial Number :Planck O : -6278Planck R2 : 0.011133576Raw Value Median : 18122Raw Value Range : 1124Date/Time Original : 2017:10:27 13:07:46.241-06:00Focus Step Count : 1522Focus Distance : 17.2 mRaw Thermal Image Width : 17920Raw Thermal Image Height : 21846Warning : Unrecognized FLIR RawThermalImage data formatRaw Thermal Image Type : DATRaw Thermal Image : (Binary data 153568 bytes, use -b option to extract)Peak Spectral Sensitivity : 10.3 umthis part is obviously an issueCode:Raw Thermal Image Width : 17920Raw Thermal Image Height : 21846Warning : Unrecognized FLIR RawThermalImage data formatRaw Thermal Image Type : DATRaw Thermal Image : (Binary data 153568 bytes, use -b option to extract)Peak Spectral Sensitivity : 10.3 um

author: eg14
author_rank: Contributor
timestamp: October 27, 2017, 08:46:25 pm
content: Closer.... need to figure out a way to deal with the 3840 bytes of crap at the end of each frame

author: cq-317
author_rank: Contributor
timestamp: September 16, 2018, 07:50:44 am
content: I read this post, but there is still a little understanding. Can someone help me?1.When connectinge4 wifivia usb, flir tools can choose whether to signal or video, and flir tools+ can record video directly. Does the method of modifying libuvc not apply toe4 wifi?2.After getting the raw data stream through uvc, how can I convert it into Seq file?

author: tmbinc
author_rank: Frequent Contributor
timestamp: September 17, 2018, 12:44:39 am
content: I don't have a Flir E4 Wifi. Can the Flir E4 Wifi record 14-bit raw data, as opposed to remapped (colorized) video as visible on the screen?

author: cq-317
author_rank: Contributor
timestamp: September 17, 2018, 06:56:24 am
content: Yes, if you connect e4 wifi via usb, you can choose to display 14-bit raw data or video as visible on the screen in flir tools（ver 5.0 or higher）, which looks like the same as E40 (I don't have E40), but if you want to record it as seq file, You can only use flir tools+. The seq files recorded with flir tools+ can be opened in flir tools and edited. Flir tools+ is a paid software that can be used for free for one month.

author: Bud
author_rank: Super Contributor
timestamp: September 17, 2018, 11:54:39 am
content: But this topic does not have to do with flirtools. Not sure why you are asking this question here.

author: cq-317
author_rank: Contributor
timestamp: September 17, 2018, 01:51:36 pm
content: I mean that flir tools can read the 14-bit raw data from E4 WIFI. Does the method of patch libuvc have been invalidated? Can we use the driver of flir directly to get the 14-bit raw data in e4 wifi without patch for libuvc? .

author: Bud
author_rank: Super Contributor
timestamp: September 17, 2018, 08:20:30 pm
content: Sorry i still do not understand why you bringing flir tools to this thread. The work done in this thread did not involve flir tools. Perhaps you are asking a generic question "if flir tools can do this or that", in which case you better open a separate topic in Thermal Imaging subforum.

author: tmbinc
author_rank: Frequent Contributor
timestamp: September 18, 2018, 09:46:30 pm
content: I think it's a valid question - the original "idea" of this thread was that Flir tools didn't offer to capture raw data, but yet we found a method to do so.If FLIR tools can now (I assume with a newer device firmware) do this, then yes, this deprecates this "hack". I would be curious how the FLIR tools to it - maybe they fixed the UVC descriptor?

author: Bud
author_rank: Super Contributor
timestamp: September 18, 2018, 10:45:00 pm
content: I see. I did not try fiir tools with E4 wifi. Still, this solution in this topic is much more portable, if to run on a Pi.

author: Uno winner
author_rank: Newbie
timestamp: October 16, 2018, 09:37:15 am
content: Does it work with flir i5 cam ?

author: asmart
author_rank: Newbie
timestamp: November 30, 2018, 06:49:29 am
content: For other model with radiometric IR video streaming over WiFi (such as T640), is this approach also works for RTSP connection? e.g contained raw data in each frame?

author: Spirit532
author_rank: Frequent Contributor
timestamp: December 06, 2018, 02:29:56 am
content: RTSP is a compressed MJPEG stream, so no, it wouldn't work.You need USB.