yashsingh1/xdaforums huggingface dataset:
[General Accessories] CommanderROR:
Hi everyone,

Thermal imaging has been a long-time dream for me, and I guess for many other as well. However, even the most simple TI cameras tend to be too expensive to justify if you want then just for private use.

It seems like their might be a way out of that now...

A Company called Melexis has released a cheap and low-resolution Thermopile Array (Melexis MLX 90620), that could be the ideal base for an extremely basic Thermal Camera. It's 16x4 Pixel Sensor won't win any prizes for building inspection or detailed thermal analysis, but it is cheap (if you buy a single unit it's around 60$ from Digikey or Futureelectronics, should be in stock this week at Futureelectronics according to Melexis) and comes complete with built-in signal amp, RAM, EEPROM and digital I2C output.

I came across this idea through the Kickstarter Project "Sensordrone" that gives you an easy way to hook up Sensors to your Smartphone via Bluetooth. They are planning to build this TI Sensor as an addon Module to their Sensordrone, but they need somebody to help build the app that turns the raw data coming from the sensor into colourful pixels... ;)

Connecting the Thermopile Array to the Sensordrone is as easy as connecting the 4 pins to the Addon Connector pins. Adding a simple Silicon Diode to reduce the power from 3.3v down to the recommended 2.6v is the only additional step.

You could also hook the Thermopile Array up to a standard Arduino or the Android ADK board I guess if you don't want to go for the Sensordrone.

So it all comes down to Software, and this is where I hope somebody in this fantastic community might help. If you would like to see this happen and (unlike me) have the required coding skills, just go ahead, contact the Sensorcon Team (developers of the Sensordrone) or build your own ADK kit with Thermal Imaging.

TecSpecs for the Melexis MLX90620 can be found on their website together with a demo video. The Sensordrone can be found on Kickstarter (just enter the name) and specs for their unit are available there or on the Sensorcon forums.

I hope somebody takes this up and makes my (and I hope many others') long time dream real! Once the cheap and low-res version works, going for the more high-resolution Heimann Thermopile arrays with 32x31 and 64x62 might be the next step to an affordable Smartphone Thermocam.
[General Accessories] Mardaso:
I had the same idea and ordered the sensor a few days ago.

Got a message from Future electronics that they get the sensor on the 23rd and than it will take 5 to 7 days for me to receive it.

So I will have to wait a little bit longer and than start programming.

I have some ideas to combine it with the phone camera to make it a real thermographic camera.

I don't know if i will use the Sensordrone or a standalone bluetooth solution. With the later I have more control over the firmware.
[General Accessories] CommanderROR:
Thanks for replying Mardaso.

The idea of combining it with the smartphone cmos camera is great, I had thought about that as well, but am not sure how it could be achieved. You'd either have to synchronize the FOV, which is pretty much impossible given the parameters of the Melexis module or you'd have to track the area the sensor is "scanning" with the smartphone and overlay the image then. This might work with LEDs or laser dots, but it would be difficult.

But maybe I'm missing something here and it might not be that complicated after all.

My idea for increasing the effective resolution was in using motion sensors and "image stitching" to create a larger and more detailed image with the low-resolution sensor. This would work much like the "panoramic" feature in Android smartphones. It works perfectly for my Nexus. This might be rather challenging to program though...but maybe also worth a try.

I think this would maybe all be relevant for later on if/when the basic functionality is there... ;)

I'm hoping you'll give the Sensordrone a try since I ordered one and would like to see Theraml Imaging on it ASAP...or if you go with a standalone Bluetooth module, would you provide your sourcecodes for the app so that it can be easily adapted for use with the Sensordrone?

The main advantage of the Sensordrone from a dev. point of view is that it takes care of the low-level stuff for you. You'd get the required raw data from the sensor easily and could concentrate on the features of the app...

Maybe you could contact Sensorcon, the creators of the Sensordrone, and see whether you'd like to work with them on this project. They are really nice guys. Are you going to do the initial work on an Arduino based ADK btw?

Whatever you do, I'm glad that you're picking this up. Your work will be useful in bringing a "world's first thermal imaging device" to the Smartphone. Since the Sensorcon folks are also in contact with Heimann Sensors for a later-to-come higher resolution version of the the TI module there is definitely a future for this and your software could be the basis for all that...:good:

I'd like to offer my assisitance, but since I am absolutle yno good at programming I'll probably not be able to do anything useful...:rolleyes:
[General Accessories] KRAZYADROIDMASTER:
That would be really cool!!!

Can the Bluetooth handle that much data???
[General Accessories] CommanderROR:
Bluetooth should not be a problem. Even BT 2.1 has about 3Mbps capacity. With the low resolution of the Thermopile arrays that is a non-issue.

@Mardaso

which FOV option die you order? 40 degree or 60?
[General Accessories] Mardaso:
I ordered: MLX90620ESF-BAB-000-TU so the 60 degree version. They had only this one listed and the evaluation board.

I want to start programming asap. So when the sensors arrive I will start first with a controller (arduino or pic(32)) sending data serial to a PC.

After that go for bluetooth and an android app.

When looking at the data sheet there is a lot of calculation to be done before you get the actual temperature. These calculations could be handled by the Pic32 processor. But we could always use the sensordrone as a gateway for the serial data.

I want to use the motion sensors from the phone to "stitch" the data.

Maybe we will need some kind of holder to align the thermo sensor and the camera pictures so you won't need an extra laser or manual alignment. Just some thoughts.
[General Accessories] CommanderROR:
Futureelectronics seems to have the 40 deg sensor listed now. I'm not sure, but I think that might be better for image stitching...not sure though.

Aligning the sensor with the phone for motion sensor useage...I'd probably go with a bit of double-sided sticky tape... ;)

At least for initial trials that could work well. It would not overlay the "real" image with the Thermal image but would create a more high-res thermal image. That might be easier as a start.

About the calculations...I read that data sheet too, but I'm not sure what is done in the sensor itself and what you have to do in software. It's rather confusing in some passages... ;)

As far as I can tell, calibration values are stored in the EEPROM and can be changed (but I would leave those well alone) and the 65 data packages are written to RAM from where they can be read. 64 temp readings for T(Object) and 1 for T(Case). I'll try to figure out what I can from the Spec sheet, but it's not exaclty my field of expertise...^^

If it gets really complicated you could always ask Melexis for advice. They are really helpful.

The Output from the Sensor should be handled as I2C, that would be easiest and would also be the way the Sensordrone receives data. I'm thinking about getting an Arduino ADK board and a breadboard so that I can at least help with testing, but I'll wait a bit...not sure whether it's worth the money if the Sensordrone comes in October and can do the same and more...

I just reread the spec-sheet, and it seems that the calculations really are not done internally but have to be performed each time you read data from the sensor.

I'd suggest doing those calculations in the app rather than in the microcontroller since that would make things fully compatible with the Sensordrone which as far as I know only passed the data through.

Btw...have you thought about refresh rates? The default is 1Hz (1fps) which is ok for tryouts, but probably not very satisfying for actual use. I' suggest something like 12Hz for general use, maybe more for "painting" a larger image with the "stitching" method. However, I do not know how much the accuracy suffers then...
[General Accessories] CommanderROR:
OK...the Sensors and Evaluation Boards are now listed "in Stock", so you should receive yours soon.

I'm looking forward to following your progress.

If I understand correctly, you'd need to initiate a PTAT read and a Full Frame read to get the raw data from RAM. In addition, you'd have to read the Constants from EEPROM since you need them for calculation.

One constant is set to 25°C, I'm not sure whether the others have to be read each or if they can be stored somewhere in the app and read from there after the first initialization (since they are constants they should not change).

Values are sotred in two's format, so you have to convert them to "real" numbers before using them...my god this is complicated. I think I'll make lunch and leave the calculating bit to somebody who knows what they are doing... ;)
[General Accessories] Mardaso:
Yes! got an email that they have been shipped.
[General Accessories] CommanderROR:
Great news!

Post when it arrives, will you? And don't hook it up to 5V... ;)
[General Accessories] cybersteveUK:
MLX90620 progress

Its worse than that!

Firstly, the chip is designed to run at 2.6V, despite what is says on some (though not all) of the pages of the manual. So,

you will need to get 2.6V from somewhere - I used a LM317 variable voltage regulator. Then you will need level changers

between the 2.6 and your 3.3 or 5v of your microprocessor.

The first thing to do is to extract the contents of the eeprom that is built into the chip. Not difficult, because it is a

22AA02 Mircochip device.

I was disappointed to find that I couldn't get all the information from the EEPROM which is supposed to be there - notably

the temperature gradient coefficient (TGC) and all the "Ta dependence of IR pixels offset" and the "individual sensitivity

coefficients" which all came out as 0! As the other variables came out, I don't think this is a fault of my code.

I have no confidence in the values I did get out either.... The first thing you need to do is to get the temperature of the

chip itself (Ta), as the 64 pixel readings are relative to that. You read a value called PTAT from the Melexis and run it

through some maths. I did that and got crazy values, even when I put the configuration into a spreadsheet. So, I have put

my Melexis in a temperature controlled oven and taken a number of readings from room temperature up to 60 degrees C to

create my own formula.

The next job is to read the 64 channels and the 'compensation pixel' value. I failed to get these out in one I2C session,

but that might be my code. I get the data pixel by pixel at the moment.

If you had confidence in the configuration variables, you could then start to run this data through the 'maths' that is

well documented in the manual (section 7.4.2). There is no way that I can get that maths to run in my target MCU - when

was the last time you tried to calculate 2 to power of 42 in an 8-bit micro!

Not the easiest thing to program for!
[General Accessories] CommanderROR:
Thanks for your input!

getting the voltage down from 3.3v to 2.6 should be easy with at silicon diode, they have a drop of 0.7v built-in and cost less than 1$.

Getting it from 5v would require more work. According to the spec-sheet the sensor should work at voltages between 2.4 and 3.6v (if I remember correctly) but I'd definitely try to run it at it's optimal voltage of 2.6v

The math is indeed strange, there is one step I don't understand where they convert something and I can't figure out what it is they are converting too and from...but I'm not good at math anyway so I guess there'll be somebody out there who gets this...I'll keep looking at it though.

mardaso is the one who plans to write the app for this, if you'd be willing to join in then I'm sure this could go better and smoother. I won't be much help, but I'll try my best to assist... ;)

edit:

This is what I could not work out:

from page 16 in the spec-sheet: 256.26 + 120 = 6776 decimal value

The example hex values convert to 26 and 120 without trouble...but I have no idea how the result of the addition yields 6776

edit2: I figured it out..sort of.

run the results, for examle 6776 trhough this converter http://www.statman.info/conversions/hexadecimal.html and you'll see something interesting about the values... ;)

Doesn't really make sense to me, but it does work...
[General Accessories] CommanderROR:
@cybersteveUK

Could you post the values you get from the sensor and eeprom? Then we can all go through the data together and try to find out how to translate it into useable temp readings.

Thanks.
[General Accessories] Mardaso:
Got the sensors! I hope that I will have some time to test them next week.
[General Accessories] amnipar:
another sensordrone backer and mlx90620 owner

Hi there,

I just registered to this forum to let you know that I wish to participate in this effort too. I am also a sensordrone backer on kickstarter, and I just received my melexis mlx90620 sensor. To start with, I plan to use it with my pandaboard, which has more than enough power to handle the math. When I get the sensor working, I will try to register the sensor data with a camera to overlay the thermal data on the rgb image.

I don't own an android phone yet, but I'm planning to get a Galaxy S3 later. For using the sensor with the android phone, I guess I would start by connecting it with the sensordrone via some simple electronics and using sensordrone's bluetooth connection to send the data to the phone for processing. Later a standalone small bluetooth module that could be attached to the phone would be nice.. Registering the sensor data with the camera image would require that the module stays somewhat stationary in relation to the phone's camera.

I don't know yet when I have time to test the module, hopefully in the next couple of weeks. Meanwhile, should we set up a project and some version control in some place like github? It would be good to have some place to store the data we gather, documentation, and the code we write.
[General Accessories] CommanderROR:
Welcome on board!

Feel free to start a project on google code or similar. So far cybersteve seems to have put the most work into this, but he has not replied in this thread apart from his initial post, so I don't know if he is willing to cooperate.
[General Accessories] Mardaso:
update: Did some first testing. Ram register values looking good but eeprom readings are NOK.

@cybersteveUK Did you get any further? My eeprom readings give all 249(dec) 2s complement => -7 except address 0 = 160 and address 87 = 49. Not something I expected. Tomorrow I will take the logic analyser to see if the I2C eeprom code is ok.
[General Accessories] Mardaso:
Good and bad news.

The good news is that the code is OK and the bad news is that the eeprom of one sensor is not.

Tried it with the second sensor and all readings are as expected.

At least I can start coding but I have to see if Melexis will exchange the other sensor.
[General Accessories] CommanderROR:
Getting it exchanged should not be a problem. Futureelectronics should take it back without a problem.

I wonder if cybersteveUK also got a damaged sensor since his readings seemed to be off as well and what that says about the quality if this first batch...
[General Accessories] KRAZYADROIDMASTER:
One you get this project working, Are you going to make a YOUTUBE video of it working???

Thanks in advanced.
[General Accessories] CommanderROR:
@ Mardaso

Can you post some of the data you get from the sensor? Just the two's values of the eeprom and the ptat, compensation and at least one IR pixel value?
[General Accessories] cybersteveUK:
More on MLX90620

I now have tried 6 devices - they all have invalid eeprom values and I am just ignoring that and calibrating myself to get started. I have put the device in a temperature controlled oven from ambient to 60 degC and measured the PTAT value at a number of steps in this range. I then create a formula that relates the PTAT values to chip temperature.

Until we get real EEPROM data, there is no point in trying to implement the Melexis maths (phew...)

The values for the 64 pixels are relative to chip temperature -ve below chip temperature and +ve above. I have a black body radiator that will run from ambient to 200 deg C and beyond, so I have run the device in that and got readings of the pixels into a spreadsheet, subtracting the chip temperature value. I then created formula for that (labfit) and fed it back into my code. Add the chip temperature value and you get a reasonable value for object temperature.

Bodge, I know, but I have got to get this thing going.

I connect the UART pins of my processor to a Roving Networks bluetooth module and transmit pixel/temperature strings.

I have written a bluetooth Android application which listens to the strings and displays a 16*4 colour patch which changes colour according to the 'ironbow' colour representation scheme.

My current issue is with reading all 64 pixels in one go. I can't get this to work. I can get 64 individual reads to work. has anyone more success than me in this respect??

Here is my code - any idea why the second function doesn't work???

************** this doesn't work *******************

void readMelexisMatrix(void)

{

U8 row;

U8 col;

U16 lsb;

U16 msb;

fnI2C_SendStart();

fnI2C_SendByte(0xC0); //device address * 2

fnI2C_SendByte(2); //command 2

fnI2C_SendByte(0);

fnI2C_SendByte(1); //address step

fnI2C_SendByte(0x40); //0x40number of reads

fnI2C_SendStart();

fnI2C_SendByte(0xC1);

for (col = 0;col<16;col++)

{

for (row=0;row<4;row++)

{

lsb= fnI2C_ReceiveByte();

msb= fnI2C_ReceiveByte();

msb = msb & 0x00ff;

msb= msb<<8;

msb = msb & 0xFF00;

msb=msb + lsb;

IRvalue[((row * 16) + col)] = msb;

} //next col

} //

fnI2C_SendStop();

}

********* 64 of these does work *************************

U16	readMelexis(U8 add, U8 andprint)

{

U16 lsb;

U16 msb;

U16 reply;

fnI2C_SendStart();

fnI2C_SendByte(0xC0); //device address * 2

fnI2C_SendByte(2); //command 2

fnI2C_SendByte(add);

fnI2C_SendByte(0); //address step

fnI2C_SendByte(1); //number of reads

fnI2C_SendStart();

fnI2C_SendByte(0xC1);

lsb = fnI2C_ReceiveByte();

msb = fnI2C_ReceiveByte();

reply = (msb<<8) + lsb;

fnI2C_SendStop();

return reply;

}
[General Accessories] Mardaso:
I think I had also problems when reading more than 48 pixels.

But at that moment I had more/other problems so i will have to try again.

My MikroBasic code for this:

Sub procedure read_IR_ALL_MLX90620()

I2C1_Start() ' issue I2C start signal

I2C1_Wr(0xC0) ' send byte via I2C (device address + W)

I2C1_Wr(0x02) ' send byte (command)

I2C1_Wr(0x00) ' send byte Address

I2C1_Wr(0x01) ' send byte Address step

I2C1_Wr(0x40) ' send byte number of reads

I2C1_Repeated_Start() ' issue I2C signal repeated start

I2C1_Wr(0xC1) ' send byte (device address + R)

' convert data to 16 bits

for xl = 0 to 63

PIX_LSB = I2C1_Rd(1) ' Read the data (acknowledge)

PIX_MSB = I2C1_Rd(1) ' Read the data (acknowledge)

IRDATA[xl] = (PIX_MSB << 8) + PIX_LSB

next xl

I2C1_Stop()

End sub

and for 1 pixel:

Sub procedure read_IR_Pix_MLX90620(Dim pixel as byte)

I2C1_Start() ' issue I2C start signal

I2C1_Wr(0xC0) ' send byte via I2C (device address + W)

I2C1_Wr(0x02) ' send byte (command)

I2C1_Wr(pixel) ' send byte Address

I2C1_Wr(0x00) ' send byte Address step

I2C1_Wr(0x01) ' send byte number of reads

I2C1_Repeated_Start() ' issue I2C signal repeated start

I2C1_Wr(0xC1) ' send byte (device address + R)

PIX_LSB = I2C1_Rd(1) ' Read the data (acknowledge)

PIX_MSB = I2C1_Rd(1) ' Read the data (acknowledge)

I2C1_Stop()

IRDATA[pixel] = (PIX_MSB << 8) + PIX_LSB

End sub

Attached a screenshot of the VB.Net program I made for testing the sensor.

Colors are not ok but this will give you an idea.

PTAT and 1st IR pixel data + conversion (=big fonts)

In the Rx field you can see the string I'm receiving from the uC. (R144 = 0x90 = PTAT register)

At the moment I'm not using the eeprom data but the values of the calculation example of the datasheet.

So actual values will be a little bit different.
[General Accessories] CommanderROR:
Thanks for the updates guys!

Looks like you're getting somewhere with this. Shame about the eeprom problems though...has one of you tried to contact Melexis about this yet? I'd like to hear what they have to say about these issues. I you haven't done so yet, I can try to get through to their tech department, who knows, perhaps they can help.

@mardaso

just curious...are the example values a lot different from the "real" eeprom parameters? If they are close that might be a way to revive the corrupted eeproms should there be trouble getting them exchanged...
[General Accessories] Mardaso:
I will see if i can make a dump of the eeprom and upload it.

I feel lucky to have at least one working one.

Didn't have time to contact Melexis but they must have more complains about the eeprom.

Feel free to contact them.
[General Accessories] CommanderROR:
Thanks, will do.
[General Accessories] Mardaso:
Attached the eeprom data and IR data.

IR Data format: R144DDDDDD145DDDDDDIDDDDDD</64x/>X

R= start register data , D=data string, I = start IR data (64 values) X= end of data

144=PTAT register, 145=Compensation pixel register.

Good luck!
[General Accessories] CommanderROR:
@mardaso

thanks for posting the data. Maybe CybersteveUK can make use of it somehow.

I've just sent a message to my contact at Melexis, let's see what they say...

UPDATE: Melexis just got back to me on the issue and wanted more details. I asked them to follow this thread and perhaps register and offer some input. Let's see if they do...

If not, then you might have to contact them directly, I'll give you the direct contact address then, but we'll see if they choose the public route first.
[General Accessories] CommanderROR:
OK,

if you have a sensor that you think might be givign you faulty readings, then please send an eeprom dump to Melexis.

Luc will check the dump and see if the eeprom is corrupted. I'll PM his email address to those who need it. He also informed me that Melexis will provide sample code for microcontrollers sometime at the end of this month, but I guess you guys will have finished that stage by then... ;)
[General Accessories] Mardaso:
Thanks!

Got the mail address before you removed it and did sent Luc the eeprom data of the bad and good sensor.
[General Accessories] cybersteveUK:
Melexis eeprom dumps

For Melexis suuport, I have been asked to go through Paul Bundy Marketing Director - Clere Electronic Components Ltd, who are agents forn Melexis in the UK. (Melexis forum is defunct.)

I have sent him EEPROM dumps of all the devices I have.

Interestingly, the EEPROM dump in this forum has meaningful data for 'IR pixel individuaol offset coefficients' and 'individual ta dependence of IR pixels offset'. I think Mardaso has a better device than mine. But, entry 0xD8 (216) is zero. This is the Thermal gradient coefficient. Does this make sense??

I am still unable to read all 64 pixels in one hit. Mardaso's code looks the same as mine!
[General Accessories] CommanderROR:
@cybersteveUK

Luc from Melexis told me that some values show 0 at the moment because they are reserved for later use. Maybe the thermal gradient is one of those? I guess we'll know more when the eeprom dumps have been checked.
[General Accessories] Mardaso:
I didn't do anything with de eeprom data but TGC = 0 makes no sense.

There's also a RAM register 0x91 => TGC sensor result.

I hope the final data-sheet will make things clear.