yashsingh1/xdaforums huggingface dataset:
[Touch HD General] Mach2003:
Actually I have case 3, my noted code change above swaps them. I actually get Alt=0, Alt84=397.5 from msapi, and F9 = 397.5, F11 = 0 from COM4.

Case 4 has no examples among us three.

Ok so replace my terms 'geoid' with 'Ellipsoid", I have not taken the time to understand this properly.

Actually I don't care if F11 is correct, but I agree that using your table, and determining if the original data needs a correction would be the way to go.

In my case though, it seems that both COM and msapi are, in fact, NOT sending the correct ASL, or my Garmin BT unit is sending the wrong ASL. BT data is consitantly 16 meters higher than internal port. Or is the correct ASL actually the sum of F9 + F11?
[Touch HD General] rpettipher:
Mach2003 I agree with cybermaus - DO NOT alter the code for the altitude in GPS mod driver until and unless we understand exactly what is going on.

The existing code works for field 9 which is all that matters for most applications.

I am pretty sure that trying to use the values to determine which case we are in will not work, because, depending on location, the values are indistinguishable. Your algorithm will just get more and more complicated trying to exclude all the special cases.

For instance, with your code:

if( (fabs(F9) > 0.1) && (fabs(F11) > 0.1) )

F11 -= F9;

if F11 already has the correct separation value then it will be wrong after this (unless I am missing something)

A possible approach is to find out which software generates which configuration then test that. Is it just the version of WM which causes the differences? But to do that you need a bigger sample size.

You say:

This means that the Internal value of 397.5 is the height above the ellipsoid, and the height above sea level is 397.5 + 16 = 413.5.

It seems that the device on COM2 is behaving correctly.

This is what we all see from COM4, but not what we (cybermaus and myself) see from the MSAPI.

I also tried your gpstest program.

I am not sure what you expect.

In the text file produced (I do not find a storage card\mach2003 directory) I see GPGGA messages from COM4 which are the same as I normally read from COM4, with height above ellipsoid in field 9 and and 0 in field 11.

I do not see any values for the maspi.

cybermaus, have you checked if the NMEA messages generated by the GPS Mod Driver are correct (GPGGA field 11 in particular) with your WM 6.5?

mach2003 it seems you are getting different values from the MSAPI from those of cybermaus and myself. Have you tried running cybermaus's MyGps program?

Must send this before another message arrives.
[Touch HD General] Mach2003:
Right, good point, but we don't have this case (number 4) from msapi, do we?

And according to that post from teh GPSID author, he intended it to be this way, or did I misread it.

It is the OEM build version of gspdriver.dll, as far as I can see. So no detecting the ROM version will be of little help.

I'll check the code, it could be I wrote that test software for Diamond only, card is "Internal Storage" on Diamond.

The MsApi data is not being logged into the text file.

All three of us have a different set of values from msapi, don't we.

I have not tried it, but I have inspected my logs, and know what I am writing to them, so am SURE of the contents of those two structure values from GPS_POSITION.

/*******/

I'm still confused. If my mapping application reads F9 as altitude from COM2, it gets 413. If it reads F9 from COM4 it gets 397, but you say both are correct? How???
[Touch HD General] rpettipher:
I think I can reply to that one.

COM2 seems to be a correctly behaving device which means that F9 is the height above sea level and f11 is the geoid separation, i.e the difference between the height above sea level (or geoid) and the height above the ellipsoid, so to get the height above the ellipsoid (should we need it) we simply add F9 and F11 and get 397.

COM4 in an incorrectly behaving device and gives only the height above the ellipsoid (which we usually do not want) as 397. So the two devices agree, even though the information is different.

This is case 1 which cybermaus has, where flAltitudeWRTEllipsoid correctly (according to John Spaith) gives the geoid separation.

So it seems, and if I understand you correctly you get the same values from the msapi as from com4, so your modified driver does not give you correct altitudes as it does for cybermaus and myself. This seems somewhat ironic.
[Touch HD General] Mach2003:
I locate the path to 'storage card' from these two registies, where should I be looking for your devices? MyRegOpenRead assumes HKLM, and other NULL defaults.

if( MyRegOpenRead( L"\\System\\StorageManager\\Profiles\\moviNAND", &hkey ) == ERROR_SUCCESS ) {
if( MyReadValue( hkey, L"Folder", &GpsFile[1], _MAX_PATH ) != ERROR_SUCCESS )
GpsFile[1] = 0;
}
if( GpsFile[1] == 0 ) {
if( MyRegOpenRead( L"\\System\\StorageManager\\Profiles\\SDMemory", &hkey ) == ERROR_SUCCESS ) {
if( MyReadValue( hkey, L"Folder", &GpsFile[1], _MAX_PATH ) != ERROR_SUCCESS )
GpsFile[1] = 0;
}
}
[Touch HD General] Mach2003:
I've been looking over some old files beta tester's have sent to me (note that these logs are not sync'ed, they are each from a different time):

HD2, Internal COM4: 10.4,M,47.0,M, and another time: 45.3,M,47.0,M

Same HD2, MsApi: 22.9,M,69.9,M, and another time: 30.2,M,77.2,M

Different HD2 MsApi: 527.2,M,571.2,M, 490.9,M,534.9,M

BT COM: 475.5,M,44.6,M

Internal COM4: 521.6,M,44.0, 516.8,M,44.0,M

Samsung OmniaII: Internal COM: 23.0,M,, ;40.0,M,,,

MsApi: -15.5,M,0.0,M, -26.5,M,0.0,M

It looks like HTC fixed the bug by the time HD2 came out :)

Also MsApi *should* have output F11 changed to (F11-F9), as documented by John Spaith in order to match what NMEA spec says COM should have.
[Touch HD General] Mach2003:
So based on our findings so far (msapi only, not COM)...

Because I am a lazy typer: Alt = GPS_POSITION.flAltitudeWRTSeaLevel

Alt84 = GPS_POSITION.flAltitudeWRTEllipsoid

Table = Correction value from cybermaus 'published' table

I could correct the F9 and F11 value on Diamond with the table (if you share the complete one) [test:Alt==0, Alt84!=0]

I could fix HD2 and rpettipher's HD: (F11=Alt84-Alt), [test: both values exist]

I could fix Samsung with table, F9 = Alt+Table, F11 = table. [test:Samsung, Alt84==0]

These fixes would break F11 for cybermaus, but the prized F9 would still be correct. (You do get the table value in Alt84, right)

When reported Alt was actually 0.0, the fixes break only for my type:

Diamond would be 0,0, so no fix, F9 and F11 would both be zero, F9 should be -table, F11 should be table.

All other HTC, Alt84 would be assumed to be HeightAboveElipsoid, Table would fix both F9, and F11 correctly (it would pass test as Diamond), cybermaus gets Alt84=Table, F9=Alt84 - Table or Zero so it would be correct, F11=Table so unchanged, still correct.

Samsung, F9=0+Table, is correct, F11=Table, still correct.

When Reported Alt84 was actually 0.0

Diamond, Again no fix applied (both zero), so F9 is 0, should be -Table, F11 is 0, should be table.

All other HTC, no fix applied, Alt84=0, F9 is correct, F11 is zero, should be table.

Samsung: Always has zero here, so no change.

Any comments?
[Touch HD General] cybermaus:
Here you'll find the latest revision of matrix and its header file

Parameter 3 & 4 of the function is what you would call F9 and F11

About the calculations: If the gpsVP matrix is used, you have to understand it is an approximation. The values in the Matrix are correct (barring typing mistakes), but it has a 10 degree resolution, and values in-between are interpolated.

Also, as a disclaimer, I probably should notify you that until this thread started, I never had feedback (not positive nor negative) about the working of this matrix. Other gpsVP users accepted it without question, which could be seen as positive (it worked) or neutral (no-one really cares).

On the other hand, I think the interpolation yields a value correct within 1 to 2 meters in all but the steepest situations. This is mostly at the edges of the continental plateaus, but overland in some places like the west-Andes. Correcting it further, would require a 5degree or 1 degree matrix, but it would become huge (1 degree: 360*360=130.000 values)

So I advise that the matrix is first used to check if the chipset driver provides separation, and only if it does not, use the actual matrix value itself. So if a suspected separation is within 5 meters of the matrix, than it is accepted as the correct separation.

Also, I still advocate that if a device/OS has been found to behave in a certain way, it is recorded as such for the duration of the program run, the rest of the day, or even until re-installation. This to avoid adding fluctuations to the altitude in real near 0 meter altitude and near 0 meter separation situations.

---

And finally some terms, since you indicated some confusion with it:

(as I understand them, but I think I got this down)

The WGS84 Ellipsoid is just that. A mathematical function, essentially a 3D elipse, like the ones you calculated in high school. And if Earth had a perfect homogeneous gravity, it would be the form of Earths sea-level.

They found however the Earth sea-level shape was rather more erratic. This is due to local variations in its gravity and mass. Deeper larger masses give large slow changing variations. Masses just under the surface give smaller but steeper variations. And so they came to the Geoid form. So the Geoid can be seen equivalent as the sea-level. NASA and math people talk about a Geoid, but NMEA definitions require sea-level.

Picture 1: Places with small but steep changes in the sealevel/Geoid

Picture 2: Full height of Geoid above Ellipsoid.

Ref pictures: http://cddis.nasa.gov/926/egm96/egm96.html
[Touch HD General] cybermaus:
Yes we do. My device is giving me the separation value. (Currently, who knows what happens next time I flash a custom ROM)

I think you misread. The author intended separation (the actual) F11 value, but the complainer argues that the documentation warrants full altitude (F9+F11) and the author is then in doubt and will consult an NMEA specialist.

But at this point, it does not matter what the author intended, or even if his discussions with an NMEA specialist yielded a firm standpoint from MS. We now know we have a variation of implementations around, and according to yourself, that variation is not just under control of MS, but also of the driver writers that may or may not have certain versions of MS template, and reuse those versions even if MS "fixes" the template. In other words, its a mess.

Its "Storage Card" on my device

The following registry point to it:

HKLM\System\StorageManager\Profiles\SDMemory

HKLM\System\StorageManager\Profiles\PCMCIA

HKLM\System\StorageManager\Profiles\SDMMC

Well, not both are correct, but you can see they are calculated from the same base:

COM4 "forgets" to apply the correction, so gets the GSM WGS84 value of 397M

COM2 behaves nicely, so corrects it to EGM96/ASL 413M, but provides you with the correction value of -16 so you can undo the correction if you feel like it.

So they do not both give the same value, but you can see they are related.
[Touch HD General] Mach2003:
First of all thanks for the explanation, now I understand!

Second thanks for the source code.

Finally, thanks for another great idea, I can detect your MsApi data type by using your function/table, and comparing it to the value in Alt84.

I will take your advise, if or when, I make these changes to ModDriver. I'll keep a reg entry for the device, update it on first MsApi use after a soft reset (when driver gets loaded by windows), and allow user override via a setting.
[Touch HD General] Mach2003:
Please check if HKLM\System\StorageManager\Profiles\moviNAND exists.

And if the "Folder" value under ..\SDMemory has "Storage Card" in it.

This also affects my Track Logging Feature in ModDriver, not just the test app I posted here.
[Touch HD General] cybermaus:
Hmm, that first line would indicate F11 only a separation (it stays stable at local 47 while the person climbs 15 meters.

But that second line, if we assume that F11 is then a confirmed separation only value, could only be correct if that person then traveled to Indonesia (70M) and on to New Guinea (77M)

You happen to know where this guy was (sure you do, you have his NMEA logs)

Well, here the HD2 is following rpettipher full altitude in MSAPI, but it's COM4 value is the correct NMEA separation. Thats a new permutation in itself, but consistent at least.

But than the BT COM disagrees with both, and wants to deduct an extra 44 meters. Is that BT measurement at the same time?

And another permutation. Both COM and MSAPI give zero for the correction, but they give different altitudes. One would guess one is giving the corrected value, but which one. But then it is followed up by the 2nd measurement, where the COM claims to be ascending while the MSAPI claims to be descending.

What to make of that?

I would not call that fixed. Its an even bigger mess. That first HD2 is different than that 2nd HD2. And it seems like Samsung is also messing it up just like HTC.
[Touch HD General] cybermaus:
Yes, moviNAND exists, and has folder value "moviNAND". But I do not have such a folder, and I suspect I do not have that type of memory.

All 3 registry entries I mentiond have folder value "Storage Card"
[Touch HD General] cybermaus:
May I suggest tuning it to not write the registry value on first GSM sample. GSM fluctuates a bit, especially in the beginning, and you would not want to make a decision on that first erratic sample. Probably best update the registry only if 60 samples consistently yield the same case, at least 60 samples into the activation of the GPS.

While this point has not been reached, the value of the previous soft-reset can be used.
[Touch HD General] Mach2003:
The first HD2, First line is clearly labled Internal COM4!

The second line is MsApi, and Alt84 contains full altitude.

NONE of the values were taken from the same log files, not even from the same day! I do know they did not travel FAR from home though. They can not be compared to each other, only in content.

The repaired COM4 is what I was reffering to as "fixed".

This could be a coincidence he may be 44 meters higher for this log entry. I'll inspect more of his BT logs to see.

[Guess] It looks like the msapi Alt is ASL - Correction, while COM is ASL.
[Touch HD General] Mach2003:
Whew, that explains why I don't find your card. I search for moviNAND first, I have moviNAND in Diamond, and SDMemory exits too On my custom ROM I point it to "Internal Storage", but stock ROM's point it elsewhere.

I'll to a test write to the path to verify it.
[Touch HD General] cybermaus:
True for altitude, but if any value is a pure separation value, you would have to travel a significant amount to change that. So in your case, with a separation value of -16, even if you traveled a hundred miles, it still be -16, and if you traveled a thousand miles, it probably be -14 or -18 at most.

So a change from +40 to +70M. Thats a good trip. And a +77M value? That only exists in New Guinea and in in maybe Iceland.

Anyway, no need to guess, you have the log files, which have the lat/Lon, so you can always check what should be the separation by using NGA's lookup
[Touch HD General] Mach2003:
I agree, so the 70m value is ASL+Correction on msapi, and the 40 value is correction alone on COM4. Same as the other HD2.
[Touch HD General] cybermaus:
I see what your saying, yes, I was reading that HD2 wrong, sorry.
[Touch HD General] cybermaus:
Which leads me to possibly another, simpler way to determine what value is given by MSAPI AltW84.

If multiple successive readings show Alt is changing, but AltW84 stays zero, we may assume AltW84 is invalid.

If multiple successive readings show Alt is changing, but AltW84 stays constant, than you may assume AltW84 is separation only (especially if within the -100+80 bandwidth)

If multiple successive readings show Alt is changing, and (Alt-Alt84) stays constant, than you may assume AltW84 is full W84 altitude and (Alt-Alt84) is the separation.

If multiple successive readings show Alt is zero, and Alt84 is non-zero, then AltW84 is the only altitude we have, copy to Alt, and zero out Alt84
[Touch HD General] Mach2003:
NP,

Here is the test app that reads all ports at the same time, AND writes csv files to the card :), well it should anyway I can't test it here.

You have to run ModSettings, and enable both msapi and COM4 and then exit ModSettings (unless you have them both checked now).

Run the test app for long enough to get a good fix, there is no display changes. Exit the test app, and inspect the ModDriver0.csv (msapi) and ModDriver4.csv (COM4).
[Touch HD General] Mach2003:
Except for me the last case is not the best solution, using your function would give me true altitude. And those poor Samsung folks don't have a solution either.
[Touch HD General] cybermaus:
Yes, but the problem is, if one is zero then we do not know if the other is AltSL or Alt84.

A week ago I thought I knew, based on the devices I have. I thought that if F11 was 0, then no correction was done, ergo, F9 was uncorrected Alt84

Now, I am not certain anymore. And if F9 is already AltSL, and you think you still need to correct it with the matrix, then it is wrong again.

But maybe my week old assumption is still good for *most* of the 'one value zero' situations. Maybe the 'one zero, the other is AltSL' is only a theoretical scenario, and no one actually has it.
[Touch HD General] cybermaus:
So this routine is better you your liking:
[Touch HD General] rpettipher:
Tried the new gpstest.

Found your files in my top level directory (not on the storage card).

They confirm what we had already discovered, namely

COM4 Alt = height above ellipsoid, Alt84 = 0

MSAPI Alt = Height above geoid, Alt84 = height above ellipsoid.
[Touch HD General] cybermaus:
Just now received my new phone by DHL. Its a Chinese clone of the Diamond2, model T5388++. I bought it because its a WM6.5 with Double SIM. Its a clone, so I am not expecting a good build, but HTC does not have any double sim phones.

So when I check, I get this:

MSAPI: Correct Alt=AltSL, Alt84 is separation value

COM6: Correct AltSL in F9, Correct separation value in F11

The ad claimed it was a Sirf chipset, so I had expected this. Of course, the ad actually said it was a Srif shipset, which could be bad spelling, but could also mean some clone chip. Just like they sell NOKLA, HYC, CiPhone, SANSUMG and Blackgrape phones.

EDIT: You'll be happy to know your driver works on the clone too. And indeed, while it does create a \Storage Card\Mach2003 folder, it puts the csv files in the root \
[Touch HD General] Mach2003:
Here is a test application that fixes teh path on card.

It also adds the corrections discussed to both msapi and com port data.

Each port has its own mode.

There is no registry yet, each start of the application re tests the input ports.

10 samples are taken, from first fix to determine the mode.

If all is working as planned, all csv files on card should ALWAYS show correct F9 and F11 data.
[Touch HD General] cybermaus:
Indeed, the csv are in the folder this time :)

Looking at the results, I am assuming that you are intending to put the Full Alt84 into that 2nd MSAPI field. Despite all that has been discussed, I still doubt if that is what MS is intending to be in that value. But OK, I submit.

My HD is indeed giving the correct values in both MSAPI and COM output. Somewhat. The COM output is clearly using the 45 meters from my matrix, while the MSAPI calculates it own 47 meter value.

While not applicable to truly separate inputs, we know that COM4 and MSAPI is really the same. So though the true COM4 has a zero, and thus you must use the matrix, this could be an exception, because you can use the separation value from MSAPI.

EDIT: I just realize: Even if they are fully different COM inputs, the separation is constant for a location. So if *any* input yields a separation value, you can use that over the matrix for any other input.

The new clone phone is giving the good values too, though it seems that you do not correct the MSAPI Alt84 field from 47M separation to its full altitude. Also, the clone was reading a little low, but I already noticed its antenna is not so good as the HD.

I attach the files, but will remove them as soon as you have them, because they contain my location a little too precise.

BTW: I sold my HD, so once I receive funds, I must ship it. Likely I will not be able to test after tomorrow.
[Touch HD General] rpettipher:
Tried the new gpstest and it does seem to be doing as advertised (though I am indoors and stationary)

The MASPI output shows Alt = 523 and Alt84 = 573 (as before) for the first 10 readings then switches to Alt = 523 and Alt84 = 50 for the remaining readings (give or take a few tenths of a metre). (It is normally a bit lower than that, but I am indoors)

The COM4 output shows Alt = 522 and Alt84 = 51 continuously.

So both give the height above the geoid, followed by the geoid separation, as appears to be required by the specification.

The 51 is presumably calculated from the cybermaus matrix, and is only 1 metre different from the chip supplied value.

So my congratulations to you. I hope it works for the other configurations.

I must try it on the move tomorrow.
[Touch HD General] Mach2003:
I am on the road right now, but have the logs to look at later go ahead and remove them now. I had intended alt84 to be separation.
[Touch HD General] Mach2003:
cybermaus, I don't understand your HD log for msapi at all. I don't have a case where Alt is added to seperation and put into alt84. Also the first reading should be different than all the others, if I am modifing the data. From, the log file alone it looks as though the HD is sending the Alt84 value as Alt + Seperation, same as the other HD here. And since Alt84 is the same for the first 10 readings, I assume it is seperation, and of course it is not. Try the test a few times, perhaps while moving the device around (during the innital 10 seconds) to induce a change in that field. Are you real sure that your HD is sending seperation?

Edit for clarity.
[Touch HD General] Mach2003:
I can't easily apply the msapi correction to COM4. In ModDriver only one port is active at a time, in the case of the two ways to read the internal GPS, the lowerst order port will always be used, the other one will never be used (if one works so will the other).

Yes I am using your matrix for seperation when ever it is needed, and am assuming a COM string without F11 present, must also be Altitude Above Ellipsoid, although it is possible that a COM string will have true altitude and not fill in F11 we have no examples of that to date have we? At least this way we will be right for all the devices we currently know about.

Of a note, my matrix value is -18, while my BT reports -16, the same error amount as your location.