yashsingh1/xdaforums huggingface dataset:
[Touch HD General] rpettipher:
GPS height error

Thanks marder.

Could you try running visualgps with the log activated, then look at the log file for the message $GPGGA? If the 11th field, between the two Ms, is always 0 then it proves the point.

I remain very surprised at the lack of complaints about this, if it is such a common problem.
[Touch HD General] cybermaus:
But why would people complain? Or even notice?

The height error is absolute, but in differential measurements, it still works well. In other words, if you have a height, and you walk up a hill, it will give you a higher height (most of the time, and within margin of error). And most people are not aware of their absolute altitude anyway. Lets face it, if you live in Moscow, and the device tells you you are at 200mtrs, why would you distrust it?

I only noticed because I live at sea-level (Holland). You only noticed because you live in an area with above average separation, and got annoyed with two devices giving you 2 different values. And marder probably scratched his head when encountering these "xxxx above sea" markers which you get when hicking in the mountains. He would not encounter such markers in the city he lives.

Face it, if you had not had you Mio, would you have noticed? Or be able to confirm it?

Of course, once we know, we think it is unacceptable. Thats human nature, myself included.

But there are other people who did see it. Searching for keywords geoid and gps shows this thread, and the program appears also to have a fixed geoid position.
[Touch HD General] stuntdouble:
Like I said before, did you try different radios? As this affects the functions of the GPS. I had this problem too but flashing to the latest radio cleared it right up, along with making the gps now lock on with a strong signal within 8 seconds. Going through the complicated route of coding things to fix it is a bit pointless when others have fixed it before you. I'm not saying it will definitely clear it up for you, but taking 20mins to try a few radios is probably worth your time.

Oh and a good way to tell if it's accurate or not, for anyone else wondering, is to check on google earth for the "elevation".
[Touch HD General] rpettipher:
GPS height error

cybermaus, I expect you are right - people do not notice.

I certainly did notice, independently of the Mio, by comparing the heights marked beside the autoroute that I use with those displayed by the HTC.

But you are right in that without the experience of the Mio I might well have assumed that it was due to some inherent inaccuracy in the GPS system and not searched further (and not found the thread at http://www.gpspassion.com/forumsen/topic.asp?TOPIC_ID=10915 where this is all explained).

stuntdouble, thanks for the input. I assume that flashing new radios is trivial for you, but somewhat more traumatic for people like me, aware that a problem during the process can end with a dead phone. Could you perhaps tell me

1. Does it actually correct the problem i.e. does the NMEA message $GPGGA give a non-zero value in field 11?

2. Do I lose all my programs and data in the process

3. Which of these many radios do you recommend

4. Where do I find it/them

5. Are you sure it will work on my phone (which is apparently of Malaysian origin)?

6. What is the procedure?

Sorry for the interrogation but this is not as easy for me as it is for you.

For information I have:

ROM version	1.18.707.3 (23358) WWE

ROM date	12/10/08

Radio version	1.09.25.14

Protocol version	52.49a.25.26U
[Touch HD General] stuntdouble:
1. Yes. All seemingly corrected.

2. No programs are touched as long as it's a successful flash. Radio is separate from the rest of the device's memory. Just backup your device with Sprite Backup or SPB Backup before you try it, in case you do need to do a hard reset. This way you can get your device back to exactly the way it was before.

3. The one in my sig fixed mine. However every device is different so you would need to find the best radio for your own device. I'd start with the latest and work backwards though if I were you.

4. Get them here.

5. If it doesn't work, it doesn't work. You can always re-flash your old radio if you wish. However I suggest you ask in the radio thread before trying, which radio would be best for your operator. There will be plenty of people who will know this and have tried it before you have I'm sure. The radio info is based on your operator and not on country of origin for the phone.

6. Easy procedure is in link above.

I'm not guaranteeing this will fix your problem, but it really is easy to do and well worth attempting as like I said it fixed it easily for me. It also increased battery life dramatically, and the faster gps fix like I mentioned before.

I understand that it may seem difficult on first read. I felt that way too when I first tried it. But it really is very simple and I was wondering what I was scared of after I tried it. Just follow the instructions in that thread carefully and there won't be any problems. If you use windows vista/7 then make sure you run as admin if you are flashing via the pc. Good luck.
[Touch HD General] cybermaus:
Sorry guys.

I have followed most of the radio updates, and never noticed this being fixed. I have to admit, I never checked since I updated to 1.17 radio, so I checked just now, and it still reports 0 meters separation and at 43 meters altitude (at the coastline of Holland...) So for clarity, I logged out the NMEA messages, and all $GPGGA mssages give 43.5,M,0,M for field 9 through 12. On the latest 1.17 radio.

While theoretically possible they fix it in the radio, it still has not been done, not even at 1.17. I guess you either have the luck of living in a region where geoid and ellipsoid are very close to each other, and/or you are not aware of your true absolute altitude.
[Touch HD General] rpettipher:
GPS height error

Thanks guys now I am confused.

I suppose we need to know what "seemingly corrected" means.

Has stuntdouble actually seen the NMEA messages with the correct data?
[Touch HD General] rpettipher:
Rom update

I do not know about taking "20mins to try a few radios" - I have spent several times that amount just reading some of the information relating to flashing!!

But then I am a slow reader.

Following the links I eventually found what is apparently the 1.17 radio rom on a support page of the HTC site.

I am puzzled to see that it is described as for South Africa only, and concerned to see that it says it is only for ROM version less than or equal to 1.14.421.5. when my ROM is 1.18.707.3.

Should I be concerned and should I also be concerned about the caution when starting the program that it will destroy all the information on my phone?

Just being cautious, especially as I do not know what improvements if any I will see.
[Touch HD General] wrxppp:
I always thought when running VisualGPS that my HD was reading a bit high for where I live, but it never impacts on what the phone is used for.

I've dusted down a trusty old Holux BT321 and placed it by a window next to the Touch HD.

The Holux shows 91.32m, the HD shows 160m.

I've pulled VisualGPS logs for both units but what info should I paste up to be meaningful to someone who knows what to look for?
[Touch HD General] rpettipher:
GPS height error

Welcome to the discussion wrxppp.

If you have the logs just look for the lines starting $GPGGA, then for the two Ms on the line.

The value before the first M (field 9) should be the corrected height above sea level and the value between the Ms (field 11) should be the difference between the ellipsoid and sea level which is use to make the correction.

On my HTC Touch HD (and perhaps everyone's - that is what I am trying to find out) the value in field 9 is the height above the ellipsoid, and field 11 contains 0.

The height given is, in my case, around 50m too high.

On my Mio, which uses the Sirftech chip, field 9 contains a pretty accurate height above sea level, and field 11 contains 50.5.

So what do you see?
[Touch HD General] cybermaus:
70 mtr, interesting, that is among the higher values, may I inquire whereabouts you live? Also, would you like to test gpsVP and report back to check the correction?

And if you want to go forward, you'll probably have to spend a lot more. Nowadays I can flash in 20 minutes, but that first time also took a while. A few tips: Dont look for the full ROM in HTC website, but for the split of radio somewhere in XDA, Touch HD forum and use/read USPL for flashing.

But lets not turn this thread into a flash discussion thread, there are plenty of those already.

Edit: Some references:

NASA page: http://cddis.nasa.gov/926/egm96/egm96.html

Calculate : http://earth-info.nga.mil/GandG/wgs84/gravitymod/egm96/intpt.html

Picture:
[Touch HD General] wrxppp:
From the logs I get:

Holux 91.32m with difference of 47.12m

Touch HD 159.8m with difference of 0m

Location is in the UK IRO 52N, 0.35W

I'll give gpsVP a go later.
[Touch HD General] rpettipher:
GPS height error

OK, 3 is a small sample size but we have 3 out of 3 with incorrect NMEA messages, so I think that I can abandon looking for a fault on my unit.

Which brings me back to my post #4 where I found a small program called MyGPS (at http://htcwiki.wetpaint.com/thread/1525057/Free+GPS+software) based on the GPSSample in the MS SDK. This uses the MS API to access the GPS info and gets the correct value for the altitude (currently it is giving me 1680 feet = 512m, whereas ozi explorer is giving 560m - the correct answer is about 510m, but I am indoors).

Does anyone know of other programs which use the API rather than the NMEA messages?

Obviously the correction is in the chipset somewhere. Is there some option to configure the NMEA message in different ways?
[Touch HD General] stuntdouble:
Ok once you get going it will take that long to do it.

Like I said before your best bet is to ask in the radio thread for help. And no it's not just for South Africa, as I'm from the UK and it works perfectly with my operator.

It might be hard to understand some of it, but as these forums are meant for developers and like-minded souls, that's just the way it is. It's best to take it slow, and be cautious because if you want anything to work on these forums it requires a fair bit of reading.
[Touch HD General] cybermaus:
:eek:

Ok, when you mentioned MyGPS, I checked it out, and I found a program of that name from an Italian maker, and the program was wholly unimpressive, used com port, and gave incorrect height.

This time, you provided a link. Its a different MyGPS. It does use the intermediate driver, it provides source and it *DOES* give correct altitude.

Again: :eek:

I had checked out the intermediate driver before, but decided not to use it because I did not realize that it provided its own sealevel correction. Had I realized that, I would have gone a different route a year ago, I can tell you.

I (speed)read the MyGPS source code, and re-read the MS Intermediate driver reference. MyGPS does not do any calculations. I changed my opinion and currently believe Microsoft does do there own sealevel correction. It is not very clear, but the only hint is that the MS documentation mentions 'sealevel' a few times, whereas they do not mention the EGM96 'geoid'. So they may use their own routines.

Now I am in doubt, what to do. I cannot just recode gpsVP to use it, as others are in effort of recoding it to Java/Linux, so we should move away from MS specific API's. Maybe I should build a new 'virtual' com driver program, that translate the intermediate driver back into a serial NMEA, so it is useable for all GPS programs. Maybe I should do nothing, as gpsVP is working as is, and I am quite happy with it.

But, I confirm: MyGPS is giving the correct altitude, but the source does not have any calculations, so the correct altitude must come out of the MS GPS Intermediate Driver. Given that the GpsSone of Qualcom chipset does not seem to provide that altitude correction, MS must calculate it themselves.
[Touch HD General] rpettipher:
GPS height error

I thought I had already sent this reply but it seems to have got lost. Apologies if it turns up as a duplicate.

Thanks for the excellent analysis cybermaus.

Sorry about the confusion with MyGPS - not a very original name - and sorry to give you some difficult decisions.

Not sure if there is anywhere to go from here - I will try HTC and even microsoft, but I am not optimistic about getting any sensible replies.

Thanks again
[Touch HD General] wrxppp:
GPSVP results

Fired up GPSVP on the PC and Touch HD and get these results:

PC 96m alt. with 47m geoid sep.

Touch HD 92m alt. with 45m geoid sep.

Is there a manual for GPSVP? I can't bring up any map (trying to use Google.com sat)
[Touch HD General] cybermaus:
Make sure you read the FAQ and Wiki with the gpsVP page. Anyway, you should have set in the menu the "internet" and "allow download" options. And also you should set your cache folder.

EDIT: With the PC, you were using the Holux again? Than it fits, the gpsVP routine should detect the separation is already there, and then not insert its own value. So the 47(.12)mtr is the Holux value, and the 45 meters is my calculated value.

It means the "auto" mechanism is functioning well, thanks for testing.

BTW: according to NASA, the value should be 46.75m. But I think it is overkill to display centimeters if the rest of the GPS has a few meters variation in its measurements, that is why gpsVP only displays whole meters.
[Touch HD General] wrxppp:
Yes, it was the Holux unit again. It must be about five years old so no idea what chip it uses. I was surprised the battery still held a charge.

I missed the "allow download" tick box so all is working.

nick
[Touch HD General] rpettipher:
This was the response I have received from HTC. It might go somewhere eventually.

On a more optimistic note this was the reply I got from Des Newman, author of the excellent Ozi Explorer:

By which he does mean that he is calculating the geoid separation in the program.:)

So I will be able to continue to use Ozi on the HTC.
[Touch HD General] cybermaus:
Cool. But I have another solution for you:

http://xdaforums.com/showthread.php?t=571266

This guy has already written a program to map GPSID back into a COM port. He may have done it for different reasons (LAG time) but chances are it solves your (and mine) altitude problem.

Also NaviComputer is also using GPSID, and in all honesty a much nicer program than gpsVP. It cannot do Garmin maps, and has less functions, but it does cache maps, and does this with a much nicer look.
[Touch HD General] rpettipher:
Many thanks cybermaus.

His program seems to be working ok on my HD, though I have not tried it on the move yet. It does indeed correct for the altitude, though there is still an error in the NMEA message generated, as I have indicated in the thread you sent.

Perhaps he should send it to Qualcomm and HTC to indicate what they should be doing.

Well my ozi explorer will be working quicker than I had expected.

Thanks again.
[Touch HD General] cybermaus:
NOTE: The discussion went to this thread for a moment, but I am splicing it back in.

:eek:

So in your case, the field *does* give the absolute height from Elipsoid?

Would it be possible that the programmers at Microsoft also are confused about the field, and tried to "correct" it is some of the builds? And then some other ones "corrected" it back? Or am I jumping to conclusions?

If so it will become difficult to correct? Luckily, it is a bit academic, as the altitude you are interested is good at least.

What is your ROM version? Mine is 6.5 build 23554

Your GSPID_HTC.dll? Mine is 72440 bytes

Your gpsid.dll? Mine is 51670 bytes

Your gpsdriver.dll? Mine 79608 bytes

Your GPSAux.dll? Mine is 31992 bytes

Your gpsapi.dll? Mine is 2921 bytes

Your reported registry driver version HKLM\Drivers\Versions\GPSID? Mine is 2009.1.9.0

The download of the MyGPS sample program here
[Touch HD General] rpettipher:
Good idea to bring the discussion back here, we do not want to hijack the GPS Mod Driver thread - it is big enough as it is.

Indeed it does - just as I had become convinced that this field should contain the separation.

I have:

ROM version? 6.1 (Build 207571.4.0) [Yours is 6.5 build 23554]

Your GSPID_HTC.dll? 68856 bytes 2/2/09 [Yours is 72440 bytes]

Your gpsid.dll? 50916 bytes 16/9/08 [Yours is 51670 bytes]

Your gpsdriver.dll? 77560 bytes 2/2/09 [Yours 79608 bytes]

Your GPSAux.dll? 30160 bytes 12/10/08 [Yours is 31992 bytes]

Your gpsapi.dll? 3012 bytes 16/9/08 [Yours is 2921 bytes]

Your reported registry driver version HKLM\Drivers\Versions\GPSID?

I do not have this key.

I do have HKLM\Drivers\Versions\builtins\GPSID but this does not have a version item.

[Yours is 2009.1.9.0 ]

Looks as if they got it wrong in WM6.1 then corrected it in 6.5, despite what John Spaith said in his blog.

I am sure he could tell us if he were still around.

If mach2003 is using WM6.5 then perhaps the NMEA messages he generates for you are correct (?).
[Touch HD General] Mach2003:
I am using 6.5.5 on my Diamond.

gpsdriver.dll is an OEM file (Created by HTC)

gpsid_HTC.dll (or in some devices like HD2 it's renamed to gpsid.dll) is OEM. Look under HKLM/Devices/BuiltIn/Gpsid for the DLL listed.

gpsid.dll is a SYS (created by M$) file, but never used (I don't even include the sys file in my builds)

gpsapi.dll is SYS file.

GpsAux.dll is QuickGps file, no affect on this topic.

gpsapi.dll provides the interface to the GPSGetPosition call, but that dll just forwards the request to the GPS0: device, which is handled by gpsdriver.dll.

gpsid_HTC.dll handles the device GPD0:, and GPD1: as documented by M$, and *should* be providing the COM interface to GPSID, although I have not got around to debugging this one, it does export COM and GPD functions.

So the bottom line is this: gpsdriver.dll is the interface to RIL to "form" gps data. It has a function to build GPS_POSITION structure, since ths dll is modified by OEM, any bugs may be different on each OEM build. gpsid_HTC.dll is also modified by the OEM, and provides the COM interface to gpsid. It in turn also reads raw data from gpsdriver.dll, either through the same function call as gpsapi, or though a different one (I don't know which), but in any case, again HTC can introduce/correct/modify any bugs with each bulid.

In case you don't know: M$ provides original the source for these dll's to the OEM's and can and does update them from time to time, and the OEM's rebuild them to interface with the device hardware.

Edit: Added this, bugs can exist in any of these layers:

radio->ril->gpsdriver->gpsid->application

radio->ril->gpsdriver->gpsapi->application

or even

radio->ril->gpsdriver->special_application_aware_of_driver
[Touch HD General] Mach2003:
I think I need to make this code change to get mapping msapi input to COM output working for almost everyone:

// some implementation differences have been found
// alt = ASL, Alt84 = Altitude difference to geoid
// alt = ASL, Alt84 = Altitude to geoid
// alt = 0, Alt84 = Altitude to geoid
// alt = ASL, Alt84 = 0

/* Notes *
GpsPos is M$ GPS_POSITION structure
F9 will map to NMEA field 9 in GGA sentence
F11 will map to NMEA field 11 in GGA sentence
*/

if( gp->dwValidFields & GPS_VALID_ALTITUDE_WRT_SEA_LEVEL ) {
F9 = GpsPos.flAltitudeWRTSeaLevel;
}
else {
if( gp->dwValidFields & GPS_VALID_ALTITUDE_WRT_ELLIPSOID ) {
F9 = GpsPos.flAltitudeWRTEllipsoid;
}
}
if( gp->dwValidFields & GPS_VALID_ALTITUDE_WRT_ELLIPSOID ) {
F11 = GpsPos.flAltitudeWRTEllipsoid;
}
else {
if( gp->dwValidFields & GPS_VALID_ALTITUDE_WRT_SEA_LEVEL ) {
F11 = GpsPos.flAltitudeWRTSeaLevel;
}
}
// if both are now the same value, set the correct output values
if( fabs( F9 - F11 ) < 0.1 )
F11 = 0.0;

// we could have the case where F9 = ASL, and F11 is ASL + geoid difference
// But NMEA just wants the difference in F11
// I'll assume that if both fields are valid, then F11 is the sum, as reported by M$ GPSID author
if( (fabs(F9) > 0.1) && (fabs(F11) > 0.1) )
F11 -= F9;

// now say that both fields are valid
gp->dwValidFields |= GPS_VALID_ALTITUDE_WRT_SEA_LEVEL;
gp->dwValidFields |= GPS_VALID_ALTITUDE_WRT_ELLIPSOID;
[Touch HD General] Mach2003:
Ok last post of the day, I promise!

I have a test application (attached) that reads all ports enabled in ModDriver (for me that is MsApi, COM4(GPSID), and COM2(BT), and logs the data from each of them into a CSV file on the card \Mach2003 folder. It also logs raw data to the device's \Mach2003 folder as TestLog.txt.

I ran the test here just now, I get exactly the same vaues from MsApi and COM4 (note that the mods from the post above are installed), and the values from COM2 are as shown before:

Internal: 397.5, 0

COM2: 413.5,-16

The values from COM2 when added together are ALWAYS nearly the same as the Internal GPS reports with either COM4 or msapi.

I'd be interested in what differences other devices and ROMS have.
[Touch HD General] cybermaus:
I am not sure I agree to your code changes. But before we discuss that, lets ensure we have the correct cases:

// some implementation differences have been found
// alt = ASL, Alt84 = Altitude difference to geoid
// alt = ASL, Alt84 = Altitude to geoid
// alt = 0, Alt84 = Altitude to geoid
// alt = ASL, Alt84 = 0

You may be confusing Geoid and Ellipsoid here.

I expect you have my device as case 1 and rpettipher as case 2

But my device has:

// alt = ASL, Alt84 = Altitude difference above Ellipsoid

And rpettipher has:

// alt = ASL, Alt84 = Altitude above Ellipsoid

If your device had Internal: 397.5, 0 and COM2: 413.5,-16

then that would suggest that indeed you have

// alt = ASL, Alt84 = 0

So who has case 3?. I hope no-one! We may need to select a larger sample-set than just us 3.

If so, the important thing we must realize is that in all 3 cases, MSAPI reports the correct ASL. And frankly, finding the correct Alt84 correction is far less critical. You may simply deside to do nothing, or at least nothing urgent. It is already working as altitude correction for the GPSone chipset.

-----

Having said that, if you do want to correct, a simple single GPS sample by sample correction is not going to work for locations that are between -80M and +100M altitude.

The reason is, you cannot judge if such value is an full altitude, or a separation value only.

I think a correction routine should be more stable over time. Assuming a single combination of hardware/OS at least has consistent behavior, I would suggest this:

If ever samples are consistently (> 100 samples) above 100M or below -80M (Israeli Dead Sea), a registry value should be set for that field to mark it is not a separation but an absolute altitude. This will ease your determination between cases 1 and 2. Even low-land people like myself only have to go hiking up a hill once. (It even allows for manual tweaking of the registry if I fail to do even that)

Similarly, a value could naturally go to 0 occasionally. So a second registry value should indicate a field has been non-zero. This should avoid single samples suddenly being seen as case 3 or 4. The GPS readings is fluctuating enough as it is, lets avoid adding spikes just because you are crossing a real 0.0 altitude or real 0.0 separation area.

Unfortunately, in case 4 (your case) you still cannot correct field 11. You'd have to copy the gpsVP matrix. But if you do that, than the -80/+100M rule of the previous one could be made more strict, just check if the chipset value differed more than 5 meters from the gpsVP matrix, instead of allowing the full -80/+100 bandwidth.

----

But as I said, I think that may be overshooting the target. If I understand correct, we've only seen cases 1, 2 and 4, and all of those have a good already corrected field 9 altitude. Field 11 is not so important really. At the very least it can wait until more people report their findings.
[Touch HD General] cybermaus:
I am not successful in running your test program.

It does generate a log, but it is a txt file, and also the inside does not look like a csv. Also, while that log does show the output from COM4, I cannot find the output from MSAPI.
[Touch HD General] Mach2003:
CSV is created on your storage card \Mach2003 folder. Log TXT file is created in device \Mach2003 folder