yashsingh1/xdaforums huggingface dataset:
[Android Apps and Games] tofu:
Thank you for your response! I did not mention the things listed probably because I felt they are obvious. Of course I have 1. installed UnifiedNlp for GAPPS-free devices (the one for android 5.0 and above), 2. enabled the localgsmNlpBackend, 3. downloaded the database several times (sometimes with my countrycode and sometimes the whole world).. yet, no success!
[Android Apps and Games] n76:
Sorry to belabor the obvious but sometimes it is overlooked. :)

Does the settings screen for the backend show "current database" date?

Do you have access to sqlite on a computer other than your cellphone? If so, we can grab the database from your phone and do some queries to see if there is actually data for your area, etc.

If not, I can get you a copy of local-GSM-backend with debug logging turned on and we can check logcat to see what is happening.
[Android Apps and Games] tofu:
Yes, the backend shows current database date.

I think I could install sqlite, but I've used it only once, so it would be quite time consuming to start learning, unless you can provide me some instructions?

The debug version of loca-GSM-backend sounds easier. Could we try that one?

I still add one more thing: I cannot get GSMLocationNlpBackend nor OpenBmapNlpBackend work either. Only thing that works everytime, is LocalWifiNlpBackend.
[Android Apps and Games] n76:
I will get you a version with debug turned on shortly. In the meantime, I think the discussion should continue on the bug report you opened rather than on this more general discussion forum.

https://github.com/n76/Local-GSM-Backend/issues/30
[Android Apps and Games] bilbonotry:
I think you need an MNC filter
[Android Apps and Games] barturblits:
Did you generate the db file through the settings dialog on your android phone or through cmd line with lacells-creator from n76 on a linux box? The former did not work for me, was really buggy, process was stalling etc and it had to download the main file for each try which is only allowed once every 24hours. lacells-creator worked perfectly though. With that you only have to download the main file once and use that to make selective db files.
[Android Apps and Games] tofu:
According to what I've read, it is not necessary.

---------- Post added at 08:32 AM ---------- Previous post was at 08:27 AM ----------

I generated it through gsm-backend settings dialog. It takes quite a lot of time, around 30-60 minutes depending on database size (and you cannot leave it on the background, hence phone cannot be used for anything else while generating), but at least for me the database creation seems to work. But thanks for the tip. I might try generating the database the way you suggested - let's see what we'll find out with the debug version of the backend. Discussion continues at https://github.com/n76/Local-GSM-Backend/issues/30
[Android Apps and Games] 1plus:
not sure if I should report in this thread, but it is easier I guess.

I did try unifiednlp on my wifi-only samsung tablet running on CM12.1. Not gapps-free but with the latest xposed installed.

It does work in the sense that UnifiedNLP works as a middleware. Can't same the same about backends though.

I haven't tried GSM backends for obvious reasons, as for wifi-capable backends the only working one was Apple.

I tried Mozilla, openb and localwifi. Mozilla does return a location but that's just a centre of a city with 50km or so accuracy range. Openb does not even return any location data at all. With apple backend enabled though the location accuracy is about same as normal (i.e. google NLP).
[Android Apps and Games] n76:
The local WiFi backend builds up its database when you have a good GPS lock. Basically it remembers where you have been so that you don't need to wait for the GPS "time to first fix" to have a reasonable good position the next time you start up a map or other app that wants location. If your tablet does not have a GPS receiver or if you wish to get a RF based location in an area that is new to you it will not suit your needs.

In my area the Mozilla and the Apple backends give roughly comparable results. It could be that volunteers contributing to Mozilla (and openb) are not very active in your area. I am not surprised that the Apple database is good for your area, it is my understanding that anyone with an iOS device is basically an involuntary contributor and there are a lot of iPhones out there. You could, of course, load the Mozilla stumbler and help make their data better. Same, I guess, goes for openb except that I haven't looked into them so I don't know how their data acquisition works.
[Android Apps and Games] 1plus:
well, that would be weird not to have any of these location services in this slightly overcrowded city :)

1. Location acquisition with openbmap works via their application called Radiobeacon.

2. Location service via that backend was not working whatsoever, but it was eventually fixed. Apparently their f-droid version is no good, I used a test apk downloaded directly from radiocell.org (their new domain). Now location service works.

3. Mozilla backend most likely does not work. Most likely due to device being wi-fi only? It is funny that it does identify the city but puts location in the middle with 50km or so range accuracy.

4. Is it possible to enable logging in the middleware application to see which applications requested location data and what they received? Just making sure google location service does not interfere.
[Android Apps and Games] n76:
UnifiedNLP outputs to logcat information about what backends it used and the results it got from those backends. It does not, however, show what apps were requesting the data. I don't think it has access to information about the requesting apps so I don't see how it could log that.
[Android Apps and Games] 1plus:
interesting. any idea if it is possible to see that only UnifiedNLP gets these location requests?

I.e. i'm trying to avoid using normal location service in non-gapps-free environment. Unfortunately for main devices I'm unable to remove gapps just yet due to reliance on gmail address :sad: Been using that gmail address for a few years, cannot just change it easily to anything else. Main privacy concern is location service though.
[Android Apps and Games] nussbaum:
Gmail addresses do not depend on device or gapps. Lots of people use K9 or others as mailclient.
[Android Apps and Games] cantenna:
This is pretty awesome, thank you for this. I'm having problems running this on Cm 12.1 5.1.1 , does this work with this version of android yet? If not I can happily downgrade. Thank you to anyone who replies:)
[Android Apps and Games] n76:
I ran UnifiedNlp on my Galaxy Nexus (maguro) using CM11 and with ZMoD 5.1.1 so I think it should run okay on CM12.1 (ZMoD is a largely AOSP ROM from the same fellow who does the CM12.1 support work for the maguro). I am currently running GmsCore which includes the UnifiedNlp functionality as well as some other stuff that makes life nicer like maps API v2.

https://github.com/microg/android_packages_apps_GmsCore
[Android Apps and Games] cantenna:
Thanks for the tip :)
[Android Apps and Games] Ultramanoid:
Working on my devices using 5.1.1 ( CM too ) just fine so far. ( O_o )
[Android Apps and Games] cantenna:
Can you guys recommend some apps I can use to test things are working correctly?

Edit; found satstat

Edit2; satstat only reports my carrier , limitation of my radio I guess.

Anyone suggest apps I can installed to confirm functionality?

Edit 3; read all pages good info!

Satstat only displays my registered network. Okay?

But, I have multiple blue and red dots on compass, does that mean gsm location is working? Thought the app had to display multiple gsm carriers other than my own to equal confirmed gsm location functionality but that does not appear to be the case, correct?
[Android Apps and Games] n76:
GSM satellites are government owned and operated and are independent of the mobile carrier you have. Or don't have (there are non-cell phone GPS receivers which have no carrier at all).

Satstat has a number of displays, one I like is accessible via a little world map icon and it shows two points with error circles around them. The red one is the position (and reported accuracy) determined by GPS and the blue one the position (and reported accuracy) based on network location. I am not sure if you need to install the Maps v1 API library first, but on my phone these are shown on an overlay over a tiled map.
[Android Apps and Games] bilbonotry:
You can use osmand or maps web version (or any app/site that asks for your location)
[Android Apps and Games] Goranius:
I have installed and it works fine. I have a gaps ROM.

Enviado desde mi MI 2S mediante Tapatalk
[Android Apps and Games] Xspeed:
The app works great under 4.3 on Xperia SP here, after the removal of everything from Google and Qualcomm that is location-related.

Theoretical question: would it be possible to make a Google Network Location backend for UnifiedNlp? Other backends work fine, but their coverage even combined isn't comparable with Google's...
[Android Apps and Games] Cookie Ninja:
In an unrooted phone googles NLP service is available (not for free, but for a fee) and in return, Google's servers ply you with ads. In a rooted phone without GMs core services, play services etc, where can they earn a revenue? Check up Google's map SDK and services, not free, available at a premium for map builders and users....I'm not sure of the details, but it caused a stink in 2012-13, causing apple to split and float their own mapping service on iDevices. So maybe NO GOOGLE CRAP! This is a FLOSS service with zilch revenue gathering opportunity and most probably, with a rooted phone, there may be a system level ad blocking by using hosts...

Dead Cookies leave no trails...
[Android Apps and Games] Nicolas Love:
Hi.

Please help.

I have KatKiss Android 6.0 Marshmallow. UnifiedNlp doesn't work there. It worked fine in Android 5.0 Lollipop.

Is there something in KatKiss ROM? Or UnifiedNlp is not compatible with Android 6.0?

It's great app for geolocation. Thank you, MaR-V-iN. I hope it can work in Marshmallow.
[Android Apps and Games] derbestimmer:
i have exactly the same problem. help is appreciated.

Gesendet von meinem Nexus 5 mit Tapatalk
[Android Apps and Games] 1plus:
It works on my nogapps CM13 the same way as on my other device nogapps CM12.1

Mind that I have a feeling that Mozilla plugin is somewhat glitchy. Best results from opebmap. I've seen that MLS plugin crashing a few times even.
[Android Apps and Games] Nicolas Love:
The problem is it doesn't work with any plugins, not only Mozilla plugin.
[Android Apps and Games] elmex1000:
First of all: Many thanks to MaR-V-iN for this absolutely fantastic piece of software. I'm astonished.

I'm using GSM Location Service since the first version days on several different phone's an tablet's. I am traveling quite a lot and was always happy to see that it worked wherever I was. Thats the reason for always downloading the whole db from Mozilla and OpenCellID (ie. without any MCC filters).

The last update I made was in October 2015. The 20th to be precise. Total lacells.db size by that time was 1.99 GB.

Today I found the time to update again. I ended up with a lacells.db size of 2.08 GB. Nice to see that the data is still growing.

However: GSM Location Service doesn't seem to like the new file. I tried it on different ROM's (eg. Sony Xperia 5.1.1, CM12.1). Under General Database, at the bottom in the app, the new file date from today is shown correctly. When I go back to the UnifiedNlp Module selection screen everything is still ok. But when I leave UnifiedNlp the GSM Location Service crashes.

And one more thing: The lacells.db file in the .nogapps folder gets deleted!

Again, the 1.99 GB file dated from October runs fine. But not the 2.08 GB new thingy. For me this looks like a limitation of the file size.

Can anyone confirm this?
[Android Apps and Games] n76:
Are you using the offline lacells generation script or using the built-in generation in the backend?

Can you get a logcat for when the backend crashes?

Posting this as a bug at https://github.com/n76/Local-GSM-Backend would be a good thing as it would help me track it.
[Android Apps and Games] elmex1000:
n76, thanks for your reply.

Up to now I always used the built-in generation feature. But I know of the scripts and I will try to do it with the lacells-creator-mod-master. Will report back.

Sorry for the possibly lazy question: How do you want me to get the logcat? Or in other words what cmd line should I use to get what you need?

For the bug report: Wilco. Or better.... done.

Oh, and I have to say sorry: Of course a big thank you goes out to you as well for the Local GSM Backend! I just realized that you are the dev behind it. Keep up the good work!
[Android Apps and Games] n76:
Assuming you have an Android development system installed on your computer, the typical (Unix) command line would be:

$ adb logcat > logdata.txt

That will capture the log continuously. Once you've caught the time when the forced close/crash occurs, use ctrl-C to terminate the capture.
[Android Apps and Games] elmex1000:
Ok. So no filter.... The whole thing.
[Android Apps and Games] n76:
You could probably filter but the simple filters that get the GSM backend log text don't get all the lines on a crash dump. So I usually just capture it all and then examine the resulting mess with an editor to find what I want.
[Android Apps and Games] elmex1000:
Ok. Done. Its on GitHub.

---------- Post added at 08:40 PM ---------- Previous post was at 08:04 PM ----------

More to the above:

I just created the lacells.db with the separate script. I had to use the gen_lacells_mergedMod one. Your original lacells-creator does not work for me. Throws some getopt error (-- o) upon launch, no matter what command line I use (I'm on OS X 10.10 there).

Anyway. The modded thingy produces the lacells.db well. However, and this is the interesting and possibly disappointing part:

The file is only 1.88 GB! It works as expected. So not really helpful regarding the original problem, but very interesting to know.
[Android Apps and Games] n76:
It may be a size issue.

Looks like SQLite is returning an error code 14 which seems like it can have several causes including unable to write to a file, unable to open file and file is corrupt. In any case the backend is handling this very poorly. If on write (database generation) it should not show a successful completion as the file would be corrupt. And on the location processing if that error is returned it should probably revert to the previous database file. And in any case, it should not fault.

One difference between the online and offline generation is the online generation leaves the duplicate towers between OCID and Mozilla in the database with the results merged on lookup while the offline generation merges the data from the two sources. So the offline created database will be smaller as you have seen.

For what its worth, it seems that Mozilla is picking up all the tower information from OCID. At least that is what appears to happen when I build a database. If that observation is correct, then you really only need to use Mozilla as a source as it will give you all of Mozilla's data as well as all of OCID's data.
[Android Apps and Games] elmex1000:
Thanks for the time invested to look into it. I appreciate it.

[MENTION=535343]all[/MENTION]: It might help if someone could confirm either that its working for him with lacells.db > 2GB or whether he has the same issue. Because at some point in time we definitely run into this problem... :)
[Android Apps and Games] EPT-3:
I installed UnifiedNlp (v1.4.0) and downloaded from F-Droid the only 2 available Backends (mozilla/ichnaea v1.1.0 and NominatimGeocoderBackend v1.1.0) for my ROM (CM 7.2 stable). Phone has NOGAPPS.

Altough OsmAnd+ functions perfect with GPS still no chance with GSM/WiFi, possibly (?) because of the lack of the GSM/Local GSM Backends.

However, in another smartphone (without NOGAPPS as well) running kitkat, GSM location works perfect.

Any chance for us with Android 2..3.7 having a workaround with a compatible GSM/Local GSM Backend?

Thanks!
[Android Apps and Games] n76:
Not sure why I picked the targetSdkVersion I did as it seems to compile fine for API 9 and API 10. I've pushed a v1.0.6 of the local GSM backend with the targetSdkVersion changed to github. If you get your copy from F-Droid it will probably take a few days to show up.
[Android Apps and Games] ahrion:
Go into the Apps and pick the Mozilla backend and enable all permissions. I had the same issue until I did that.
[Android Apps and Games] Nicolas Love:
Thank you, ahrion! You solved the problem of non working UnifiedNlp Wi-Fi Location in Marshmallow I had been having for many months!!!!!!!!! :D
[Android Apps and Games] amnher:
Any recommendation for good config back end? I tried all the backends but only Mozilla works and sadly it is not accurate?
[Android Apps and Games] n76:
Accuracy of the mobile/cell phone tower based backends is dependent on the data available to them. If you use the Mozilla stumbler program https://f-droid.org/repository/browse/?fdfilter=mozilla&fdid=org.mozilla.mozstumbler you can get the towers (and WLAN/WiFi access points) near you into their database and improve your accuracy. The Mobile/cell tower data collected for Mozilla is freely downloadable and can also then be used by the local GSM backend if you desire to run with data disabled (or simply reduce your data fees).
[Android Apps and Games] amnher:
OK. Does apple back end works for you?

I am kinda hesitant to do the offline one because of space constraint. I have read I need to download a certain amount like around 1gb+ for offline. I have an unlimited internet connection. And I use it when I was using gapps before. So I am looking for an accurate assisted GPS just like google servers
[Android Apps and Games] n76:
The Apple backend works about the same for me as the Mozilla backend. I normally use the offline local GSM and WiFi backends as I normally have data turned off.

The current data size for the local GSM backend on my phone for all of the United States is about 228MB. That is the app info shows that as the data size which is mostly the tower database. The size of the database on your phone will depend on which countries you download data for and how complete the tower information for those countries in Mozilla or OpenCellId.

The current data size for the local WiFi database on my phone is about 1.5 MB and has about 10.3K access points in it. This depends on how many WiFi/WLAN access points there are in areas that you frequent as it learns the location by observing WiFi signals when you have some application open that uses the GPS (the backend will not turn on the GPS). When going some place for the first time I normally have a navigation app open so the local WiFi backend is building its database as I go. So the phone's local database has information for most places I've been and definitely has data for everywhere I frequent. So typically when I start an app that wants my location it has a fairly accurate position immediately. The only time I might have an issue is on arrival in a new city where I haven't been through the airport before. But if that area has had its mobile/cell towers mapped, I can still get an reasonable location to help my GPS acquire (my phone has a lousy GPS).

Disclosure: I am the author of the local GSM and local WiFi backends.
[Android Apps and Games] amnher:
[MENTION=6204913]n76[/MENTION]

If I am using a VPN, will that affect the local gsm and WiFi back ends or other backends? Apple backend has never worked for me in my observation for the past two days of use

By the way I am from the Philippines
[Android Apps and Games] n76:
Use of a VPN should not affect network location: All the backends I've looked at base their location on the mobile/cell towers and WiFi/WLAN access points the phone detects, not on the IP address assigned by your carrier. A VPN that allows their traffic should not affect the results. But the Apple and Mozilla backends do need to talk to their respective servers and could be blocked by a firewall.

My local GSM and WiFi backends to not use network data for position estimation at all and cannot be blocked by a VPN or firewall. The local GSM backend will use data when you ask it to download tower information but it only does that when you tell it to and you'd want to do that when you have fast and cheap data (probably WiFi connection) as the data files it scans are quite large. My WiFi backend does not use data at all for any reason.

I believe that Apple builds their database using information collected by iPhones in the area. I suppose there are parts of the world where the number of iPhone users is low and the cell tower data is thus incomplete but it surprises me a little.

There were areas where the data in OpenCellId and/or Mozilla was sparse. When I first started using tower data I found that areas near where I lived in a pretty high tech part of the world were not mapped. That situation has improved dramatically and even one person running a stumbler while driving around can fill out their mobile/cell tower database very quickly. If your area is not well mapped you can correct it by putting a stumbler from one or the other of those projects on your phone and submitting data. (Mozilla seems to have a data sharing agreement with OpenCellID now so towers in the OpenCellID database are also in the Mozilla database.) Mozilla has a status map showing where data has been collected so you can see how well you are covered. https://location.services.mozilla.com/map#6/7.144/126.079
[Android Apps and Games] amnher:
[MENTION=6204913]n76[/MENTION]

thank you for this very detailed explanation. I'm sory if I had so many questions as this is my first time not to rely on google to access my location(Aside from GPS hardware of course). I know this might sound silly but will there be a bad effect if i have four backends checked? e.g. apple, mozilla, local wifi, local gsm
[Android Apps and Games] n76:
There should be no issue having all checked. The Unified NLP software will take the result from the backend that claims it has the best accuracy. Having two network based backends (Apple and Mozilla for example) will mean network traffic for both but if you have a low cost data plan that should not be an issue.
[Android Apps and Games] amnher:
@ n76 Ok now mozilla and apple works accurately as I am on wifi right now though not 100% all the time but better than data connection.

With regards to GSM local as I am interested in offline mode, I already updated the database and filtered with MCC = 515 ( Philippines) but blank MNC to cover all networks. The download has taken time but already finished. I can't seem to make it work. I have unchecked. Mozilla, apple and wifi location services. Tried both wifi and gsm checked but no result.
[Android Apps and Games] n76:
It might be nice to have a few lines from a command like

adb logcat | grep NlpLocationBackendFuser

to see what the various backends are reporting.

It is possible that the cell towers in your area are not yet in the Mozilla and/or OpenCellID databases. You can use an app like satstat to show you the MCC, MNC, LAC and Cell ID(s) of the cell towers your phone can see. You can then use that information to look up the tower at OpenCellId and see if the tower(s) are in their database. If the tower(s) was/were only just added it can take a day to get into the downloadable csv file the backend uses for populating its internal database. There may be some way of displaying Mozilla's data for the tower on a web page and that would be a good thing too but I am no aware of how to do that other than downloading the latest csv file and opening it with a text editor (too big for the spreadsheet software that I have).
[Android Apps and Games] amnher:
I'm on travel right now I'll try to logcat soon. By the it work suddenly as I am on another area but its a radius one see screenshot. I can't get an accurate position