yashsingh1/xdaforums huggingface dataset:
[Miscellaneous Android Development] steveglowplunk:
Disable Tracking Services

This is a set of scripts that disables privacy invasive tracking components in all user installed apps (service/broadcast receiver/activity/content provider).

Root permission is required to run the scripts.

How it works:

UpdateExodus.sh downloads tracker names from Exodus Privacy.

DisableTracking.sh dumps a list of components of all installed apps, compare it against the list of trackers name, and disable those components (using the 'pm disable' command) if they match.

Download: Google Drive link

Check instructions.txt for details on how to use the scripts.

The latest version is 1.8.

You are recommended to backup your apps before running the scripts.

You are also recommended to do a full nandroid backup before running the scripts.

I hope you will find this useful :)

The script was originally based on the one here: https://xdaforums.com/android/development/script-disable-fk-services-trackers-apps-t4074427
[Miscellaneous Android Development] steveglowplunk:
Update log

Version 1.5:

-DisableTracking.sh now uses a database to store processed app names and versions.

This means if an app was updated, it will check for trackers in the updated app again.

-If there was a deleted app, DisableTracking.sh will now delete its record from ProcessedApps.db automatically. No more manual deletion.

-Note: if you've used version 1.4 or below and don't want the script to process all apps again, run CreateAppDB.sh.

It will create a ProcessedApps.db with records of all currently installed apps.

Version 1.6:

-DisableTracking.sh now supports checking for trackers in system apps. Check the usage section for how to use.

-Improved logic of DisableTracking.sh to reduce problems. Thanks to hyborian for pointing them out.

-Added checks for root permission and presence of extra binaries.

-The new script 'InstallReq.sh' will automatically install missing extra binaries from Termux.

Check the prerequisites section for how to use.

-UpdateExodus.sh will no long generate exodustrackers_extracted.txt. The content is stored in memory instead of a file.

-MergeExodus.sh is no longer needed. The functions are...merged into UpdateExodus.sh.

Version 1.7:

-DisableTracking.sh now shows the app name as seen in the launcher when processing an app (will be empty if the app name is not in Engish)

-DisableTracking.sh will skip apps in whitelist.txt.

-UpdateExodus.sh now shows new tracker names that were not previously present in the old MergedTrackers.txt.

Version 1.8:

-Small fix in DisableTracking.sh that stops it from creating duplicated records in ProcessedApps.db

(in line 165, grep -qx is chnaged to grep -q)

If you've used version 1.6 or above, you are recommended to delete ProcessedApps.db located in the script's directory and run CreateAppDB.sh to regenerate a ProcessedApps.db, although the existing ProcessedApps.db with duplicated records may still work.
[Miscellaneous Android Development] steveglowplunk:
reserved 3
[Miscellaneous Android Development] hyborian:
Thanks, fantastic tool
[Miscellaneous Android Development] hyborian:
del
[Miscellaneous Android Development] Random-Tester:
Hello I got a little issue with your tools ( it says to i need to verify my internet connection but I have wifi on. I have tried in mobile data still same ...

(When I open the web site manually it work but during the script execution I have this issue any other method?
[Miscellaneous Android Development] steveglowplunk:
Ok try running the script in attachment (remove .txt in the name), copy/screenshot the output and post it here.
[Miscellaneous Android Development] Random-Tester:
Hi thnx for fast answers !! Unfortunately nothing new

I got this in terminal :

$ su

:/data/data/com.termux/files/home # /system/bin/UpdateExodus.sh

curl not found

wget not found

jq is a tracked alias for /sbin/jq

Checking connection with https://etip.exodus-privacy.eu.org/trackers/export

/system/bin/UpdateExodus.sh[26]: curl: inaccessible or not found

Network check status is empty

Unable to connect with https://etip.exodus-privacy.eu.org/trackers/export. Please check you internet connection.

No changes done.

Script will exit now.
[Miscellaneous Android Development] steveglowplunk:
Yes, the purpose of the script was to find out missing binaries, and it says you're missing curl and wget. This could be because you don't have busybox installed (wget is included in busybox) and you're running stock rom (curl should be included in a custom rom such as LineageOS).

What you can do is download those binaries from Termux. (Use the command 'pkg install curl', same for wget), and copy them to /sbin, just like what you'd do with jq.

curl is used for checking connection status. wget is used for downloading the trackers json file.
[Miscellaneous Android Development] phantom146:
After executing everything perfectly, I ran to a weird instance where everytime it scans an app, a command sqlite3: inaccessible or not found was shown. I was wondering if this shows because I performed disable f*ck services script first then this script? are they supposed to do the same? does that mean the apps I'm scanning right now already have their services/receivers disabled thats why disabletracking.sh can't detect them? thank you!
[Miscellaneous Android Development] steveglowplunk:
That means sqlite3 is not installed (I thought it's supposed to be included in Android?). Verify it by running 'type sqlite3' in the terminal.

Maybe you can try downloading sqlite (without the 3) from Termux with the 'pkg install' command, then copy the downloaded binaries to /sbin (similar to what you'd do with jq). Or you can use version 1.4 for now.

sqlite3 is used for writing a database that stores processed apps names and versions, so that they will be skipped in the next run. The script will still disable tracking components that haven't been disabled already.
[Miscellaneous Android Development] phantom146:
This worked well. Thank you.

Do you plan on implementing something for sqlite, jq and aapt to stick on sbin for the next update?

Also, some of the apps I'm scanning shows:

Exception occured while executing:

java.lang.illegalArgumentException: component class bugsnag.com etc. Etc. Does not exist in bla bla app.

I assume exodus detected that a certain app has trackers and tries to disable it but since I used disablefohkservice script before this, I guess it was already disabled and disabletracking.sh can't find it now?
[Miscellaneous Android Development] steveglowplunk:
The error you mentioned is normal. That component cannot be toggled.

Also, please try running the script attached in Termux as a normal user (root permission will be required half-way through the script) by running 'bash /path/to/script/InstallReq.sh' . It will check if aapt, jq, curl or sqlite3 are missing and automatically download them, then copy them to /system/xbin. (binaries there should retain across reboots). You are recommended to backup at least your system partition in a recovery. Although tested on my own phone and it shouldn't overwrite anything important in /system, I'm not responsible for any damage caused.

Edit: small change to the script to avoid problems with checking curl
[Miscellaneous Android Development] steveglowplunk:
Updated script to version 1.6. Check update log for details.
[Miscellaneous Android Development] hyborian:
I just added a 2> /dev/null to the disable line so I don't see it ...

---------- Post added at 10:10 AM ---------- Previous post was at 10:09 AM ----------

Great, will try it ...
[Miscellaneous Android Development] hyborian:
All good :good:, I just added code to delete old app versions from processed db, and to recreate DBFullList variable again after the sqlite changes.
[Miscellaneous Android Development] steveglowplunk:
Um... That doesn't sound necessary?

The version field in the database is updated after processing an updated app. There shouldn't be old app versions left behind?

Also, DBFullList doesn't need to be updated after processing an app. If an app's name doesn't exist in DBFullList, then it must be a new app. If it exists in DBFullList and was processed, then it must be an updated app. Not updating the DBFullList variable after processing each apps shouldn't affect the functions.
[Miscellaneous Android Development] hyborian:
Lol, you are right, I was looking at my old processed db which had apps with different versions. I should just have erased it and started fresh.

Anyway, it was nice practice for me. learned something while doing it when I couldn't grep the darn version number of Google Play Store (Version: 20.2.09-all [0] [PR] 311811225 (82020900) in DBFullList. I finally realized it had metacharacters [ ] so changed the grep from -qx to -qFx, and all ok now; otherwise, it will get processed again even if it was already done.

Thanks for the script, really great, beats manually doing it using MyAndroidPro or Blocker.
[Miscellaneous Android Development] steveglowplunk:
Ah I see. I'll add that -F option in the next update.
[Miscellaneous Android Development] alipendier:
[MENTION=6944590]steveglowplunk[/MENTION]
[MENTION=8671532]hyborian[/MENTION]

This collaboration is really admired and appreciated, thanks! :good::cool:
[Miscellaneous Android Development] xontax:
Really useful, thanks a lot
[Miscellaneous Android Development] caltkevv:
How to fix error:

Unable to connect with https://etip.exodus-privacy.eu.org/trackers/export. Please check you internet connection. ?
[Miscellaneous Android Development] steveglowplunk:
It means either you really can't connect to Exodus Privacy or the curl binary is missing (check by running this command: 'type curl')

If it's the second case, run InstallReq.sh in Termux to fix that (check the 'Prerequisite' section in instructions). Doing a full backup first is highly recommended.

If curl is not missing, post your output of this command: 'curl -sI https://etip.exodus-privacy.eu.org/trackers/export'

BTW I just realised there's a typo in the error message, I'll note that down.
[Miscellaneous Android Development] DirtyAngelicaSecured:
Cool! Does the script block more stuff than AdGuard Premium?
[Miscellaneous Android Development] steveglowplunk:
Which AdGuard do you mean?

Anyway, this script doesn't only disable ad services in apps, but also disable analytic services listed on Exodus Privacy's website.
[Miscellaneous Android Development] caltkevv:
https://privatebin.net/?c40f4fb8bfc2585f#JEFTmph5QKapcMQxAzVFdxSga4AaZWB7FPDnQvMC3LJw
[Miscellaneous Android Development] steveglowplunk:
Are you sure you typed the command correctly? It's not supposed the output the json file.
[Miscellaneous Android Development] caltkevv:
You're right:

Was equally dumbfounded by that output; anyway:

HTTP/2 200

server: nginx

date: Sat, 06 Jun 2020 16:19:13 GMT

content-type: application/json

content-length: 61019

vary: Accept-Encoding

content-disposition: attachment; filename=trackers.json

x-frame-options: SAMEORIGIN

x-frame-options: SAMEORIGIN

x-content-type-options: nosniff

x-xss-protection: 1; mode=block

strict-transport-security: max-age=31536000; includeSubdomains;

access-control-allow-origin: none

content-security-policy: default-src 'self'; style-src 'self' 'unsafe-inline' stackpath.bootstrapcdn.com; script-src 'self' 'unsafe-inline' stackpath.bootstrapcdn.com code.jquery.com cdn.jsdelivr.net; img-src 'self' data:; font-src 'self'; base-uri 'none'; frame-ancestors 'none';

referrer-policy: strict-origin-when-cross-origin

feature-policy: geolocation none;midi none;notifications self;push none;sync-xhr self;microphone none;camera none;magnetometer none;gyroscope none;speaker self;vibrate none;fullscreen self;payment none;
[Miscellaneous Android Development] steveglowplunk:
Weird, it should work if you're using the latest version of UpdateExodus.sh. I'm not sure how it could be fixed.

One workaround is to remove the network check (line 48 to 56 in UpdateExodus.sh)

The network check works by checking for the word '200' in the output of curl. Your output contains '200' so I have no idea what's broken.
[Miscellaneous Android Development] xontax:
Is their a way to implement this on a routine? E.g. plugin the phone to charging cable or after a restart.

The outcome of this method is great. Had 4 days on battery without charging instead of 2 days without the script. App tracking really is a massive battery drain.
[Miscellaneous Android Development] steveglowplunk:
Try using something like tasker. It's only one command anyway (su -c sh /path/to/script/DisableTracking.sh).
[Miscellaneous Android Development] DirtyAngelicaSecured:
Does this method disable persistent Google Play Services for location, analytics, etc? It is the main Android platform process and it cannot be stopped.
[Miscellaneous Android Development] steveglowplunk:
Theoretically no. If you want to reduce tracking to the minimum, you should switch to a custom rom and use microG.
[Miscellaneous Android Development] DirtyAngelicaSecured:
Ah... Huge thanks for the effort to reduce the amount of tracking. Also, your list can be easily imported into AdGuard User Rules :D.
[Miscellaneous Android Development] caltkevv:
Removing the network check worked. Thank you.
[Miscellaneous Android Development] adriansticoid0312:
Something's wrong with Advanced Download Manager. The script disabled all the necessary components and left enabled all the tracking components.
[Miscellaneous Android Development] hyborian:
I installed that app just to try after reading your post. It sure is a unique one, almost all components are disabled. It's like it is treated as one giant tracking machine lol. Uninstalled the app through other means, as it disappeared from the launcher.

Anyway, just unblock them all using Blocker ... it shouldn't get processed again unless a new version comes.
[Miscellaneous Android Development] adriansticoid0312:
Yes, almost all are disabled, but "InstallTrackerReceiver" was left enabled. One I would expect the script would disable.

It disappeared from your launcher because it thought it doesn't have any launchable activities. In my case, it didn't disappear, but when I tap it, it takes me straight to the app info in settings.
[Miscellaneous Android Development] hyborian:
You can check the merged tracker file to see if that tracker is there; otherwise, add it in Custom Tracker file. You need the full name of the tracker I think, which I can see if I use MyAndroid Pro, I don't know about Blocker.

Maybe next feature, if inclined, can be a file to include apps to be ignored, or even apps with their tracking components one wants to ignore. I don't need it with my system now, but it might be useful. I do have an app SendAnywhere where I have to unblock 2 "trackers" or it won't work. I just modified the script to watch out for it, it's only 1 app anyway.
[Miscellaneous Android Development] radogost:
Huge thank you for the script. Works great! I wonder how it could be automated without any external apps like Tasker. Once again thank you for your time.
[Miscellaneous Android Development] steveglowplunk:
Yes you are right. This is because most components of ADM contains the word 'com.dv.', which coincides with one of the tracker names. As hyborian suggested, you could use something like Blocker to re-enable them. Since ADM is already in ProcessedApps.db, it won't be processed again in the next run and so won't cause problems.

I should probably make a whitelist function in the future.
[Miscellaneous Android Development] adriansticoid0312:
Or probably don't process "com.dv" if the app's package name is com.dv.adm?
[Miscellaneous Android Development] steveglowplunk:
Updated to version 1.7.

I have added a whitelist function that skips processing Advance Download Manager now.
[Miscellaneous Android Development] xontax:
Just a little script to safe some time..

sh run.sh

#!/bin/bash
bash /sdcard/download/InstallReq.sh 2> /dev/null
su -c sh /sdcard/download/UpdateExodus.sh 2> /dev/null
su -c sh /sdcard/download/DisableTracking.sh 2> /dev/null
[Miscellaneous Android Development] DragonPitbull:
Many thanks for you of the script. I will try it out soon.
[Miscellaneous Android Development] steveglowplunk:
Updated to version 1.8.

-Small fix in DisableTracking.sh that stops it from creating duplicated records in ProcessedApps.db

(in line 165, grep -qx is chnaged to grep -q)

If you've used version 1.6 or above, you are recommended to delete ProcessedApps.db located in the script's directory and run CreateAppDB.sh to regenerate a ProcessedApps.db, although the existing ProcessedApps.db with duplicated records may still work.
[Miscellaneous Android Development] xontax:
Bug: !!

Please check your external update tracker, it will update .com and disable all packages
[Miscellaneous Android Development] steveglowplunk:
Huh? I don't understand

Edit: uh oh you're right. I'll investigate later.
[Miscellaneous Android Development] xontax:
Its a bit hard to explain. All packages *.com are going to be disabled.

Thanks for your help

Edit: there is space in

com. hreesixtydialog\.sdk\.

which should be removed
[Miscellaneous Android Development] steveglowplunk:
Just updated UpdateExodus.sh in the Version 1.8 folder in Google Drive. Delete the old MergedTrackers.txt before using.
[Miscellaneous Android Development] steveglowplunk:
Updated UpdateExodus.sh in the Version 1.8 folder in Google Drive. Added a quote at line 63 after the 'cat' command to prevent problems with directory names containing spaces.
[Miscellaneous Android Development] hyborian:
Just some code at the end to remove ...
[Miscellaneous Android Development] steveglowplunk:
I wrote those so that people can know the script has finished.
[Miscellaneous Android Development] hyborian:
It's extra code, duplicated.

End of if ... else became "fickers.txt" instead of fi
[Miscellaneous Android Development] steveglowplunk:
Huh those aren't in the original script, it should end at line 101.
[Miscellaneous Android Development] hyborian:
I got it from 1.8 folder of google drive, and it executed but showed the error at the end ... Let me download again and see

---------- Post added at 07:28 AM ---------- Previous post was at 07:19 AM ----------

You are right, i downloaded again and it's ok. Sorry about that, i dont know how it happened *scratch head*
[Miscellaneous Android Development] problemathic:
$ bash /data/local/tmp/InstallReq.sh

bash: /data/local/tmp/InstallReq.sh: Permission denied
[Miscellaneous Android Development] steveglowplunk:
That sounds like the script is missing execution permission

Try:

su

chmod +x /data/local/tmp/InstallReq.sh
[Miscellaneous Android Development] problemathic:
its fine now. the script wont run on su. only on termux normal
[Miscellaneous Android Development] steveglowplunk:
Yeah, as I stated in the instructions post.

What the script does is:

1. Detect if aapt, jq, curl and sqlite3 are installed

2. If not, install them to termux's own storage as a normal user (root user won't work at this part)

3. Copy the downloaded binaries to /system/xbin so that DisableTracking.sh can use them
[Miscellaneous Android Development] problemathic:
its fine now bro. check attached file. the script is working really great.
[Miscellaneous Android Development] steveglowplunk:
Great, I hope you find it useful :)
[Miscellaneous Android Development] problemathic:
no need to do anything right ater every reboot?
[Miscellaneous Android Development] steveglowplunk:
If DisableTrackers.sh or UpdateExodus.sh don't say you're missing any binaries, you don't need to do anything. But if they do, run InstallReq.sh again.
[Miscellaneous Android Development] problemathic:
1st reboot says i have missing aapt binaries then i run the installer, update and disable scripts.

2nd reboot says i have missing jq binaries then did the same process again.

3rd reboot says i have again missing jq binaries..

i guess this has to be done every reboot to make sure all are working. thanks for this bro. it surely helps
[Miscellaneous Android Development] laclac:
I confirm these script doesn't work for me. ( InstallReq.sh, DisableTrackers.sh or UpdateExodus.sh).

I think the problem is with sqlite3. The script can't install it.

And manually with the command "pkg install sqlite3"

I have the lessage:" Unable to locate package sqlite3"

With aapt, jq or curl, I don't have the problem.
[Miscellaneous Android Development] steveglowplunk:
My bad, sqlite3 belongs to a a package called 'sqlite', and the script was trying to install sqlite and then copy a binary called 'sqlite' (which doesn't exist). I've made some changes to the script (see attachment). Remove the .txt extension to use it

EDIT: I just realised the script won't properly detect whether sqlite3 exists outside of Termux. Please use InstallReqv2.sh instead.
[Miscellaneous Android Development] sixisixsix:
If I replace all of the mentions of "xbin" and replace with "sbin" (because I use magisk) will the script automatically copy to sbin instead of xbin? I am assuming it would, but do I also need to change " #!/system/bin/bash" to sbin as well?

Lastly can you help me make a termux shortcut to install all necessary files to sbin then run disabletrackers.sh?

Forgive my ignorance, I'm still in the very early learning stages of linux cli.

Thank you so much for this script! I really would like to run the script against system apps, but while whitelisting play store, play services, google framework etc. Because I have shelter installed (work profile) when I run a twrp backup it is unable to decrypt work profile so I a afraid to try it before I have a solid backup.
[Miscellaneous Android Development] steveglowplunk:
Yes it will, but it won't make much difference because magisk resets sbin after every reboot anyway.

You can actually just ignore the bang line because you have to specify running the script with bash anyway (you should get 'permission denied' if you run the script directly like './InstallReq.sh' because it doesn't have the execution permission, and you can't change permissions for files stored in the internal storage somehow).

You could probably try Termux's home launcher widget.

I still strongly recommend a backup before running my script. Although I've tested the scripts on my phone and made sure they work as intended, I can't guarantee they'll work perfectly on your device.
[Miscellaneous Android Development] hyborian:
I have no xbin too, and I copy the binaries to /system/bin ...
[Miscellaneous Android Development] steveglowplunk:
Huh maybe that's because I'm still on android 8.1
[Miscellaneous Android Development] hyborian:
Maybe it's just rom related. Like I have xbin on miui android 10, no xbin on custom rom A10. Moved on to A11 rom, and no xbin too.
[Miscellaneous Android Development] sixisixsix:
On oneplus 7pro if I run installreq.sh it doesn't copy to the correct place and changing all "xbin" to "sbin" broke the script. For now I just download to termux directory and drag drop through gui lol.
[Miscellaneous Android Development] steveglowplunk:
Huh send me your version of the modified script. I don't think changing xbin to sbin could break anything.
[Miscellaneous Android Development] geekborg:
GUI

Hi,

is a GUI planned for these scripts ?

It would be nice to see an apk that quickly installs and enable the script at each boot once you give the root permissions.

Easily access the logs, see what services were disabled, and easily re-enable a service in the case of an application bug.
[Miscellaneous Android Development] steveglowplunk:
Actually there are already two beautifully made apps with GUI that can do the same thing as my scripts.

One being App Manager (https://github.com/MuntashirAkon/AppManager)

And another one being Warden (https://gitlab.com/AuroraOSS/AppWarden)

My scripts only exist as a simple, barebone alternative.
[Miscellaneous Android Development] hyborian:
Use App Manager as pointed out by steveglowpunk. It uses IFW method to block components, so the apps can't undo the blocking (Warden is just using the usual pm disable). Furthermore, it takes less than a minute to block all apps (less than 10 seconds for what would usually take almost an hour or more for me). Plus it really is a great App Manager.