yashsingh1/xdaforums huggingface dataset:
[Thunderbolt Android Development] yareally:
I put together a small shell script to make it easy for anyone rooted to block ads and check to see if they have the latest version I keep of adhosts. If after checking you do not have the latest, it will download it for you automatically.

What does this script do (in a nutshell):

1) Checks to see if you have the latest version of ad hosts for adblocking from me

2) If you do not have the latest, it will download it for you

3) If you have the latest, it just exits out, no changes

4) Installs the script to your /system/xbin directory so the next time you just have to type:

su
blockads

and it will run the script for you :)

How to run this?

Get this if you are using an AOSP ROM:

1) Download it from http://dl.dropbox.com/u/24904191/blockads

Try this if you are using a SENSE ROM (thanks chrootz for the fixes):

1) http://dl.dropbox.com/u/24904191/sense/blockads

*** If you have troubles getting the script to work, try updating your system to the latest version of busybox (stick it in /system/xbin and overwrite the current version of busybox, then change the permissions to 755 and owner and group to root) ***

2) Move it to /data/local/tmp (if you dont have it, create it) from a file explorer with root permissions (must set the system in r/w mode) or from the terminal shell

3) make sure the script has the right permissions by typing:

su
chmod 755 /data/local/tmp/blockads

in the terminal shell (in the emulator or whatever you use to get to it)

4) run the script by typing

/data/local/tmp/blockads

5) let it do its thing

After its done you can run it again anytime from the terminal by typing

su
blockads

as already mentioned.

I'm working on making it a GUI app next (with more features), but this should tie anyone who wants it over until then :)

Anyone who finds any adservers and wants me to add them to the list, feel free to list them here and I will add them.

Latest updates:

2011-12-01: Updated to block carrieriq
[Thunderbolt Android Development] newter55:
Tried this out and got this:

sh-3.2# su

sh-3.2# /data/local/tmp/blockads

Fetching host file via hostname

wget: bad address 'dl.dropbox.com'

update could not be completed, sorry :(

sh-3.2#
[Thunderbolt Android Development] yareally:
I forgot some ROMs and such have a busybox version that doesn't work right with hostnames in wget. I'll see if I can work around that and update the script :)

EDIT: updated to work around by using the IP for those that their busybox wget does not work correctly with hostnames. Only issue with that is the IP may change.
[Thunderbolt Android Development] GermanGuy:
Ok. I am stuck. I actually don't have a hosts file under "etc", only a hosts.bak. With root explorer mounted as r/w I am still not able to rename the "host.bak" file to "hosts" - error message "read-only file/directory". I created a blank "hosts" text file, put in on the sdcard and with terminal emulator still can't cp to "etc" because of "read-only directory" error message. Any other suggestions?
[Thunderbolt Android Development] DroidTh3ory:
Top right corner of Root Explorer .... Mount RW. :)
[Thunderbolt Android Development] spotmark:
Does this do anything differently then AdFree, from the market?
[Thunderbolt Android Development] newter55:
Updated blockads script to test this am and:

sh-3.2$ export PATH=/data/local/bin:$PATH

sh-3.2$ su

sh-3.2# blockads

Fetching host file via hostname

wget: bad address 'dl.dropbox.com'

The busybox version of wget on your system does not work correctly with

hostnames. Trying workaround with latest known IP to the hostname...

Connecting to 184.73.229.87 (184.73.229.87:80)

hosts 100% |***********| 641k 00:00:00 ETA

backing up old host file and updating

failed on 'hosts' - Cross-device link

md5sum: can't open '/system/etc/hosts': No such file or directory

/system/xbin/blockads: line 38: [: 8172d35d1b72871f7e1ef4e709920f0e: unary operator expected

Unable to chmod /system/etc/hosts: No such file or directory

Unable to chmod /system/etc/hosts: No such file or directory

remounting filesystem in r/o mode...

Cleaning up files...

rm failed for -f, No such file or directory

update complete!

sh-3.2#

The end result is the existing hosts file being renamed hosts.bak and obviously no new hosts filebeing being created.

EDIT.... The above is testing on my inspire running busybox 1.19. Tried using the script on my sensation running bb 1.18.4 and it works fine.

Edit edit... Installed bb 1.18.4 on inspire and result is the same as posted above.
[Thunderbolt Android Development] GermanGuy:
That's what I did, but it does not work - still does not let me modify read-only files.
[Thunderbolt Android Development] twistedumbrella:
After reading over the code and seeing the comments, I can only guess you had the modified version of wget on your device during testing, which does support host names. Before I continue, I am not trying to rain on your parade, but... By downloading the file, checking the md5, comparing it to the existing file, replacing if needed, and then cleaning up... you went about 5 steps beyond what just doing an "if server file is newer" check would do by including a custom wget. Even though you have to put in the code to install and verify the wget, the trade is that you no longer have to download the file every time, check it every time, and worry about making sure the md5 matches.

If you decide you want to switch over to that method, you can check out the option already included in http://xdaforums.com/showthread.php?t=1092128 by downloading the live install version to view the actual script. Just a recommendation to optimize the process. Nice work, though, you definitely found a way to get the job done.

As for the difference between this and Adfree. There is no real difference. Adfree checks the server, does a callback, and prompts the user with updates. Adfree just requires an entire shell to turn it into an app, but will automatically run on boot.
[Thunderbolt Android Development] spotmark:
So, for those of you not comfortable running ADB, or Terminal Emulator, just install AdFree, and let it do everything for you.
[Thunderbolt Android Development] twistedumbrella:
Well, you also have the option of most included ROM toolkits, ScriptFusion, and a lot of preinstalled adblock scripts, but the big difference is the hosts file itself. Sounds like this one is custom, whereas mine takes bamf and adfree and merges them. This one may end up blocking more ads, but the only way to know would be to sit down and compare them all.
[Thunderbolt Android Development] spotmark:
Ah, I see.
[Thunderbolt Android Development] yareally:
Thank you for your comments everyone :). I'll refine the script a bit more and also deal with a permanent workaround for not having a wget that checks hostsnames by testing with a ping and getting the IP first, using that and then doing wget. This started as a way for me to deal with having to move my ad host file each time I update it on my phone to just giving it out to everyone.

The only other difference than what was mentioned between my script and adfree as far as I know is I include a wider range of adhosts according to those that have used both. I have never used adfree, but people that have said I blocked more ads than they do and I also use an invalid IP address to block (0.0.0.0) instead of the loopback address (127.0.0.1) so it kills them a bit faster.

If anyone is still having problems that weren't resolved with it from above, let me know as well. I tested it against busybox 1.16.2 on my own phone, but it seems no 2 busybox version are alike :)

I havent tried this, but this one does include busybox 1.19 for those that don't want to manually install it. https://market.android.com/details?id=com.jrummy.busybox.installer&feature=related_apps
[Thunderbolt Android Development] twistedumbrella:
Hit me up if I can be any help. It's good to have a better list available. Adfree has never been very thorough.
[Thunderbolt Android Development] chrootz:
+1 I exam yareally's hosts, This is much faster killing them and I glad you finally developing the script update for the hosts. I am going test your script update on 2 of the roms A05PTH3ORY and Gengeritis 3D VII. Thanks - Keep it up :D
[Thunderbolt Android Development] yareally:
Added about 10-15 more hosts today.
[Thunderbolt Android Development] Zomb!e:
Thanks for the script. I'm going to give it a shot in a little bit. AdFree hasn't worked right in months. I've been using Absolute System to block ads, but I think I'll give this a try. Thanks again.
[Thunderbolt Android Development] yareally:
I'm about 50% done working on a basic gui app to do what the script does + some other features, just to give an update. GUI app will be free to any xda members of course :)

Things I am putting into it:

Choice between no blocking, partial blocking (basically in browser only stuff in case you want to show developers some love by not blocking their application ads), and full blocking.

Updating like the script does with just a touch of a button.

Optional ability to download a host block list for your desktop pc.

*features that may come later*

-ability to send me hosts from the app to add to the block list

-ability to add your own hosts on the fly to the list (either for personal routing or for adblocking, since adblocking is only the secondary use of the hosts file). Unfortunately, you will have to figure out the hosts yourself, since there's no easy way to tell really automatically.

Any other suggestions are welcomed
[Thunderbolt Android Development] Mustang302LX:
Sounds like AdFree but much better and will work better as well. Can't wait!!
[Thunderbolt Android Development] Camaros_Kill:
Just tried this today on gingeritis 3d beta8 and it doesnt seem to work as i get the same error someone else posted a few posts back. Something about no such file or directory system/etc/hosts. I reverted adfree and uninstalled it and renamed the hosts file to hosts from hosts.bak but when I Su adblock it gets renamed to hosts.bak and terminal shows that same error
[Thunderbolt Android Development] chrootz:
Same here, It failed on new version Gingeritis 3D VIII and I had do the manually by download the hosts then put in /system/etc.
[Thunderbolt Android Development] yareally:
Weird that it does that. i guess busybox versions change quite a bit and it's hard to account for them all when you do certain things. Bare with it if you can for now and just manually get the hosts if the script doesnt work from here until I finish the gui thing.
[Thunderbolt Android Development] chrootz:
I am running Busybox 1.19 verison on Gingeritis 3D VIII with latest version.

Right now, I am testing and tweak. Hope I can solve the problem and will upload the file for you to check it out before offically. ;)
[Thunderbolt Android Development] yareally:
If you can debug why the script doesnt work for me on sense roms always, that would be great. I haven't run sense in a while and cyanogen comes with an older version of busybox (1.16), but im avoiding touching it, because for once, hostnames work in wget :)
[Thunderbolt Android Development] chrootz:
OK, I fixed and it working this time. I am using latest Gingeritis 3D VIII (3.0 sense and latest base 2.11.605.0).

This is for 1.19 version of the Busybox. It apparently the busybox does not allow use mv caused error "Cross-device link" then use copy `cp` and the busybox's rm parameter really limited. I mean -f parameter is not support so use rm <file>.

I switch mv to cp instead and rid -f parameter for the rm.

I uploaded blockads.txt and all you need rid .txt (mv blockads.txt blockads) due xda doesn't allow me upload the valid extension. arggh Check it out :D

Here the 2 errors involved. Cross-device link and rm -f

Cross-device link error:

#./blockads
md5sum: can't open '/system/etc/hosts': No such file or directory
Fetching host file via hostname
wget: bad address 'dl.dropbox.com'
The busybox version of wget on your system does not work correctly with
hostnames. Trying workaround with latest known IP to the hostname...
Connecting to 184.73.229.87 (184.73.229.87:80)
hosts 100% |*******************************| 641k 0:00:00 ETA
[: ==: argument expected
backing up old host file and updating
failed on '/system/etc/hosts' - No such file or directory
failed on 'hosts' - Cross-device link
md5sum: can't open '/system/etc/hosts': No such file or directory
[: !=: argument expected
Unable to chmod /system/etc/hosts: No such file or directory
Unable to chmod /system/etc/hosts: No such file or directory
remounting filesystem in r/o mode...
Cleaning up files...
update complete!
#

rm -f error

# rm -f
rm failed for -f, No such file or directory
# rm
rm [-rR] <target>
#
[Thunderbolt Android Development] Camaros_Kill:
Okay, so i renamed your modded blockads file and replaced the one in the OP. then i fixed permissions and tried to update the file. I got the same rm failed for -f, no such file or directory that you got. now what?? lol i feel like a noob not understanding what to do here.
[Thunderbolt Android Development] chrootz:
@Camaros_Kill- It looked like you did not replace your current blockads. Download blockads.txt to your sdcard and make sure unmount sdcard when you are complete download the blockads.txt to your sdcard.

Try this on your terminal emulator. Let me know if you still having problem.

su
cd /data/local/tmp
rm blockads
cp /sdcard/blockads.txt .
mv blockads.txt blockads
chown root blockads
chmod 744 blockads
./blockads
[Thunderbolt Android Development] yareally:
Thanks for debugging it for me chrootz on sense roms. I added it to the OP and gave you credit for the fixes :)
[Thunderbolt Android Development] Camaros_Kill:
if i see hostfile is up to date, exiting...im good right? in system/etc i see hosts and hosts.bak, so it all seems good. thanks for the help!
[Thunderbolt Android Development] yareally:
If that's all you see and nothing that looks like an error message, you're all set :)
[Thunderbolt Android Development] chrootz:
Yup, Your set and working correctly. ;)
[Thunderbolt Android Development] jimmydene84:
Nice job man!
[Thunderbolt Android Development] chrootz:
Welcome, Gladly help and AOSP and Sense supposed be same. Have you test the script on your AOSP without `-f` parameter for RM?

It supposed not matter if it's in AOSP or Sense. Both are supposed be same. I mean it's Busybox is Busybox except different version. I think the newer Busybox went cut fat off (reduce bytes by unnecessary add more parameters code in the function programming).

I will test on AOSP CM7 and OMFGB later on. I used be hardcore in AOSP till I stopped using it due battery life was bad.
[Thunderbolt Android Development] amwenguer:
Got 2 thunderbolts. Mine running shiftaosp, and no issue. By girls running shiftsense and I'm getting this error:

/data/local/tmp/blockads

Fetching host file via hostname

wget: bad address 'dl.dropbox.com'

The busybox version of wget on your system does not work correctly with

hostnames. Trying workaround with latest known IP to the hostname...

Connecting to 184.73.229.87 (184.73.229.87:80)

wget: server returned error: HTTP/1.1 503 Service Unavailable: Back-end server is at capacity

update could not be completed, sorry :(

I did update busybox installer pro on hers
[Thunderbolt Android Development] yareally:
Looks like the IP address no longer works for it or at least not at the moment (the updated busybox is probably what stopped the hostname from working though). I updated the IP fallback for it though. Switch the IP in your script to http://50.17.213.175/u/24904191/hosts on line 12 or redownload it again. Whichever is more convenient for you.

Might want to try the script made for sense that has some modifications for her instead. The busybox installer version is more similar to that, since it's statically compiled. Both versions work exactly the same on the user level though.
[Thunderbolt Android Development] yareally:
Oh yeah, it should work fine without the -f on rm. I'm just used to typing -f since i symbolically link rm to interactively prompt me and was looking out for anyone else that might do that too. I'll merge them back together since there's no need for 2 scripts.
[Thunderbolt Android Development] amwenguer:
I actually downloaded the sense version to start with. I tried the fix you posted on page 3 with blockads.txt as well. Had a previous version of busybox that I thoughtwas outdated and that is why I purchased and used the newest pro installer.

Brain is a little fries from rooting hers today and setting it up. Will try your work around tomorrow. Thanks for the very quick reply.
[Thunderbolt Android Development] yareally:
No problem :)

Some reason most of the AOSP roms use an older busybox version, which does have the benefit of being more compatible it seems with a normal shell in linux (which is where I was debugging first before testing on the phone). I had no idea they modded it in later version to remove a field from the remove command. Odd but I guess they felt it wasn't needed.
[Thunderbolt Android Development] amwenguer:
I redownloaded (sense version), and ran it the same way in term emulator. Got the same error. Think ill try the aosp version, or update the host file. Any suggestions? Ill reread the first few posts to figure out how to update
[Thunderbolt Android Development] yareally:
Try seeing if http://50.17.213.175/u/24904191/hosts will open in your browser. I'm thinking it's an issue with drop box.
[Thunderbolt Android Development] amwenguer:
Downloaded the new host file and placed in system/etc, fixed permissions, changed the owner/group and then tried again with the sense version. Got this:

# export PATH=/data/local/bin:$PATH

# su

# chmod 755 /data/local/tmp/blockads

# /data/local/tmp/blockads

Fetching host file via hostname

wget: bad address 'dl.dropbox.com'

The busybox version of wget on your system does not work correctly with

hostnames. Trying workaround with latest known IP to the hostname...

Connecting to 184.73.229.87 (184.73.229.87:80)

wget: server returned error: HTTP/1.1 503 Service Unavailable: Back-end server is at capacity

update could not be completed, sorry :(

#

Should I try and revert back to an older busybox? Which?

Thanks for the assistance!
[Thunderbolt Android Development] yareally:
Try using 1.16.2 if you can.

Also updated the list a bit more, adding some nasty tracker that facebook uses.
[Thunderbolt Android Development] amwenguer:
You mentioned an app you were developing. You releasing it soon?

Not having much luck on sense.
[Thunderbolt Android Development] yareally:
Next couple weeks I'll have something basic out, or sooner. Just working on it in my spare time. You should be able to set up the host file with the shell in adb if you have adb set up in the mean time if you don't want to wait. Stick the host file on your PC in the same directory as adb, then do the below:

From the command line type:

adb push hosts /data/local/tmp/hosts
adb shell
mount -o remount,rw -t ext3 /dev/block/mtdblock5 /system
mv /system/etc/hosts /system/etc/hosts.bak
mv /data/local/tmp/hosts /system/etc/hosts
chmod 644 /system/etc/hosts
chown root /system/etc/hosts
mount -o remount,ro -t ext3 /dev/block/mtdblock5 /system
exit
[Thunderbolt Android Development] amwenguer:
Ahhhhhhh. Just updated ROM (still aosp which works just fine normally)... and I'm getting..

sh-4.1# /data/local/tmp/blockads

Fetching host file via hostname

wget: bad address 'dl.dropbox.com'

The busybox version of wget on your system does not work correctly with

hostnames. Trying workaround with latest known IP to the hostname...

Connecting to 50.17.213.175 (50.17.213.175:80)

wget: server returned error: HTTP/1.1 404 Not Found

update could not be completed, sorry :(

sh-4.1#

Will try busybox version you recommended. Never had a problem on aosp
[Thunderbolt Android Development] yareally:
Updated the list with about 30-40 more adservers.

I got sidetracked from doing other stuff, but trying to wrap up the gui adblock app now (at least the initial version).
[Thunderbolt Android Development] amwenguer:
Much thanks! Definitely prefer this over adblock and AdAway. Looking forward to the app and new things you're working on.
[Thunderbolt Android Development] phoney6:
Where can I download just the host file? Thanks!
[Thunderbolt Android Development] yareally:
https://gist.github.com/8cbcc2dd3778972b2799
[Thunderbolt Android Development] yareally:
Updated to block carrieriq data and requests from being sent out of your phone.
[Thunderbolt Android Development] Wingman520:
getting this same issue on the sense rom i'm running (gingerbreation 1.0) i'll just wait for the basic program to come out and try it then :) definetely want to try this! hate adblock since it doesn't block as many ads anymore like it used to
[Thunderbolt Android Development] yareally:
Trying to get it out over xmas break when I have lots of free time (which starts after this week).

This should make things work for you if you get the proper version of busybox and overwrite the one on your device.

http://busybox.net/downloads/binaries/latest/busybox-armv6l

Get that and stick in in /system/xbin with root explorer or es file explorer and overwrite the busybox version there. After that, change the permissions on it back to 755 via your same explorer app and if possible change the owner and group to root.

After that, redownload the script once again.
[Thunderbolt Android Development] Wingman520:
not seeing a busybox labeled file in system/xbin

read on a search and one said df was busybox and i looked and saw a few files labeled busybox next to it (using root explorer) not sure which one to replace and don't wanna replace the wrong one to mess it up
[Thunderbolt Android Development] yareally:
screen shot whatever you see. that will help.
[Thunderbolt Android Development] Wingman520:
did one better since there was so much in this file... did a screen capture link for ya

http://dl.dropbox.com/u/28575641/cast20111213_025032.mp4

you can d/l it or view if u have quicktime installed since it was done in .mp4 format just fyi
[Thunderbolt Android Development] yareally:
Hmmm...it looks like busybox is missing for you (if you didnt set it that way, I would speak to the creator of your ROM, because it should be in that directory and perhaps it's a mistake on their part), despite things being symbolically linked to it. It shouldn't hurt to just stick the busybox binary from the link in the OP in your xbin directory. Just set the permissions to 755 (rwxr-xr-x) and the user and group to root.

Also, before you do the above, check your /system/bin directory as well. It really should be in xbin, but never know.

you can also try using adb

adb shell ls -l /system/xbin

and paste what dumps out.
[Thunderbolt Android Development] Wingman520:
It's in system/bin so just copy paste after renaming file then change permissions and such then try again? If so will do tonight and post back
[Thunderbolt Android Development] yareally:
Yep, if it's in /system/bin just overwrite it with the updated busybox, keeping the same permissions, user and group that the current one has. Just in case of course, back up your current one by making a copy of it (even if there isn't an issue, it's just a good practice). That should do it all.
[Thunderbolt Android Development] Wingman520:
Works like a charm thanks for the support! :-)
[Thunderbolt Android Development] yareally:
glad you're all set :)
[Thunderbolt Android Development] amwenguer:
New phone and the download for tthe aosp version isn't working. Who can upload it on here? Hoooooook it up. Please.

This nexus needs ads blocked, this way.
[Thunderbolt Android Development] rrrrrrredbelly:
not sure if this list is still being updated, been awhile since last post to thread. cant run this script. i've copied the 'blockads' file to the /data/local/tmp folder, ran chmod 755, but when i try to run it, says file not found. before I had to send the phone out for a warranty repair (phone came back un-rooted), I was able to run this with no problems.

any ideas? am i missing something obvious?

i made sure i have latest busybox, and that the blockads file is spelled correctly with no file extention.

thanks

dan