yashsingh1/xdaforums huggingface dataset:
[Android Apps and Games] Mike B.:
As usual, the first step should always be to look at the DNS Request Log. To spot the hosts that need to be whitelisted, temporarily disable AdAway first. Once you see which hosts are accessed, you have to mostly guess which ones contain the ones you wish to block/unblock (the hosts names are sometimes helpful, but not always). It's often easier to simply access the same page from a browser without any "blocking" (such as on your laptop/desktop) and see the exact location of the elements you want to see. Most browsers can show you what a link points to by hovering over that link. If you're able to do so, you can also look at the page's source (including any external CSS or scripts), but it can get very time consuming. It's not a science, it's an art! ;)
[Android Apps and Games] arteaga323:
Yes and that's exactly what I did, literally everything you had mentioned back when we were trying to solve my problem, I had already done it, however, though I did get a dozen links when I checked the log, not one of them were the ones mrRobinson told me to whitelist. I had whitelisted all of them, didn't work, somehow he got a different link that I didn't get and I just don't know how that happened or what I missed.
[Android Apps and Games] Mike B.:
You need to be sneaky. Those links may have been hidden in CSS or scripts and not readily available from a cursory look at the log. It also helps to use as few source files as possible and look at your generated hosts file to compare those entries to the list of DNS requests. It's not a quick or easy process. No magic involved, just lots of time and a little bit of luck. When you ask someone else to do that for you, it also means that they have to spend a lot of time looking at sites with content they may not be too thrilled about.

Once again, none of that has much to do with AdAway and it get a bit tough trying to "support" every site and app out there that pushes ads. The support areas for whatever source files you use would be much more suitable venues... ;)
[Android Apps and Games] arteaga323:
Thanks I'm still lost about how to go around and find the culprits but nonetheless, thanks
[Android Apps and Games] Mike B.:
The true "culprits" are actually the source files you selected. The more hosts they block, the more likely they are to block something you want. The easiest way to eliminate problems is to use as few source files as possible, and stick to the least aggressive ones. Even then, you may still want to see a few hosts that other people wish to block. One person's annoying ad is another's useful information. If you had the time, you could start with a blank slate and only blacklist the hosts that you actually encounter during your browsing, but that's not a realistic approach for most people.

It's important to remember that each user has his/her own browsing habits, interests, and tolerance for "junk". For example, I would never suspect any problem with Facebook since I never use that thing, and I could have gone a lifetime without ever trying to see some of the videos missing from the site you mentioned. You start with default source files that are "good enough" for many users, then if you need more you have no choice but to do some research yourself, following the many suggestions made in this thread and other places.

If you want the easy approach, when something apparently needs to be whitelisted, just get rid of as many source files as needed to make the desired content appear, and don't bother with daily updates. You might be amazed at how efficient one single small source file can be. ;)
[Android Apps and Games] arteaga323:
Ohhh, the science behind ad blocking is astounding lol thanks for your help!
[Android Apps and Games] Mike B.:
Not much science is involved. Each host (name) has a numeric address (IP); when you don't want to see a host, assign it an address that doesn't exist (127.0.0.1). That's DNS resolution and ad blocking in one sentence! It's just like giving someone a fake phone number so that they don't bother you later... ;)
[Android Apps and Games] arteaga323:
Yeah I know that part, my thing was even know I can't seem to decipher how mrRobinson got those addresses and not me, I wonder if he got them by doing the method you described earlier with the CSS/script thing
[Android Apps and Games] Mike B.:
I don't know what [MENTION=3355398]mrRobinson[/MENTION] did, but glancing again at your war site's HTML, I see that their videos are using JW Player (through the WordPress "jwplayer.js" plugin script) and if the source files you are using block subdomains of "jwpsrv.com" or "jwpltx.com", those would be reasonable suspects. Google Analytics/Tags are everywhere and are always suspects.

The point is that you MUST do some sleuthing, even if only by narrowing the problem down to a single source file. If it were easy, those monetizing schemes would never have taken hold as they (unfortunately) have. Many of you won't remember, but at one time the web was a place where one could seek information without having to fight unwanted junk shoved down one's virtual throat! ;)
[Android Apps and Games] arteaga323:
Wow, I never thought of it that way, I think if anyone ever made a " How to AdAway for Dummies" book or a "How's it Made" episode about AdAway, I would buy them lol this is truly confusing and hard! Lol I need to do some serious research.
[Android Apps and Games] croques:
thepiratebay.se and themobilebay.org appear to have their top level domain name servers removed from the TLD Registry by the Registrars.

An easy workaround is to add:

thepiratebay.se 141.101.118.194
themobilebay.org 104.27.159.69

To the 'Redirection' pane under 'Your Lists in adaway.

On a PC just add to the system/hosts file; Linux /etc/hosts
[Android Apps and Games] Mike B.:
You can add any redirections you wish to your hosts file, with or without AdAway, but make sure you know what you're doing else you could inadvertently go to unfriendly places you didn't mean to visit.

For members who might not understand how that works, it means that if, for example, you add to your hosts file a line that says "search.google.com 204.79.197.200", whenever you try to search with Google you are taken to Bing instead without any warning. Bad things can happen with redirections and that's why under AdAway you have to specifically enable that "feature". Proceed with caution!
[Android Apps and Games] DrPglixq:
it says app is not installed

---------- Post added at 12:51 PM ---------- Previous post was at 12:30 PM ----------

okay i already installed it, i manually move it to my system/app directory :)
[Android Apps and Games] Mike B.:
Which "it" said that on which device/ROM? How did you (attempt to) install which AdAway version? If you don't need the optional autoupdate of source files, you can even install it to external storage if your configuration allows, and there should be absolutely no need to move anything manually.
[Android Apps and Games] DrPglixq:
i tried both version that is posted on the first post,

i am using Slim Fresh rom,

tried to install it in phone storage, but i got "App is not installed" so i tried another version but the results are the same, then i tried it to install it on my sdcard, still there is no luck for me

so i decided to install it via push method, i pushed it on my /root/system/app directory, then do a reboot,

And now i am using it and there is no error encountered :)
[Android Apps and Games] Mike B.:
Since you forgot to mention which device/ROM/root version you are using, it's difficult to guess what happened but your symptoms are consistent with having an old AdAway version with a different signature (such as something from F-Droid) that would need to be uninstalled before installing any version from this thread. If you need help trying to understand what the problem was, we'll need a little bit more information but if you're happy and AdAway is happy, we're happy... ;)
[Android Apps and Games] cognitivedissonance:
Can someone help me what to add to my whitelist? I am viewing www.landsend.com and browsing the products. When AdAway lists are applied (the default ones only btw) the little selector that chooses different colors does not work. When I revert the hosts list to normal, the selector works. I've tried using the "Log DNS Requests" feature to figure out what I need to whitelist. To do this, I loaded a product page with the hosts applied, then added the requests I saw to the whitelist (I even modified a few of the to me more general, like *.landsend.com.) After adding the new entries to the whitelist and applying them, I'll reload the product page. The color selector still does not work, and when I view the DNS Request Log, there are no new entries. If I revert the hosts file, reload the page and then look at the DNS Request Log, there are a whole crapload of entries, and I have absolutely no clue which of them I need to add to the whitelist, but I definitely don't want to add them all because some are definitely for ads. Given that there at about 30 entries, it would take an unreasonable amount of time and effort to keep adding them to the whitelist and removing them in order to see which needed to be allowed, and which didn't. Can anyone help me figure this out?
[Android Apps and Games] .psd:
A few days ago I got a weird notification which I think was from adaway, but it went away as soon as I pulled down there yay to check it. The next day I checked my FK updater and noticed I was on kernel r7 despite never updating from r6 and having no auto update enabled (because you lose root and other things can **** up, if you don't reinstall SuperSU after the Kernel).

Anyway I never had to reinstall SuperSU which is weird also. But the issue is that since then, adaway hasn't found any updates, and I'm wondering if it's just coincidence because prior to that weird stuff it had been updating every day. So I'm curious, if you guys check your hosts in adaway, when was your most recent update? My last was January 28, it was the hosts-file.net one.
[Android Apps and Games] Telyx:
hosts-file.net 1/28

pgl.yoyo.org 1/26

mpvs.org 1/3

adaway.org 8/25/2015
[Android Apps and Games] Mike B.:
We've explained the required steps many times in this thread. Perhaps the most important piece of information you can give if you want us to help is the exact actual URL of whatever it is that you need to whitelist. Yes, it can be time consuming, but then it would be just as time consuming for us as it would be for you... ;)

The only landsend host blocked by the current default files is "affiliate.landsend.com" (in the hosts-file.net list). If whitelisting "*landsend.com" doesn't work, then clearly that content is hosted somewhere else. Open the page that give you trouble from a browser that shows locations, and find out where your color selector lives. If it is coming from a script you''ll have to find out where that script lives by looking at the page source. From glancing at that site, I see that it uses scripts form omniture.com, a site that is blocked in the current version of the pgl-yoyo.org source file. Whitelisting "*omniture.com" may or may not help, but realistically you'll simply have to go through the 30 or so sites you mentioned earlier until you find the correct one(s). If that's still too much work for you, disable some of your source files until you get the results you want.

That said, I just now went to "landsend.com" on a phone using AdAway and only the 4 default source files, then selected "shop men's". It showed me some shirts. I picked the first one, selecting "show all colors" and lo and behold, I can see 8 colors and picking any one changes the color of the picture, which seems to be the correct behavior.

Remember, this has NOTHING to do with AdAway which seem to be working fine for you. It's about what you decide to block or not block! )

---------- Post added at 11:40 AM ---------- Previous post was at 11:35 AM ----------

Can you be more specific? What exact AdAway notification did you receive? If you have any doubt about which are the current versions of the default source files, you can easily check from any browser:

http://adaway.org/hosts.txt

http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext

http://hosts-file.net/.\ad_servers.txt

http://winhelp2002.mvps.org/hosts.txt
[Android Apps and Games] androcraze:
OMG! AdAway FC'd! OMG!

Actually, sorta dumb on my part....

Updated to the latest MM 6.0.1_r11. Installed Root and AdAway. All is good...

Tried importing my whitelist, and AdAway gave me an FC.

I knew immediately the issue, went to Settings -> Apps, and gave AdAway "Storage" permissions. Fixed.

Why is AdAway crashing? It should either:

1. Give an error message "unable to import file. No permissions granted".

2. Request to add a "Storage" permission.

I can live with it as-is, I just figured it might be fix-able.
[Android Apps and Games] Mike B.:
A workaround might be possible, but MM 6.01 is not very good at telling apps ahead of time. I see similar reports for quite a few apps.
[Android Apps and Games] androcraze:
Most apps, even the F-Droid clan, do. Odd.

I didn't see it reported else where in this thread, so I thought I'd mention it.

---------- Post added at 16:03 ---------- Previous post was at 15:16 ----------

Just some guesses to try:

analytics.analytics-egain.com

assets.adobedtm.com

www.googleadservices.com

www.google-analytics.com

dsa.csdata1.com

b.monetate.net

cdn.mercent.com

login.dotomi.com

dnn506yrbagrg.cloudfront.net

the "assets.adobedtm.com" is commented out on "winhelp2002.mvps.org/hosts.txt" as causing issues with Land'sEnd, yet it's active in "hosts-file.net/download/hosts.txt"

Could be none of the above, one of the above, or some.
[Android Apps and Games] Mike B.:
I don't quite understand the reference to F-Droid since AFAIK they don't develop anything and merely compile/distribute, but if you have a repeatable unexpected crash in AdAway, posting the relevant log or starting an issue on GitHub would be helpful.
[Android Apps and Games] androcraze:
I just mentioned F-Droid as it's a mixed-bag of smaller applications, many of them older. It was a bad example.

I spoke too soon, as the bug report is on github and marked as a "bug":

https://github.com/Free-Software-for-Android/AdAway/issues/670
[Android Apps and Games] Mike B.:
That issue refers to inability to export the lists when storage permission is not granted (more of an annoyance), but it doesn't involve an FC (much more serious, but might not be anything AdAway can control). If you don't log it, it doesn't exist! :)
[Android Apps and Games] androcraze:
OK. Grabbed a 5000+ line log file with MatLog.

I have a "--------- beginning of crash" line. How far back do I go from there? Or is only that point and forward relevant?

I'm not even sure where to post it. Do I need to sweat PII?

How about this truncated section:

http://www.mediafire.com/view/9722m027ov9uxno/AdAway-StorageCrashLog.txt
[Android Apps and Games] Mike B.:
I'm sure [MENTION=3355398]mrRobinson[/MENTION] will contact you if he needs more information.
[Android Apps and Games] androcraze:
Interesting.... Google actually has a developer guide for requesting permissions:

https://developer.android.com/training/permissions/index.html

https://developer.android.com/training/permissions/requesting.html
[Android Apps and Games] infatality:
Very interesting.. That's what I had been looking for..
[Android Apps and Games] rcll:
Does anyone know a good solution to whitelist sponsored Google Search ad links? Like when you google 'Newegg', the top link that pops up is an ad link to Newegg.com

Using this example Google seems to link to googleadservices.com then to dartsearch.net then to doubleclick.net before finally to Newegg.com

Whitelisting *googleadservices.com *dartsearch.net and especially *doubleclick.net would seem to let in many other unwanted ads. Is there a better solution for allowing just Google Search sponsored ad links?
[Android Apps and Games] Mike B.:
For many such common adservers, you have to decide which is worse for you: missing out on "sponsored" ads, or seeing countless unwanted ads. Realistically, you can usually safely ignore/blacklist the "sponsored" links and still get the information you want from the "unsponsored"(?) links.
[Android Apps and Games] androcraze:
These are the only "doubleclick" listed that directly reference "googleads" listed in the four hosts files:

googleads2.g.doubleclick.net

googleads.g.doubleclick.net

www.doubleclickbygoogle.com

These are the only "googleadservices" listed in the four hosts files:

4.afs.googleadservices.com

googleadservices.com

pagead2.googleadservices.com

partner.googleadservices.com

www.googleadservices.com

These are the only "dartsearch" listed in the four hosts files:

clickserve.dartsearch.net

clickserve.dartsearch.net

clickserve.eu.dartsearch.net

clickserve.uk.dartsearch.net

clickserve.us2.dartsearch.net

That's a small list. I don't know that the doubleclick list is complete; there may be more used by Googleads/Dartsearch. I'm seeing over 12,000 unique host entries for doubleclick for the four host files.
[Android Apps and Games] Leonardohernandez98:
If i use this app can it block me from watching videos? Sorry for noob question.
[Android Apps and Games] kye04:
Hi. I have a problem: i recently installed several CM12.1 roms. And i noticed something: adaway, no matter wich version, not working. After i install app it says Disabled. If i press Download and install , it downloads the files and it installs them. I went even to the hosts file in the etc folder and opened it with a file editor, the file has the addresees inseted everything seems to be fine. Yet, no adds are blocked...

It looks like the hosts file in the etc folder is not actually the real host file.

Can you please help me ?

Thanks.
[Android Apps and Games] Mike B.:
AdAway is simply a tool to add the hosts you do NOT want to access to your hosts file. If a video is accessed via DNS (by host name, not host IP), you can block that host. AdAway itself doesn't care what the hosts is or what contents it sends you. I think you need to read a little about about what AdAway is and does then you can decide if it would be useful to you.

---------- Post added at 08:11 AM ---------- Previous post was at 08:05 AM ----------

From your description is seems AdAway is working perfectly (adding the desired hosts to the hosts file). If none of your ads are blocked, that suggests that you are using a proxy (in your APN perhaps?) that bypasses the hosts file, a common oversight. As long as the hosts file is correct, AdAway itself doesn't need to be running, but the "Disabled" message is a bit confusing. Make sure you have a full root and no other conflicting "ad blocker" or multiple /system/etc/hosts.* files.
[Android Apps and Games] ahfu25:
Most likely the rom you're using already has a ad blocking feature. This happened to me on a few roms that I later found to already have ad blocking.
[Android Apps and Games] kye04:
Indeed the rom has this feature built in. Yet when I try to update the adaway feature it's saying cannot update.

Strange is that some commercials are banned and some not....
[Android Apps and Games] Mike B.:
Your builtin "ad blocker" is probably some (old?) incompatible version of AdAway. As suggested earlier, get rid of it (as well as of any extra '/system/etc/host.* files), reboot, then the current and correct AdAway as well as all future updates from this thread should install and run just fine.
[Android Apps and Games] den10:
I have a HTC M8S and the system partition is not r/w (no s-off). The adaway app cannot apply the host files and there's no way to change preferences. I can however flash the zip from twrp and they seem to stick.

Is there anyway to update hosts without manually flashing new builds? Also the latest hosts.zip file I found was early 2015. Where could I find newer hosts.zip?

Thanks.
[Android Apps and Games] Mike B.:
If your system configuration allows it, you might be able to use a symlink in /data instead of the actual /system/etc/hosts. For details, see the AdAway builtin Help and the "Target hosts file" preference.

If you find no other way to tame your system You could use AdAway to prepare the hosts file that you wish to use on another device, then "manually" move it into place from TWRP, but I'm not sure what you mean by "flash the zip"(?), sorry.

The optional default source files suggested by AdAway can be checked from any browser. See THIS post for details. There is no compelling reason for source files to be updated all that often, and 2015 is not exactly ancient history. ;)

In general you might find it beneficial to first search this thread since those issues have been addressed countless time previously
[Android Apps and Games] den10:
Symlink worked!

Thank you for your help. I actually did search for this and countless possible solutions came up for not quite the same device. For a while I did not even realise that unlocking the bootloader and installing a recovery + root does not equal /system partition r/w.

While searching, I found this guide and like you mentioned, creating a symbolic link from /system/etc/hosts to /data/data/hosts was the way to go. The app now updates and applies hosts perfectly without fc or rebooting.

For anyone else having the same problem with HTC phones that come with /system partition write protection, this is the way to go about it (without kernel extensions to make the entire /system partition r/w).
[Android Apps and Games] Sandivisi:
[MENTION=4343728]Mike B.[/MENTION] what i can add this https://raw.githubusercontent.com/heradhis/indonesianadblockrules/master/subscriptions/abpindo.txt to host source?
[Android Apps and Games] Mike B.:
The list to which you link is

NOT

a suitable source file and cannot be used for inclusion in a hosts file, either with or without AdAway. It seems to be intended for "AdBlock Plus" or similar firewall type of applications but it has

no useful content

for AdAway or local DNS resolution, sorry ;)
[Android Apps and Games] croques:
Agreed: so look it up yourself. Thepiratebay.org use cloudfare as their hosts.

Below is console nameserver lookup using a specific cloudfare host

nslookup thepiratebay.se SOFIA.NS.CLOUDFLARE.COM
Server:	SOFIA.NS.CLOUDFLARE.COM
Address:	173.245.58.223#53

Name:	thepiratebay.se
Address: 141.101.118.195
Name:	thepiratebay.se
Address: 141.101.118.194
[Android Apps and Games] Mike B.:
I'm actually rather familiar with The Pirate Bay and Cloudflare, but the point is that since this has nothing to do with AdAway, this AdAway thread is not the best place to discuss those issues. Members get confused enough as it is dealing solely with genuine AdAway matters...

My recommendation stands:

avoid using DNS redirection

unless you're positive you know what you're doing. You can still directly access "thepiratebay.se" through most registrars if you so desire, and in this particular example there is little to gain by using the IP unless you know how to bypass Cloudflare "security", an entirely different set of headaches, still completely unrelated to AdAway! ;)

Moving on...
[Android Apps and Games] Balajumb:
Thanks a ton mate
[Android Apps and Games] stokholm:
I'm having trouble getting the systemless patch from OP to work on my Nexus 6. I know other people are too:

https://www.reddit.com/r/nexus6/comments/3xcuz0/systemless_root_261_and_adaway_w_the_systemless/

https://www.reddit.com/r/nexus6/comments/3yehji/anyone_have_adaway_working_with_systemless_root/

Their proposed "solution" is to modify the system partition. :eek:

Output of 'mount' reveals that the 'bind' command from the /su/su.d/ script is being run. But when I apply the host file in AdAway it complains that my system partition is full. (...which it is, but it shouldn't matter since /system/etc/hosts is bound to /su/etc/hosts).

So am I doing something wrong? I'm using the latest beta of SuperSU systemless (2.67).