yashsingh1/xdaforums huggingface dataset:
[Android Apps and Games] hp420:
not servers...domains. i manually added lines with 4 domains hosting ads i was still getting to the hosts file adaway created...it's not 4 domains in the entire hosts file lol that would be incredible, you're right!
[Android Apps and Games] Mike B.:
Yes, technically you are correct, the hosts file uses domains and subdomains, not "servers", but adding only 4 domains to solve all your ad-related issues is still quite a feat. In any case, that's what the blacklist is designed for and your experience is not unusual at all as it's common for new ad locations to appear once in a while. Note that the hosts file you posted also has MANY entries of questionable validity (permanently redirected duplicates, one-time randomly-generated URLs, some not accessible from your location, etc.). For the overwhelming majority of users, simply loading the default source files suggested by AdAway (at most, and no obvious need to even insist on updating daily) and adding a handful of black and white list entries suffices.
[Android Apps and Games] hp420:
like i said, i used the default hosts, easylist and fanboy's list to make a host file with adaway, and that didn't work, so i used that same host file, added 4 new domains and saved it. So that's why there are a whole crap-ton of them in there ;) only 4 are mine. for all i know the rest of the ads i should have seen were being blocked already, but the ones i added polished off pretty much everything left.
[Android Apps and Games] Mike B.:
Since you haven't revealed which 4 entries you added, I can't comment further, but it seems as if you simply used AdAway correctly as intended and documented, except that you may not even have had to add the easylist source files at all. The file you posted includes invalid entries (http://*, https://*), countless duplicates, etc. which may be coming from those additional sources. The shorter and cleaner the hosts file, the snappier your device will be, but In summary AdAway is behaving as it should, which is always good news and a far cry from your earlier "NOTHING works anymore" statement... ;)
[Android Apps and Games] hp420:
my additions:

127.0.0.1 http://doubleclick.net

127.0.0.1 http://cpmstar.com

127.0.0.1 http://www.googleadservices.com

127.0.0.1 https://optimized-by.rubiconproject.com
[Android Apps and Games] Mike B.:
FYI, those are incorrectly formatted. A hosts file entry should include a numeric IP followed by whitespace followed by a canonical hostname optionally followed by whitespace-separated aliases. No "http://", "https://" or anything of the sort. For example:

127.0.0.1 googleadservices.com www.googleadservices.com ads.googleadservices.com

Even with the correct syntax, your additions are a bit puzzling since, for example, no ad is typically served from "doubleclick.net" (they usually come from ad.doubleclick.net.*.*), and "www.googleadservices.com", "cpmstar.com" and "optimized-by.rubiconproject.com" are already included in the default source files, but if you and your device are happy, that's all that matters... ;)
[Android Apps and Games] fjdifede:
When you open your APNs even if not selected you need to remove proxy and port or you going to get ads anyway, just leave alone the one for MMS!
[Android Apps and Games] dev_don:
I have the latest version of adaway from fdroid but I am seeing adverts across the web

these appear to be doubleclick adverts, I see an MPU (300x250) advert on this page:

http://www.theguardian.com/football/2015/sep/24/juventus-frosinone-serie-a-match-report

Using Oppo Find 7a, CM 5.1.1 build 12.1-20150917

Anyone else with the same issue
[Android Apps and Games] hp420:
proxy, port, mms proxy & mms port all say "not set"

right, but if I put doubleclick.net instead of ad.doubleclick.net it will block the entire doubleclick domain, rather than only ad.doubleclick.net wouldn't it?

thank you for these correction, though.
[Android Apps and Games] Mike B.:
No. ;)

If you simply use AdAway as documented, you won't have to worry about any of those issues. It can help you find out which additional hosts to block or allow if the defaults don't satisfy you, and it even lets you know if you try to use an invalid entry.
[Android Apps and Games] fmc000:
No.
[Android Apps and Games] hp420:
I seem to be getting ads again. I'm really frustrated about all this. I've never had issues doing anything but clicking 'update' and rebooting, but seriously, for the last 2-3 months I've been bombarded with ads.

....but here's the strange part...if I use any browser on any of the 4 android devices I own I get ads like crazy...I mean EVERYWHERE, like adaway isn't even there. I have one on jb 4.1 (nexus s), one on kk 4.4.4 (nexus 4), and two on 5.1.1 (tmo lg g2 and unlocked moto g3)....all are identical. Ads ALL OVER the page!!

Now....If I use the same sim card in every device and tether any one of those 4 devices to my debian desktop with chrome, I see NOT ONE ad anywhere.

So I can rule out tmobile proxy. The internet isn't only from the same carrier, but even the same sim card.

The hosts I added to adaway...easylist and fanboy"s...those are the only two I use on my desktop. Using those two alone works on my desktop, though, and not any of my androids.

I have the same internet source, and the same hosts on every single web-enable device I own, but my desktop has no ads and my androids ALL have ads like mad. There are no variables besides the app being used to create, merge and apply the hosts file and the general android operating system....although android is a debian fork, so we're pretty close still.

I've tried adaway, adfree, and minmin...plus minmin WITH adaway or adfree. Nothing has worked.

One thing I can say is working, though....in-app ads are 100% blocked. I only see ads in web browsers, and even then, only using android.

---------- Post added at 11:59 AM ---------- Previous post was at 11:46 AM ----------

See, now that's very strange, because I just get a grayed-out box where the ad should be, between the paragraphs.
[Android Apps and Games] Mike B.:
I'm positive there's more to the story. Forget about AdAway for now (delete it temporarily if you want) and concentrate on your hosts file. Completely uninstall any other "Ad blocker" you have. Those are often the culprits. Reboot to recovery, clear cache, then reboot . Make sure your hosts file still contains all the entries you wish to block. Also verify that your APNs do not have a proxy (I know, same SIM, but that doesn't mean anything). Make sure your browsers (Chrome?) don't have a "data saver" or "optimizer" function enabled. Get rid of any "booster" app you may have. Mostly, do not assume that "there are no variables besides the app being used to create, merge and apply the hosts file" because obviously there are. My first guess would be a forgotten Chrome "data saver" or a conflicting "ad blocker". If you still have trouble without AdAway (but with the same hosts file) , we can at least safely conclude that your problem is unrelated to AdAway and needs not take over this AdAway thread. Happy hunting! ;)
[Android Apps and Games] hp420:
uhhhh....data saver, you say, huh?? lol :cyclops: *mega-facepalm*

How in the flying f*#k does putting more ads on my page save data?

So far, so good....thanks!!!

Do you have a wallet account where I can send you a few bucks as a thanks? I don't have paypal.
[Android Apps and Games] Mike B.:
The Chrome "data saver" is simply a proxy.

I wish someone would have posted that

* if you use Chrome and still see ads on web pages, remember to disable the "data saver" or "reduced data usage" browser option​

Oh, wait, someone did, right in the OP!

If you have extra money, feel free to donate to [MENTION=3355398]mrRobinson[/MENTION]. To thank me, just read the OP next time! ;)
[Android Apps and Games] hp420:
I suppose I deserved that ;)
[Android Apps and Games] Cookie Ninja:
A long series of posts, retorts and insinuation to learn one simple rule... Never trust Google. Never. Ever. They manage to sneak in ads in the guise of saving you data. Root uninstall Chrome, use Firefox and you're good....

Dead Cookies leave no trails...
[Android Apps and Games] Ultramanoid:
You know that Firefox for Android comes bundled with the AdMob library, I suppose. And you're okay with that ? They are not showing ads ( yet; but can siphon your data as well as Google Analytics or Flurry do... ) but it says a lot about how much you can trust Mozilla too these days.

You want real privacy, compile your own browser. Both Chromium and Firefox make source available, otherwise, you cannot trust anyone, period.
[Android Apps and Games] Mike B.:
Brief OT comment: If you want real privacy don't do/type/show/share anything "private" online. All those evil corporations you people complain about only know the information that YOU provide! ;)
[Android Apps and Games] Mike B.:
Whatever is on F-Droid is not the same as the preferred latest AdAway available from the OP on here to which you should consider updating, but your issue might be simply a failure to block the URLS that you don't wish to see. Countless ads come from a variety of doubleclick-related hosts. The specific page you mention appears to grab data from ubads.g.doubleclick.net, googleads.g.doubleclick.net, and possibly others but only YOU can tell since it depends on your location, ISP, login, browser, etc. That's what the AdAway DNS Request Log is for. Review the online help or search this thread for details on how to use it to pinpoint the URLs that you want to blacklist.
[Android Apps and Games] Abu-7abash:
Any workaround for this on 5.1.1?
[Android Apps and Games] Pkt_Lnt:
There is!

Try doing what it says......

"Please read HELP for more information"
[Android Apps and Games] Matheus França:
Nice Work
[Android Apps and Games] mrRobinson:
Sorry, I don't take donations either. I like being grumpy when i want and if I take money I'd feel an obligation to be nice. :D
[Android Apps and Games] mrRobinson:
Dev Build

New Dev Build. This one won't go to the OP since I want our more adv users to test. If this looks good we will get 3.0.3 next.

The thing I want to test is a new preference I added called "Enable IPv6".

Basically I was updating the internal webserver to its latest release (another change) and was able to compile it for IPv6 support. But the hosts file didn't support writing any entries as ::1 (IPv6 localhost) so a webserver listening on ::1 would be pointless. I do see that ad networks support IPv6 at this time however luckily we don't get served many ads over IPv6 (yet). So to help future proof this, that new preference will allow you to enable IPv6 localhost entries in the hosts file. so if your source hosts file has

127.0.0.1 adbasedsite.com

127.0.0.1 weservead.com

...

then with that checkbox enabled your hosts file will have

127.0.0.1 adbasedsite.com

::1 adbasedsite.com

127.0.0.1 weservead.com

::1 weservead.com

this way if any of the ad sites start serving ads over IPv6 they are now blocked and the new webserver will also do its job for IPv4 or IPv6.

Keep in mind this will double the size of your hosts file. It is more important if you check that box to follow [MENTION=4343728]Mike B.[/MENTION] 's advice and use the default srcs and only add what you need to block anything else. the way it is implemented it will not double the download size or increase parsing. All that still happens as it was just when it is getting written it will write the 127.0.0.1 entry then the ::1 entry, then loop.

Anyway this gets us blocking IPv6 served ads once they start showing up.

After reading all that, Download AdAway-release_Build-Sep.24.2015.apk
[Android Apps and Games] swat4samp:
Sorry for late reply. Took a log after turning on webserver and tcpdump in log DNS requests tab.

pastebin.com/PRyX9j0E
[Android Apps and Games] mrRobinson:
It's not showing why the webserver would be stopping. Try the dev build just above.
[Android Apps and Games] RichardEb:
Is it safe to use any block list source or can a "bad" source list redirect traffic to an attacker?
[Android Apps and Games] swat4samp:
Yes, i tried it many times, still it didn't show anything in log. Will try new version and report it.
[Android Apps and Games] rpr69:
Would it be wrong to say I told you so. :p
[Android Apps and Games] Mike B.:
Is there anything in particular that needs testing? The generated hosts file looks correct but it's difficult to check if the webserver properly redirects IPv6 requests. Did you find any usable URLs suitable for testing?

---------- Post added at 09:46 AM ---------- Previous post was at 09:40 AM ----------

A source file can potentially redirect URLS to something besides the local host if you enable the "Allow redirection" AdAway option.The point is that if you have any doubts about a source file, stay away from it. It's almost never necessary and seldom recommended to use additional source files, and redirection should only be enabled when you're absolutely positive that you know what you're doing.

---------- Post added at 10:04 AM ---------- Previous post was at 09:46 AM ----------

I have no interest in defending Google or anyone else, but you might be misunderstanding the issue. The Chrome "data saver" proxy doesn't sneak in ads. it merely redirects all request to a central location where the data you request may already have been cached, saving the additional back-and-forth data traffic that would be otherwise required for each packet your browser requests. It's a common practice and is the de facto standard for satellite internet, satellite phones, etc. It does save some bandwidth especially when accessing common sites, but one of the drawbacks is that since all requests go through an external proxy, they can't be processed by the local DNS resolution scheme, including the hosts file, and therefore DNS-based "ad blockers" can't do their job.
[Android Apps and Games] mrRobinson:
If i expand on what MIke B. responded with, if you aren't using the redirection lists, actually even if you are adaway does a level of sanitation on hosts source. For example if you hosts source was:

# Facebook blockers
127.0.0.1 localhost #IPv4 localhost
::1 localhost #IPv6 localhost
#
# AD SERVERS START HERE
#
0.0.0.0 facebook.com
127.0.0.1 www.facebook.com
219.23.83.154 login.facebook.com
219.23.83.155 evil.redirect.facebook.com
127.0.0.1 www.fbcdn.net
::1 www.google.com
::1 babylove.com

you'll notice adaway will apply that to your system as:

127.0.0.1 static.ak.facebook.com
::1 static.ak.facebook.com
127.0.0.1 www.google.com
::1 www.google.com
127.0.0.1 soundwave.mobile.yahoo.com
::1 soundwave.mobile.yahoo.com
127.0.0.1 babylove.com
::1 babylove.com
127.0.0.1 www.facebook.com
::1 www.facebook.com
127.0.0.1 graph.facebook.com
::1 graph.facebook.com
127.0.0.1 facebook.com
::1 facebook.com
127.0.0.1 s-static.ak.facebook.com
::1 s-static.ak.facebook.com
127.0.0.1 connect.facebook.net
::1 connect.facebook.net
127.0.0.1 www.fbcdn.net
::1 www.fbcdn.net

notice how the login.facebook.com and evilredirect hostnames didn't make it in? Adaway ignored those hostnames because they didn't start with 127.0.0.1, ::1, or 0.0.0.0

Adaway will only capture hostnames behind those 3 addresses and write them to the hosts file in front of 127.0.0.1 only or if you check the ipv6 box you'll also get ::1 now as well. So no evil redirect can happen.

I did test the webserver with wget and an android version of curl and i can confirm it is working properly. For the Gen pop i'd just like more than my device to confirm that

1. when the check box is checked, you do see ::1 entries and that and vice versa, when unchecked and re-applied they go away.

2. checking or unchecking the box doesn't lead to any weird behavior, (no new crashes, hangs, etc.)
[Android Apps and Games] Mike B.:
It's nice that AdAway can protect users from obvious evil redirections, and that by itself might be a good reason to NOT try to directly add unknown source lists to the hosts file without a thorough understanding of the consequences.

I too tried some basic stuff with wget/curl shell tests, but until we're all bombarded by IPv6 ads I guess there's not much more we can do. Nothing unusual or unholy showed up on my Note3/ShinTo/Lollipop/kinda-stock/OH1-based/debloated/deodexed/ daily driver. So far so good... ;)
[Android Apps and Games] Flavien125:
Ram?

hello, is there a ram increase use? sorry for my English , I use google translation
[Android Apps and Games] xdarthanonx:
No, not directly. But if you make a huge hosts file, you might have problems. Just try it with the standard sources and see how it goes :)
[Android Apps and Games] IronTechmonkey:
You can download one or more of the host lists and then view the #of lines in the resultant hosts file in an editor and check the size of the file to get a relative (and fairly loose idea of impact. For instance, a host file that uses all available sources accessible by the app is a few hundred thousand lines long while the host list from Adaway (the smallest) is only a few hundred entries long.
[Android Apps and Games] mrRobinson:
AdAway will use memory during the downloading, parsing, building, applying phase. Once it writes the hosts file it does not run. My recent updates to the webserver took it from about 692k of resident memory to about 392k of resident memory. So the newest dev build will save you 300k of mem if you use the webserver.
[Android Apps and Games] hjjiang:
i alwas wondered, is it possible to have an option that can export/import Host Source URLs?? there is a backup for the whitelist/blacklist but we cant export the source urls. makes it easier when i transfer to phones or redo my phone instead of just copy and pasting the urls
[Android Apps and Games] skidaway:
Everything is working as it should on my N6 running Chroma. I've toggled the IPV6 a few times, downloaded and redownloaded lists, and everything seemed to function fine.

Thanks for thinking of future issues, and implementing solutions for them now.
[Android Apps and Games] BEDBOI2:
Since AdAway modifies a system file, does it mean it breaks OTA? Or is it simple to revert to the original host file?
[Android Apps and Games] ChiDi9901:
[MENTION=5611888]BEDBOI2[/MENTION]

Most of the time having the device rooted already breaks OTA. But this also depends on what kind of device you have. Some devices of some manufacturers can still OTA update while having a rooted phone. Other manufacturers will allow you to update a rooted phone, but will overwrite system files, which will likely reset the host file to default. Then you have to reapply Adaway again.
[Android Apps and Games] IronTechmonkey:
A bit more info \/ \/ \/ \/
[Android Apps and Games] BEDBOI2:
I have a Nexus 5 and (for now) I'm on 4.4.4; until now I just followed "Section C" on http://xdaforums.com/google-nexus-5/general/info-nexus-5-ota-help-desk-t2523217 and this way I'm able to revert Xposed installation, but if AdAway replace a system file how would I apply OTA again without flashing the system.img? If it's easy to put back the original host it would be great, and I don't mind applying again AdAway after root.
[Android Apps and Games] IronTechmonkey:
The restoration of stock recovery is also an important part of that equation. Also, i believe the "normal" path from...

- custom recovery/ROM

- to stock recovery/ROM

- to applying OTA

- to installing custom recovery/ROM.

...precludes the preservation of such things. That said, it is fairly easy to backup the entire hosts file or the text within. I'm not sure if restoring Adaway from TiBu restores the host file because I always re-download at that moment but thanks to your question I'm curious and may check that out further.
[Android Apps and Games] mrRobinson:
If you click the button to disable adaway it reverts the hosts file. That's all you have to do.
[Android Apps and Games] RyomaEchizen:
Twitter app can't connect and Twitter mobile (for browser) works great

What can i put on whitelist for Twitter app works again?
[Android Apps and Games] Mike B.:
The real question is which hosts are you blocking. As usual, the best approach is to start with the default AdAway source files, then use the DNS Request Log to see which hosts, if any, your application needs to access. Note that you should first verify that your app's failure to connect is related to the hosts file. You didn't specify which error message you get, but a simple test involves temporarily disabling AdAway, rebooting, then trying your app again. Since you say "again", I suspect that it was working for you at one time, so maybe you could try to remember what you changed (new source file, perhaps?) that triggered the problem.
[Android Apps and Games] RyomaEchizen:
Yes. I put new sources files because AdAway default source files are not very effective with Argentina websites

Twitter says "Cannot retrieve tweets at this time. Please try again later"

No problem with default AdAway source files. I will try dns request log to see wich hosts needs Twitter app :)

Thanks
[Android Apps and Games] mrRobinson:
when applying hosts file, you see some text output like Downloading... Applying... under that Parsing and merging, then Building... etc.

I was trying to increase performance on those or make adaway more efficient esp in the writing area since if you enable IPv6 it will write twice the lines. the parsing and merging is most of the work and where people without enough memory crash on the mahakala source that I hate, so those people will still crash there. Anyway i decided to use it for my test, since my device get's past it, plus 7 other large sources. AND i enabled IPv6 so adaway had to write each line twice, once with 127.0.0.1 and once with ::1.

With that huge 85MB hosts file the Building phase of Applying took 42 seconds. After my change it took 17 seconds. So when the new version comes out, don't fear enabling IPv6 cause of twice the writes, it'll be at least 2.5x faster than it ever was. Using the default sources, even with IPv6, you used to see the "building..." maybe for a couple seconds, it should go by quite quickly now. I prob won't release 3.0.3 till after i can get hands on 5x and 6p just to confirm no new issues.
[Android Apps and Games] mrRobinson:
ok, I'd like to confirm/refute a thought i have for the MOAAB thing and why it crashes on some people's phones.. So i need to ask the community here 2 things.

1. can someone temp host this https://www.androidfilehost.com/?fid=24052804347823905 somewhere where adaway would be able to load it? Can't use afh cause of the timer. Its a modified copy of the moaab thing on adblock.mahakala.is to test a theory I have.

2. once temp up somewhere can someone who's adaway crashes when loading the one from the mahakala site try the one uploaded in step 1?

thanks!
[Android Apps and Games] Flash_Aholic:
So recently I installed adaway cause I was tired of all the ads lol but now every time I update the host files it puts me in a boot loop for some reason.. I'm on a HTC desire 510 virgin mobile s-on raptorrom v2 twrp 2.8.6, all I gotta do is reflash the boot img but jus curious if anyone else has had this problem lol
[Android Apps and Games] Mike B.:
If you still have S-ON, have you tried the workaround detailed under the AdAway Help menu (create a symlink in /data since access to your /system/etc is restricted)?
[Android Apps and Games] Flash_Aholic:
Have not lol sorry bout that will try here in a sec, thanks I'll repot back asap
[Android Apps and Games] nuumuun:
hosts file is accessible at http://exoblob.net/hosts.txt
[Android Apps and Games] mrRobinson:
Can someone load http://adblock.mahakala.is as their hosts source. See if their phone crashes on it. Then remove it and use the one above and report back.
[Android Apps and Games] rbird2:
Running AdAway 3.0.2

Both http://adblock.mahakala.is and http://exoblob.net/hosts.txt took quite a while to download and then Adaway crashed during "Parsing & merging hosts files" which was running very slow. Had to reboot phone (Samsung Galaxy S4) to get AdAway to work correctly again.
[Android Apps and Games] shaqman89:
Second this. Also using Adaway 3.0.2, but with Ace2 running VanirKK.
[Android Apps and Games] HAVOK83:
I'm using http://adblock.mahakala.is/

And it's working perfectly fine