yashsingh1/xdaforums huggingface dataset:
[Magisk] eoh7678:
Strangely, when booting into TWRP and using ADB, it has no trouble finding and executing the busybox commands, so I did the following while in TWRP:

adb push Unified-Hosts-Adblock_v2.3-Test.zip /tmp/adblock.zip
adb shell
mkdir /tmp/zip
unzip /tmp/adblock.zip META-INF/* -d /tmp/zip
sh /tmp/zip/META-INF/com/google/android/update-binary 3 1 /tmp/adblock.zip 2>/dev/null | grep ui_print

Idea taken from here: https://xdaforums.com/showpost.php?p=72604085&postcount=10323

After rebooting, the module shows up in Magisk Manager, and cat /system/etc/hosts shows a very full hosts file!!

So a workaround does seem to exist.
[Magisk] Zackptg5:
Are there any files in the busybox folder? Looks like there aren't from the logs.
[Magisk] Zackptg5:
Odd. Whatever works I guess but at this point, the problem appears to not be specific to this module
[Magisk] Zackptg5:
Pushed v2.3 update. Some bug fixes for pixels/AB devices. Thanks to [MENTION=1276115]eoh7678[/MENTION] and [MENTION=7622794]gunner6643[/MENTION] for troubleshooting
[Magisk] eoh7678:
marlin:/ $ ls -R /magisk/busybox-ndk/
/magisk/busybox-ndk/:
auto_mount module.prop system

/magisk/busybox-ndk/system:
xbin

/magisk/busybox-ndk/system/xbin:
[ factor link powertop tail
[[ fakeidentd ln printenv tar
acpid false loadfont printf tcpsvd
adjtimex fatattr loadkmap ps tee
ar fbset logread pscan telnet
arp fbsplash losetup pstree telnetd
arping fdflush ls pwd test
ash fdformat lsattr pwdx tftp
awk fdisk lsmod raidautorun tftpd
base64 fgconsole lsof rdate time
basename fgrep lspci rdev timeout
bbconfig find lsscsi readlink top
beep findfs lsusb readprofile touch
blkdiscard flock lzcat realpath tr
blkid fold lzma reboot traceroute
blockdev free lzop reformime traceroute6
brctl freeramdisk lzopcat renice true
bunzip2 fsck makedevs reset truncate
busybox fsck.minix makemime resize tty
bzcat fsfreeze man rev ttysize
bzip2 fstrim md5sum rfkill tunctl
cal fsync mesg rm tune2fs
cat ftpd microcom rmdir ubiattach
chat ftpget mkdir rmmod ubidetach
chattr ftpput mkdosfs route ubimkvol
chgrp fuser mke2fs rtcwake ubirename
chmod getopt mkfifo run-parts ubirmvol
chown grep mkfs.ext2 rx ubirsvol
chroot groups mkfs.minix script ubiupdatevol
chrt gunzip mkfs.reiser scriptreplay udhcpc
chvt gzip mkfs.vfat sed udhcpc6
cksum hd mknod sendmail udhcpd
clear hdparm mkswap seq udpsvd
cmp head mktemp setconsole uevent
comm hexdump modinfo setfont umount
conspy hostname modprobe setkeycodes uname
cp httpd more setlogcons uncompress
cpio hush mount setpriv unexpand
crond hwclock mountpoint setserial uniq
crontab id mpstat setsid unix2dos
cttyhack ifconfig mv setuidgid unlink
cut ifdown nameif sha1sum unlzma
date ifenslave nanddump sha256sum unlzop
dc ifplugd nandwrite sha3sum unxz
dd ifup nbd-client sha512sum unzip
deallocvt inetd nc showkey uptime
depmod inotifyd netstat shred usleep
devmem insmod nice shuf uudecode
df install nl slattach uuencode
dhcprelay ionice nmeter sleep vconfig
diff iostat nohup smemcap vi
dirname ip nslookup sort volname
dmesg ipaddr od split watch
dnsd ipcalc openvt ssl_client watchdog
dnsdomainname ipcrm partprobe start-stop-daemon wc
dos2unix ipcs paste stat wget
du iplink patch strings which
dumpkmap ipneigh pgrep stty whoami
dumpleases iproute pidof sum whois
echo iprule ping svc xargs
ed iptunnel ping6 swapoff xxd
egrep kbd_mode pipe_progress swapon xz
eject kill pivot_root switch_root xzcat
env killall pkill sync yes
ether-wake killall5 pmap sysctl zcat
expand klogd popmaildir syslogd zcip
expr less poweroff tac
marlin:/ $
[Magisk] Zackptg5:
Looked at the busybox module installer and it doesn't use the magisk template at all which is why it works. With the new v4 template for magisk modules, almost all of the functions are no longer contained in the module's update-binary but are rather contained in /data/magisk/util_functions.sh script which the update-binary calls. It's this function that isn't finding the busybox commands and throwing errors.

Edit: I believe the issues is that busybox isn't present during the install. Since it's installed to magisk img, it isn't yet mounted when the module is being flashed. So can you uninstall and then reinstall the busybox to the system? you can just add the words 'uninstall' and 'system' to the filename for osm0sis's busybox. Then try flashing the module again
[Magisk] TONGEXHISPEED:
I flash module in twrp when restart and go to terminal.

i try

Su

Hosts

But it tell me hosts not found

Please guild me!

My device op5/oos 4.5.7/magisk13.3
[Magisk] g40q90:
Hello, I'm having problems updating/installing v2.3.

I obtain the message "! /data/magisk isn't setup properly!" in magisk manager and also in TWRP.

I installed v2.0 in magisk 12 without problems.

Using Magisk 13.3 on a LG G5 h840 with data encryption.

here is the TWRP log. Hope it helps! Thank you
[Magisk] Didgeridoohan:
Try a clean re-install of Magisk (uninstall everything and install again).
[Magisk] g40q90:
Already tried it but the error persist..

During Magisk installation I read "Data unavailable, using cache workaround"

and during Unified Host v2.3 installation "/data/magisk isn't setup properly!" so I think the problem here is that the module don't search for Cache/Magisk if Data/Magisk isn't present.

Other modules install fine.
[Magisk] Broken_Anima:
all fixed!

sorry my problem is fixed but I don't know how to delete posts. mod please delete.
[Magisk] Zackptg5:
Its case sensitive so you'd need to type

su
hosts
[Magisk] TONGEXHISPEED:
same result!
[Magisk] g40q90:
I add the v2.0 log for comparation:
[Magisk] Zackptg5:
Hmmm. This isn't mod specific. This is actually a part of the magisk module template v4. That's really odd. Magisk has for a while (and continues to) install to /cache if it can't install to /data. Yet for some reason, the capability for installing modules to a /cache based magisk install is absent from the latest template. I'll look into it and post a test build in a little bit. Thanks for the logs (they did help :))
[Magisk] firmanhadiii:
Can this module block ads vidio on YouTube?
[Magisk] Redfred_:
Hey guys I flashed this module correctly and enabled fgps in su -> host (terminal) I then rebooted and the ads are hidden as expected. :svetius:

However... When I try to visit a social media website or app i.e facebook.com / Facebook app it rejects connection within chrome displaying the following message:

err_connection_refused

When I disable or uninstall this mod social mediw websites / apps are fully functional again.

Any ideas?

Thank you in advance. :fingers-crossed:
[Magisk] oux:
no

s is for social, which blocks social media. try fgp and it should work
[Magisk] g40q90:
I tried to reply but I'm not sure if the pm arrived to you. If it's not, let me know and I'll answer here
[Magisk] Zackptg5:
Here's a test build. Post the recovery log and how it works. Thanks!
[Magisk] g40q90:
Installation failed again! Here's the log.

Just asking, maybe is also the case to inform topjohnwu about this issue with template v4? Becouse I'm having problem installing other modules also, but Unified Hosts is the most important for me!
[Magisk] Zackptg5:
Already opened an issue for it here: https://github.com/topjohnwu/magisk-module-template/issues/25

Think I found the issue, try this build.
[Magisk] yung40oz84:
Install worked fine for me on Angler, DU, Magisk 13.3 official. When I apply hosts what does this mean? Cannot verify certificate? Will that affect anything?
[Magisk] g40q90:
Olee it works great now!! thank you really much :)

Here's the log just for clarity
[Magisk] Zackptg5:
Sweet! Thanks for the reports!
[Magisk] Zackptg5:
Version 2.4 is live!

Readded support for magisk installs located in /cache. Thanks to [MENTION=4247782]g40q90[/MENTION] for troubleshooting
[Magisk] Redfred_:
Thank you this worked, appreciate the help. :)
[Magisk] SDMU:
I get this message " do not use spaces" , I am not . Please help . Thank You !

Edit:- got it , I used "m" as well , "f" and "g" include "m" .
[Magisk] Didgeridoohan:
[MENTION=6037748]Zackptg5[/MENTION] Maybe you should add a sed command to remove m from $INPUT if the input is more than one letter...
[Magisk] Erick.criminal:
Is the host automatically updated?
[Magisk] Zackptg5:
I'll keep that in mind for the next update

Nope. You'll need to rerun the script once or twice a week
[Magisk] CypherMK:
So, does anyone know?
[Magisk] Zackptg5:
Best way is to use an adblocker such as ublock origin in your web browser to determine the website(s) you need to whitelist
[Magisk] Zackptg5:
V2.5 released! See OP for changelog
[Magisk] Zackptg5:
You need to install busybox (system install)
[Magisk] Didgeridoohan:
[MENTION=2084155]spambayes[/MENTION]

That looks like an issue with Lollipop. Give the latest unofficial snapshot a try. I've had some success with some of the later code when testing Magisk on Lollipop.
[Magisk] spambayes:
Busybox is installed. As it turned out, Didgeridoohan's post at https://xdaforums.com/showpost.php?p=73344559&postcount=437 is right on the money, the problem was caused by the latest Magisk v13.3 with Lollipop. I tried the latest snapshot and the problem was gone and the Adblocker module installed without any issues.

---------- Post added at 06:12 AM ---------- Previous post was at 05:17 AM ----------

You're right. It was an issue Lollipop has with the latest stable Magisk v13.3. Installed build 170806 for both Magisk and Magisk Manager and the problem went away. I was able to flash Unified Hosts Adblocker v2.5 without any errors and it is working fine. :)
[Magisk] MrBrazil1911:
I'm having a problem with the latest hosts file applying.

https://goo.gl/photos/YtaV5RfH4YjbkegF7

I'm also no longer able to apply my whitelist so I'm again no longer able to see my YouTube history. Issue just started when I tried to update yesterday. https://goo.gl/photos/pXBaVhZH1Jf1ttqe6
[Magisk] Zackptg5:
The first picture is normal, hosts did apply. The second one isn't. Do you have a folder named temp on your sdcard? If so, delete it or move it somewhere else
[Magisk] MrBrazil1911:
Yes, there is a folder labeled 'temp'. I was previously instructed to place the whitelist file directly to Sdcard. Should I place it somewhere else?

And the reason I am thinking hosts did not apply properly is because when I run SU Hosts, it tells me I have updates again where as before it would say No Updates.
[Magisk] Zackptg5:
When you apply the whitelist, the script creates and then removes a file called 'temp' and places it on sdcard as an intermediary file. If you have a folder called temp, this process will fail and your whitelist won't apply. So you need to get rid of anything named 'temp' on root of sdcard for it to work

A good way to check if the hosts file applied is to navigate to /magisk/.core/ and open the hosts file. It has the date on the top. The script displays the latest date at the top so you can compare these
[Magisk] MrBrazil1911:
Thanks to your tip, it is back working correctly; however, when I went to /magisk/.core/ and open the hosts file, I didn't see anything in there.

Edit: Apparently, something isn't letting me view it with the installed texts editors. I can view it on another phone using ES File Manager, but can't with Solid Explorer on my other phone. I've made sure I can access hidden files, so I think Solid Explorer is the problem.
[Magisk] Zackptg5:
The hosts file is very large and many text editors can't open it as a result. I use FX file explorer and the built in text editor says that the file exceeds available memory so I have to use a different app
[Magisk] Didgeridoohan:
[MENTION=6037748]Zackptg5[/MENTION]

I have another idea for you. A way for users of your module to automate updating the hosts file through something like Tasker.

Add input variables to the script. For example by entering "hosts m", the script would run the master option and then quit. Running "hosts fps b w" would run master+fakenews+porn+social, then add the blacklist, then the whitelist and then quit.

Basicly, check if $1, $2 and $3 are set and then run the script accordingly. If they aren't set, run the script as normal.
[Magisk] Zackptg5:
Oooooo, I like that. That'll be the next update, thanks!
[Magisk] Zackptg5:
Version 2.6 is live! Added the argument idea of [MENTION=4667597]Didgeridoohan[/MENTION] so automation will be a piece of cake, and made some fixes so users who have files/folders on sdcard named temp and such won't have any more issues with the script :)
[Magisk] Didgeridoohan:
That was quick! :highfive: Did a quick test and it seems to work great.
[Magisk] Zackptg5:
Sweet! I just made a single block automate flow for updating if you use it instead of tasker :D

It's on the OP
[Magisk] atarek:
Hello,

Perfect.

Thanks for your Great effort and fast implementation to keep module better and enhanced. :o:svetius:

Regards,

Ahmed
[Magisk] Didgeridoohan:
Here's a simple Tasker profile to automatically update the hosts file once a week. Defaults to only setting the master option but can easily be changed according to the new one-line command (see the OP). Just change the shell line in the Task to your preferred setup.

Set and forget.
[Magisk] Zackptg5:
Linked to the OP, thanks!
[Magisk] Jotabeta:
without Tasker

Is it possible to use it without Tasker? Sorry my friend but I don't understand how install and use it. My english it's s..t
[Magisk] Zackptg5:
Of course. All of the old stuff still applies. There's some new stuff added to the OP for the new features (hint: look under 'Usage')
[Magisk] Ashish Lahare:
Unable to link my Facebook profile with Clash Royale game... What do I need to whitelist ?

I have applied all the hosts... Also needed social host to be applied.. Thank you..
[Magisk] Zackptg5:
I don't use the social filter so I don't know exactly. There's 2 different ways to find out:

Install ublock origin to your web browser and see what sites need whitelisted while on the page

Install adaway, click on the 3 dots, and choose log DNS requests. Start the tcpdump service, try the Facebook thing, then check back on the log from it
[Magisk] Jotabeta:
Sorry but (I'm a newbie) I don't understend how to install the file "....update automate flow". When I open the zip with the file manager There is a .flo. I don't know what I must do. Can you help me? It's a strange extension for me and I haven't an app to open it. Many Thanks
[Magisk] Zackptg5:
It's for users of the automate app. If you don't use it, then don't worry about it
[Magisk] puma9S:
Hey, can someone please give an example of how the whitelist.txt should look like?

I've placed mine in the root section of my SD card but in the terminal it says "no whitelist detected"
[Magisk] Didgeridoohan:
It should be named "whitelist", no .txt...
[Magisk] puma9S:
Ah, nice that worked!

Can you also tell me how to add websites?

atm it's like this, but it seems not to work..

www.xysite.com
www.yzsite.com
[Magisk] Didgeridoohan:
You need to match the exact lines in the hosts file.

0.0.0.0 www.xysite.com

Or use regex.
[Magisk] yung40oz84:
Module makes new beta 13.5 lose root. I can reproduce over and over. I can install all modules and when install this and reboot there is not root. Also, even if I use adblock I lose root, so it may be related to new beta, not entirely sure. U can find my logs and posts in the magisk beta thread on second to last page. Running DU, Flash kernel. Thanks.
[Magisk] Zackptg5:
Can you post the link to your post on that thread for others to see too? There's some changes on the module template in 13.5 I haven't looked at yet, thanks!
[Magisk] yung40oz84:
https://xdaforums.com/apps/magisk/beta-magisk-v13-0-0980cb6-t3618589/post73397845

---------- Post added at 07:12 PM ---------- Previous post was at 07:11 PM ----------

https://xdaforums.com/apps/magisk/beta-magisk-v13-0-0980cb6-t3618589/post73399000
[Magisk] Zackptg5:
[MENTION=6044187]yung40oz84[/MENTION]

Hey everyone, current beta 13.5 has issues with systemless hosts so stay on 13.3 stable for now
[Magisk] oux:
yesterday I flashed pure fusion os rom on my oneplus 3 and then installed the module but got ads. I started with fgp, but also tried just m since I got that suggestion before. tried reapply a few times and ofcourse rebooted the device several times. Magisk 13.3 and Magisk manager 5.1.1. systemless hosts enabled. my hosts file is 1.37 mb. also tried to wipe cache/dalvik
[Magisk] hitmancho:
I have an issue adding to the whitelist: I create the file, I type in:

0.0.0.0 www.ad.fly

0.0.0.0 www.bit.ly

and everything downloads correctly from the terminal. I use the "su -c hosts m w" command. But I still can't open any links from those ad sites again. Also, what does "regex" and "wildcard" mean in the context of whitelist. I can't understand anything from online results.
[Magisk] Didgeridoohan:
Try removing www.

0.0.0.0 bit.ly

And search for "grep regex" to find out how to use the regex version of whitelist.
[Magisk] hitmancho:
That worked, I think. I can open the sites now.
[Magisk] pinoholo:
Is it safe to install with latest beta 13.6?
[Magisk] Zackptg5:
Seems to work fine on my end
[Magisk] Zackptg5:
Version 2.7 update! Added feature where the if you already have, for example, v2.7 of the module installed and you flash the 2.7 installer again, it'll uninstall instead of install. Thanks to [MENTION=4667597]Didgeridoohan[/MENTION] for the idea (I saw your github issue on it :))
[Magisk] Didgeridoohan:
Thanks, but all credit for that idea should go to [MENTION=7526972]Rorsch_X1[/MENTION].
[Magisk] DAusRon:
Hi [MENTION=6037748]Zackptg5[/MENTION]

Could you give any thought on this?
[Magisk] Zackptg5:
Your log shows that the hosts file is properly mounted so it "should" work but any issues with systemless hosts are from magisk, not this module.

I don't support the beta builds because they're betas and I'd rather not sift through all of the random issues that may pop up (like this one). I've tried the the betas and have had weird stuff like this happen too
[Magisk] DAusRon:
thanks mate. I'm pretty sure it's not your module's bug as the log already showed it's properly mounted. Maybe it's from the manager 5.2 itself.

Anyway thanks for the heads up.
[Magisk] alvaro363901:
I have a problem. I update the hosts file and works well, but suddenly it stops working and ads appear.
[Magisk] error404:
Hey guys I'm having trouble with white list. I created a file named whitelist in /sdcard and wrote this in the file:

0.0.0.0 a d f . ly

And also tried

0.0.0.0 *a d f . ly

(without the spaces of course).

Tried to apply whitelist and regex, but the sites still blocked. Am I doing something wrong?
[Magisk] Zackptg5:
You're missing something :)

The '*' tells grep to repeat any number of the preceding character. Your preceding character is a space so it's not what you'd want. The '.' when used in front of the asterisk means 'any character'. So in your case, try:

.*a d f . l y.*

and use the 'wr' option.

However, there's no variants of 'a d f . l y' in the unified hosts files so you could just do it the old fashioned way:

0.0.0.0. a d f . l y

and use the 'w' option.

(without the spaces. That's some weird xda blocking right there)
[Magisk] LSS4181:
The module works good but I need some help with diagnosing the causes of certain apps that were broken because of the hosts file.

The truth is, on recent Android versions some apps seems to be able to either detect the presence of a modified hosts file, or whether certain links failed to load, and redirect to either an error page, or for some apps, a blank page.

Before using it I was on AdAway and had this issue, and looks like this module also has it after applying the hosts file. Back then it took me a whole night figuring out what could be causing the issues (I tried using alternative webviews, enabling multiprocess webviews and such at that time, thought it might be due to WebView incompatibility)... until I found out that disabling AdAway fixed it.

As such, I need a good way to find out what exactly was tripping the page loading detection so I can isolate the problems and deal with them separately.
[Magisk] Zackptg5:
Sounds like you need to whitelist some sites.

First, remove any hosts modifications (disable and then reenable systemless hosts). Then in adaway, open the 3 dot menu and go to 'log DNS requests'. Start tcpdump, open apps in question, then go back that adaway page and open the tcpdump log to see what sites it's trying to access. Add those to your whitelist, apply host filters and whitelist and see what happens. It's trial and error really at this point
[Magisk] Mrwhitepantz:
Can someone explain what the exactly the different hosts (fakenews, gambling, social, porn) block or why I wouldn't want to block everything? Are they just different ad providers?
[Magisk] Zackptg5:
They block exactly what they say they block. Note that all hosts include the master one as well (adware/malware).

You may not want to use the social filter cause it'll block all social networking such as Facebook and twitter
[Magisk] Megakaban_:
I've made an app GUI for this module. You can download it from GP https://play.google.com/store/apps/details?id=ru.dixl0f0s.unifiedhostsmanager or from attachment

If you are getting error without description, just try restarting app
[Magisk] Stefan6183:
Hi,

I installed Unified_Hosts_Adblock_v2.7.zip from Magisk 13.3.

After rebooting, I am getting a bootloop.

I uninstalled it in TWRP using Unified_Hosts_Adblock_v2.7.zip and Android 7.0 boots again.

I have a Samsung Galaxy S8 (model SM-G950F) with unofficial Xposed also installed (classic version, not as Magisk module).

Does someone have an idea why I am getting the bootloops after installing the Unified_Hosts_Adblock module?
[Magisk] Zackptg5:
Nice!

Without logs, not really but it's probably Xposed related. If you're using magisk, you should probably be using the Xposed module instead of the system install
[Magisk] Stefan6183:
Unfortunately, I could not get the Magisk Xposed module to run - I also got bootloops there.

This is why I'm using the classic (system install) version.

Maybe I'll try to disable Xposed and install Unified Hosts Adblocker afterwards.

If I keep getting bootloops: How can I create logs that help to analyze the problem using TWRP (I cannot get into Android due to the bootloop)?
[Magisk] Zackptg5:
Try it without xposed. See this guide for taking logcats via adb: https://xdaforums.com/showthread.php?t=1726238
[Magisk] Legendary Cryptid:
I keep getting errors like this every time I try to use systemless hosts.

--2017-08-23 21:14:02--
[Magisk] LSS4181:
Thanks. I was eventually able to figure out the cause with AdAway's log DNS requests. After searching I found out my problem with that app was with www.google-analytics.com alone.

It seems on recent Android versions, blocking www.google-analytics.com can trip some certain in-app or in-page mechanisms to redirect users to a page loading error page or simply abort page loading. Strangely enough, on an old device of mine running Android 4.4 (KitKat) this never happened.

EDIT: This is only one particular case... but it seems true that with the recent Android versions (mine is 7.1) app or page designers can now include mechanisms that will trip if certain links (be it ads or tracking services) fail to load.
[Magisk] Reynt:
Need some help. Did I understand the whitelist fuction wrong here?

I really like this host adblock but I really need to visit a website which blocks users with adblock filters. How can I use this website again? Whitelist doesn't work.
[Magisk] Zackptg5:
Some sites have some kind of JavaScript or something that detects if adblockers are running. This module isn't a browser extension or anything so I dont know if you'd need to whitelist all domains running in that website or not to get around that
[Magisk] Sophia.xda:
Thanks its work. Good job ?
[Magisk] Didgeridoohan:
Without you telling us the details about what you've done so far it's gonna be impossible to answer that question.