Repository: qutebrowser/qutebrowser

Issue:
Use URL patterns for content.host_blocking.whitelist
For historical reasons, the `content.host_blocking.whitelist` setting uses [fnmatch](https://docs.python.org/3/library/fnmatch.html) (see `is_whitelisted_host` in `browser/adblock.py`).

Now that we have [URL patterns](https://github.com/qutebrowser/qutebrowser/blob/master/qutebrowser/utils/urlmatch.py), it should use those instead.
Use URL patterns for content.host_blocking.whitelist
For historical reasons, the `content.host_blocking.whitelist` setting uses [fnmatch](https://docs.python.org/3/library/fnmatch.html) (see `is_whitelisted_host` in `browser/adblock.py`).

Now that we have [URL patterns](https://github.com/qutebrowser/qutebrowser/blob/master/qutebrowser/utils/urlmatch.py), it should use those instead.


Discussion:
Can i work on this issue ?

@sumit9696 Nobody has worked on this yet, and it should be a good newcomer issue - so please do, and feel free to ask if you need help!
Ok.. Thanks...
Can u elaborate me this issue a little?
This should probably build on top of https://github.com/qutebrowser/qutebrowser/pull/4181/commits/7fb222464000de15dac945b40712615fd546d9c3 so that custom validation of the patterns doesn't need to be done. At which point the `config.val.content.host_blocking.whitelist` setting can be changed to be a list of url patterns instead on strings.
The whitelist check is here https://github.com/qutebrowser/qutebrowser/blob/25c189d8d469b123a9284847e48f4cbfc5a2fa0b/qutebrowser/browser/adblock.py#L71 and apart from the validiation that'll have to be changed to be something like `urlmatch.UrlPattern(pattern).matches(url)`, and the caller noods to be modified to pass in the whole url instead of just the host.
That way people can block whole domains and whitelist individual pages.
Can i work on this issue ?

@sumit9696 Nobody has worked on this yet, and it should be a good newcomer issue - so please do, and feel free to ask if you need help!
Ok.. Thanks...
Can u elaborate me this issue a little?
This should probably build on top of https://github.com/qutebrowser/qutebrowser/pull/4181/commits/7fb222464000de15dac945b40712615fd546d9c3 so that custom validation of the patterns doesn't need to be done. At which point the `config.val.content.host_blocking.whitelist` setting can be changed to be a list of url patterns instead on strings.
The whitelist check is here https://github.com/qutebrowser/qutebrowser/blob/25c189d8d469b123a9284847e48f4cbfc5a2fa0b/qutebrowser/browser/adblock.py#L71 and apart from the validiation that'll have to be changed to be something like `urlmatch.UrlPattern(pattern).matches(url)`, and the caller noods to be modified to pass in the whole url instead of just the host.
That way people can block whole domains and whitelist individual pages.

Resolving patch:
diff --git a/qutebrowser/browser/adblock.py b/qutebrowser/browser/adblock.py
--- a/qutebrowser/browser/adblock.py
+++ b/qutebrowser/browser/adblock.py
@@ -24,11 +24,10 @@
 import functools
 import posixpath
 import zipfile
-import fnmatch
 
 from qutebrowser.browser import downloads
 from qutebrowser.config import config
-from qutebrowser.utils import objreg, standarddir, log, message
+from qutebrowser.utils import objreg, standarddir, log, message, urlmatch
 from qutebrowser.commands import cmdutils
 
 
@@ -61,14 +60,14 @@ def get_fileobj(byte_io):
     return byte_io
 
 
-def is_whitelisted_host(host):
-    """Check if the given host is on the adblock whitelist.
+def is_whitelisted_url(url):
+    """Check if the given url is on the adblock whitelist.
 
     Args:
-        host: The host of the request as string.
+        url: The url to check in QUrl form.
     """
     for pattern in config.val.content.host_blocking.whitelist:
-        if fnmatch.fnmatch(host, pattern.lower()):
+        if pattern.matches(url):
             return True
     return False
 
@@ -118,7 +117,7 @@ def is_blocked(self, url):
         host = url.host()
         return ((host in self._blocked_hosts or
                  host in self._config_blocked_hosts) and
-                not is_whitelisted_host(host))
+                not is_whitelisted_url(url))
 
     def _read_hosts_file(self, filename, target):
         """Read hosts from the given filename.


Test patch:
diff --git a/tests/unit/browser/test_adblock.py b/tests/unit/browser/test_adblock.py
--- a/tests/unit/browser/test_adblock.py
+++ b/tests/unit/browser/test_adblock.py
@@ -33,7 +33,7 @@
 
 # TODO See ../utils/test_standarddirutils for OSError and caplog assertion
 
-WHITELISTED_HOSTS = ('qutebrowser.org', 'mediumhost.io')
+WHITELISTED_HOSTS = ('qutebrowser.org', 'mediumhost.io', 'http://*.edu')
 
 BLOCKLIST_HOSTS = ('localhost',
                    'mediumhost.io',
@@ -50,7 +50,8 @@
                  'http://ads.worsthostever.net',
                  'http://goodhost.gov',
                  'ftp://verygoodhost.com',
-                 'http://qutebrowser.org')
+                 'http://qutebrowser.org',
+                 'http://veryverygoodhost.edu')
 
 
 class BaseDirStub: