Package: adblockparser
adblockparseradblockparseris a package for working withAdblock Plusfilter rules.
It can parse Adblock Plus filters and match URLs against them.Installationpip install adblockparserPython 2.7 and Python 3.3+ are supported.If you plan to use this library with a large number of filters
installingpyre2library is highly recommended: the speedup
for a list of defaultEasyListfilters can be greater than 1000x.pip install ‘re2 >= 0.2.21’Note that pyre2 library requires C++re2library installed.
On OS X you can get it using homebrew (brew install re2).UsageTo learn about Adblock Plus filter syntax check these links:https://adblockplus.org/en/filter-cheatsheethttps://adblockplus.org/en/filtersGet filter rules somewhere: write them manually, read lines from a file
downloaded fromEasyList, etc.:>>> raw_rules = [
...     "||ads.example.com^",
...     "@@||ads.example.com/notbanner^$~script",
... ]CreateAdblockRulesinstance from rule strings:>>> from adblockparser import AdblockRules
>>> rules = AdblockRules(raw_rules)Use this instance to check if an URL should be blocked or not:>>> rules.should_block("http://ads.example.com")
TrueRules with options are ignored unless you pass a dict with options values:>>> rules.should_block("http://ads.example.com/notbanner")
True
>>> rules.should_block("http://ads.example.com/notbanner", {'script': False})
False
>>> rules.should_block("http://ads.example.com/notbanner", {'script': True})
TrueConsult with Adblock Plusdocsfor options description. These options allow to write filters that depend
on some external information not available in URL itself.PerformanceRegex enginesAdblockRulesclass creates a huge regex to match filters that
don’t use options.pyre2library works better than stdlib’s re
with such regexes. If you havepyre2installed thenAdblockRulesshould work faster, and the speedup can be dramatic - more than 1000x
in some cases.Sometimes pyre2 prints something likere2/dfa.cc:459: DFA out of memory: prog size 270515 mem 1713850to stderr.
Give re2 library more memory to fix that:>>> rules = AdblockRules(raw_rules, use_re2=True, max_mem=512*1024*1024)  # doctest: +SKIPMake sure you are using re2 0.2.20 installed from PyPI, it doesn’t work.Parsing rules with optionsRules that have options are currently matched in a loop, one-by-one.
Also, they are checked for compatibility with options passed by user:
for example, if user didn’t pass ‘script’ option (with aTrueorFalsevalue), all rules involvingscriptare discarded.This is slow if you have thousands of such rules. To make it work faster,
explicitly list all options you want to support inAdblockRulesconstructor,
disable skipping of unsupported rules, and always pass a dict with all options
toshould_blockmethod:>>> rules = AdblockRules(
...    raw_rules,
...    supported_options=['script', 'domain'],
...    skip_unsupported_rules=False
... )
>>> options = {'script': False, 'domain': 'www.mystartpage.com'}
>>> rules.should_block("http://ads.example.com/notbanner", options)
FalseThis way rules with unsupported options will be filtered once, whenAdblockRulesinstance is created.LimitationsThere are some known limitations of the current implementation:element hiding rules are ignored;matching URLs against a large number of filters can be slow-ish,
especially ifpyre2is not installed and many filter options are enabled;match-casefilter option is not properly supported (it is ignored);documentfilter option is not properly supported;rules are not validatedbeforeparsing, so invalid rules may raise
inconsistent exceptions or silently work incorrectly.It is possible to remove all these limitations. Pull requests are welcome
if you want to make it happen sooner!Contributingsource code:https://github.com/scrapinghub/adblockparserissue tracker:https://github.com/scrapinghub/adblockparser/issuesIn order to run tests, installtoxand typetoxfrom the source checkout.The license is MIT.Changes0.7 (2016-10-17)Fixed parsing issue with recent easylist.txt;fixed a link to easylist (thankshttps://github.com/limonte).0.6 (2016-09-10)Added support for regex rules (thankshttps://github.com/mlyko).0.5 (2016-03-04)Fixed an issue with blank lines in filter files
(thankshttps://github.com/skrypka);fixed an issue with applying rules with ‘domain’ option
when domain doesn’t have a dot (e.g. ‘localhost’);Python 2.6 and Python 3.2 support is dropped;
adblockparser likely still work in these interpreters,
but this is no longer checked by tests.0.4 (2015-03-29)AdblockRule now caches the compiled regexes (thankshttps://github.com/mozbugbox);Fixed an issue with “domain” option handling
(thankshttps://github.com/nbraemfor the bug report and a test case);cleanups and test improvements.0.3 (2014-07-11)Switch to setuptools;better__repr__forAdblockRule;Python 3.4 support is confirmed;testing improvements.0.2 (2014-03-20)This release provides much fasterAdblockRules.should_block()method
for rules without options and rules with ‘domain’ option.better combined regex for option-less rules that makes re2 library
always use DFA without falling back to NFA;an index for rules with domains;paramsmethod arguments are renamed tooptionsfor consistency.0.1.1 (2014-03-11)By defaultAdblockRulesautodetects re2 library and uses
it if a compatible version is detected.0.1 (2014-03-03)Initial release.