Package: psl-dns
DNS-based Public Suffix List handling for PythonThis Python package provides aPSLclass forquerying the Public
Suffix List (PSL)via the DNS. By utilizing
the library, one can retrieve information about the public suffix
status of a domain as well as the PSL rules governing it. There is also
a corresponding command-line tool,psl-dns_query, enabling convenient
use of the library from the shell.Public suffix information is based on DNS lookups only; no rule
matching is performed at lookup time. To make this possible, the PSL
rules have been encoded in the DNS itself (currently under the
DNSSEC-enabled zonequery.publicsuffix.zone). This facilitates easy
querying without the need to keep the PSL at hand. The PSL zone is
maintained bySSEand usually updated once
a day.TheParserclass (along with thepsl-dns_parsecommand) is used to
iterate over aPSL fileand convert the ruleset into a list of DNS Resource Record sets for
submission to the DNS operator. The tool adds an extraTXTrecord at
the root of the PSL zone, containing the parsing timestamp as well as
the PSL file SHA-256 hash for currentness checking.The package also contains thepsl-dns_checkcommand (based on theCheckerclass) to iterate over a PSL file and query the DNS for each
rule encountered, to verify whether the PSL zone contents are in
agreement with the file. (Note that DNS caching may cause update
delays; after a zone update, you may be receiving outdated information
until the TTL of the PSL DNS records expires. To make sure, specify one
of the PSL zone's authoritative servers as theresolverargument.)Note:DNS resolvers learn about the domains that get queried, so
depending on the use case, using this service may not be up to your
privacy standards. It is possible though to set up a private copy of
the query zone and configure a local resolver to avoid query leaks.UsagePythonThe following examples show how to query the PSL via DNS using thePSLclass. For advanced use, please refer to the source.Example use cases for theParserandCheckerclasses can be found
in the scripts underpsl/commands/.Initialize>>>frompsl_dnsimportPSL>>>psl=PSL()If your system resolver does not supportPTRrecords, you can set
another resolver during initialization:PSL(resolver='...')Query public suffix status of a domain (for the rules, see below)>>>psl.is_public_suffix('com')True>>>psl.is_public_suffix('checkip.dedyn.io')False>>>psl.is_public_suffix('takatsu.kawasaki.jp')True>>>psl.is_public_suffix('www.ikuoufukushi.takatsu.kawasaki.jp')False>>>psl.is_public_suffix('city.kawasaki.jp')False>>>psl.is_public_suffix('www.library.city.kawasaki.jp')FalseGet the public suffix for a domain>>>psl.get_public_suffix('com')'com'>>>psl.get_public_suffix('checkip.dedyn.io')'dedyn.io'The following examples are based on PSL wildcard rules. Wildcard labels
are expanded into the respective labels of the domain of interest:>>>psl.get_public_suffix('takatsu.kawasaki.jp')# Wildcard *.kawasaki.jp'takatsu.kawasaki.jp'>>>psl.get_public_suffix('www.ikuoufukushi.takatsu.kawasaki.jp')# same'takatsu.kawasaki.jp'>>>psl.get_public_suffix('city.kawasaki.jp')# Wildcard exception'jp'>>>psl.get_public_suffix('www.library.city.kawasaki.jp')# same'jp'If the queried domain has a trailing dot, the dot is preserved in the
response. Furthermore, IDDA mode is preserved so that Unicode queries
return Unicode responses, and Punycode queries return Punycode responses:>>>psl.get_public_suffix('www.xn--55qx5d.cn')'xn--55qx5d.cn'>>>psl.get_public_suffix('www.公司.cn.')'公司.cn.'Get the set of rules applicable for a domain>>>psl.get_rules('com'){'com'}>>>psl.get_rules('checkip.dedyn.io'){'dedyn.io'}>>>psl.get_rules('takatsu.kawasaki.jp'){'*.kawasaki.jp'}>>>psl.get_rules('www.ikuoufukushi.takatsu.kawasaki.jp'){'*.kawasaki.jp'}>>>psl.get_rules('city.kawasaki.jp')# Note wildcard exception{'jp','!city.kawasaki.jp','*.kawasaki.jp'}>>>psl.get_rules('www.library.city.kawasaki.jp')# same{'jp','!city.kawasaki.jp','*.kawasaki.jp'}Rules are always returned in Unicode encoding and without a trailing
dot, consistent with the encoding in the Public Suffix List itself:>>>psl.get_rules('www.xn--55qx5d.cn.'){'公司.cn'}Command linepsl-dns_queryThis is a command-line interface to thePSLclass demonstrated in the
previous section.$psl-dns_query-h
usage:psl-dns_query[-h][--zoneZONE][--resolverRESOLVER][--timeoutTIMEOUT][-l][-c][-v]domain

QuerythePSLviaDNSandcheckthePSLstatusofadomain.

Returnsthetheword"public"or"private",followedbythepublic
suffixthatcoversthequerieddomain.IDNAmodeandtrailingdots(ifgiven)arepreserved.

Optionally,thesetofapplicablerulesandthePSLchecksumcanbe
displayed.

Exitcodes:0(public)or1(private).

positionalarguments:domainDomaintoquery

optionalarguments:-h,--helpshowthishelpmessageandexit--zoneZONEPSLzonetouse(default:query.publicsuffix.zone)--resolverRESOLVERDNSresolvertouseinsteadofsystemresolver(default:None)--timeoutTIMEOUTDNSquerytimeout(seconds)(default:5)-lShowsetofapplicablerules(default:False)-cShowPSLchecksum(default:False)-v,--verboseIncreaseoutputverbosity(default:0)Retrieve status and public suffix# Plain$psl-dns_querycom
publiccom# Same, followed by the set of relevant rules, in no particular order$psl-dns_querywww.ck-l
private*
*.ck
!www.ck
*psl-dns_parse$psl-dns_parse-h
usage:psl-dns_parse[-h][--zoneZONE][--formatFORMAT][-l][-v]psl_file

PrintrulesfromaPublicSuffixList(PSL)fileinDNSRRsetsformat.

positionalarguments:psl_filePathtoPSLfile

optionalarguments:-h,--helpshowthishelpmessageandexit--zoneZONEPSLzonetouse(default:query.publicsuffix.zone)--formatFORMATOutputformattouse(default:deSEC)-lListavailableformats(default:False)-v,--verboseIncreaseoutputverbosity(default:0)Convert current PSL file to deSEC RRsets# Note: This produces very long output$timepsl-dns_parse<(curlhttps://publicsuffix.org/list/public_suffix_list.dat)|jq.[{"subname":"ac","ttl":86400,"type":"PTR","records":["ac."]},...# shortened for readability{"subname":"","ttl":86400,"type":"TXT","records":["\"1555895008 d205f587d61c6bbf05bec818776da1dd030ce68f2e8912fea732158b9a33cc54\""]}]real0m1.262s
user0m0.475s
sys0m0.239spsl-dns_check$psl-dns_check-h
usage:psl-dns_check[-h][--resolverRESOLVER][--timeoutTIMEOUT][--zoneZONE][-v]psl_file

CheckrulesfromthePublicSuffixList(PSL)viaDNSandoutput
inconsistencies.

positionalarguments:psl_filePathtoPSLfile

optionalarguments:-h,--helpshowthishelpmessageandexit--resolverRESOLVERDNSresolvertouseinsteadofsystemresolver(default:None)--timeoutTIMEOUTDNSquerytimeout(inseconds)(default:5)--zoneZONEPSLzonetouse(default:query.publicsuffix.zone)-v,--verboseIncreaseoutputverbosity(default:0)Verifying the correctness of the PSL zone$timepsl-dns_check-v<(curlhttps://publicsuffix.org/list/public_suffix_list.dat)...# shortened for readabilityINFO:psl:Queryingforzone.id.query.publicsuffix.zone.TXT
INFO:psl:Queryingforzone.id.query.publicsuffix.zone.PTR
INFO:psl:Queryingforquery.publicsuffix.zone.TXT
WARNING:psl:Hashmismatch!InputPSLfileappearstodifferfromremoteversion.8684ruleswith3inconsistenciesfound

real13m42.366s
user0m38.560s
sys0m8.383s