Package: cfspflat
Cloudflare SPF FlattenerFinding sender-policy-flattener (spflat)A few weeks ago one of our SaaS vendors added a couple of additional DNS entries to their SPF record (without telling us...).We started to get reports of email deliver failures. Looking into it we discovered the number of DNS entries in our SPF record was the problem. The SaaS vendors change put us over the maximum of 10.We've decided future SaaS vendors and partners will send from a subdomain, but for existing ones there are too many actors - both internal and external to accomplish this change any time soon.Hence we started researching SPF flattening as the best solution, but were concerned about the vendors making changes to the sender records.Looking for a solution to that probkem we stumbled upon thesender-policy-flattenerproject.  This tool -spflat- solved many problems and has some excellent features and characteristics.The project is opensourceIt will query your list of approved sendors and flatten these into a list of appropriately formatted and sized spf records composed ofip4andip6spf entries.Provide these entries as a chain of spfincludeentries.Should any of your the senders dns entries change, it sends email indicating the changes and the the corrections required to your spf records.spflatworks reliably and accuratelyCan run entirely from a JSON configuration file.Written in Python, and easy to install and configure.sendier-policy-flattnerdid90%of what we desired for flattening and managing our SPF records.Requirements that led to this projectAfter usingsplatfor a bit we come up with a wish list - the other10%of what we needed.Here is the wishlist of requirements we came up with:The ability to runspflatin the command line, without the email being sent. (a--no-emailswitch)Our public DNS zones are in Cloudflare. We wantedspflattoAUTOMATICALLYupdate the necessary resource records in Cloudflare when a  change is detected. (a--updateswitch).Ability to havespflatre-create the SPF zone records in Cloudflare DNS for zone repair (and initial configuration.) (a--force-updateswitch)Have the output status file (sums file) update ONLY if the zones are updated.originalspflatreplaces the sums file each time it ran, so in monitoring mode a change was reported only once.sender-policy-flatteneris a fine tool, so we did not want to reinvent the wheel:Utilizedspflatby supplementing its functionality.A supplement, not a replacement.Keep the changes opensource:Recognize the debt tosender-policy-flattenerMake the changes avialable under the same licenses.To limit confusion, change the command fromspflattocfspflatHencecfspflat-CloudFlare Sender Policy Flattenerwas created.cfspflat- Cloudflare Sender Policy FlattenerQuick overview:%cfspflat-h
usage:cfspflat[-h][-cCONFIG][-oOUTPUT][--update-records][--force-update][--no-email]AscriptthatcrawlsandcompactsSPFrecordsintoIPnetworks.Thishelpsto
avoidexceedingtheDNSlookuplimitoftheSenderPolicyFramework(SPF)https://tools.ietf.org/html/rfc7208#section-4.6.4

optionalarguments:-h,--helpshowthishelpmessageandexit-cCONFIG,--configCONFIGName/pathofJSONconfigurationfile(default:spfs.json)-oOUTPUT,--outputOUTPUTName/pathofoutputfile(defaultspf_sums.json)--update-recordsUpdateSPFrecordsinCloudFlare--force-updateForceanupdateofSPFrecordsinCloudflare--no-emaildon'tsendtheemailThesender-policy-flattenermodule is installed as part ofcfspflatThe existing core ofspflatis kept mostly intact, so the basic features are maintained bycfspflat.The changes to accomodatecfspflatwere in the parameter handling and adding the Cloudflare updates to the processing look.Thecloudflarelibrary, with some abstraction classes, is used to make the zone updates.cfspflateliminates many of the command arguments of spflat in favor of using the json config file.Cloudflare TXT records are automatically generated and updated when the configuration changes.Withcfspflatyou can completely automate your SPF flattening using cfspflat with the--updateswitch in a cron job, even silently with the--no-emailswitchInstalling and Configuring  cfspflat1. pip install the cfspflat%pipinstallcfspflatBut it's advisable to do this in its own venv:%python3-mvenvspfenv
%sourcespfenv/bin/activate
%pipinstallcfspflatpip will install the prerequisites, including thesender-policy-flattner(spflat),dnspython,netaddr, andcloudflarepython modules.The executable is installed in bin of the venv ascfspflat2. Create an anchor SPF record for the zone apex in CloudflareCreate the TXT SPF record on zone apex used (e.g. example.com), At the end of this anchor record include the first SPF record that slpat will write - spf0.example.comwe also include our ownip4andip6entries in this anchor record.example.com TXT "v=spf1 mx include:spf0.example.com -all"This anchor record is never changed bycfspflat. It's purpose is to link to the first SPF record in the chain thatcfspflatmanages.2. Edit the cfspflat configuration fileCreate a spfs.json file.  Add all the entries required:cfspflatuses the same configuration and sums file formats as the originalspflat.If you already use spflat you can use thos files as is with cfspflat.There is one extension - the "update_subject" entry containing the subject of the email sent when cfspflat has updated your SPF records.  This message will contain the same detail spflat provides.spfs.jsonis the default name of the config file, but it can be specified with the--configswitch.Here is an example config file:Example spfs.json configuration file:{"sending domains":{"example.edu":{"amazonses.com":"txt","goodwebsolutions.net":"txt",....moresenderspf'shere...."delivery.xyz.com":"txt","spf.protection.outlook.com":"txt"}},"resolvers":["1.1.1.1","8.8.8.8"],"email":{"to":"dnsadmins@example.com","from":"spf_monitor@example.com","subject":"[WARNING] SPF Records for {zone} have changed and should be updated.","update_subject":"[NOTICE] SPF Records for {zone} have been updated.","server":"smtp.example.com"},"output":"monitor_sums.json"}Config file detailsThesending domainssection isrequiredand contains sending domain entries: this is your sender domain (e.g. example.com forj.smith@example.com, noreply.example.com fordeals@noreply.example.com)  There can be multiple sending domains in the config file.Each sending domain contains dns spf records for the dnsincluderecords of your approved senders.These dns names are resolved and flattened:These entries are in the key-value pairs of  : .Record type can be "txt" (other SPF records),  "A" and "AAAA" records (for specific hosts).Theresolverssection isoptional(using the system default DNS resolvers if none are supplied)Theemailstanza isrequiredand is global (i.e. for allsending domains).  This section includes:subject(optional)is the email subject if a change was detected but no updates were made. The default message is the one shown in the example.update_subject(optional)is the email subject if a change was detected and the dns records were updated. The default message is shown in the example.to- isrequired- this is the destination for emails sent bycfspflatfrom- isrequired- the source email of the messagescfspflatsendsserver- isrequired- your mail relay.outputis the file that maintains the existing state and checksum of sender records. If this is not specifiedspfs_sum.jsonis used.Output file detailsTheoutputfile is a JSON file only updated if it is new (empty) or the records have been updated.Likewise the default output file isspf_sums.jsonbut can be changed in the config file or by the--outputswitch.This contains the list of flattened spf records and a checksum used to assess changes in senders records.Because you recieve emails of detected changes or updates, there is little reason to care about the output file.3. Create a credentials file for CloudflareThere are a couple of locations and formats for the API credentials Cloudfare requires.Consult cloudflare documentation and the dashboard for creating API credentialsConsult the python-cloudflare site for documentation on passing credentials.For simplicity you can use the.cloudflare.cfgfile in either your home directory or in the directory you will run cfspflat.[CloudFlare]
email = dnsadmin@example.org
token = 1234567890abc...abcIt should go without saying - protect the API keys and this file.It's also possible to pass the credentials as environment variables.4. Runcfspflatto build the sums file and SPF entriesRun cfspflat twice:%cfspflat--no-email
%cfspflat--forceThe first time constructs the base records and the second time forces the dns updates.With force update the DNS records are created even if a change hasn't been detected.A list of the records will be sent to your email.5. Automatecfspflatto the level you are comfortable withYou are up and running:You can runcfspflatin advisory mode (likespflat) sending you emails notifying of changesOr you can run it with the--update-recordsswitch and update your records automatically whenever they change (still giving you notifications of the changes made.)Example email formatExample fromsender-policy-flattenerREADME: