Package: secrules-parsing
OWASP CRS Rules parserIncomplete parser model and sample application for parsingCore Rule Setwritten in the ModSecurity DSL SecRule language. It uses the python librarytextXfor parsing.How to use it (CLI):Install dependencies
Dependencies can be installed system-wide, or just for your user (using--user).System-wide:sudopipinstallsecrules-parsingUser:pipinstall--usersecrules-parsingExecutesecrules-parserspecifying the location of the files you want to scan using the -f/--files argument. This takes wildcards or individual files.$ secrules-parser -c -f /owasp-crs/rules/*.confAdd flags to accomplish needed tasks:-h, --help:Description:show the help message and exitExample:$ secrules-parser -h-r, --regex:Description:Extract regular expressions from rules fileExample:$ secrules-parser --regex -f /owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
{"/owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf": [{"920100": ["^(?i:(?:[a-z]{3,10}\\s+(?:\\w{3,7}?://[\\w\\-\\./]*(?::\\d+)?)?/[^?#]*(?:\\?[^#\\s]*)?(?:#[\\S]*)?|connect (?:\\d{1,3}\\.){3}\\d{1,3}\\.?(?::\\d+)?|options \\*)\\s+[\\w\\./]+|get /[^?#]*(?:\\?[^#\\s]*)?(?:#[\\S]*)?)$"]}, {"920120": ["(?<!&(?:[aAoOuUyY]uml)|&(?:[aAeEiIoOuU]circ)|&(?:[eEiIoOuUyY]acute)|&(?:[aAeEiIoOuU]grave)|&(?:[cC]cedil)|&(?:[aAnNoO]tilde)|&(?:amp)|&(?:apos));|['\\\"=]"]}, {"920160": ["^\\d+$"]}, {"920170": ["^(?:GET|HEAD)$"]}, {"920171": ["^(?:GET|HEAD)$"]}, {"920180": ["^POST$"]}, {"920190": ["(\\d+)\\-(\\d+)\\,"]}, {"920210": ["\\b(?:keep-alive|close),\\s?(?:keep-alive|close)\\b"]}, {"920220": ["\\%(?:(?!$|\\W)|[0-9a-fA-F]{2}|u[0-9a-fA-F]{4})"]}, {"920240": ["^(?:application\\/x-www-form-urlencoded|text\\/xml)(?:;(?:\\s?charset\\s?=\\s?[\\w\\d\\-]{1,18})?)??$"]}, {"920260": ["\\%u[fF]{2}[0-9a-fA-F]{2}"]}, {"920290": ["^$"]}, {"920310": ["^$"]}, {"920311": ["^$"]}, {"920330": ["^$"]}, {"920340": ["^0$"]}, {"920350": ["^[\\d.:]+$"]}, {"920420": ["^(?:GET|HEAD|PROPFIND|OPTIONS)$"]}, {"920440": ["\\.(.*)$"]}, {"920450": ["^.*$"]}, {"920200": ["^bytes=(?:(?:\\d+)?\\-(?:\\d+)?\\s*,?\\s*){6}"]}, {"920230": ["\\%((?!$|\\W)|[0-9a-fA-F]{2}|u[0-9a-fA-F]{4})"]}, {"920121": ["['\\\";=]"]}, {"920460": ["(?<!\\Q\\\\\\E)\\Q\\\\\\E[cdeghijklmpqwxyz123456789]"]}]}-c, --correctness:Description:Check the validity of the syntaxExample:$ secrules-parser -c -f /owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
Syntax OK: ../../../rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf-v, --verboseDescription:Print verbose messagesExample:$ secrules-parser -c -v -f /owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
...-o FILE, --output FILEDescription:Output results to fileExample:$ secrules-parser -c -o out.json -f /owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf--output-type github | plainDescription:Desired output format. Useful if running from Github Actions and you want annotated outputExample:$ secrules-parser -c --output-type github -f /owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.confHow to use it (API):process_rules(list files)Takes a list of file path's and returns modelsimportglobimportosfromsecrules_parsingimportparser# Extract all of our pathingfiles=glob.glob("../../rules/*.conf")# Pass absolute paths because of module locationfiles=[os.path.abspath(path)forpathinfiles]models=parser.process_rules(files)get_correctness(list files, list models)importglobimportosfromsecrules_parsingimportparser# Extract all of our pathingfiles=glob.glob("../../rules/*.conf")# Pass absolute paths because of module locationfiles=[os.path.abspath(path)forpathinfiles]models=parser.process_rules(files)parser.get_correctness(files,models)DevelopmentIf you want to modify this module, follow these steps:Clone this repository:git clone git@github.com:coreruleset/secrules_parsing.gitDo not forget to install dependencies usingpoetry:poetry installfirst!Edit and change the files you want.Write tests! Tests are in thetestssubdirectoryCreate a PRhere, and ask for review!MiscTo visualize the syntax tree, use:textx visualize secrules.tx
dot -Tpng -O secrules.tx.dotThen review the generated PNG modsec.tx.dot.png!Please file anissueif you find a bug or you want some feature added.