Package: pySigma
pySigmapySigmais a python library that parses and converts Sigma rules into queries. It is a replacement
for the legacy Sigma toolchain (sigmac) with a much cleaner design and is almost fully tested.
Backends for support of conversion into query languages and processing pipelines for transforming
rule for log data models are separated into dedicated projects to keep pySigma itself slim and
vendor-agnostic. See theRelated Projectssection below to get an overview.Getting StartedTo start usingpySigma, install it using your python package manager of choice. Examples:pip install pysigma
pipenv install pysigma
poetry add pysigmaDocumentation with some usage examples can be foundhere.Autodiscovery and Migration of Backend PluginsPreviously, in order to export the objects (backends, pipelines, and validators) of the plugin, it was required to manually export them using global variables in the__init__.pyfile of the corresponding module. However, this manual export is no longer necessary as pySigma now employs a better mechanism for autodiscovery to locate the exported objects. Consequently, the use of global variables becomes redundant and can be eliminated. Nevertheless, to ensure compatibility with older versions of pySigma and sigma-cli, support for global variables will be retained for the time being. However, it is important to note that in a future release of pySigma, the global variables will be completely phased out.Pipeline changes after v0.9.11In the previous implementation, pipelines were defined as functions that returned aProcessingPipelineobject. However, this approach presented a challenge for autodiscovery because the functions lacked type hints, making it difficult to determine their return type. To address this issue, a global variable namedpipelineswas introduced in the__init__.pyfile of the plugin's pipeline module. While this workaround allowed the user to manually export the pipeline functions, it was not an ideal solution.Following discussionshere, a decision was made to introduce a class decorator that could be applied to the pipeline functions. This decorator serves two purposes: it allows the functions to be treated as pipeline objects and provides a convenient way for autodiscovery to locate them. Additionally, this approach enables a gradual migration from pipeline functions to pipeline classes without breaking backward compatibility.By adopting this decorator-based approach, the need for global variables and manual exporting is eliminated. It simplifies the autodiscovery process and enhances code organization. It is important to note that while the previous approach of using global variables will be supported for the time being, it will eventually be phased out in a future release of pySigma.Here is the revised example illustrating the use of the class decorator:fromsigma.pipelines.baseimportPipeline@Pipelinedefpipeline_1():returnProcessingPipeline(...# pipeline code goes here)classPipeline_2(Pipeline):defapply(self):returnProcessingPipeline(...# pipeline code goes here)Both pipelines can still be used in the same manner as before. There is no difference between the two approaches because thePipeline_2class can be instantiated and used as a pipeline object, likePipeline_2()()orPipeline_2().apply(). In other words, when the class is instantiated, an object of thePipelineclass is returned. Calling the object itself will automatically run theapplymethod, which returns aProcessingPipelineobject. This behavior aligns with the functionality of thepipeline_1function, which also returns aProcessingPipelineobject. This consistency results in a cleaner and more streamlined approach for autodiscovery and facilitates the gradual migration of pipeline functions to classes.pySigma before v0.9.11The backend pluginautodiscoveryfunctionality has been added, eliminating the need for manual registration of plugins insigma-cli. However, some backends may not function with the updated sigma-cli version. To address this issue, plugin developers should make the following changes to their backends:In thesigma/backends/my_awesome_backend/__init__.pyfile, add abackendsglobal variable that references the backend class:from.my_awesome_backendimportMyAwesomeBackendbackends={"my_awesome_backend":MyAwesomeBackend,}In thesigma/pipelines/my_awesome_pipelines/__init__.pyfile, add apipelinesglobal variable that lists the available pipelines:from.my_awesome_pipelinesimportpipeline_1,pipeline_2pipelines={"pipeline_1":pipeline_1,"pipeline_2":pipeline_2,}(Optional) If your backend hasValidators(used withsigma check): In thesigma/pipelines/my_awesome_validators/__init__.pyfile, add avalidatorsglobal variable that lists the available pipelines:validators={"validator_1":MyFirstValidator,"validator_2":MySecondValidator,}Finally, submit a pull request to thepySigma-plugin-directoryand update the version compatibility of your backend plugin with pySigma.By following these steps, your backend plugin will be compatible with newer versions of pySigma and sigma-cli, allowing for autodiscovery and migration of backend plugins.Create Your Own Backend for pySigmaThe creation of a backend has become much easier with pySigma. We recommend using the "Cookie Cutter Template" and reviewing the existing backends listed in the "Related Projects" section of this README.pySigma Cookie Cutter TemplateFeaturespySigmabrings a number of additional features compared tosigmac, as well as some changes.Modifier comparison between pySigma and sigmacModifierUsesigmac legacycontainsthe value is matched anywhere in the field (strings and regular expressions)XstartswithThe value is expected at the beginning of the field's content (strings and regular expressions)XendswithThe value is expected at the end of the field's content (strings and regular expressions)XexistsThe field exists (yes/true) in the matched event or doesn't exist (no/false)base64The value is encoded with Base64Xbase64offsetIf a value might appear somewhere in a base64-encoded value the representation might change depending on the position in the overall valueXwidetransforms value to UTF16-LE encodingXrevalue is handled as regular expression by backendsXiRegular expression ignore case modifierignorecaseRegular expression ignore case modifiermRegular expression multiline modifiermultilineRegular expression multiline modifiersRegular expression dot matches all modifierdotallRegular expression dot matches all modifiercidrvalue is handled as an IPv4 CIDR by backendsallThis modifier changes OR logic to ANDXltField is less than the valuelteField is less or egal than the valuegtField is Greater than the valuegteField is Greater or egal than the valueexpandModifier for expansion of placeholders in values. It replaces placeholder strings (%something%)Backends comparison between pySigma and sigmacOn 2022/04/10sigmac BackendsObservationpySigmaalaAzure Log Analytics Queriesala-ruleAzure Log Analytics RulearcsightArcSight saved searcharcsight-esmArcSight ESM saved searchathenaSQL querycarbonblackConverts Sigma rule into CarbonBlack query stringchronicleGoogle Chronicle YARA-LcrowdstrikeCrowdStrike Search Processing Language (SPL)pySigma-pipeline-crowdstrikecsharpCSharp Regex in LINQ querydatadog-logsDatadog log search querydevoDevo queryee-outliersee-outlierselastalertElastAlert QS queryelastalert-dslElastAlert DSL queryes-dslElasticsearch DSL querypySigma-backend-elasticsearches-eqlElasticsearch EQL queryes-qsElasticsearch query string. Only searches, no aggregationspySigma-backend-elasticsearches-qs-lrLucene query string for LogRhythm. Only searches, no aggregationses-ruleElastic SIEM lucene queryes-rule-eqlElastic SIEM EQL queryfieldlistList all fieldnames from given Sigma rules for creation of a field mapping configurationfireeye-helixFireEye Helix Query LanguagefortisiemBase class for Fortisem backends that generate one text-based expression from a Sigma rulegraylogGraylog query string. Only searches, no aggregationsgrepGenerates Perl compatible regular expressions and puts 'grep -P' around ithawkHAWK searchhumioHumio querykibanaKibana JSON Configuration files (searches only)kibana-ndjsonKibana JSON Configuration files (searches only)pySigma-backend-elasticsearchlaceworkLacework Policy PlatformlimacharlieLimaCharlie D&R ruleslogiqLOGIQ event rule api payloadlogpointLogPoint querymdatpMicrosoft Defender ATP Hunting QueriespySigma-backend-microsoft365defendernetwitnessNetWitness saved searchnetwitness-eplRSA NetWitness EPLes-qs (proxied)OpenSearch search query string. Only searches, no aggregationspySigma-backend-opensearch(proxied bypySigma-backend-elasticsearch)es-dsl (proxied)OpenSearch DSL querypySigma-backend-opensearch(proxied bypySigma-backend-elasticsearch)opensearch-monitorOpenSearch monitors and ElasticRule are in Elastic Common SchemapySigma-backend-opensearchpowershellPowerShell event log cmdletspySigma-backend-powershellqradarIBM Qradar AQLpySigma-backend-QRadar-AQLqualysQualys saved searchsentinel-ruleAzure Sentinel scheduled alert rule ARM templatesplunkSplunk Search Processing Language (SPL)pySigma-backend-splunksplunkdmSplunk syntax leveraging Datamodel accelerationsplunkxmlXML used for Splunk Dashboard PanelssqlSQL querysqliteSQL query for SQLitestixSTIX patternsumologicSumoLogic querysumologic-cseSumoLogic CSE querysumologic-cse-ruleSumoLogic CSE querysysmonsysmon XML configurationuberagentuberAgent ESA Threat Detection EnginepySigma-backend-uberAgentxpack-watcherX-Pack Watcher JSON for alertingOverviewConversion OverviewPipelinesMore details are described inthe documentation.TestingpySigma uses pytest as testing framework. Simply runpytestto run all tests. Runpytest --cov=sigmato get a coverage report.BuildingTo build your own package runpoetry build.LintingTo lint the code runpoetry run black. To check for linting errors runpoetry run black --check.This project also usespre-commit, which is installed by poetry as part of dev dependencies. To install the git hooks runpoetry run pre-commit installafter cloning the repository and installing the dependencies.ContributingPull requests are welcome. Please feel free to lodge any issues/PRs as discussion points.Thisblog postby Micah Babinski explains the process from a developer's perspective.MaintainersThe project is currently maintained by:Thomas Patzkethomas@patzke.orgFrançois HubautRelated ProjectspySigma isn't a monolithic library attempting to support everything but the core. Support for target
query languages and log data models is provided by additional packages that extend pySigma:sigma-cli: a command line interface for conversion of
Sigma rules based on pySigma.pySigma-backend-splunkpySigma-pipeline-sysmonpySigma-pipeline-crowdstrikeAll packages can also be installed from PyPI if not mentioned otherwise by the Python package
manager of your choice.LicenseGNU Lesser General Public License v2.1. For details, please see the full license filelocated here.