Package: contentctl
contentctlIntroductionSecurity Is HardAnyone who has managed aSOCwill tell you it's hard work.  SOC Managers, Detection Engineers, and Cybersecurity Professionals must understand and manage countless tools and data sources while ensuring the reliability and security of their network and applications.  At the same time, they must be able to detect and react to data breeches, vulnerabilities, and performance degradation in minutes - not seconds or hours.
These responsibilities leave little time for writing new content - let alone documentation, maintenance, and testing of legacy content.  Existing solutions like Wikis, JIRA tickets, Excel Spreadsheets, and "asking that one team member who knows everything" don't scale and spread information across various systems.contentctl Makes ItEasyLess Hardcontentctl is a single application that support the full cycle of security content development (each of the links below will redirect to the appropriate section of the README/Wiki):Create Content Packs- Version-Controlled collections of searches, documentation, and test dataAdd New Content- searches, macros, lookups, and other contentStatically ValidateContent Against a Well-Defined BaselineBuildinga Content Pack for your target - Use a supported output format or design your ownTestyour Content Pack on a running Splunk InstanceDeployyour Content Pack to Splunk Cloud Instance, via REST API, or create a Splunkbase Application for manual deploymentEcosystemProjectDescriptionSplunk Attack RangeEasily deploy a preconfigured Splunk Environment locally or on AWS containing a Splunk Instance, Windows and Linux Machines, and Attacker Tools like Kali Linux.  Automatically simulate attacks or run your ownPurpleSharp Attack SimulationOpen source adversary simulation tool for Windows Active Directory environments (integrated into Attack Range)Red Canary Atomic Red TeamLibrary of attack simulations mapped to the MITRE ATT&CK® framework (integrated into Attack Range)Splunk Attack DataRepository of Attack Simulation Data for writing and Testing DetectionsSplunk Security ContentSplunk Threat Research Team's Content included in theEnterprise Security Content Update App (ESCU)Splunk contentctlGenerate, validate, build, test, and deploy custom Security ContentSigmaHQ Sigma RulesOfficial Repository for Sigma Rules. These rules are an excellent starting point for new content.Workflowgraph TD
starting_state --contentctl init--> contentPack 
newContent{new_content} --contentctl new sigma content--> contentPack
newContent{new_content} --contentctl new spl content--> contentPack
contentPack --contentctl validate--> ValidatedContent
ValidatedContent --contentctl build--> SplunkContent
ValidatedContent --contentctl docs--> WebsiteDocumentation
ValidatedContent --contentctl docs--> MITRE_mapDocumentation
SplunkContent --contentctl test via container-->TestResults
TestResults --contentctl deploy-->SplunkRESTAPI
TestResults --contentctl deploy-->SplunkbaseApp
TestResults --contentctl deploy-->SplunkCloudInstanceInstallationRequirementsTesting is run usingGitHub Hosted Runners.  Ubuntu22.04 is the recommended configuration.Operating SystemSupportedPassing Intregration TestsWindows 10YesNot tested yetWindows 11YesNot tested yetUnuntu 20.04 LTSYesYES (locally + GitHub Actions)*Ubuntu 22.04 LTSYesYES (locally + GitHub Actions)macOS Montery 12YesNot tested yetmacOS Ventura 13Yes (Intel and Apple Silicon)Yes (locally)RequirementSupportedDescriptionPassing Integration TestsPython <3.9NoNo support planned.  contentctl tool uses modern language constructs not supported ion Python3.8 and belowN/APython 3.9Yescontentctl tool is written in PythonYes (locally + GitHub Actions)Python 3.10Yescontentctl tool is written in PythonYes (locally + GitHub Actions)Python 3.11Yescontentctl tool is written in PythonYes (locally + GitHub Actions)Docker (local)YesA running Splunk Server is required for Dynamic Testing.  contentctl can automatically create, configure, and destroy this server as a Splunk container during the lifetime of a test.(locally + GitHub Actions)Docker (remote)PlannedA running Splunk Server is required for Dynamic Testing.  contentctl can automatically create, configure, and destroy this server as a Splunk container during the lifetime of a test.NoBecause contentctl uses a large number of dependencies, it's recommended to install contentctl in a virtual environment.
The instructions below using "poetry" ALSO follow this recommendation.  Poetry will automatically create a new virtual environment for you.It is typically recommended to install poetry to the Global Python Environment.Install via pip (recommended):python3.9 -m venv .venv
source .venv/bin/activate
pip install contentctlFrom Source (advanced)git clone https://github.com/splunk/contentctl
cd contentctl
python3.9 -m pip install poetry
poetry install
poetry shell
contentctl --helpUsagecontentctl initCreates a new Content Pack in the current directory as well as a configuration file calledcontentctl.ymlwhich contains a number of important configuration options.
The content pack contains a wide variety of content types:detections- A piece of content that wraps and enriches a Splunk Search.Example Detectionbaselines- This content is not currently supported.lookups- Static files, such as CSVs, that can be loaded into Splunk for use in lookup commands.Example Lookupmacros- Common code that is re-used across a wide set of detections.Example Macrostories- Analytic Stories Allow a developer to tie together a group of detections, for example detections that all relate to Ransomware.Example Analytic Storycontentctl validatePerforms static validation on all of the content in this Content Pack.  Writing validation is extremely complex.  Each piece of content can one or dozens of fields ranging from free text to numbers to references to other content.  contentctl's build in validation ensures that a number of conditions are met:Required fields in content are definedValues in fields are appropriate.  For example, if content references a Datamodel, then that Datamodel must be valid. Similarly, numeric values that must fall within a certain range must be validated.If a piece of content references other content, then it exists.  For example, if a detection makes use of a macro then the existence and correctness of the macro must be validated.There are no conflicts between content.  For example, two detections cannot have the same name.
If any of these conditions are not met, then a descriptive error will be printed and contentctl will return a nonzero exit code.  This makes it suitable for running in an automated workflow or other CICD context.  Instead of exiting on the first error,contentctl validatewill continue validating all content so that it can present all relevant error before exiting.contentctl buildGenerates Content Packs in the output format defined in thecontentctl.ymlconfiguration file.  These outputs may include {splunk_app, api, ba}.  Whencontentctl buildruns, it first performs acontentctl validatein order to ensure that a valid app is generated.  Note that it isNOTrequired or recommended to runcontentctl validateseparately if the intention is to build a Content Pack.contentctl reportThis section is under active development.  It will allow you to aMITRE Mapshowing your coverage of MITRE Techniques.  The link here shows the generation of that MITRE Map using the content in Splunk'ssecurity_content repo.contentctl new [--type TYPE]Choose TYPE {detection, story} to create new content for the Content Pack.  The tool will interactively ask a series of questions required for generating a basic piece of content and automatically add it to the Content Pack.contentctl inspectThis section is under development.  It will enable the user to perform an appinspect of the content pack in preparation for deployment onto a Splunk Instance or via Splunk Cloud.contentctl deployThe reason to build content is so that it can be deployed to your environment.  However, deploying content to multiple servers and different types of infrastructure can be tricky and time-consuming.  contentctl makes this easy by supporting a number of different deployment mechanisms. Deployment targets can be defined incontentctl.yml.Deploy via API - Using the REST API, individual pieces of content are deployed to a running server.  This is a great way to deploy all of the content in a content pack, but can also be used to deploy individual peices of content.Deploy to Splunk Cloud via ACS - Using the Automated Private App Vetting (APAV) Feature of the Admin Config Service (ACS), Splunk Cloud customers can easily deploy custom apps to their environments.contentctl docsThis section is under active development.  It will allow you to easily generate verboseweb-based documentationfrom all of your content.  The link here shows the generation of that documentation using the content in Splunk'ssecurity_content repo.contentctl testThe static validation performed bycontentctl validatecan only take you so far.  While it's powerful, and fast, it can only tell determine if the content issyntacticallycorrect.contentctl testcan test your content on real Splunk Infrastructure to ensure there are no errors in your SPL, raw data can be properly ingested/processed/accelerated on your server, your search finds the event(s) you're looking for in raw data, and even provides high-level runtime performance metrics about your searches.    As part of this workflow, a large number of apps will be installed by default.  You can see, and edit, the list of apps in theapp_default.ymlfile.  We plan on making this even easier to update in the future, so stay tuned for updates.
The following diagram shows this workflow at a high level.graph TD
SplunkContainer--start-->SplunkInstance
SplunkServer-->SplunkInstance
SplunkInstance--Install Apps-->ConfiguredInstance
SplunkInstance--Configure Settings-->ConfiguredInstance
ContentPack--Deploy-->ConfiguredInstance
ConfiguredInstance--Start Test-->Testing
Testing--Test Detection-->Detection

Detection--Replay Raw Data-->Result
subgraph Test Indidivual Detection
	Detection--Execute Search-->Result	
end
Result--Cleanup-->Testing
Testing--Cleanup-->TestResultsTesting ModesThere are a number of different testing modes that control which content will be tested.  This can be controlled with the--mode {}option at the command lineall - This will test all of the content in the content pack. Please note that by default only detections marked production will be tested (detections marked as deprecated or experimental, for example, will be ignored).selected - Detections whose relative paths are provided on the command line will be tested.  This is particularly useful if you would like to troubleshoot or update just a handful of detections and can save a significant amount of time.  For example, to test two detections use the following commandcontentctl test --mode selected --detections_list detections/detection_one.yml detections/detection_two.ymlchanges - (not currently supported) If you have a large number of detections and use a branching workflow to create new content or update content, then this is an easy way to automatically find and test only that content automatically.  This prevents you from needing to explicitly list the subset of content to test using "selected"Testing Behaviorcontentctl test's default mode allows it to quickly test all content with requiring user interaction.  This makes it suitable for local tests as well as CI/CD workflows.  However, users often want to troubleshoot a test if it fails.  contentctl allows you to change the tool's behavior if and/or when a test fails:--behavior never_pause - The default behavior.  If a test does not pass, the tool begins the next test immediately--behavior pause_on_failure - If a test fails, then additional information about that test, and the raw SPL of the test, is printed to the terminal.  A user may then click (or CMD+Click) the "LINK" to begin interactively debugging the failed test on the Splunk Server.  Note that the credentials for the server are printed out at the very beginning of the test.  After you have finished debugging the failure, hit "Return" in the terminal to move on to the next test. The attack_data for this test remains loaded on the server for debugging until the user moves on to the next test.--behavior always_pause - Similar to pause_on_failure, but this pauses after every test regardless of whether it passes or fails.init- Initilialize a new repo from scratch so you can easily add your own content to a custom application.new- Creates new content (detection, story)validate- Validates written contentbuild- Builds an application suitable for deployment on a search head using Slim, the Splunk Packaging Toolkitdeploy- Deploy the security content pack to a Splunk Serverdocs- Create documentation as Markdownreporting- Create different reporting files such as a Mitre ATT&CK overlayAcronymsAcronymMeaningDescriptionSOCSecurity Operation CenterDescription of a SoCDaCDetection as CodeA systematic approach applying DevOps priciples to Detection Engineering. DaC enables Continuous Integration and Continuous Delivery of Detectionsa via automated validation, testing, and deploymentCICDContinuous Integration/Continuous DeliveryA modern DevOps practice that encourages users to make small, frequent changes which are automatically tested and deployed. This contrasts with legacy approaches that emphasize large changes which may be manually tested an infrequently deployed.ACSAdmin Config ServiceTheSplunk Admin Config Serviceis a cloud-native API that provides programmatic self-service administration capabilities for Splunk Cloud Platform. One of its features, Automated Private App Vetting (APAV) enables the installation of custom app on Splunk Cloud instances.APAVAutomated Private App VettingAutomated Private App Vettingenables admins to use the ACS API to install, upgrade, and uninstall apps directly on your Splunk Cloud Platform deployment without assistance from Splunk Support.LicenseCopyright 2023 Splunk Inc.Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.