Package: ostorlab
Ostorlab Scan Orchestration Engine aka OXOOstorlab is a security scanning framework built for modularity, scalability and simplicity.Ostorlab Engine combines specialized tools to work cohesively to find vulnerabilities and perform actions like recon, enumeration, fingerprinting ...RequirementsDocker is required to run scans locally. To install docker, please follow theseinstructions.InstallingOstorlab ships as a Python package on pypi. To install it, simply run the following command if you havepipalready
installed.pipinstall-UostorlabGetting StartedOstorlab ships with a store that boasts dozens of agents, from network scanning agents like nmap, openvas, nuclei or
tsunami,
web scanner like Zap, web fingerprinting tools like Whatweb and Wappalyzer, DNS brute forcing like Subfinder and Dnsx,
malware file scanning like Virustotal and much more.To run any of these tools combined, simply run the following command:Ostorlab CLI is accessible also using theoxoalias for convenience.ostorlabscanrun--install--agentagent/ostorlab/nmap--agentagent/ostorlab/openvas--agentagent/ostorlab/tsunami--agentagent/ostorlab/nucleiip8.8.8.8oroxoscanrun--install--agentagent/ostorlab/nmap--agentagent/ostorlab/openvas--agentagent/ostorlab/tsunami--agentagent/ostorlab/nucleiip8.8.8.8This command will download and install the following scanning agents:agent/ostorlab/nmapagent/ostorlab/tsunamiagent/ostorlab/nucleiagent/ostorlab/openvasAnd will scan the target IP address8.8.8.8.Agents are shipped as standard docker images.To check the scan status, run:ostorlabscanlistOnce the scan has completed, to access the scan results, run:ostorlabvulnzlist--scan-id<scan-id>
ostorlabvulnzdescribe--vuln-id<vuln-id>ExamplesAgents accept argument to tweak their behavior, like setting the default ports for nmap or passing the API key to
Virustotal. Agents composition can also be saved to a YAML file for latter use. See below some examplesRun a network scan:# file: agent_group.yamlkind:AgentGroupdescription:Grouping of 6 agents.agents:-key:agent/ostorlab/nmapargs:-name:portstype:stringdescription:List of ports to scan.value:'22,443,80'-key:agent/ostorlab/tsunamiargs:[]-key:agent/ostorlab/openvasargs:[]-key:agent/ostorlab/nucleiargs:-name:template_urlstype:arraydescription:List of template urls to run. These will be fetched by the agentand passed to Nuclei.value:""-name:use_default_templatestype:booleandescription:use nuclei's default templates to scan.value:trueostorlabscanrun--install-gagent_group.yamlip8.8.8.88.8.4.4Run a web scan:# file: agent_group.yamlkind:AgentGroupdescription:Grouping of 4 agents.agents:-key:agent/ostorlab/zapargs:[]-key:agent/ostorlab/whatwebargs:[]ostorlabscanrun--install-gagent_group.yamldomain-nameexample.comList all subdomains, resolve their IP addresses and run a network scan:# file: agent_group.yamlkind:AgentGroupdescription:Grouping of 5 agents.agents:-key:agent/ostorlab/nmapargs:[]port_mapping:[]-key:agent/ostorlab/subfinderargs:[]-key:agent/ostorlab/dnsxargs:[]ostorlabscanrun--install-gagent_group.yamldomain-nameexample.comBrute force all TLDs of a subdomain and resolve their IP addresses:# file: agent_group.yamlkind:AgentGroupdescription:Grouping of 5 agents.agents:-key:agent/ostorlab/nmapargs:[]-key:agent/ostorlab/all_tldsargs:[]-key:agent/ostorlab/subfinderargs:[]-key:agent/ostorlab/dnsxargs:[]ostorlabscanrun--install-gagent_group.yamldomain-nameexample.comScanning a list of assets is supported using a Yaml asset definition file:Inject domain name assetsexample.comandexample_2.comostorlabscanrun--install-gagent_group.yaml-aassets_group.yaml# file: asset_group.yamldescription:Target group definitionkind:targetGroupname:master_scanassets:domain:-name:"example.com"-name:"example_2.com"Inject IOS store assets with the bundle_idscom.example.appandcom.example_2.appostorlabscanrun--install-gagent_group.yaml-aassets_group.yaml# file: asset_group.yamldescription:Target group definitionkind:targetGroupname:master_scanassets:iosStore:-bundle_id:"com.example.app"-bundle_id:"com.example_2.app"Here is a more encompassing example:ostorlabscanrun--install-gagent_group.yaml-aassets_group.yaml# file: asset_group.yamldescription:Target group definition for the NSAkind:targetGroupname:master_scanassets:androidStore:-package_name:"com.example.app"-package_name:"com.example_2.app"androidApkFile:-path:/home/organisation/assets/application.apk-path:/home/organisation/assets/application.apk-url:https://exanple.storage.com/production.apk-url:https://exanple.storage.com/dev.apkandroidAabFile:-path:/home/organisation/assets/application.aab-path:/home/organisation/assets/application.aab-url:https://exanple.storage.com/production.aab-url:https://exanple.storage.com/dev.aabiosStore:-bundle_id:"com.example.app"-bundle_id:"com.example_2.app"iosFile:-path:/home/organisation/assets/application.ipa-path:/home/organisation/assets/application.ipa-url:https://exanple.storage.com/production.ipa-url:https://exanple.storage.com/dev.ipalink:-url:"https://www.nasa.gov/humans-in-space/artemis/"method:"GET"-url:"https://www.nasa.gov/specials/artemis/"method:"POST"domain:-name:"example.co"-name:"example_2.dev"ip:-host:"10.21.11.11"mask:30-host:0.1.2.1The PitchTesting for even the most simple vulnerabilities often requires chaining multiple tools. Take for instance scanning
for a Log4J bug, this requires:CrawlingPath brute forcingRequest Injection Point fuzzingCallback interceptionTools will often re-invent the wheel by poorly re-implementing all of these and then add their detection. These often
results in poor detectors as most of these are complex tasks that requires specialized tools.Ostorlab offers the ease of chaining the specialized tools to focus on perform the required task, offering increased
detection, faster delivery.To do that, Ostorlab provides on the following:A simple, yet powerful SDK to make simple cases effortless while supporting the complex one, like distributed locking,
QPS limiting, multiple instance parallelization ...A battle-tested framework that has been powering Ostorlab Platform for years and used to perform complex dynamic
analysis setup and demanding static analysis workloads running on multiple machines.Performant and scalable design, thanks to the use of message queue with dynamic routing, binary and compact message
serialisation with protobuf, universal file format using docker image and resilient deployment thanks to docker swarm.A store of agents that makes it easy to use and discover tools to add your toolset.An automated builder to take the hassle away of building and publishing.A GUI to prepare and write down your tool collection setup.Focus on documentation, multiple tutorials and upcoming videos and conference presentations.A ready to use one-click template repo to get started.AssetsOstorlab supports scanning of multiple asset types, below is the list of currently supported:AssetDescriptionagentRun scan for agent. This is used for agents scanning themselves (meta-scanning :).ipRun scan for IP address or an IP range .linkRun scan for web link accepting a URL, method, headers and request body.fileRun scan for a generic file.android-aabRun scan for an Android .AAB package file.android-apkRun scan for an Android .APK package file.ios-ipaRun scan for iOS .IPA file.domain-nameRun scan for Domain Name asset with specifying protocol or port.The StoreOstorlab lists all agents on a public store where you can search and also publish your own agents.In addition, the store, a graphical agent group builder is also available to compose multiple agents and see how
they would interact with each other.The builder also helps with generating the agent group YAML file to set special arguments that can be passed to agents
to control their behavior.Publish your first AgentTo write your first agent, you can check out a full
tutorialhere.The steps are basically the following:Clone a template agent with all files already setup.Change thetemplate_agent.pyfile to add your logic.Change theDockerfileadding any extra building steps.Change theostorlab.yamladding selectors, documentation, image, license.Publish on the store.Profit!Once you have written your agent, you can publish it on the store for others to use and discover it. The store
will handle agent building and will automatically pick up new releases from your git repo.Ideas for Agents to buildImplementation of popular tools like:semgrepfor source code scanning.nbtscan: Scans for open NETBIOS nameservers on your target’s network.onesixtyone: Fast scanner to find publicly exposed SNMP services.Retire.js: Scanner detecting the use of JavaScript libraries with known
vulnerabilities.snallygaster: Finds file leaks and other security problems on HTTP servers.testssl.sh: Identify various TLS/SSL weaknesses, including Heartbleed, CRIME and ROBOT.TruffleHog: Searches through git repositories for high entropystrings and secrets, digging deep into commit history.cve-bin-tool: Scan binaries for vulnerable components.XSStrike: XSS web vulnerability scanner with generative payload.Subjack: Subdomain takeover scanning tool.DnsReaper: Subdomain takeover scanning tool.RoadmapAdd support for ARM64 agentsAdd Kubernetes-based runtimeSupport for Mypyc to improve performanceReduce start-time (switch from slow starting services to faster alternatives like NATS or Redis Queues)Support for running agent as standalone tools