How to Write Your Own Subdomain Enumeration Script for Better Recon « Null Byte :: WonderHowTo

There are tons of tools out there that do all kinds of recon, but it can be hard to narrow down what to use. A great way to be more efficient is by taking advantage of scripting. This doesn't have to mean writing everything from scratch — it can simply mean integrating existing tools into a single, comprehensive script. Luckily, it's easy to create your own subdomain enumeration script for better recon.Step 1: Install DependenciesBefore we begin, there are a few things we need to install and set up for everything to work properly. First, make sureGo and Subfinderare installed on the system. Second, we'll be using a tool calledassetfinderfor additional subdomain recon; we can get the latest release from GitHub with:~# wget https://github.com/tomnomnom/assetfinder/releases/download/v0.1.0/assetfinder-linux-amd64-0.1.0.tgz

--2021-04-28 15:00:12--  https://github.com/tomnomnom/assetfinder/releases/download/v0.1.0/assetfinder-linux-amd64-0.1.0.tgz
Resolving github.com (github.com)... 140.82.114.4
Connecting to github.com (github.com)|140.82.114.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/193392376/6e64a200-d33f-11e9-9d79-2165e6e68bb1?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210428%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210428T200012Z&X-Amz-Expires=300&X-Amz-Signature=3704ee96ec028f1ac8de3a3af870351ff434bdbd1150e3893a2cd02d43113b71&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=193392376&response-content-disposition=attachment%3B%20filename%3Dassetfinder-linux-amd64-0.1.0.tgz&response-content-type=application%2Foctet-stream [following]
--2021-04-28 15:00:12--  https://github-production-release-asset-2e65be.s3.amazonaws.com/193392376/6e64a200-d33f-11e9-9d79-2165e6e68bb1?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210428%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210428T200012Z&X-Amz-Expires=300&X-Amz-Signature=3704ee96ec028f1ac8de3a3af870351ff434bdbd1150e3893a2cd02d43113b71&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=193392376&response-content-disposition=attachment%3B%20filename%3Dassetfinder-linux-amd64-0.1.0.tgz&response-content-type=application%2Foctet-stream
Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.217.46.132
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.217.46.132|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3739744 (3.6M) [application/octet-stream]
Saving to: ‘assetfinder-linux-amd64-0.1.0.tgz’

assetfinder-linux-amd64-0.1.0.tgz         100%[=====================================================================================>]   3.57M  1.78MB/s    in 2.0s

2021-04-28 15:00:14 (1.78 MB/s) - ‘assetfinder-linux-amd64-0.1.0.tgz’ saved [3739744/3739744]And usetarto extract the binary:~# tar xzf assetfinder-linux-amd64-0.1.0.tgzThen, move assetfinder to a directory in our path:~# mv assetfinder /usr/local/bin/Third, we need a tool calledhttprobe, which will allow us to filter live hosts in our results. Grab the release from GitHub with:~# wget https://github.com/tomnomnom/httprobe/releases/download/v0.1.2/httprobe-linux-amd64-0.1.2.tgz

--2021-04-28 15:05:40--  https://github.com/tomnomnom/httprobe/releases/download/v0.1.2/httprobe-linux-amd64-0.1.2.tgz
Resolving github.com (github.com)... 140.82.114.4
Connecting to github.com (github.com)|140.82.114.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/80510806/d4c97700-afc2-11e9-9a18-8f50cc10ac23?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210428%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210428T200541Z&X-Amz-Expires=300&X-Amz-Signature=35781254f155f3fd67a026f17035c7fa9f0124feed26e08a305266c73eff08f0&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=80510806&response-content-disposition=attachment%3B%20filename%3Dhttprobe-linux-amd64-0.1.2.tgz&response-content-type=application%2Foctet-stream [following]
--2021-04-28 15:05:41--  https://github-production-release-asset-2e65be.s3.amazonaws.com/80510806/d4c97700-afc2-11e9-9a18-8f50cc10ac23?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210428%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210428T200541Z&X-Amz-Expires=300&X-Amz-Signature=35781254f155f3fd67a026f17035c7fa9f0124feed26e08a305266c73eff08f0&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=80510806&response-content-disposition=attachment%3B%20filename%3Dhttprobe-linux-amd64-0.1.2.tgz&response-content-type=application%2Foctet-stream
Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.217.44.212
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.217.44.212|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3555994 (3.4M) [application/octet-stream]
Saving to: ‘httprobe-linux-amd64-0.1.2.tgz’

httprobe-linux-amd64-0.1.2.tgz            100%[=====================================================================================>]   3.39M  1.61MB/s    in 2.1s

2021-04-28 15:05:43 (1.61 MB/s) - ‘httprobe-linux-amd64-0.1.2.tgz’ saved [3555994/3555994]Unpack the binary:~# tar xzf httprobe-linux-amd64-0.1.2.tgzAnd move it to a directory in our path:~# mv httprobe /usr/local/bin/Fourth, we need to configure a couple of things for Go. First,make a directorycalledgo:~# mkdir /usr/local/goUse the following command to set the GOPATH environmental variable:~# go env -w GOPATH=/usr/local/goWe can confirm that we set it correctly:~# go env GOPATH

/usr/local/goThen, we need to add GOPATH to our path. Use the following command, creating the/bindirectory if it doesn't already exist:~# export PATH=$PATH:$(go env GOPATH)/binNext, we can make our changes persistent by adding the configuration to our.bashrcfile:~# echo 'export GOPATH=/usr/local/go' >> ~/.bashrcUse the following command to source the file, making it persistent:~# . ~/.bashrcLast, we need a tool calledSubjack; we'll get into what this tool does later on, but for now, we can install it with thego getcommand:~# go get github.com/haccer/subjackThat will automatically install it in our GOPATH and make it ready to use. And that should be everything we need, so now let's start our script.Step 2: Start the ScriptTo begin, create a script and open it with your favoritetext editor:~# nano subrecon.shThe first line we need, called a shebang or hashbang, will point to the system's interpreter. This tells the system how to run the file; in this case, it is aBash script:#!/bin/bashNext, we will make sure the user supplies input to the script, and if not, prints a usage example and exits. Use aconditional if-thenblock:if [ -z $1 ]
then
        echo './subrecon.sh <list of domains>'
        exit 1
fiThe$1is the argument passed to the script, and the-zoption returns true if the string is null. So basically, this says if no argument is supplied, show the usage and exit. The argument we'll pass in is a list of domains.Don't Miss:How to Uncover Hidden Subdomains to Reveal Internal Services with CT-ExposerStep 3: Enumerate SubdomainsThe first action our script will take is enumerating subdomains:echo 'FINDING SUBDOMAINS...'

while read $line
do
        for var in $line
        do
                echo 'enumerating:' $var

                subfinder -silent -d $var > out1
                cat out1 >> subs1

                assetfinder -subs-only $var > out2
                cat out2 >> subs2

                rm out1 out2
        done
done < $1This will use a while loop to read input from our file containing a list of domains, use a variable to display the current domain being enumerated, and gather results from both Subfinder and assetfinder.The next section will combine the results, remove any duplicates, and save the output to a file calledall_subs:sort -u subs1 subs2 > all_subs
rm subs1 subs2
echo 'saved subdomains to all_subs'Step 4: Determine Live HostsThe next part of the script will determine which hosts from the previous results are live. This is extremely useful for cutting down the time it takes to go through everything since hosts that are down are usually of no interest.This will take the list of subdomains and use httprobe to filter out live hosts, saving the results to a file calledlive_subs:echo 'FINDING LIVE HOSTS...'

cat all_subs | httprobe > live_subs
echo 'saved live hosts to live_subs'Step 5: Test for Subdomain TakeoverSubdomain takeoveris the process of registering a domain name to gain control over another domain. This happens when a host, usually a subdomain, points to a service that is no longer in use. The most common scenario is when a subdomain points to another domain, theDNS recordexpires, and the domain is available to be registered by someone else. Anyone who can successfully register the domain now has full control over the subdomain.Don't Miss:How to Quickly Look Up the Valid Subdomains for Any WebsiteIn some cases, this type of attack is not possible due to verification methods, but you'd be surprised by how many services are vulnerable to subdomain takeover.Amazon S3 buckets, GitHub pages, Heroku, Shopify, andMicrosoft Azureare all prone to this attack in some shape or form.Subjack is a handy tool that will test a list of subdomains for potential takeover. Here, we will use the-wflag for an input file and the-aflag to send requests to every URL:echo 'CHECKING FOR SUBDOMAIN TAKEOVER...'

subjack -w all_subs -a

echo 'DONE'If anything in our list is vulnerable to subdomain takeover, the results will show on-screen along with the associated service.Step 6: Review the ScriptThe final script should look something like this:#!/bin/bash

if [ -z $1 ]
then
        echo './subrecon.sh <list of domains>'
        exit 1
fi

echo 'FINDING SUBDOMAINS...'

while read line
do
        for var in $line
        do
                echo 'enumerating:' $var

                subfinder -silent -d $var > out1
                cat out1 >> subs1

                assetfinder -subs-only $var > out2
                cat out2 >> subs2

                rm out1 out2
        done
done < $1

sort -u subs1 subs2 > all_subs
rm subs1 subs2
echo 'saved subdomains to all_subs'

echo 'FINDING LIVE HOSTS...'

cat all_subs | httprobe > live_subs
echo 'saved live hosts to live_subs'

echo 'CHECKING FOR SUBDOMAIN TAKEOVER...'

subjack -w all_subs -a

echo 'DONE'Now it's time to test it out. Save the script, then make it executable:~# chmod +x subrecon.shAnd run it, supplying a list of domains to enumerate:~# ./subrecon.sh domains.txt

FINDING SUBDOMAINS...
enumerating: wonderhowto.com
saved subdomains to all_subs
FINDING LIVE HOSTS...
saved live hosts to live_subs
CHECKING FOR SUBDOMAIN TAKEOVER...
DONEThis is a good start, but the beauty of this script is that it can easily be expanded. Anything useful forrecon, especially subdomain recon, can be added to make the process of enumeration unique.Wrapping UpIn this tutorial, we learned how to write our own subdomain enumeration script in Bash. First, we installed some dependencies and got started on our script. Next, we used Subfinder and assetfinder to discover subdomains and combine the results, and filtered out live hosts with httprobe. Finally, we utilized Subjack to check for potential subdomain takeover.Don't Miss:How to Discover Hidden Subdomains on Any Website with SubfinderWant to start making money as a white hat hacker?Jump-start your hacking career with our2020 Premium Ethical Hacking Certification Training Bundlefrom the newNull Byte Shopand get over 60 hours of training from cybersecurity professionals.Buy Now (90% off) >Other worthwhile deals to check out:97% off The Ultimate 2021 White Hat Hacker Certification Bundle99% off The 2021 All-in-One Data Scientist Mega Bundle98% off The 2021 Premium Learn To Code Certification Bundle62% off MindMaster Mind Mapping Software: Perpetual LicenseCover image byChristina Morillo/PexelsRelatedHow To:Quickly Look Up the Valid Subdomains for Any WebsiteAdvanced Nmap:Top 5 Intrusive Nmap Scripts Hackers & Pentesters Should KnowDon't Be a Script-Kiddie part2:Building an Auto-Exploiter Bash ScriptHack Like a Pro:Reconnaissance with Recon-Ng, Part 1 (Getting Started)Hack Like a Pro:How to Use Maltego to Do Network ReconnaissanceHack Like a Pro:Using the Nmap Scripting Engine (NSE) for ReconnaissanceHow To:Conduct Recon on a Web Target with Python ToolsHow To:Uncover Hidden Subdomains to Reveal Internal Services with CT-ExposerHow To:Get Started Writing Your Own NSE Scripts for NmapHow To:Use WebTech to Discover What Technologies a Website UsesHow To:Bypass Facebook's HSTSHow To:Force Safari to Automatically Show Reader View for Specific Websites on Your iPhoneHow To:Enumerate NetBIOS Shares with NBTScan & Nmap Scripting EngineHow To:Discover Hidden Subdomains on Any Website with SubfinderHow To:Find & Exploit SUID Binaries with SUID3NUMHack Like a Pro:Metasploit for the Aspiring Hacker, Part 14 (Creating Resource Script Files)Don't Be a Script-Kiddie part1:Introduction to Shell ScriptHack Like a Pro:Abusing DNS for ReconnaissanceHow To:Write better action beats in a screenplayHow To:Attack a Vulnerable Practice Computer: A Guide from Scan to ShellHow To:Use Photon Scanner to Scrape Web OSINT DataHow To:Script Writing Basics How to Structure a Film Script lWeekend Homework:How to Become a Null Byte Contributor (3/9/2012)How To:Enable Code Syntax Highlighting for Python in the Nano Text EditorNews:The Money Making MacroForbes Exploited:XSS Vulnerabilities Allow Phishers to Hijack Sessions & Steal LoginsNews:INTRO TO SCREENWRITINGNews:AsteroidsHow To:Bash (Shell) Scripting for BeginnersNews:Wario Ware DIYHow To:Start a PoemHow To:Screenplay Format Guide (PDF)