BlackWidow - Web Application Spider - GeeksforGeeks

23 Aug, 2021
Gathering Information about the target domain is very important for making the penetration testing or bug bounty hunting process successful. Collecting Subdomains, Unique URLs, URLs with Parameters, Emails of the target domain, etc information is very crucial, so to get this Information we have a tool named BlackWidow.
BlackWidow tool is a python language-based tool used for web-based application spider and gathers lots of information. Inject-X fuzzer is used in this tool for scanning Dynamic URLs for common OWASP vulnerabilities. BlackWidow tool can be used in the initial steps of web-based application vulnerability assessment for the Information Gathering phase. Inject-X fuzzer has the capability to detect XSS, SWL Injection, HTML code injection, etc. vulnerabilities.
It gathers subdomains, unique dynamic URLs, emails, phone numbers.It uses an Inject-X fuzzer for the detection of vulnerability.It can store the results in dedicated text files in sorted order.It is developed in Python Language.It is open-source and free to use.
It gathers subdomains, unique dynamic URLs, emails, phone numbers.
It uses an Inject-X fuzzer for the detection of vulnerability.
It can store the results in dedicated text files in sorted order.
It is developed in Python Language.
It is open-source and free to use.
Step 1: Check whether Python Environment is Established or not, use the following command:
python3
Step 2: Open up your Kali Linux terminal and move to Desktop using the following command.
cd Desktop
Step 3: You are on Desktop now create a new directory called Black-Widow using the following command. In this directory, we will complete the installation of the Black-Widow tool.
mkdir Black-Widow
Step 4: Now switch to the Black-Widow directory using the following command.
cd Black-Widow
Step 5: Now you have to install the tool. You have to clone the tool from Github.
sudo git clone https://github.com/1N3/BlackWidow.git
Step 6: The tool has been downloaded successfully in the Black-Widow directory. Now list out the contents of the tool by using the below command.
ls
Step 7: You can observe that there is a new directory created of the BlackWidow tool that has been generated while we were installing the tool. Now move to that directory using the below command:
cd BlackWidow 
Step 8: Once again to discover the contents of the tool, use the below command.
ls
Step 9: Download the required packages for running the tool, use the following command.
sudo pip3 install -r requirements.txt
Step 10: Now we are done with our installation, Use the below command to view the help (gives a better understanding of tool) index of the tool.
python3 blackwidow -h
In this example, we will be performing Full URL Spider. We have provided the target Full URL using the -u tag (https://geeksforgeeks.org) 
sudo python3 blackwidow -u https://geeksforgeeks.org
In the below Screenshot, we are getting our scan results, Subdomains of geeksforgeeks.org are displayed in the below Screenshot.
In the below Screenshot, dynamic URLs found on geeksforgeeks.org are mentioned below in the screenshot, Basically, these are parameterized URLs that can be tested for XSS and many more vulnerabilities.
In the below screenshot, the files created by the tool while scanning is shown. Each file has its own results like subdomains, emails, URLs, etc.
In the below Screenshot, You can see that Emails of geeksforgeeks.org are saved in the emails.txt file, this makes the task easier.
In this example, rather than providing a full URL (like Example 1), we are specifying only domain names to scan. -d tag is used along with the target domain name.
sudo python3 blackwidow -d geeksforgeeks.org
Domain name to spider
In the below Screenshot, we have got the results of our Domain name Spider.
In this example, we will scan the domain to a certain depth. 5 levels of depth will be scanned as we have specified a depth of 5 using the -l tag.
sudo python3 blackwidow -d geeksforgeeks.org -l 5
In the below Screenshot, the results of the depth of 5 are displayed.
In this example, We will scan for all the dynamic URLs detected on the geeksforgeeks.org domain. In the below Screenshot, the list of unique_dynaminc URLs list is displayed.
sudo python3 blackwidow -d geeksforgeeks.org -s /usr/share/blackwidow/geeksforgeeks.org_80/geeksforgeeks.org_80-dynamic-unique.txt
In the below Screenshot, we are starting our Scan on dynamic URLs which are provided in the form of list.
In the below screenshot, you can see that the results of the above scan are displayed.
In this example, we will perform a scan on the domain specified with the port number. We have specified Port 80, so only Port 80 URLs will be scanned.
sudo python3 blackwidow -d geeksforgeeks.org -p 80
In the below Screenshot, we have got the results of our above Example Port of the URL.
In this example, we will be displayed in a more verbose format. -v tag is used along with the confirmation (y/n).
sudo python3 blackwidow -d geeksforgeeks.org -v y
In the below Screenshot, the results of verbose format are displayed.
Kali-Linux
Linux-Tools
Linux-Unix
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
nohup Command in Linux with Examples
Thread functions in C/C++
scp command in Linux with Examples
mv command in Linux with examples
chown command in Linux with Examples
SED command in Linux | Set 2
Docker - COPY Instruction
Array Basics in Shell Scripting | Set 1
Basic Operators in Shell Scripting
Named Pipe or FIFO with example C program