How to Use WebTech to Discover What Technologies a Website Uses « Null Byte :: WonderHowTo

Correctly identifying the underlying technologies that run on a website gives pentesters a considerable advantage when preparing an attack. Whether you're testing out the defenses of a large corporation or playing the latest CTF, figuring out what technologies a site uses is a crucial pen-tester skill.Knowing the technology and codebase used to build a site can speed things up by eliminating potential attack vectors or exploits that we know won't work. But it can also reduce the chances of your penetration being detected by raising fewer alarms.Today we will be exploring a tool called WebTech to bring these technologies to light.Why WebTech?WebTechis an open-sourcePythontool used to identify web technologies. You can utilize it in a variety of ways, including as acommand linetool, as aBurp Suiteextension, and as a Python library forscripting.There are other tools available to accomplish this goal, such asWappalyzer, a commonly used browser extension, or other online alternatives likeW3Techsor this one fromPentest-Tools. But WebTech stands out in a way that is extremely modular and easy to use, especially when it comes toPythonscripting.Thereconnaissance phaseis essential in the ethical hacking or penetration testing process. The old saying "measure twice, cut once" holds here.Don't Miss:Hack Like A Pro — Python Scripting for the Aspiring Hacker, Part 1Install & Basic UsageIn order to use WebTech, we must first install it. While you can use WebTech on any operating system that supports Python, I'll be showing you here how it works inKali Linux(or any other Debian-based distro).First, make sure you have Python installed on your device — we can check with thewhichcommand:~# which python

/usr/bin/pythonIf you don't see any output from this, install it with thepackage manager:~# apt-get install pythonThen, installpip, a package management system for Python, with the following command:~# apt-get install python-pipNow we can finally install WebTech:~# pip install webtech

Collecting webtech
  Downloading https://files.pythonhosted.org/packages/a7/66/3bd231369ca661e76fa863546c2d7d8c73fd214fc018dcee37ff32a368d8/webtech-1.2.7.tar.gz (103kB)
    100% |████████████████████████████████| 112kB 1.5MB/s
Requirement already satisfied: requests in /usr/lib/python2.7/dist-packages (from webtech) (2.21.0)
Building wheels for collected packages: webtech
  Running setup.py bdist_wheel for webtech ... done
  Stored in directory: /root/.cache/pip/wheels/36/0d/d6/67a0bbbfd449ecb578cac82c098668ef032dbd513640257c94
Successfully built webtech
Installing collected packages: webtech
Successfully installed webtech-1.2.7Simply typewebtechin the terminal to display its usage and options:~# webtech

No URL(s) given!
Usage: webtech [options]

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -u URLS, --urls=URLS  url(s) to scan
  --urls-file=URLS_FILE, --ul=URLS_FILE
                        url(s) list file to scan
  --user-agent=USER_AGENT, --ua=USER_AGENT
                        use this user agent
  --random-user-agent, --rua
                        use a random user agent
  --database-file=DATABASE_FILE, --db=DATABASE_FILE
                        custom database file
  --json, --oj          output json-encoded report
  --grep, --og          output grepable report
  --update-db, --udb    force update of remote db files
  --timeout=TIMEOUT     maximum timeout for scrape requestsThe most basic usage of the tool is with the-uflag to specify aURLto scan:~# webtech -u https://null-byte.wonderhowto.com/

Target URL: https://null-byte.wonderhowto.com/
Detected technologies:
    - jQuery 1.7
    - Google Font API
    - comScore
Detected the following interesting custom headers:
    - Server: WonderHowTo
    - X-UA-Compatible: IE=Edge,chrome=1
    - X-Server-Name: APP02Here we can see the technologies it detected that are used by the site, as well as a few interesting headers. This information can be useful when preparing an attack, since cutting down unneeded extra variables can drastically decrease the time needed to be successful.This tool also lets us specify a customuser agent, which can sometimes be utilized to probe the site for different responses depending on how it's set up. User agents are a means of identifying the browser and OS to the web server, sent as a string of text in HTTP headers.Here is a databaseof virtually any user agent you can think of to help with your probing.Find the user agent you want to test and add it into this string after your URL--ua='[USERAGENTCODE]'instead of USERAGENTCODE. Here's what that looks like:~# webtech -u https://null-byte.wonderhowto.com/ --ua='Mozilla/5.0 (Linux; Android 6.0.1; SM-G920V Build/MMB29K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.98 Mobile Safari/537.36'

Target URL: https://null-byte.wonderhowto.com/
Detected technologies:
    - jQuery 1.7
    - Google Font API
    - comScore
Detected the following interesting custom headers:
    - Server: WonderHowTo
    - X-UA-Compatible: IE=Edge,chrome=1
    - X-Server-Name: APP02Here we set the user agent to mimic aSamsung Galaxy S6, although we did not get any different results. Sometimes, for example, if there is a bug in a specific browser, the server will send a different response depending on the user agent.We could also set a random user agent if we wanted to, using the--random-user-agentflag, which will randomly select a user agent to use.WebTech's command line usage is definitely helpful when identifying the underlying technologies of a website, but where it really shines is its ability to be used in scripts.Scripting with WebTechYou can use WebTech in any Python script by merely importing the library. This flexibility is the most useful feature of the tool, in my opinion, since you can integrate it into any other script where determining what a website is built on is important.For example, WebTech could be used as part of a largerfingerprintingscript alongsideport scans and service enumeration. It would also be useful inexploit scripts, where the details of the exploit change slightly depending on the platform.Let's write a quick demo script to show it in action. Create your Python file with yourtext editorof choice — in this case, I'll usenanosince it is easy:~# nano scan.pyThe first line should tell the script how to run by pointing to our Python binary:#!/usr/bin/pythonNext, we need an import statement to import the WebTech library:import webtechThen we can start a new instance with thewtvariable:wt = webtech.WebTech()And start a scan of the desired URL (I had to also set a short timeout or it wouldn't work) and save that to theresultsvariable:results = wt.start_from_url('https://null-byte.wonderhowto.com/', timeout=1)Finally, we can print the results of the scan to the screen:print resultsThe final script should look like this:#!/usr/bin/python

import webtech

wt = webtech.WebTech()

results = wt.start_from_url('https://null-byte.wonderhowto.com/', timeout=1)
print resultsWe can now run our script with thepythoncommand we set up above:~# python scan.py

Target URL: https://null-byte.wonderhowto.com/
Detected technologies:
    - jQuery 1.7
    - Google Font API
    - comScore
Detected the following interesting custom headers:
    - Server: WonderHowTo
    - X-UA-Compatible: IE=Edge,chrome=1
    - X-Server-Name: APP03Here we can see that we've obtained the same results as before. Keep in mind, this was just a simple proof-of-concept — we could make this a lot more robust if we wanted.What this does is prove how potentially useful WebTech could be when integrating it into other scripts or tools.Wrapping UpIn this article, we learned about WebTech — a Python tool used to identify website technologies.We saw how to install it with pip and run it from the command line with ease. We also explored its true power, the ability to be used as a Python library, by writing our little script. WebTech makes it easy to get to know your target during the recon phase, giving you the advantage when planning an attack.Want to start making money as a white hat hacker?Jump-start your hacking career with our2020 Premium Ethical Hacking Certification Training Bundlefrom the newNull Byte Shopand get over 60 hours of training from cybersecurity professionals.Buy Now (90% off) >Other worthwhile deals to check out:97% off The Ultimate 2021 White Hat Hacker Certification Bundle99% off The 2021 All-in-One Data Scientist Mega Bundle98% off The 2021 Premium Learn To Code Certification Bundle62% off MindMaster Mind Mapping Software: Perpetual LicenseCover image byNegative Space/Pexels; Screenshots by drd_/Null ByteRelatedNews:Microsoft Project "Oxford"Hack Like a Pro:How to Hack Web Apps, Part 7 (Finding Hidden Objects with DIRB)How To:Use Maltego to Fingerprint an Entire Network Using Only a Domain NameMarket Reality:Business Interest High, Adoption Low for Augmented RealityHow To:Learn More in Class by Leaving Your Laptop at HomeDev Report:IBM & Unity Partner to Offer AI Tool That Could Make Augmented Reality Apps SmarterNews:Google Home's New 'Listen' Tab Makes It Easy to Discover Music You'll LoveNews:New Messenger Discover Feature Is a Blast from the Past — with a TwistApple AR:iPhone X Face-Masking App Gives New Meaning to 'Incognito Mode'Hack Like a Pro:How to Find Website Vulnerabilities Using WiktoHow To:Teach your kids how to use their first digital cameraNews:Augmented Reality App Created by NYU Students Can Translate Sign Language in Real TimeHack Like a Pro:How to Hack Web Apps, Part 5 (Finding Vulnerable WordPress Websites)Android for Hackers:How to Scan Websites for Vulnerabilities Using an Android Phone Without RootHow To:Conduct OSINT Recon on a Target Domain with Raccoon ScannerNews:L'Oreal Acquires Leading Augmented Reality Makeup Provider ModiFaceForbes Exploited:XSS Vulnerabilities Allow Phishers to Hijack Sessions & Steal LoginsThe Tabletop Demonstrator:A PVC Filmmaking ContraptionNews:What Is Technology?News:Awesome LEGO JewelryA Kit-of-No-Parts:"Crafting" Electronics at MITNews:Discover The Secrests To Custom Paint and AirbrushingNews:More Unreleased goodies discoveredNews:Are u depressed?News:Yes, more 3D.News:Magisto App for iPhone Released at CESNews:Two Earth-Sized Planets Discovered by NASANews:Bugzilla Cross Site Request ForgeryKinotopic:How to Make Cinemagraphs Using Your iPhoneNews:The Top 32 Authors to Follow on Google+News:The MERCURY Line - Palmistry - JIM WINTERFROM THE HUBBLE:Sun Devours PlanetNews:It's our organic world! We are all living in it!News:Panasonic's New Millimeter-Wave Radar Technology Could Save Lives at IntersectionsNews:Figure Sculpting Demo- Zeus' iPhone (Calling Down The Thunder)How To:Clay sculpting tutorial - Plato's iPadNews:Technology of 3DNews:Congresswoman Roybal-Allard serving California's 34th Congressional DistrictNews:Save Me!News:SOPA and PIPA Blackout!