How to Use Facial Recognition to Conduct OSINT Analysis on Individuals & Companies « Null Byte :: WonderHowTo

Analysis of photographs and social connections can be a huge component ofsocial engineering. Understanding who a person is, as well as who they know, can establish links within a company. This information could be used by hackers to execute elaborate social engineering attacks.OSINT(open-source intelligence) is a term used to refer to the gathering, evaluation, and application of publicly available data. To quote theCentral Intelligence Agency, "Information does not have to be secret to be valuable. Whether in the blogs we browse, the broadcasts we watch, or the specialized journals we read, there is an endless supply of information that contributes to our understanding of the world."One of the most common forms of such information found on the internet is not stored in plaintext, but rather in photographic content. Traditionally, these images had to be filtered manually by a human viewer to determine and evaluate their content, but with the availability of facial recognition software, this whole process can be automated.Don't Miss:How to Use Kismet to Watch Wi-Fi User Activity Through WallsFacial recognition has generated a lot of news and discussion regarding the use ofbiometrics, as well as the effects of this technology on data collection and privacy. While this technology is frequently used on platforms such as Facebook and is gaining widespread use in law enforcement worldwide, the backend technologies which allow one to use facial recognition for their own purposes are now easily accessible and available for use in free and open-source implementations.In this tutorial, we'll use theFace RecognitionPython library to analyze and discover images based on the people within them. We'll look at how batches of images can be crawled for and discovered, as well as look at how to generate statistics and conclusions about an individual or website.Step 1: Install the PrerequisitesThe Python library we'll be using,Face Recognition, is available directly through the Pip package manager onLinuxandmacOS. While the package isn't officially supported onWindows, aninstallation guideis available, as well as apreconfigured virtual machine image.Before beginning installation, make sure your operating system is up to date. On Debian-based Linux distros such as Ubuntu andKali, repositories can be updated and new software installed by running the command below.sudoapt-getupdate && apt-get upgradeWhile Pip is included by default in some distributions, it can also be installed by running the command below. This command will fetch the newest version of the package and install it.sudo apt-get install python-pipOnce Pip is installed, we can use it to install the Face Recognition library by running the command below.sudo pip install face_recognitionOnce the Pip installation completes, the library should be ready to use!Step 2: Organize Data for AnalysisThe library used for facial recognition being implemented in Python means it can be implemented into other applications with relative ease. This means that the addition of facial recognition function could be added to live webcam streams, web crawlers, or other formats where images or videos containing faces are displayed digitally. This, however, requires the writing of additional original code. Instead, in this tutorial, we'll use the included command line tool and combine it with other system utilities to perform similar tasks without the need to write new scripts.The included command line tool is intended to work with two batches of files. One folder will contain a set of recognized faces, the other a set of images which the user desires to compare to this set.To create a set of images to filter for specific individual faces, we can recursively download a website, such as the one below, using Wget. In real-world scenarios, a pentester or researcher may instead target a specific business's domains, image galleries, or even social media or other sources to try to download as many potentially relevant images as possible.Image viaThe White HouseWget can be operated from the command line and be used to download anything from a single page or image to an entire website. A command such as the one shown below will follow every link within a given domain and download all content available on that page. This command can be quit at any time by pressingCtrl+C, otherwise, it will run until the given request completes.wget -p -r -E -e robots=off --convert-links -U mozilla --level 1whitehouse.govThe-pflag specifies that additional content such as images, stylesheets, and scripts are also downloaded by the tool.The-Eflag mandates that each file downloaded is saved with its full file extension, which will make it easier for us to filter for images.The-rflag is for recursion so that links and other directories are also downloaded.Later in the string, the level of recursion is specified as--level 1. This ensures that links from the index page are followed once. To allow links to be followed to an increased depth, the number can be increased from 1 or the flag can be removed to attempt infinite recursion.If one wished to make sure that no files are to be downloaded from other websites, the--domainflag can be used, followed by the domain which the command should be restricted to.Keep in mind that recursive requests can be blocked or cause issues on some servers, so you may wish to limit your recursion or cap your request speed by using the-wflag to create a delay between requests.Using the-e robots=offavoids the request from being dismissed or blocked as an undesired web crawler.For the same reason, the-U mozillaargument is used to specify that a Mozilla useragent should be used so that the tool appears to be a web browser.The--convert-linksflag reformats links so that the page can be more easily utilized offline. While this is not necessary for the purpose of filtering the images downloaded from the site, it may make it easier to identify where the images were located on the webpage retroactively.Don't Miss:How to Use Maltego to Research & Mine Data Like an AnalystOnce we've let the program run for the desired duration, we can make duplicates of all of the image files in a new directory to be able to more easily work with them. First, create a new directory by running the command below.mkdirunknown_facesNext, copy all image files which were retrieved from the Wget request into this new directory by running the following two commands, the first for JPG files, and the second for PNG files. Theurlfolderstring should be replaced with the name of the folder which Wget created, usually corresponding to the URL or domain used in the initial command.cp urlfolder**/*.jpg unknown_faces/cp urlfolder**/*.jpg unknown_faces/Thecpcommand copies any matching files, followed by the directory argument, in this case,whitehouse.gov. The ** (double asterisk) following this directory specifies recursion, followed by a * (wildcard) for any JPG or PNG images. The last parameter of the string specifies that files should be copied to theunknown_facesdirectory.Once these images are moved to this new directory, we can uselsto view the files within this folder. The command below will list the contents of theunknown_facesfolder.ls unknown_faces/We can also forward the output of this command using a|(pipe) redirection operator in order to get a count of how many image files were discovered. This may be useful later in order to identify the frequency of a given face within a set of images. This can be done by running the command below, where the output of the previous command is sent towcwith the-lflag to count the number of lines.ls unknown_faces/ | wc -lThe output of this command shows that there are 178 image files which were saved by Wget. Once these images are ready to use, we can create a positive identification list for the faces we would like to find within this dataset.Step 3: Prepare Identification ListTo begin preparing a list of known identities and faces, create a new folder outside of the directory of the images you wish to parse. From the Wget-created directory, one can move up one directory by runningcd ../, create a new directory titledknown_facesby runningmkdir known_faces, and finally move into this directory by runningcd known_faces.Once this directory is created, one can save any files they wish to be used as a positive identification list to this folder. Clear photographs of any person are ideal for facial recognition, such as those often used as profile photos on "Contact" webpages. Ensure sure that only one face is visible in any known face photo. One may also wish to name the files something which corresponds to the person in the photo, as the filename of the photo will be returned when any positive identification is made.Once these directories are prepared, theface_recognitioncommand line utility is ready to be run.Step 4: Use Facial Recognition to Identify IndividualsThe command line utility included with the Python Face Recognition library is relatively simple to run, requiring only two parameters. The first is a directory of known faces, and the second is a directory containing faces to attempt to identify. With these folders titled according to the previous steps, the command below will run the tool.face_recognition known_faces/ unknown_faces/As the program runs, it will return filenames followed by either a positive identification of a specific person, a negative identification of an unknown person, or no persons found if there are no faces in the image. To make these results slightly more useful for automated data manipulation, we can run the same command followed by a>redirector to send the output of the program to a text file, such as in the example below.face_recognition known_faces/ unknown_faces/ > results.txtWe can use this text file as a reference to analyze and utilize the results of the facial recognition matches.Step 5: Analyze Facial Recognition MatchesOne possible way to understand the data results from the facial recognition tool is to look for all of the files which matched with a specifically identified face. We can usegrepand theresults.txttext file to see each match made with a given face. The command below will return every line in the text file which was identified with the nameDonald Trump.grep "Donald Trump" results.txtWe can also use a combination of other system utilities to determine the frequency of each result of the program by running the command below.catresults.txt | sed 's/.*,//' | sort | uniq -cFirst, runningcat results.txtreads the contents ofresults.txtas standard input. This is sent tosed, where everything before and including the comma in each line is removed. This list of results is then sent tosortwhere each result is grouped together. Finally, this is used as input foruniqwhich as a result of the-cparameter counts the number of occurrences of each line.This produces a list where each result is listed with its corresponding frequency, allowing for simple analysis of larger datasets.If we wish to have a look at the specific images identified by the facial recognition tool, we can use a similar technique. We can use the command below to produce a new text file which is a list of filenames which were tagged with a given result by the program and saved in the previous output file.grep "Donald Trump" results.txt | sed 's/,.*$//' > identified.txtIn this case, all images identified asDonald Trumpare searched for usinggrepin theresults.txttext file. These results are sent as standard input tosed, where everything including and following the comma is removed from each line. Finally, the redirection operator>sends the output to a text file namedidentified.txt.If we run a command likehead identified.txt, we'll see that the file only contains filenames. We can use this filename list as input for a short script to copy each of these files to a new folder. We can create a new script file using Nano by running the commandnano identified.sh.#!/bin/bashmkdir identifiedwhile read line; dofilename=$(echo $line | sed 's/.*\///')cp $line identified/$filenamedone < identified.txtThe first line of the script,#!/bin/bash, declares the script as being intended for Bash.The second line creates a new directory namedidentifiedwhere all of the positively identified images will be moved to.The third line, in combination with the last line, reads the text fileidentified.txtas input such that each line of the file is modified individually within the while statement.The first line of the while statement, the fourth line of the code, creates a variable namedfilename. This variable is assigned to the filename of the file without the directory location, usingsedto remove any characters before the slash in each line.This variable is utilized in the last line, where each file within theidentified.txttext file is copied to theidentifieddirectory and assigned its original filename.Don't Miss:How Hackers Cover Their Tracks on an Exploited Linux Server with Shell ScriptingOnce this script is written out in Nano, it can be saved by pressingCtrl+O, andCtrl+Xcan be used to then exit the program. After the script is saved asidentified.sh, we can add execution privileges to the file by running the command below.chmod+x identified.shNow, we can run the script itself by simply typing./identified.shand pressingenter/return.Once the script has finished running, you can navigate to the "identified" directory and see all of the images which were discovered that match your specific query! This automatically filtered process expediates a traditionally time-intensive problem requiring considerable human interaction to identify and sort images.Protecting Against Facial RecognitionUnfortunately, the only real protection against facial recognition is to limit the exposure of your face, online or otherwise. This can mean using stricter privacy settings for accounts, choosing not to use social media, or even attempting to conceal one's face in public. Many of these precautions are relatively inconvenient.In general, the dangers of facial recognition are more about the data attached to a given instance of an online profile or photo than the photo itself. Remember that whenever you choose to place any pictures or information online, you can never be completely certain of who may end up having access to them.Thanks for reading! If you have any questions, you can leave a comment below or hit me up on Twitter at@tahkion.Don't Miss:How to Stop Facial Recognition Software from Finding Out Who You Are on CameraFollow Null Byte onTwitterandGoogle+Follow WonderHowTo onFacebook,Twitter,Pinterest, andGoogle+Want to start making money as a white hat hacker?Jump-start your hacking career with our2020 Premium Ethical Hacking Certification Training Bundlefrom the newNull Byte Shopand get over 60 hours of training from cybersecurity professionals.Buy Now (90% off) >Other worthwhile deals to check out:97% off The Ultimate 2021 White Hat Hacker Certification Bundle99% off The 2021 All-in-One Data Scientist Mega Bundle98% off The 2021 Premium Learn To Code Certification Bundle62% off MindMaster Mind Mapping Software: Perpetual LicenseCover image byvchalup/123RF; Screenshots by Takhion/Null ByteRelatedHow To:Find Identifying Information from a Phone Number Using OSINT ToolsHow To:Stop Facial Recognition Software from Finding Out Who You Are on CameraHow To:Use Maltego to Target Company Email Addresses That May Be Vulnerable from Third-Party BreachesHow To:Use the Buscador OSINT VM for Conducting Online InvestigationsNews:Blippar Brings Facial Recognition AR Profiles to Mobile AppHow To:Change Your Android Screen's Orientation Using Your Face Instead of the Device's AngleRecon:How to Research a Person or Organization Using the Operative FrameworkNews:Mobile AR App Octi Evolves into Social Network via Facial Recognition TechHow To:Use Photon Scanner to Scrape Web OSINT DataNews:Image Recognition Expected to Generate $40 Billion by 2021News:Galaxy S9 Tipped to Have 'Intelligent Scan' Facial Unlocking — Iris & Face Recognition CombinedHow To:Brute-Force SSH, FTP, VNC & More with BruteDumHow To:Hunt Down Social Media Accounts by Usernames with SherlockHow To:Use Google Search Operators to Find Elusive InformationNews:Your Face Could Unlock the New iPhone 8News:AR Makes Its Broadway Debut on Makeup AppNews:Google Challenges Apple's iPhone X Series with Face Unlock & Air Gestures on Pixel 4News:Microsoft Releases Snapchat-Like Photo Editor on iOSHow To:Find OSINT Data on License Plate Numbers with SkiptracerMarket Reality:Apple, Snap, & Google Execs Talk AR, Form Takes AR to the Pool, & Octi Pivots to AR Social NetworkingNews:Vuzix Blade Teams with Facial Recognition Company to Provide Law Enforcement SolutionHow To:Disable Facebook facial recognition for photosNews:Google Photos Gets Smart with New AI & Sharing FeaturesHow To:Find Passwords in Exposed Log Files with Google DorksNews:Blippar's Mobile AR Business Collapses After Funding Falls ThroughApple AR:Warby Parker Uses Face ID in iPhone X to Measure Your Face for GlassesVideo:How to Use Maltego to Research & Mine Data Like an AnalystNews:Apple's Tim Cook, Google's Sundar Pichai, & Snap's Evan Spiegel Deliver Strategic Insights into AR in 2020 & BeyondHow To:Use Maltego to Fingerprint an Entire Network Using Only a Domain NameNews:Want to Try on Lipstick Without Leaving Your Home? Meitu Has a WayNews:Snapchat Virtually Restores London's Big Ben with Holiday-Themed, Location-Based AR LensNews:Snapchat Becomes SnapCAT with AR Lenses for Your Feline FriendsNews:Blippar Adds Landmark Recognition to Its Bag of AR TricksVaccine bombshell:Baby monkeys develop autism after routine CDC vaccinationsNews:Police Admit To Drugging Occupy Wall Street Protesters; Suspend ProgramNews:Magisto App for iPhone Released at CESNews:It-doesnt-pay-to-be-intelligentNews:9 facial masks you can make with ingredients from the kitchenToday's Tidbit:Politicians Never Forget a Face, Just Like Fuscatus WaspsNews:What Is Technology?