Package: s3recon
s3reconAmazon S3 bucket finder and crawler.Check out the s3recon docsInstallationNOTE: s3recon requires python version>=3.6$pipinstalls3reconUsageusage: s3recon [-h] [-o file] [-d] [-p] [-t seconds] [-v] [-c num] word_list [word_list ...]

positional arguments:
  word_list                      read words from one or more <word-list> files

optional arguments:
  -h, --help                     show this help message and exit
  -o file, --output file         write output to <file>
  -d, --db                       write output to database
  -p, --public                   only include 'public' buckets in the output
  -t seconds, --timeout seconds  http request timeout in <seconds> (default: 30)
  -v, --version                  show program's version number and exit
  -c num, --concurrency num      maximum <num> of concurrent requests (default: # of lcpus)Example 1: Output to a json file1. Download a word-list.TheSecListsrepository has a multitude of word-lists to choose from. For this example, let's download the sample word-list included in this repository.$curl-sSfL-o"word-list.txt""https://raw.githubusercontent.com/clarketm/s3recon/master/data/words.txt"2. Runs3recon.Executes3reconusing theword-list.txtfile and output thepublicS3 buckets to a json file namedresults.json.$s3recon"word-list.txt"-o"results.json"--public

-PRIVATEhttps://s3.sa-east-1.amazonaws.com/test-lyft
-PRIVATEhttps://s3.ap-south-1.amazonaws.com/test.amazon
+PUBLIChttps://walmart-dev.s3.us-east-1.amazonaws.com
-PRIVATEhttps://s3.ap-southeast-1.amazonaws.com/apple-prod
-PRIVATEhttps://walmart.s3.ap-southeast-1.amazonaws.com
...3. Inspect the results.Check theresults.jsonoutput file to view the S3 buckets you have discovered!$cat"results.json"{"public":{"total":12,"hits":["https://walmart-dev.s3.us-east-1.amazonaws.com","https://apple-production.s3.ap-southeast-1.amazonaws.com",...]}}Note:to includeprivatebuckets in the results omit the-p, --publicflag from the command.4. Crawl the results.Enumerate the static files located in each bucket and record the findings.Coming soon!Example 2: Output to a MongoDB database1. Download a word-list.TheSecListsrepository has a multitude of word-lists to choose from. For this example, let's download the sample word-list included in this repository.$curl-sSfL-o"word-list.txt""https://raw.githubusercontent.com/clarketm/s3recon/master/data/words.txt"2. Start an instance of MongoDB$ docker run --name "mongo" -p 27017:27017 -v "mongodb_data:/data/db" -v "mongodb_config:/data/configdb" -d mongo3. Runs3recon.Executes3reconusing theword-list.txtfile and output to MongoDB instance.$s3recon"word-list.txt"--db

-PRIVATEhttps://s3.sa-east-1.amazonaws.com/test-lyft
-PRIVATEhttps://s3.ap-south-1.amazonaws.com/test.amazon
+PUBLIChttps://walmart-dev.s3.us-east-1.amazonaws.com
-PRIVATEhttps://s3.ap-southeast-1.amazonaws.com/apple-prod
-PRIVATEhttps://walmart.s3.ap-southeast-1.amazonaws.com
...3. Inspect the results.Check the MongoDB database:s3reconcollection:hitsto view the S3 buckets you have discovered!$mongo"s3recon"--quiet--eval'db.hits.find({}, {"url": 1, "access": 1, "_id": 0}).limit(5)'{"url":"https://s3.us-east-2.amazonaws.com/apple","access":"private"}{"url":"https://s3.us-west-1.amazonaws.com/microsoft-dev","access":"private"}{"url":"https://s3.us-west-1.amazonaws.com/dev-microsoft","access":"private"}{"url":"https://s3.us-east-2.amazonaws.com/amazon","access":"private"}{"url":"https://s3.us-east-1.amazonaws.com/dev-amazon","access":"private"}4. Crawl the results.Enumerate the static files located in each bucket and record the findings.Coming soon!FAQQ: How do I configure this utility?A:s3reconcan be configure using a yaml configuration file located in either the current working directory (e.g../s3recon.yml) or your home diretory (e.g.~/s3recon.yml).The following is the list of configurable values:# s3recon.ymldatabase:{host:"0.0.0.0",...}separators:["-","_","."]environments:["","backup","backups",...]regions:["ap-northeast-1","ap-northeast-2",...]To see the full list of configurable values (and theirdefaults) please refer to thes3recon.ymlfile in this repository.Q: How do I customize the AWS regions used in the recon?A:The AWSregionscan be altered by setting theregionsarray in yours3recon.ymlconfiguration file.# s3recon.ymlregions:["us-west-2",...]Q: How do I customize the environment values used in the recon?A:Theenvironmentsare modifiers permuted with each item of theword-list(and theseparator) to construct the bucket value in request.
The value can be altered by setting theenvironmentsarray in yours3recon.ymlconfiguration file.For example, to only search lines from the word-listverbatim(i.e. without modification) you can set this value to an empty array.# s3recon.ymlenvironments:[]Q: How do I customize the MongoDB host and port?A:The databasehostandportcan be configured by altering thedatabasemap in yours3recon.ymlconfiguration file.For example,hostandportcan be set directly inside thedatabasemap# s3recon.ymldatabase:{host:"0.0.0.0",port:27017}Q: How do I use a database other than MongoDB?A:Sorry, at the moment only MongoDB is supported.Going ForwardCreatecrawlcommand to crawl public/private buckets found infindstage.Separate outfindandcrawlas subcommands.Store discovered buckets in a NoSQL database.DisclaimerThis tools is distributed for educational and security purposes. I take no responsibility and assume no liability for the manner in which this tool is used.LicenseMIT ©Travis Clarke