Project Sonar

Project Sonar by security company Rapid7 is a collection of internet-wide network scans with the aim of investigating vulnerabilities. The data is available for free, though many files (historical and most current ones) require an account.
== Datasets ==
As of 2021-06-29, the following data is available through Project Sonar:
Dataset 	Description 	Scan frequency	Data range 	Available without registration	Size 	Size growth per month
Forward DNS (FDNS) 	"DNS 'ANY', 'A', 'AAAA', 'TXT', 'MX', and 'CNAME' responses for known forward DNS names" 	Weekly 	Feb 2017 to present 	Third most recent scan 	11.1 TiB	393 GiB
Reverse DNS (RDNS) 	"DNS IPv4 PTR responses" 	Weekly 	Feb 2017 to present 	Third most recent scan 	3.14 TiB	58 GiB
HTTP GET Responses 	"Responses to HTTP/1.1 GET requests against various HTTP ports" 	Bi-weekly 	Oct 2013 to present 	Second most recent scan 	15.3 TiB	265 GiB
HTTPS GET Responses 	"Responses to HTTP/1.1 GET requests against various HTTPS ports" 	Bi-weekly 	July 2017 to present 	Second most recent scan 	11.2 TiB	299 GiB
SSL Certificates 	"X.509 certificate metadata observed when communicating with HTTPS endpoints" 	Bi-weekly 	Oct 2013 to present 	Second most recent scan 	1.24 TiB	61 GiB
More SSL Certificates (non-443) 	"X.509 certificate metadata observed when communicating with miscellaneous non-HTTPS endpoints, such as IMAPS, POP3S, etc." 	Bi-weekly 	June 2014 to present 	Second most recent scan 	450 GiB 	14.6 GiB
UDP Scans 	"UDP scan results for common UDP services across all of IPv4" 	Monthly 	June 2014 to present 	Second most recent scan 	340 GiB 	1 GiB
TCP Scans 	"SYN scan results for common TCP services across all of IPv4" 	Weekly 	Mar 2017 to present 	Second most recent scan 	680 GiB 	19 GiB
National Exposure Scans 	"Open port results for Rapid7's National Exposure reports" 	N/A 	Apr 2016 to Apr 2018 	None 	13 GiB 	N/A
Forward DNS (FDNS) -- ANY 2014-2017	"DNS 'ANY' responses for known forward DNS names from 2014-2017" 	Weekly 	Mar 2014 to Jan 2017 	None 	1.1 TiB 	N/A
Reverse DNS (RDNS) -- 2013-2017 	"DNS IPv4 PTR responses from 2013-2017" 	Weekly 	Sept 2013 to Feb 2017	None 	650 GiB 	N/A
Critical.IO Service Fingerprints 	"The Critical.IO project was designed to uncover large-scale vulnerabilities on the global IPv4 internet and scanned a number of ports across between May 2012 and March 2013."	Monthly 	May 2012 to Mar 2013 	None 	355 GiB 	N/A
Total 	Total 	Total 	Total 	Total 	46.3 TiB	1.08 TiB
(Note: The website uses the wrong units on the files. 'MB' on their list actually means MiB.)
=== No longer available (lost?) ===
* from Nov 2016. A single file of 520 MiB was available as of 2018-11-13 and has vanished as of 2021-06-29. The file was publicly accessible but is not available in the Wayback Machine and presumed .
==Closure to Public==
On February 10th, 2022, Rapid7 decided to close public access to Project Sonar. The project was closed due to "changes ... triggered by the evolving regulatory landscape"https://www.rapid7.com/blog/post/2022/02/10/evolving-how-we-share-rapid7-research-data-2/, concerning the heavy presence of IP address in their data. Further, the API that made this data searchable (https://sonar.omnisint.io/) https://github.com/Cgboal/SonarSearch has been removed.
Some past data that was previously released to the public remains available in the Wayback Machine, with the latest available files being from 2021-09-24, though many of them appear to be truncated at 2GB.
== References ==