The Top 10 Best Places to Find Datasets 📊
By: Jeff Hale
Tags: Technology, Data Science, Artificial Intelligence, Machine Learning, Data

Awesome Data

Awesome Data is a GitHub repository with a seriously impressive list of datasets separated by category. It is updated regularly.

Data Is Plural

Jeremy Singer-Vine’s Data Is Plural weekly newsletter has great fresh data sources. I’m always impressed by the quality. The archive is available here.

Kaggle Datasets

In addition to competitions, Kaggle has a huge range of datasets. Kaggle Datasets provide great summary information and previews for most datasets. You can download the data or use their platform to analyze it in a Jupyter notebook. You can also contribute your own datasets and make them public or private.

Kaggle is great for browsing or searching for a particular topic.

Data.world

Like Kaggle, Data.world provides a wide range of user-contributed datasets. It also offers a platform for companies to store and organize their data.

Google Dataset Search Tool

I think it’s safe to say that Google knows a thing or two about search. It recently added a separate search functionality for datasets through its Google Dataset Search Tool. It’s worth a shot if you’re looking for data on a particular topic or from a particular source.

OpenDaL

OpenDaL is a data aggregator that allows you to search using a variety of metadata. For example you can search based on time or search by location by selecting part of a map.

Screenshot from OpenDaL.

Pandas Data Reader

The Pandas DataReader will help you pull data from online sources into Python pandas DataFrames. Most of the data sources are financial. Here’s the list of available data sources as of October 31, 2020. 🎃

Here’s how you use it after installing it into a Python environment with pip install pandas-datareader .

import pandas_datareader as pdr

pdr.get_data_fred('GS10')

VisualData 👓

If you are looking for computer vision datasets, VisualData is a nice new source. It has some handy filtering options. Thanks to Jie Feng for reminding me of it! ***Added Nov. 2, 2020.***

Data.gov 👓

If you are looking to use the US government’s datasets, Data.gov has over 217,000 fo them! Thanks to Michael Wallace for recommending it! ***Added Nov. 4, 2020.***

Python API Wrappers 🐍

I recently updated my list of Python API wrappers to help users see whether a package is popular and being maintained. It now uses shields.io to automatically display GitHub stars and the date of the most recent commit. This list was originally forked from GitHub repo of Real Python via johnwmillr. My repo contains what I believe is the largest updated list of Python API wrappers — many of which can help you find the data you might need for a project.

APIs

Getting data from a documented API using Python might sound intimidating if you haven’t done it before, but it’s really not bad. Check out my guide to getting data from APIs here. 🚀

Make your own

When all else fails, collecting your own data can be an excellent way to create a dataset for your needs. 😉

Recap

Do you have a favorite place to find data? Awesome! Share it on Twitter or leave it in the comments! 🎉

I hope you find this tool helpful when you’re searching for data sources. If you do, please share it on your favorite social media. 🚀

I write about Python, data science, and other tech topics. If you’re into that kind of stuff read more here and subscribe to my Data Awesome newsletter for awesome monthly curated data resources.