Datasets in Python. 5 packages that provide easy access to... | by Zolzaya Luvsandorj | Towards Data Science

There are useful Python packages that allow loading publicly available datasets with just a few lines of code. In this post, we will look at 5 packages that give instant access to a range of datasets. For each package, we will look at how to check out its list of available datasets and how to load an example dataset to a pandas dataframe.
I assume the reader (👀 yes, you!) has access to and is familiar with Python including installing packages, defining functions and other basic tasks. If you are new to Python, this is a good place to get started.
I have used and tested the scripts in Python 3.7.1 in Jupyter Notebook. Let’s make sure you have the relevant packages installed before we dive in:
◼️ ️pydataset: Dataset package,◼️ ️seaborn: Data Visualisation package,◼️ ️sklearn: Machine Learning package,◼️ ️statsmodel: Statistical Model package and◼️ ️nltk: Natural Language Tool Kit package
For each package, we will inspect the shape, head and tail of an example dataset. To avoid repeating ourselves, let’s quickly make a function:
# Create a function to glimpse the datadef glimpse(df):    print(f"{df.shape[0]} rows and {df.shape[1]} columns")    display(df.head())    display(df.tail())
Alright, we are ready to dive in! 🐳
The first package we are going look at is PyDataset. It’s easy to use and gives access to over 700 datasets. The package was inspired by ease of accessing datasets in R and aimed to bring that ease in Python. Let’s check out the list of datasets:
# Import packagefrom pydataset import data# Check out datasetsdata()
This returns a dataframe containing dataset_id and title for all datasets which you can browse through. Currently, there are 757 datasets. Now, let’s load the famous iris dataset as an example:
# Load as a dataframedf = data('iris')glimpse(df)
Loading a dataset to a dataframe takes only one line once we import the package. So simple, right? Something to note is that row index starts from 1 as opposed to 0 in this dataset.
🔗 To learn more, check out PyDataset’s GitHub repository.
Seaborn is another package that provides easy access to example datasets. To find the full list of datasets, you can browse the GitHub repository or you can check it in Python like this:
# Import seabornimport seaborn as sns# Check out available datasetsprint(sns.get_dataset_names())
Currently, there are 17 datasets available. Let’s load iris dataset as an example:
# Load as a dataframedf = sns.load_dataset('iris')glimpse(df)
It also takes only one line to load a dataset as a dataframe after importing the package.
🔗 To learn more, check out documentation page for load_dataset.
Not only is scikit-learn awesome for feature engineering and building models, it also comes with toy datasets and provides easy access to download and load real world datasets. The list of toy and real datasets as well as other details are available here. You can find out more details about a dataset by scrolling through the link or referring to the individual documentation for functions. It’s worth mentioning that among the datasets, there are some toy and real image datasets such as digits dataset and Olivetti faces dataset.
Now, let’s look at how to load real dataset with an example:
# Import packagefrom sklearn.datasets import fetch_california_housing# Load data (will download the data if it's the first time loading)housing = fetch_california_housing(as_frame=True)# Create a dataframedf = housing['data'].join(housing['target'])glimpse(df)
Here’s how to load an example toy dataset, iris:
# Import packagefrom sklearn.datasets import load_iris# Load datairis = load_iris(as_frame=True)# Create a dataframedf = iris['data'].join(iris['target'])# Map target names (only for categorical target)df['target'].replace(dict(enumerate(iris['target_names'])),                      inplace=True)glimpse(df)
💡 If you get an error regarding the as_frame argument, either update your sklearn version to 0.23 or higher or use the script below:
# Import packagesimport pandas as pdfrom sklearn.datasets import load_iris# Load datairis = load_iris()# Create a dataframeX = pd.DataFrame(iris['data'], columns=iris['feature_names'])y = pd.DataFrame(iris['target'], columns=['target'])df = X.join(y)# Map target names (only for categorical target)df['target'].replace(dict(enumerate(iris['target_names'])),                      inplace=True)glimpse(df)
🔗 For more information, check out scikit-learn’s documentation page.
Another package through which we can access data is statsmodels. Available built-in datasets are listed here on their website. Let’s pick ‘United States Macroeconomic data’ as an example and load it:
# Import packageimport statsmodels.api as sm# Load data as a dataframedf = sm.datasets.macrodata.load_pandas()['data']glimpse(df)
As you may have noticed, the name we used to access ‘United States Macroeconomic data’ is macrodata. To find the equivalent name for other datasets, have a look at the end of the URL for that dataset documentation. For instance, if you click on ‘United States Macroeconomic data’ in Available Dataset section and look at the address bar in your browser, you will see ‘macrodata.html’ at the end of URL.
Statsmodels also allows loading datasets from R with the get_rdataset function. The list of available datasets are here. Using iris dataset as an example, here is how we can load the data:
# Load data as a dataframedf = sm.datasets.get_rdataset(dataname='iris', package='datasets')['data']glimpse(df)
🔗 For more information, check out documentation page for datasets.
This package is slightly different from the rest because it provides access only to text datasets. Here’s the list of text datasets available (Psst, please note some items in that list are models). Using the id, we can access the relevant text dataset from NLTK. Let’s take Sentiment Polarity Dataset as an example. Its id is movie_reviews. Let’s first download it with the following script:
# Import packageimport nltk# Download the corpus (only need to do once)nltk.download('movie_reviews')
If it is already downloaded, running this will notify that you have done so. Once downloaded, we can load the data to a dataframe like this:
# Import packagesimport pandas as pdfrom nltk.corpus import movie_reviews# Convert to dataframedocuments = []for fileid in movie_reviews.fileids():    tag, filename = fileid.split('/')    documents.append((tag, movie_reviews.raw(fileid)))df = pd.DataFrame(documents, columns=['target', 'document'])glimpse(df)
There is no one size fits all approach when converting text data from NLTK to a dataframe. This means you will need to look up the appropriate way to convert to a dataframe on a case-by-case basis.
🔗 For more information, check out this resource on accessing text corpora and lexical resources.
There you have it, 5 packages that allow easy access to datasets. Now you know how to load datasets from any of these packages. It’s possible that datasets available in these packages could change in future but you know how to find all the available datasets, anyway! 🙆
Would you like to access more content like this? Medium members get unlimited access to any articles on Medium. If you become a member using my referral link, a portion of your membership fee will directly go to support me.
Thank you for reading my post. Hope you found something useful ✂️. If you are interested, here are the links to some of my other posts:◼️️ 5 tips for pandas users◼️️️️ How to transform variables in a pandas DataFrame◼️ TF-IDF explained◼️ Supervised text classification model in Python
Bye for now 🏃💨