A Quicker Way to Download Kaggle Datasets in Google Colab | by Nabanita Roy | Towards Data Science

Kaggle is one of the best practice fields for Data Scientists and many of us like to use Google Colab to play around with datasets due availability of better data processing infrastructure. In this article, I have walked through three simple steps to download any dataset seamlessly from Kaggle with a simple configuration that would
not require you to download/upload kaggle.json again and again
make re-running jupyter notebooks smoother, even on other machine with access to your Google account and drive.
Log in to Kaggle and access your account. Scroll down to the API section:
Click on ‘Create New API Token’ and download the kaggle.json file which contains your API token.
Make note of the path to this file. Let’s call this your/path/to/kag gle.json.
Then open a new notebook in Google Colab and mount your drive by clicking on the icon as shown in the picture below.
This step is important since you can alternately use code to mount your drive and upload file from your laptop but everytime you re-run the notebook, you’d have to scroll up & browse your kaggle.json from your your machine.
!pip install -q kaggle!pip install -q kaggle-cli!mkdir -p ~/.kaggle!cp "your/path/to/kag gle.json" ~/.kaggle/!cat ~/.kaggle/kaggle.json !chmod 600 ~/.kaggle/kaggle.json# For competition datasets!kaggle competitions download -c dataset_name -p download_to_folder# For other datasets!kaggle datasets download -d user/dataset_name -p download_to_folder
Replace:
your/path/to/kag gle.json with your path to kaggle.json on drive. You can explore your drive on the left and copy the path. I have double quotes around it to avoid issues with blank spaces in folder names.
download_to_folder with the folder where you’d like to store the downloaded dataset
dataset_name and/or user/dataset_name as follow:
To download a competition Dataset: You can easily get hold of the dataset_name to use in the URL. For example, if you want to download Fake News dataset, select just fake-news from the URL :
Also, make sure to have agreed to the competition rules:
Then, your final script would look like -
!pip install -q kaggle!pip install -q kaggle-cli!mkdir -p ~/.kaggle!cp "/content/drive/My Drive/your_path_to_kaggle.json" ~/.kaggle/!cat ~/.kaggle/kaggle.json !chmod 600 ~/.kaggle/kaggle.json# For competition datasets!kaggle competitions download -c fake-news -p Dataset
If you do not accept the competition rules then you’d encounter the 403 Forbidden Error.
To download any other Dataset: Replace user_name/dataset_name with the Kaggle username and the dataset name. This can be easily extracted from the URL. For example, if you want to download US Election 2020 Tweets, you could simply copy the part after kaggle.com -
... and plug it into the script.
Therefore your final script would look like -
!pip install -q kaggle!pip install -q kaggle-cli!mkdir -p ~/.kaggle!cp "/content/drive/My Drive/your_path_to_kaggle.json" ~/.kaggle/!cat ~/.kaggle/kaggle.json !chmod 600 ~/.kaggle/kaggle.json# For other datasets!kaggle datasets download -d manchunhui/us-election-2020-tweets -p Dataset
Finally, to re-run notebooks without having the necessity to scroll up, you could comment out the entire script including codes for unzipping datasets. If you are logged into your Google account, have access to your drive, you can run your code on any machine by directly downloading data without worrying about kaggle.json configs.
There are of course other ways of downloading Kaggle datasets but this works the best for me. Hope this helps!
Thanks for visiting!
My Links: Medium | LinkedIn | GitHub