Debug Google Colab and Kaggle
By: James Fulford
Tags: Data Science, Python, Jupyter Notebook, Colab, Kaggle

Hopefully, I’ll save you from getting her headache. Photo by Aiony Haust on Unsplash.

Debug Google Colab and Kaggle

You’d imagine free public datasets would be painless

This wasn’t painless. But now that I’ve figured it out, I want to save you Google-ers out there some time.

What’s Google Colab?

Colab is a way to run Python Jupyter Notebooks on the Cloud, for free. It’s provided by Google and integrates with Google Drive. Very clean way to handle “but it doesn’t work on my computer!” issues.

See the “Resources” section at the end of this article for a link.

What’s Kaggle?

Kaggle is “Your Home for Data Science”. It hosts public datasets and competitions revolving around Data Science and Artificial Intelligence. It adds a sense of gamification to it as well. You can even share your profile to show others your involvement.

Before hitting my connection issues, I was aware of Kaggle but did not have an account yet.

After Signing Up For Kaggle

There are some tricky preconditions before you can download data.

Verify Account

You need to verify your account with a phone number that is not already used in Kaggle.

If your number is already used in Kaggle (previous owner — my mother, actually — signed up for Kaggle years ago), the Kaggle system will simply not text you a verification code, and will not tell you why. (This is probably a security feature).

If you hit a similar situation as me, the advice from Kaggle is to either use or delete the old account. The other alternative is to use someone else’s phone. Or get a new phone number.

Accept Rules

You need to accept the rules for the competition. If this errors out (the error message is pretty vague), then make sure your account is verified via phone number. See the previous section.

Kaggle API Key

With your new Kaggle account, go to “My Account”, find an API Key section and generate an API key. This will download a kaggle.json file. Open it, and you will see 2 values: a username and a key. Keep those handy.

The key value is sensitive, you need to treat it like a password. If you give someone a notebook that has this value printed out, quickly go to your “My Account” page and expire all your API Keys (same menu where you generated your API Key). Otherwise, someone can steal your Kaggle identity. At that point, your old API key will stop working, so you may want to generate a new one.

(You can put this file at `~/.kaggle/kaggle.json` and give it 600 permissions, however uploading that file every time you want to work with Kaggle in Colab would get old really fast. So, just follow what you see below)

Installing Kaggle on Google Colab

Add this Code cell for each Colab Notebook. It will install Kaggle CLI and prompt you for your credentials.

!pip install kaggle

!mkdir -p /root/.kaggle

!echo "Input contents of kaggle.json"

!read -s KAGGLE_JSON; echo "$KAGGLE_JSON" > /root/.kaggle/kaggle.json

# this is sensitive, so `-s` masks the input

!chmod 600 /root/.kaggle/kaggle.json

(The ! tells Colab to run this as a Bash command, not Python. It’s useful for installing dependencies, downloading datasets, and exploring directories.)

At this point, you can download the dataset. I’m using the 2013 dogs-vs-cats competition dataset, here’s the command I copied from the data page.

!kaggle competitions download -c dogs-vs-cats

If you get an error here (something like 401 or 403):

Double-check that your KAGGLE_USERNAME and KAGGLE_KEY is accurate. Make sure you accepted the rules to the competition. Make sure your account is verified over SMS.

Conclusion

Alright, you should now be able to download Kaggle data in Google Colab.

If you hit a different issue, or if you found this debugging resource useful, please write in to let us know!

Resources