Want to easily integrate data with python? | by Ryan Brown | Towards Data Science

Picture this: you’re working on a data science project in python with a few peers or coworkers. You haven’t bothered to link the data input or output with a SQL/python database integration, because there’s not that much data changing hands. However, there are still enough .csv files being created and updated that it’s a hassle to email, Slack, or put them in a drive folder.
I’ve personally had this happen on multiple occasions in the past few months. I didn’t want to go through the whole SQL integration process, but file transfer was still tedious.
I recently discovered that there’s a much easier way to go about this, and it can be set up by someone with no experience in platform integration in as little as 5 minutes. The solution: data.world.
data.world is an enterprise data catalog, has the world’s largest open data community, and is free for individual use. You can upload various kinds of data, code, and analysis to either a dataset or a project.
The first step is to create a free account. You can do so at data.world/login using Google, Facebook, GitHub, or Twitter. You’ll end up on a page that looks like this:
Click “+ New”, and then “Create new dataset”. Choose a name and a few other settings, and then “Create dataset”. I named mine “Medium Article Sample”. You can add a description and data now, or do it later. I added the training data from Kaggle’s Titanic competition.
Next, go to data.world/integrations (or click the “Integrations” icon at the bottom left of your screen). Either type “python” in the filter search bar, or scroll down until you see it. Click on it, click “Enable Integration”, and then “Authorize Python”.
Now that the integration is complete, you can click “Get started” and follow the steps there. I’ll include them in this article as well.
1.Using the command line (or a jupyter notebook), install the data.world package from PyPI.
pip install datadotworld[pandas]
2. Configure the package using the command line. You’ll need your data.world API token, found here (click “Get Token”). Now you’re ready to use the package!
dw configure
3. Import the package in a notebook or script.
import datadotworld as dw
There are a lot of things you can do with the datadotworld package, but here are some fundamentals that we’re looking for with this basic data integration.
First, you can read all the files in your dataset, and grab a particular file as a dataframe (depending on the filetype you may need to import it as a table or raw data; more on that here). Note that table names are all lowercase and have underscores in place of spaces.
dataset = dw.load_dataset('ryan-brown/medium-article-sample')df = ds.dataframes['titanic_training_data']
If you’re using Windows you may have trouble with encoding; I created a quick function that takes advantage of datadotworld’s .query function to read a particular table from the dataset.
def dw_get(dataset_name, table_name):    results = dw.query(dataset_name, "SELECT * FROM `{}`".format(table_name))    df = results.dataframe    return dfdf = dw_get('ryan-brown/medium-article-sample', 'titanic_training_data')
Either way, you’ll now have a dataframe in your notebook that you can clean, work with, and analyze. Suppose I decided to fill drop the “ticket” column and all rows with NaN values to clean the data. I now have the following dataframe:
I’m ready to update the file on the dataset for my peers to use in analysis for the project. Doing this is easy — just save the dataframe as a .csv file in your working directory, then call the datadotworld api client and upload files.
df.to_csv('titanic_data_clean.csv',index=False)client = dw.api_client()client.upload_files('ryan-brown/medium-article-sample', files=['titanic_data_clean.csv'])
Notice that I decided to give it a name other than titanic_training_data.csv, so it’ll show up as a new file in the dataset. However, if I’d decided to name it titanic_training_data.csv, it would replace the old titanic_training_data.csv file in the dataset (while still maintaining any labels or descriptions you may have added).
Now that we’ve uploaded the new data, let’s take a look at the data.world dataset:
You can see the cleaned data is there, and on the right there’s an update indicating that I added data using python. It’s that easy.
To give peers access so you can work on and upload data collaboratively, click on the “Contributors” tab of your dataset, then click “Invite” to invite them by email or username.
This is just one small look at the many things the datadotworld package can do and that the site can be used for as a whole; you can check out more of the library’s features in a great article written here.