Category: dataset
Question: How can I create a Dataset on Kaggle?
Answer: It’s easy to create a dataset on Kaggle and doing so is a great way to start a data science portfolio, share reproducible research, or work with collaborators on a project for work or school. You have the option to create private datasets to work solo or with invited collaborators or publish a dataset publicly to Kaggle for anyone to view, download, and analyze.

## Navigating the Dataset Interface

To publish a private or public dataset, start by navigating to the [Datasets listing](https://www.kaggle.com/datasets). There you will find a New Dataset button. Click on it to open the New Dataset modal.

The required “bare minimum” fields for uploading a dataset to Kaggle in descending order are:

- The *Title* is the name of the Dataset – e.g. what will appear in the listing when searching or browsing.
- The *URL* is the link the Dataset will live at. The slug will first auto-populate and mimic your Title. However, you can hover over the slug to change it right away.
- Finally, you may upload data from one of four sources:
  - *Your local machine* - upload files/folders via drag and drop or by selecting them in your file browser. To speed up file/folder uploads, try uploading them as a ZIP archive; the contents will be unzipped on our side to make them accessible in Notebooks.
  - *Remote Files* - enter list of public URL(s) which identify files to be imported into dataset
  - *Github Repository* - enter URL to github repository whose files will be imported into dataset
  - *Notebook Outputs* - use inbuilt search to explore publicly available files produced from Kaggle’s large repository of public Notebooks

To make your dataset more useful for your collaborators and the community it is recommended you update the following settings:

- The Sharing menu controls the Dataset’s visibility. Datasets may be Private (visible only to you and your collaborators, and to Kaggle for purposes consistent with the Kaggle Privacy Policy) or Public (visible to everyone). The default setting is Private.
- The Licence is the license the dataset is released under (relevant for public datasets). If the license you need doesn’t appear in the dropdown, select the “Other (specified in description)” option and be sure to provide information on the license when writing the dataset description (in the next step). Below is a list of common licenses.

### Common Licenses

#### Creative Commons
- CC0: Public Domain
- CC BY-NC-SA 4.0
- CC BY-SA 4.0
- CC BY-SA 3.0
- CC BY 4.0 (Attribution 4.0 International)
- CC BY-NC 4.0 (Attribution-NonCommercial 4.0 International)
- CC BY 3.0 (Attribution 3.0 Unported)
- CC BY 3.0 IGO (Attribution 3.0 IGO)
- CC BY-NC-SA 3.0 IGO (Attribution-NonCommercial-ShareAlike 3.0 IGO)
- CC BY-ND 4.0 (Attribution-NoDerivatives 4.0 International)
- CC BY-NC-ND 4.0 (Attribution-NonCommercial-NoDerivatives 4.0 International)

#### GPL
- GPL 2
- LGPL 3.0 (GNU Lesser General Public License 3.0)
- AGPL 3.0 (GNU Affero General Public License 3.0)
- FDL 1.3 (GNU Free Documentation License 1.3)

#### Open Data Commons
- Database: Open Database, Contents: Database Contents
- Database: Open Database, Contents: © Original Authors
- PDDL (ODC Public Domain Dedication and Licence)
- ODC-BY 1.0 (ODC Attribution License)

#### Community Data License
- Community Data License Agreement - Permissive - Version 1.0
- Community Data License Agreement - Sharing - Version 1.0

#### Special
- World Bank Dataset Terms of Use
- Reddit API Terms
- U.S. Government Works
- EU ODP Legal Notice

- Owner allows you to specify the dataset Owner if you belong to any Organizations. You may assign ownership to yourself or to any Organizations you are a member of (see the section “Creating and using organizations” to learn more about this feature).

Once you have provided the required information alongside your data source, click on “Create Dataset” and your dataset will start processing. Once the dataset is finished processing, you will be taken to your new dataset’s home page.

Note that if your dataset is very large (multiple gigabytes in size), processing may take a while, up to several minutes. Feel free to navigate away from the browser window whilst processing is inflight as it will continue in the background.

Your datasets has now been created! However, for truly great Datasets, the work doesn’t stop there. Once you have specified the required fields there are a few other things you should do in order to maximize your dataset’s usefulness to the community or your collaborators:

- Upload a cover image. We recommend using [unsplash.com](http://unsplash.com/) for shareable, high resolution images.
- Add a subtitle to the dataset. This is a short bit of text explaining in slightly more detail what is in it. This subtitle will appear alongside the title in the search listings.
- Add tags. Tags help users find datasets on topics they are interested in by making them easier to find.
- Add a description. The description should explain what the dataset is about in long-form text. A great description is extremely useful to Kaggle community members looking to get started with your data.
- Publish a public Notebook. Use Notebooks to show community members or your collaborators how to get started with the data. This can be something simple like an exploratory data analysis or a more complex project reproducing research using the data.

A few examples of well-formatted datasets are “[CS:GO Competitive Matchmaking Data](https://www.kaggle.com/skihikingkevin/csgo-matchmaking-damage)”, “[Yelp Dataset](https://www.kaggle.com/yelp-dataset/yelp-dataset)”, “[1.6 million UK traffic accidents](https://www.kaggle.com/daveianhickey/2000-16-traffic-flow-england-scotland-wales)”, and “[Fashion MNIST](https://www.kaggle.com/zalando-research/fashionmnist)”.

## Creating Datasets from Various Connectors

As outlined above, in addition to uploading files from your local machine, you can also create Datasets from various data sources including GitHub, remote URLs (any public file hosted on the web), and Notebook output files. These are each icons that can be found in the Dataset Upload Modal sidebar.

### GitHub and Remote File Datasets

Datasets created from a GitHub repository or hosted (remote) files are downloaded directly from the remote server to Kaggle’s cloud storage and, therefore, will consume none of your local network’s bandwidth. This makes the remote files connector a convenient solution for creating datasets from large files.

When a dataset is created from a github repository or hosted file, the publisher is able to set up automatic interval updates from the dataset’s Settings tab. Here’s an example [stock market dataset](https://www.kaggle.com/timoboz/stock-data-dow-jones) that updates daily.

Don’t want to wait for a refresh? No problem! Click the Update button within the "..." dropdown in the dataset menu header to sync your dataset immediately.

### Notebook Output File Datasets

Creating a dataset from a Notebook’s output files will let you create reproducible data pipelines. To create a dataset from a Notebook’s output files, click on the icon in the uploader and search for your Notebook. Alternatively, you can click “Create Dataset” from the Output tab on your rendered Notebook. Then, select the files you want to use in your dataset.

### Limitations

It's worth noting that for user experience and technical simplicity, a dataset can be created and versioned from exclusively one data source. That is, data sources currently can not be mixed and matched in any given dataset (for example, a dataset created from a GitHub repository can't also include files uploaded from your local machine). If you would like to use various different data sources in a Notebook you can create multiple datasets and add them both to said Notebook.

The usual technical specifications for dataset creation apply to connectors too. See the [Technical Specifications](https://www.kaggle.com/docs/datasets#technical-specifications) section for more information.

## Updating Dataset Using JSON Config

For advanced users, you may find it easier to update key parameters of your dataset by specifying the details as JSON configuration. To do this, navigate to your dataset and click Settings, followed by “JSON Config” in the menu of options on the left.

You can update any of the settings you would normally edit through the datasets user interface, such as title, collaborators, licenses, keywords and more. For a reference to the schema you can use for updating dataset settings, you can look at our documentation for the relevant actions within the Public API.

Please note, there are some subtle differences between the Public API schema and the schema supported in the JSON Config settings UI. They are as follows:

- *id* is omitted as it cannot be changed after dataset creation
- *resources* is omitted as you cannot change the uploaded files using this UI
- The *isPrivate* is an added boolean option that allows users to change the privacy of their datasets (note: public datasets can NOT be made private)
- *collaborators* is an added array of objects with shape | { “username”: string; “role”: “read” | “write”}| that can be used to specify dataset collaborators