# Loading and accessing data in a notebook #

You can integrate data into notebooks by accessing the data from a local file, from free data sets, or from a data source connection\. You load that data into a data structure or container in the notebook, for example, a pandas\.DataFrame, numpy\.array, Spark RDD, or Spark DataFrame\.

To work with data in a notebook, you can choose between the following options:

<!-- <table> -->

Recommended methods for adding data to your notebook

| Option                                                           | Recommended method                          | Requirements                                               | Details                                                                                                                                                              |
| ---------------------------------------------------------------- | ------------------------------------------- | ---------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Add data from a file on your local system                        | Add a **Code snippet** that loads your data | The file must exist as an asset in your project            | [Add a file from your local system](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/load-and-access-data.html?context=cdpaas&locale=en#add-file-local) and then [Use a code snippet to load the data](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/load-and-access-data.html?context=cdpaas&locale=en#files)                                                                               |
| Add data from a free data set from the Samples                   | Add a **Code snippet** that loads your data | The data set (file) must exist as an asset in your project | [Add a free data set from the Samples](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/load-and-access-data.html?context=cdpaas&locale=en#loadcomm) and then [Use a code snippet to load the data](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/load-and-access-data.html?context=cdpaas&locale=en#files)                                                                            |
| Load data from data source connections                           | Add a **Code snippet** that loads your data | The connection must exist as an asset in your project      | [Add a connection to your project](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/create-conn.html) and then [Add a code snippet that loads the data from your data source connection](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/load-and-access-data.html?context=cdpaas&locale=en#conns)                                            |
| Access project assets and metadata programmatically              | Use `ibm-watson-studio-lib`                 | The data asset must exist in your project                  | [Use the `ibm-watson-studio-lib` library to interact with data assets](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/using-ibm-ws-lib.html)                         |
| Create and use feature store data                                | Use `assetframe-lib` library functions      | The data asset must exist in your project                  | [Use the `assetframe-lib` library for Python to create and use feature store data](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/py_assetframe.html) |
| Access data using an API function or an operating system command | For example, use `wget`                     | N/A                                                        | [Access data using an API function or an operating system command](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/load-and-access-data.html?context=cdpaas&locale=en#api-function)                                                                                                 |

<!-- </table ""> -->

Important: Make sure that the environment in which the notebook is started has enough memory to store the data that you load to the notebook\. The environment must have significantly more memory than the total size of the data that is loaded to the notebook\. Some data frameworks, like pandas, can hold multiple copies of the data in memory\.

## Adding a file from your local system ##

To add a file from your local system to your project by using the Jupyterlab notebook editor:

<!-- <ol> -->

1.  Open your notebook in edit mode\.
2.  From the toolbar, click the **Upload asset to project** icon (![Shows the Upload asset to project icon](https://dataplatform.cloud.ibm.com/docs/content/wsj/console/images/find_data_icon.png)) and add your file\.

<!-- </ol> -->

Tip: You can also drag the file into your notebook sidebar\.

## Load data sets from the Samples ##

The data sets on the Samples contain open data\. Watch this short video to see how to work with public data sets in the Samples\.

This video provides a visual method to learn the concepts and tasks in this documentation\.

To add a data set from the Samples to your project:

<!-- <ol> -->

1.  From the IBM watsonx navigation menu, select Samples\.
2.  Find the card for the data set that you want to add\. ![A view of data sets](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/images/datasets.png)
3.  Click **Add to project**, select the project, and click **Add**\. Clicking **View project** takes you to the project Overview page\. The data asset is added to the list of data assets on the project's Assets page\.

<!-- </ol> -->

## Loading data from files ##

**Prerequisites** The file must exist as an asset in your project\. For details, see [Adding a file from your local system](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/load-and-access-data.html?context=cdpaas&locale=en#add-file-local) or [Loading a data set from the Samples](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/load-and-access-data.html?context=cdpaas&locale=en#loadcomm)\.

To load data from a project file to your notebook:

<!-- <ol> -->

1.  Open your notebook in edit mode\.
2.  Click the **Code snippets** icon (![the Code snippets icon](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/images/code-snippets-icon.png)), click **Read data**, and then select the data file from your project\. If you want to change your selection, use **Edit** icon\.
3.  From the **Load as** drop\-down list, select the load option that you prefer\. If you select **Credentials**, only file access credentials will be generated\. For details, see [Adding credentials](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/load-and-access-data.html?context=cdpaas&locale=en#adding-creds)\.
4.  Click in an empty code cell in your notebook and then click **Insert code to cell** to insert the generated code\. Alternatively, click to copy the generated code to the clipboard and then paste the code into your notebook\.

<!-- </ol> -->

The generated code serves as a quick start to begin working with a data set\. For production systems, carefully review the inserted code to determine whether to write your own code that better meets your needs\.

To learn which data structures are generated for which notebook language and data format, see [Data load support](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/data-load-support.html#file-types)\.

## Loading data from data source connections ##

**Prerequisites** Before you can load data from an IBM data service or from an external data source, you must create or add a connection to your project\. See [Adding connections to projects](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/create-conn.html)\.

To load data from an existing data source connection into a data structure in your notebook:

<!-- <ol> -->

1.  Open your notebook in edit mode\.
2.  Click the **Code snippets** icon (![the Code snippets icon](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/images/code-snippets-icon.png)), click **Read data**, and then select the data source connection from your project\.
3.  Select the schema and choose a table\. If you want to change your selection, use **Edit** icon\.
4.  Select the load option\. If you select **Credentials**, only metadata will be generated\. For details, see [Adding credentials](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/load-and-access-data.html?context=cdpaas&locale=en#adding-creds)\.
5.  Click in an empty code cell in your notebook and then insert code to the cell\. Alternatively, click to copy the generated code to the clipboard and then paste the code into your notebook\.
6.  If necessary, enter your personal credentials for locked data connections that are marked with a key icon (![the key symbol for connections with personal credentials](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/images/privatekey.png))\. This is a one\-time step that permanently unlocks the connection for you\. After you unlock the connection, the key icon is no longer displayed\. For more information, see [Adding connections to projects](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/create-conn.html)\.

<!-- </ol> -->

The generated code serves as a quick start to begin working with a connection\. For production systems, carefully review the inserted code to determine whether to write your own code that better meets your needs\.

To learn which data structures are generated for which notebook language and data format, see [Data load support](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/data-load-support.html#file-types)\.

## Adding credentials ##

You can generate your own code to access the file located in your IBM Cloud Object Storage or a file accessible through a connection\. This is useful when, for example, your file format is not supported by the snippet generation tool\. With the credentials, you can write your own code to load the data into a data structure in a notebook cell\.

To add the credentials:

<!-- <ol> -->

1.  Click the **Code snippets** icon (![the Code snippets icon](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/images/code-snippets-icon.png)) and then click **Read data**\.
2.  Click in an empty code cell in your notebook, select **Credentials** as the load option, and then load the credentials to the cell\. You can also click to copy the credentials to the clipboard and then paste them into your notebook\.
3.  Insert your credentials into the code in your notebook to access the data\. For example, see this code in a [blog for Python](https://medium.com/ibm-data-science-experience/working-with-ibm-cloud-object-storage-in-python-fe0ba8667d5f)\.

<!-- </ol> -->

## Use an API function or an operating system command to access the data ##

You can use API functions or operating system commands in your notebook to access data, for example, the `wget` command to access data by using the HTTP, HTTPS or FTP protocols\. When you use these types of API functions and commands, you must include code that sets the project access token\. See [Manually add the project access token](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/token.html)\.

For reference information about the API, see [Watson Data API](https://cloud.ibm.com/apidocs/watson-data-api)\.

**Parent topic:**[Notebooks and scripts](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/notebooks-and-scripts.html)

<!-- </article "role="article" "> -->