Managing Data Refinery flows
Managing Data Refinery flows A Data Refinery flow is an ordered set of steps to cleanse, shape, and enhance data. As you [refine your data](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/refining_data.htmlrefine) by [applying operations](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/gui_operations.html) to a data set, you dynamically build a customized Data Refinery flow that you can modify in real time and save for future use. These are actions that you can do while you refine your data: Working with the Data Refinery flow * [Save a Data Refinery flow](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/data_flows.html?context=cdpaas&locale=ensave) * [Run or schedule a job for Data Refinery flow](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/data_flows.html?context=cdpaas&locale=enjobs) * [Rename a Data Refinery flow](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/data_flows.html?context=cdpaas&locale=enrename) Steps * [Undo or redo a step](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/data_flows.html?context=cdpaas&locale=enundo) * [Edit, duplicate, insert, or delete a step](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/data_flows.html?context=cdpaas&locale=enedit-duplicate) * [View the Data Refinery flow steps in a "snapshot view"](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/data_flows.html?context=cdpaas&locale=ensnapshot) * [Export the Data Refinery flow data to a CSV file](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/data_flows.html?context=cdpaas&locale=enexport) Working with the data sets * [Change the source of a Data Refinery flow](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/data_flows.html?context=cdpaas&locale=enchange) * [Edit the sample size](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/data_flows.html?context=cdpaas&locale=ensample) * [Edit the source properties ](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/data_flows.html?context=cdpaas&locale=enedit-source) * [Change the target of a Data Refinery flow](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/data_flows.html?context=cdpaas&locale=enoutput) * [Edit the target properties](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/data_flows.html?context=cdpaas&locale=enedit-target) * [Change the name of the Data Refinery flow target](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/data_flows.html?context=cdpaas&locale=enchange-name) Actions on the project page * [Reopen a Data Refinery flow to continue working](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/data_flows.html?context=cdpaas&locale=enreopen) * [Duplicate a Data Refinery flow](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/data_flows.html?context=cdpaas&locale=enclone) * [Delete a Data Refinery flow](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/data_flows.html?context=cdpaas&locale=enremove) * [Promote a Data Refinery flow to a space](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/data_flows.html?context=cdpaas&locale=enpromote) Working with the Data Refinery flow Save a Data Refinery flow Save a Data Refinery flow by clicking the Save Data Refinery flow icon ![Save icon](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/images/save.png) in the Data Refinery toolbar. Data Refinery flows are saved to the project that you're working in. Save a Data Refinery flow so that you can continue refining a data set later. The default output of the Data Refinery flow is saved as a data asset source-file-name_shaped.csv. For example, if the source file is mydata.csv, the default name and output for the Data Refinery flow is mydata_csv_shaped. You can edit the name and add an extension by [changing the target of a Data Refinery flow](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/data_flows.html?context=cdpaas&locale=enoutput). Run or schedule a job for a Data Refinery flow Data Refinery supports large data sets, which can be time-consuming and unwieldy to refine. So that you can work quickly and efficiently, Data Refinery operates on a sample subset of rows in the data set. The sample size is 1 MB or 10,000 rows, whichever comes first. When you run a job for the Data Refinery flow, the entire data set is processed. When you run the job, you select the runtime and you can add a one-time or repeating schedule. In Data Refinery, from the Data Refinery toolbar click the Jobs icon ![the run or schedule a job icon](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/images/Run-schedule_Blue.png), and then select Save and create a job or Save and view jobs. After you save a Data Refinery flow, you can also create a job for it from the Project page. Go to the Assets tab, select the Data Refinery flow, choose New job from the overflow menu (![overflow menu](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/images/actions.png)). You must have the Admin or Editor role to view the job details or to edit or run the job. With the Viewer role for the project, you can view only the job details. For more information about jobs, see [Creating jobs in Data Refinery](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/create-job-dr.html). Rename a Data Refinery flow On the Data Refinery toolbar, open the Info pane ![info icon](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/images/info-pane.png). Or open the Flow settings ![settings icon](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/images/settings.png) and go to the General tab. Steps Undo or redo a step Click the undo (![undo icon](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/images/undo.png)) icon or the redo (![redo icon](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/images/redo.png)) icon on the toolbar. Edit, duplicate, insert, or delete a step In the Steps pane, click the overflow menu (![overflow menu](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/images/actions.png)) on the step for the operation that you want to change. Select the action (Edit, Duplicate, Insert step before, Insert step after, or Delete). * If you select Edit, Data Refinery goes into edit mode and either displays the operation to be edited on the command line or in the Operation pane. Apply the edited operation. * If you select Duplicate, the duplicated step is inserted after the selected step. Note:The Duplicate action is not available for the Join or Union operations. Data Refinery updates the Data Refinery flow to reflect the changes and reruns all the operations. View the Data Refinery flow steps in a "snapshot view" To see what your data looked like at any point in time, click a previous step to put Data Refinery into snapshot view. For example, if you click Data source, you see what your data looked like before you started refining it. Click any operation step to see what your data looked like after that operation was applied. To leave snapshot view, click Viewing step x of y or click the same step that you selected to get into snapshot view. Export the Data Refinery flow data to a CSV file Click Export (![export icon](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/images/export.png))