Azure Data Factory LAB Copy CSV
Published: 2023-06-13 19:12:28


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7l65fb8up9mf31w0haiy.png)

Azure Data Factory is the platform that solves such data scenarios. It is the cloud-based ETL and data integration service that allows you to create data-driven workflows for orchestrating data movement and transforming data at scale. Using Azure Data Factory, you can create and schedule data-driven workflows (called pipelines) that can ingest data from disparate data stores. You can build complex ETL processes that transform data visually with data flows or by using compute services such as Azure HDInsight Hadoop, Azure Databricks, and Azure SQL Database.

## Lab ADF Copy Data from CSV to CSV


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/m9c3e1h0by2kj8btfzz4.png)


## 1 STEP
Create a resource group: LAB01_ADF_COPYING_DATA_FROM_CSV_TO_CSV

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ytf9t31o500aa7t4b56u.png)

## 2 STEP
Create a storage account (blob storage)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eq28v9pl7dg7y7x9ktxe.png)

Redundancy: Locally-redundant storage (LRS)

## 3 STEP
Create a storage account (data lake)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8cxw2anr5rile9foqb2g.png)

Enable hierarchical namespace

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jukiuz5bo18dla8tvztl.png)

## 4 STEP

Inside blob storage (moviesblobst) add container 'bankmovies'

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/z5db49pl4lgfubuojiml.png)

and after that upload file csv 


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t79w5d7778a38ayc09jr.png)

## 5 STEP 

Config data lake


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9l64rr03b1m73zx2mclc.png)

**Linked services**

Linked services are much like connection strings, which define the connection information needed for the service to connect to external resources.

Is time to create a Data Factory, here we go.

## 6 STEP

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tvppqts66y0lds5gjtpj.png)

Name: LAB01ADF01

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/smqk0t735giv0qmrao23.png)

Launch studio

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9p7z0ww9nbzulwpxll50.png)

## 7 STEP

Create new Linked service to Blob Storage


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8w1bx23xl81ax81xx4uo.png)

Name: ls_blob_moviesblobst

Storage account name: moviesblobst
and test connection 


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/crkqc8qjb0o00sb5s7sf.png)

## 8 STEP

Create new Linked service to Data Lake


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7opix0w0tgal95ha32wp.png)

Name: ls_dl_moviesdatalakee
Storage account name: moviesdatalakee
and test connection 


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dftei1x5c589i1qkg6tw.png)

## 9 STEP 

Create 2 dataset origin and sink

1. Dataset Blob Storage

Dataset > New dataset > Azure Blob Storage > DelimitedText (CSV)

Name: ds_movies_bank_row_bs
Linked service: ls_blob_moviesblobst


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ikemkohwhr7st780rdlv.png)

File path you can clic in preview data

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ivbolmqeprx3jahyb279.png)

2. Dataset Data Lake

Dataset > New dataset > Azure Data Lake Storage Gen2 > DelimitedText (CSV)


Name: ds_movies_bank_raw_dl
Linked service: ls_dl_moviesdatalakee


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zoxpdyi0soe11eammze0.png)

Validate all and publish all


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8ck1t80fuuw0o4tmrl8l.png)

## 10 STEP

Generate new pipeline, name is pl_ingestion_movies_data

Activities: Move & transform > Copy data

Copy data movies

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rp3gjblcjt2pee6gyzj7.png)

Source dataset: ds_movies_bank_row_bs
Sink dataset: ds_movies_bank_raw_dl

Validate check
Debug


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qvo7qoui54227wn4j4iv.png)


Go to data lake and look the file csv with data


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/m8i49nj1y8i54knjqzz8.png)

[Source ADF](https://azure.microsoft.com/en-us/products/data-factory)

**Thanks for taking your time to read this post.**