AzureFunBytes Episode 44 - @Azure Data Factory Security with @narainabhishek
Published: 2021-05-20 19:10:25

AzureFunBytes is a weekly opportunity to learn more about the fundamentals and foundations that make up Azure. It's a chance for me to understand more about what people across the Azure organization do and how they do it. Every week we get together at 11 AM Pacific on [Microsoft LearnTV](https://cda.ms/226) and learn more about Azure.

![AzureFunBytes animation](https://res.cloudinary.com/practicaldev/image/fetch/s--Z7BxBMz1--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_66%2Cw_880/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/j2xzw2g664tj31jij13t.gif)

This is part two of our series on [Azure Data Factory](https://cda.ms/28h).  [Last time Mark helped get us on the road to understanding how to best get our data into the cloud](https://dev.to/azure/azurefunbytes-episode-43-intro-to-azure-data-factory-with-kromerbigdata-4pa8) by using the linked services and tools with Azure Data Factory. Data Factory contains a series of interconnected systems that provide a complete end-to-end platform for data engineers. Utilizing our data requires some thoughtfulness when it comes to security.  

This week on AzureFunBytes, Senior Program Manager, [Abishek Narain](https://twitter.com/narainabhishek) joins me to learn more about security best practices for Data Engineers.  

Our Agenda:

- Authentication
- Meta-data encryption (Meta-data at rest)
- Credential management 
- Data in transit
- Data at rest
- Network Security
- Azure Policy integration

With all of this in mind, we take a deeper look at how security not only impacts our applications deployed, but the data they may access and store.

{% youtube QaqLaUS1Siw %}

[00:04:46 - Let's meet Abishek](https://youtu.be/QaqLaUS1Siw?t=286)
[00:09:13 - Azure Data Factory: Best practices for data engineers](https://youtu.be/QaqLaUS1Siw?t=553)
[00:14:49 - Authentication](https://youtu.be/QaqLaUS1Siw?t=889)
[00:21:53 - Meta-data encryption (at rest)](https://youtu.be/QaqLaUS1Siw?t=1312)
[00:25:10 - Credential management](https://youtu.be/QaqLaUS1Siw?t=1510)
[00:35:57- Data in transit & data at rest](https://youtu.be/QaqLaUS1Siw?t=2157)
[00:38:33 - Network security](https://youtu.be/QaqLaUS1Siw?t=2313)
[00:43:58 - Custom RBAC roles](https://youtu.be/QaqLaUS1Siw?t=2638)
[00:46:23 - Demo Time!](https://youtu.be/QaqLaUS1Siw?t=2783)

When securing your data pipeline, there are some configurations and settings suggested by Azure you should follow.  You'll want to follow this security baseline that applies guidance from the [Azure Security Benchmark version1.0](https://docs.microsoft.com/en-us/azure/security/benchmarks/overview-v1) to Azure Data Factory. You'll also want to ensure login details to your data endpoints are protected to ensure there's no unauthorized access due to credentials existing in the wild.  There are also network security considerations that you will want to adhere to for various data stores that are accessed by Azure Data Factory, whether they are in the cloud or on-prem.  

------

Learn about Azure fundamentals with me!

Live stream is available on Twitch, YouTube, and LearnTV at 11 AM PT / 2 PM ET Thursday. You can also find the recordings here as well:

[AzureFunBytes on Twitch](https://twitch.tv/azurefunbytes)
[AzureFunBytes on YouTube](https://aka.ms/jaygordononyoutube)
[Azure DevOps YouTube Channel](https://www.youtube.com/channel/UC-ikyViYMM69joIAv7dlMsA)
[Follow AzureFunBytes on Twitter](https://twitter.com/azurefunbytes)

Useful Docs:

[AzureFunBytes Episode 43 - Intro to @Azure Data Factory with @KromerBigData](https://dev.to/azure/azurefunbytes-episode-43-intro-to-azure-data-factory-with-kromerbigdata-4pa8)
[Get $200 in free Azure Credit](https://cda.ms/219) 
[Microsoft Learn: Introduction to Azure fundamentals](https://cda.ms/243)
[Azure security baseline for Azure Data Factory](https://cda.ms/28d)
[Security considerations for data movement in Azure Data Factory](https://cda.ms/28f)
[What is Data Factory?](https://cda.ms/28h)
[Data access strategies](https://cda.ms/28j)
[Azure Key Vault secrets in pipeline activities](https://cda.ms/28k)
[Azure Policy documentation](https://cda.ms/28m)
[Compute environments supported by Azure Data Factory](https://cda.ms/28p)
[Managed identity for Data Factory](https://cda.ms/28y)
[RBAC - Role Based Authentication Control](https://cda.ms/28z)