Migrating from Oracle 2 Snowflake: Part 4
By: Rajiv Gupta
Tags: Oracle, Guides And Tutorials, Snowflake, Migration, Snow

This blog is in continuation of my previous blog Migrating from Oracle 2 Snowflake: Part 3, I would like to continue sharing further my experience migrating from Oracle to Snowflake.

Note: This is more to give an idea on what challenges one can face during Oracle to Snowflake migration. All suggestion & expression are my own, and it had no relation with my employer(past / present).

Now that we if you are following my previous 3 blogs now you had clear approach how we want to approach for Object & Data Migration.

Now it's time to set up our application flow for day to day processing. Since our application is depending on file incoming from source system on various interval, we are loosely couple from our source application. Even if we don't process incoming file for a couple of days, we can survive as we are not customer facing active application, rather our data are more used to make decision.

As we had proposed, the new architecture using Azure blog and Snowflake snowpipe for all incremental load. We created an input folder with event hub setup. On snowflake end we created Snowpipe which is listening to storage event trigger. We set up the file format, external stage, blob security and ready to ingest data the moment our ETL (Python) put something on Azure blob. Earlier PERL script was more used for file processing and SP calling as business logic was kept inside Oracle database. While migrating to Snowflake, we leverage the PYTHON programming language to push the business logic from database to ETL layer. As anyway, we are utilizing this logic via ETL layer, so we are not facing any difference in behavior.

All our code changes are part of Azure Repo GIT and all our CI/CD was set up using Azure Pipelines.

On Go live day, we put freeze on existing application processing, we already converted the table DDL beforehand and get this deployed in Snowflake using Azure pipelines. Mass data was loaded using our native approach. Data Validation was kicked off post data load. Once data is validated, we scheduled our ETL(Python) scripts to start ingesting data from source path and push that down to Azure blob storage.

And the magic started and thing started working as we had planned.

Now, this may feel simple when I am writing this blog or while you are reading this blog, but it is always challenging to segregate the required steps and keep in correct sequence. I must appreciate my team for working hard on this and helped me to deliver this program.

Hope this blog help you to get insight on how we can migrate from Oracle to Snowflake. Feel free to ask a question in the comment section if you have any doubts regarding this. Give a clap if you like the blog. Stay connected to see many more such cool stuff. Thanks for your support.

You Can Find me:

Follow me on Medium: https://rajivgupta780184.medium.com/

Follow me on Twitter: https://twitter.com/RAJIVGUPTA780

Connect with me in LinkedIn: https://www.linkedin.com/in/rajiv-gupta-618b0228/

Subscribe to my YouTube Channel: https://www.youtube.com/channel/UC8Fwkdf2d6-hnNvcrzovktg

#Keep learning #Keep Sharing #EverydayLearning.

References:-