Unlocking the Data Universe: Embarking on a Data Engineer Journey
Published: 2023-07-25 16:51:49

---
title: Unlocking the Data Universe: Embarking on a Data Engineer Journey
published: true
date: 2023-07-21 23:43:20 UTC
tags: 
canonical_url: https://chrisparaiso.dev/unlocking-the-data-universe-embarking-on-a-data-engineer-journey
---

I've been in the software engineering field since 2009 with a focus on front-end development and UI/UX. This is funny to me because when I first started learning web development back in the early 2000s, I avoided JavaScript like the plague. I would only touch the HTML and CSS of the website. The design was all I cared about. Now, JavaScript is all I do.

## The Now

Currently, my team is contracted by [Cisco](https://www.cisco.com/) to convert and migrate their [Networking Academy](https://www.netacad.com/) application to replace all unsupported Flash assets into Canvas. Everything from localized SVG translations, animations, and simulations is being converted over.

We used JSFL in Adobe Animate to export what we could, and manually update and fix what we could not export. This and the effort to break out different pieces of course content for subject matter experts to build and/or modify a given course for their students is still ongoing.

What scripting language did we use to migrate this over? I want to say Python... But it was JavaScript. Leveraging Node we didn't need to context switch due to a change in languages since we were all front-end engineers. I wanted it to be Python though.

## The Obvious Choice

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1689919123460/ce31ec6e-c900-4138-8a29-fb5af87a4f22.jpeg)

I'm at a crossroads here. It's obvious which direction I'm going given the title of this blog post but if that wasn't clear, I'm planning to shift careers into data engineering. Why? The current project I'm on has me shifting data from one place to another to be consumed by subject matter experts as I've said before. This very action has inspired me to move and transform data on a **bigger** scale but with a different impact.

I'd like to help data scientists and ML engineers get the data that they need to work their magic or analyze the data in a way that helps either the business or the world.

Python has intrigued me since being introduced to it circa Python 3 but I've never had a chance or time to work in that language.

Now, I'm making time...

## The Future

I've set up a personal learning plan for myself because I wanted to be clear on what to study next. I tend to ponder too much on "what's next?" rather than what I'm learning at this very moment. The learning plan I've created is only a start and not the end all be all of data engineering. Of course, this plan isn't set in stone and will be modified if a course is subpar.

## My Data Engineering Learning Plan

My company offers "professional development" funding every year and I've opted to subscribe to Datacamp and Udemy to learn data engineering as well as a single book for now.

### Broad Topic

- [DataCamp Data Engineer Track](https://www.datacamp.com/)

- [Data Warehouse Fundamentals for Beginners](https://www.udemy.com/course/data-warehouse-fundamentals-for-beginners/)

- [_Fundamentals of Data Engineering_](https://www.amazon.com/Fundamentals-Data-Engineering-Robust-Systems/dp/1098108302/)

The first two list items above are courses that cover a wide range of topics. The third in this list is a book I see everywhere and have been enjoying the read. Datacamp's teaching style doesn't match the way my brain learns. I see inconsistencies in teaching methods and exercises. I'll still complete this course, but that's my high-level review of Datacamp.

### Programming Languages

- [The Complete Python Bootcamp From Zero to Hero in Python](https://www.udemy.com/course/complete-python-bootcamp/)

- [The Complete SQL Bootcamp: Go from Zero to Hero](https://www.udemy.com/course/the-complete-sql-bootcamp/)

- [Python for Data Structures, Algorithms, and Interviews!](https://www.udemy.com/course/python-for-data-structures-algorithms-and-interviews/)

Python & SQL are, from what I've researched thus far, required as a data engineer.

Python is such a beautiful language. I've worked with it before, but don't feel confident to call myself proficient. I'll be changing that after the courses above are complete.

SQL joins make sense. Complex SQL joins hurt my brain to read. Hopefully, this course on SQL will help with that.

### Technologies

- [Taming Big Data with Apache Spark and Python - Hands On!](https://www.udemy.com/course/taming-big-data-with-apache-spark-hands-on/)

- [Azure Databricks & Spark For Data Engineers (PySpark / SQL)](https://www.udemy.com/course/azure-databricks-spark-core-for-data-engineers/)

- [The Complete Hands-On Introduction to Apache Airflow](https://www.udemy.com/course/the-complete-hands-on-course-to-master-apache-airflow/)

- [Apache Kafka Series - Learn Apache Kafka for Beginners v3](https://www.udemy.com/course/apache-kafka/)

- [Snowflake[A-Z] Zero to Hero Masterclass(Core+SQL+API+Visual)](https://www.udemy.com/course/snowflake-zero-to-hero-masterclass/)

- [Learn DevOps: CI/CD with Jenkins using Pipelines and Docker](https://www.udemy.com/course/learn-devops-ci-cd-with-jenkins-using-pipelines-and-docker/)

These are courses on topics I've researched that a data engineer should know. I can't speak on these technologies yet, but these courses are highly reviewed.

### Certifications

- [[AWS] Ultimate AWS Certified Cloud Practitioner - 2023](https://www.udemy.com/course/aws-certified-cloud-practitioner-new/)

- [[AWS] Ultimate AWS Certified Solutions Architect Associate SAA-C03](https://www.udemy.com/course/aws-certified-solutions-architect-associate-saa-c03/)

My company is an AWS shop. Speaking with my team lead, he has stated this would be a good starting point. But I feel I need some knowledge of how these pieces work together... maybe the courses cover that? Regardless, I'll be taking this course, the Python course and reading my _Fundamentals of Data Engineering_ book in parallel.

### Practice

- [Leetcode](https://leetcode.com/problemset/all/)

- [HackerRank](https://www.hackerrank.com/domains/sql)

Who doesn't need practice? I like puzzles. These links will scratch that part of my brain that likes puzzles.

## From here on out...

All in all, I'm pretty excited. I haven't been this excited in a while. After years of saying, "Oh, another JavaScript library/framework to learn." I can say, "Oh, a new tool that may be better than the current one." :shrug:

Off to learning!