When you run after the horse in the rabbit whole
Tags: python, redis, mysql, docker
Published: 2023-08-01 15:41:32




Welcome dear reader, I am Silent-Mobius, AKA Alex M. Schapelle, your faithful [servitor](https://warhammer40k.fandom.com/wiki/Servitor) that continues to lead with examples of learning of new topics with trials and errors.

As mentioned in our [previous encounter](https://dev.to/otomato_io/learn-redis-or-how-devops-had-to-do-back-end-work-4fi), I have started to dwell into world of databases, by exploring two main types which are NoSQL and RDBMS, mainly talking about MySQL and Redis. About why I am doing this, we'll explore later, as on this  entry we'll mostly discuss the architecture as mentioned before  and according to table of content: 

| Learn Redis With Project|Description| Roadmap Status |
|--------------|----------------------|----------------|
| Project      | Create architecture  | We are Here |
| Application  | Create application and Version control ||
| Database     | Learn basics of Redis and implement with application ||
| Micro Services | Set all local development env with docker-compose || 
| Migration to K8S | Setup k8s manifests and generate helm chart ||

Let create a small diagram that will illustrate what we are going to create:


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7rhb9dxhhqvqyy0fkli7.png)


We'll be using docker containers to pack our application, and deploy the packed images into container registry. Within Gilab-CI we'll work on our code, and in case it will pass, it will start new pipeline that will deploy on test network with docker compose the scaled application, Redis-cluster and PostgreSQL-cluster.

There will be configuration files for Redis to store data  with SQL server and in term, SQL server with be saving data to volume that will be saved under backed up volume... it seems there will be a lot of configuration to be done, either with `Python`, `Bash` or `Ansible`.

The architecture is inspired by [Alex Xu](https://twitter.com/alexxubyte) book named [System Design Interview](https://www.amazon.com/gp/product/B08CMF2CQF/ref=as_li_qf_asin_il_tl?ie=UTF8&tag=sandordargo-20&creative=9325&linkCode=as2&creativeASIN=B08CMF2CQF&linkId=6bd98a7b06a1165368754af945bae169)


As we can see there are a lot of things to setup, create, manage, provide and deploy, thus here is a list of things that I am going to use:

- Language: `Python`, mostly due to the development can be fast and in this case I'd prefer to focus on learning DB and Cache part more then learning new programming language.
- Version Control: `Git`, the application is not going to be that big, in addition today git is the de facto the best choice for code version control, which is also incorporated within CI
- CI : `Gitlab-CI`, Because my github account got locked and the support suggested to destroy the account but not open it for me, thus they lost me as a customer.
- RDBMS: `PostGreSQL`, mostly to master the automation of main and secondary installation with `Python` as well as auto-database setup or connection with new instance.
- NoSQL: `Redis`, to learn the caching mechanism provided by Alex's book
- Micro-Services: `Docker` and `Docker Compose`, for only development purposes, eventually we'll create k8s deployment

Although I've been working as IT professional for 15 years, and have been working on these systems separately, It never occurred to me that I need to connect them by myself, mostly because we used to work with waterfall methodology and guilt, along side new application version was dropped down at us to handle by development team. Our Dev team were cool guys and were always there to help, but lack of knowledge and experience of high-level programming languages made it very hard to determine to whom we should consult for aid.
When idea to learn *enough programming to do tasks popped up*, though it was there for some time, with it came uncertain choice: 
- `Perl`
- `Ruby`
- `Python`
- `Node.js`

 Which one to use ?

I chose `Ruby` which in long run, was somewhat incorrect choice, and moving to python from it was not hard but a very long and confusing journey.
Today I stumble to same dilemma, whether to use `Go` or `Rust`, to which I've chosen `Go`... but this is for another article.

All the other automation I've done, was mostly with `shell` or `Bash`, with Linux Built-in tools, which eventually turned into `Cloud-init`, `Puppet and `Ansible`, thus burdening myself with SQL or NoSQL, was never the issue for me, to what I somewhat regret.
Thus this is where I start to learn ... and hope you will learn with me. No matter what, Please Do Try To Enjoy 

Till next time