Building the Right Platform
Tags: devops, observability, monitoring, productivity
Published: 2021-10-16 10:32:28

In mid-level and big scale organizations, the mission focus is on having continuous innovation backed by stability. The former is required to stay ahead of the curve (*being better than one's competitors*) while the latter is the springboard to delivery. Thus, a strong **platform** is an apparent need for every technological company.

Thus, we land upon a concept called Platform Engineering.

### So what's it actually?
> As per [this definition](https://www.limepoint.com/platform-engineering)
_Platform engineering is the process through which enterprises adopt (*new technology and platforms*), leverage (*existing technologies and platforms*), and transform (*shift the dial on delivering value by transforming the way things are done*) cloud platforms. It is at the core of designing, building, and operating your cloud infrastructure to deliver the next generation IT ecosystem._

While the above definition is accurate, it does not really help understand on the why, what & how to build reliable platforms. Especially for teams who are looking to either augment or revamp their infrastructure from old ways to new.

Therefore, I felt I could contribute by creating one from my know-how so far. For easier understanding, the categories herewith will be structured into the following parameters: why, some points to consider, alternatives.

![begin-blog](https://memegenerator.net/img/instances/70467226/alright-now-thats-settled-lets-begin-shall-we.jpg)

To build a smooth platform, most e-commerce companies need tooling/tech/processes/workflows in the following categories.

> 
1. [Domains](#domains)
1. [Cloud Platform](#cloud-platform)
1. [Infrastructure as Code](#infrastructure-as-code)
1. [Version Control](#version-control)
1. [Packaging](#packaging)
1. [CI/CD](#cicd)
1. [Web Services](#web-services)
1. [Databases](#databases)
1. [Monitoring](#monitoring)

## Domains
![Right Domain Name](https://i2.wp.com/thisonlineworld.com/wp-content/uploads/2018/07/domain-name-meme-232x300.jpg)

**Why?:** Branding of one's product, everything on the internet begins with a domain. E.g: google.com, facebook.com, dev.to
**Some points to consider?:**
+ Domain pricing / renewals (_with ICANN fees_)
+ Whois Privacy
+ The right name and [Top Level Domain or TLD](https://www.techopedia.com/definition/1348/top-level-domain-tld) or [country-level TLD](https://www.techopedia.com/definition/1323/country-code-top-level-domain-cctld) if it's region specific. (_Impacts SEO_)

**Known tools:** [Bigrock.in](https://bigrock.in)
**Alternatives:** [Namecheap](https://www.namecheap.com), [Route53](https://aws.amazon.com/route53), [Cloudflare](https://www.cloudflare.com/products/registrar), [Google Registry](https://www.registry.google)


## Cloud Platform
![Infrastucture Diagram](https://egy4sgsh0bgexn13si3j24nh-wpengine.netdna-ssl.com/wp-content/uploads/2021/02/what-is-cloud-computing.png)

**Why?:** Gateway to the system, having the right infrastructure will be like a strong backbone.
**Some points to consider?:** 
+ Basic features such as instances/VMs, isolated networks (_not everyone will need Kubernetes or Mesos_)
+ Sufficient Capacity for one's enterprise at scale (_The last thing one wants in the middle of a frantic user rush is no hardware available_)
+ Multiple accounts with RBAC to distinguish between teams/environments.
+ Compatibility with [Infrastructure-as-Code](https://www.ibm.com/cloud/learn/infrastructure-as-code) tools
+ Effective Pricing.

**Known tools:** [Amazon Web Services (AWS)](https://aws.amazon.com/)
**Alternatives:** [Google Cloud Platform](https://cloud.google.com), [Microsoft Azure Cloud](https://azure.microsoft.com/en-in), [DigitalOcean](https://www.digitalocean.com), [Linode](https://www.linode.com)


## Infrastructure as Code
![code](https://www.hashroot.com/assets/img/services/devops/infrastructure-as-a-code.jpg)
**Why?:** Codifying the infrastructure provisioning and configuration to keep it exact across environments.
**Some points to consider?:**
+ Idempotency by changing/deleting infrastructure only once.
+ Ability to store state locally & remotely.
+ Should be vender neutral & serve as a crisp log for audit

**Known tools:** [Terraform](https://developer.hashicorp.com/terraform)
**Alternatives:** [Pulumi](https://www.pulumi.com/), [AWS Cloudformation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/Welcome.html), [Google Deployment Manager](https://cloud.google.com/deployment-manager/docs), [Azure Resource Manager](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/overview)

## Version Control
![Version Control](https://webinerds.com/app/uploads/2015/10/A-Brief-Timeline-of-Version-Control-Systems-03-770.png)

**Why?:** Managing codebases with multiple teams across varied projects in different business verticals.
**Some points to consider?:**
+ De-centralized (_avoids a single point of failure_)
+ Clear history log (_for updates/rollback_)
+ Branching/separating strategies.
+ Ease of setup

**Known tools:** Git
**Alternatives:** Mercurial, TFS, SVN

## Packaging
![Application Packages](https://cdn-media-1.freecodecamp.org/images/binzfjGViQTJIpQs3d9UUIxsgl62HClcJOac)

**Why?:** Software should be same across environments. Helps in the CI/CD step and reproduce issues to be fixed quicker
**Some points to consider?:**
+ Application storage size (_No point in having GB sized containers_)
+ Ability to generate artifacts (if any)

**Known tools:** Docker
**Alternatives:** Containerd, LXC, APT, Yum


## CI/CD
![ci-cd-flow](https://miro.medium.com/max/700/1*7k5Z3-gJaqrY7YFxQx509g.png)

**Why?:** To compile/build and deploy software easily. Manual steps can be time-consuming, error-prone and be a bottleneck.
**Some points to consider?:**
+ Ability to build/deploy multiple programming languages, frameworks
+ Support for unit/smoke tests, canary deployments.
+ Preferably independent than infrastructure (_helps avoid vendor lock-in_)
+ Self-hosted vs SaaS (factors here are cost, upgrades and maintenance)

**Known tools:** [Jenkins](https://www.jenkins.io)
**Alternatives:** [Github Actions](https://github.com/features/actions), [Gitlab](https://docs.gitlab.com/ee/ci) , [CircleCI](https://circleci.com), [TravisCI](https://travis-ci.org), [ArgoCD](https://argoproj.github.io/cd)


## Web-Services
![web-entrypoint](https://kinsta.com/wp-content/uploads/2020/08/How-a-Reverse-Proxy-Server-Works.png)

**Why?:** Helps your userbase access all the consumer endpoints of the system. For e.g: discover, checkout, payments
**Some points to consider?:**
+ Scale management (_An incorrectly setup web-service can quickly have a cascading effect in case of heavy traffic_)
+ Reverse proxying to one/multiple back-ends and fail-overs (_for High Availability_)
+ Support for HTTP/GRPC protocols.
+ Advanced features such as rate-limits, throttling, IP bans.

**Known tools:** HAProxy
**Alternatives:** NGinx, Apache, Ambassador, Envoy, Istio


## Databases
![sample DB stucture](https://3s81si1s5ygj3mzby34dq6qf-wpengine.netdna-ssl.com/wp-content/uploads/2018/09/tigergraph-database-compare.jpg)

**Why?:** To maintain state of applications and store all data such as inventory, payments and other types of relational information. Helpful for data analytics as well.
**Some points to consider?:**
+ ACID Compliance
+ Libraries/connectors for multiple programming languages.
+ Storage parameters such as in-memory or persistent to disk.
+ Ease of backup, recovery in case of DR drills.
+ Key-value stores for relatively smaller datasets.

**Known tools:** MySQL or MariaDB
**Alternatives:** PostgreSQL, MSSQL, MongoDB, Kafka, Aerospike, Redis, Memcached

## Monitoring
![Monitoring system](https://media.springernature.com/original/springer-static/image/chp%3A10.1007%2F978-1-4842-6888-9_3/MediaObjects/497207_1_En_3_Fig1_HTML.jpg)

**Why?:** A small monitored system will be more reliable than a big one without.
**Some points to consider?:**
+ Agents/libraries across multiple programming languages.
+ Ability to view application/infrastructure metrics.
+ Retention as per compliance.

**Known tools:** [NewRelic](https://newrelic.com)
**Alternatives:** [Datadog](https://www.datadoghq.com), [AppSignal](https://appsignal.com), [Instana](https://www.instana.com), [ElasticAPM](https://www.elastic.co/apm)

*TL;DR: While this is not a definitive list of things that's required to build a system but there's a good chance this blog post provides an entrypoint for you to begin with*

---

What are some pointers you would consider to build a reliable, consistent and easy-to-use platform?