Hooray, I’m an AWS Certified Pro Architect. So what?
By: Kris Peeters
Tags: AWS, Data, Certification

You’ll learn a ton about AWS

The Pro exam actually covers only about 20% of the services in AWS. So you can safely ignore things like AWS Device Farm, AWS Robomaker, AWS EventBridge, AWS Sumerian, AWS Lumberyard, … . But get ready to see every nook and cranny of EC2, ELB, Autoscaling Groups, VPC, Security Groups, NACLs, VGW, DirectConnect, EBS, IAM, S3, SQS, Lambda, Cloudfront, DynamoDB, Redshift, Fargate, Storage Gateway, …

If you work on AWS for a while, you’ve come across most things mentioned above. But I always approached it on a “need-to-know” basis. Now it was time to really get to know these services. And it gave me a much richer understanding of AWS. Here are my 3 main lessons learned. It might be trivial for some, but here it goes:

1. Federated authentication is really powerful. Although IAM allows you to create your own users, it is often much better to use an external identity provider such as Okta, Auth0 or even Active Directory. User management is really basic in AWS and in any reasonably large environment, you want to manage those centrally somewhere. Even for a small organisation such as Data Minded, we are going to look at an identity provider. As we are on Google Apps for Business, maybe Google is good enough. Although I’ve seen Okta being used at clients, and I’ve been pretty impressed with its capabilities. With Federated Authentication, a user is linked to an IAM Role, and that role will get access to certain services. Just like it’s best practice to create roles for services to talk to each other. You can ask AWS STS to generate temporary credentials for you, should you need them. If you use simple IAM users by contrast, you will get permanent credentials that you usually store in a ~/.aws/credentials file. Feels so dirty compared to having a central login system. I’ve always been avoiding the topic because it looked complex. But really, you only need to learn the flow once. And once you understand the flow, it’s relatively simple.

2. There is actually plenty of support for hybrid cloud. I’ll confess, I’ve been frustrated by clients who didn’t want to go ALL IN on cloud after we’ve done a 2 week Proof-of-Concept. Silly, right? But that’s not reality. Most IT departments are understaffed and overworked. Having fancy consultants coming in and asking for extensions of their network to the cloud, and access to a bunch of production databases, which can break if you look at them in the wrong way, is really not something on top of their mind. I don’t blame them. IT infra is a department judged by how stable they can run the things that are built by a department (dev) judged by how fast they can ship things. Services like Storage Gateway enable you to gradually move data to the cloud, while at the same time, offering a reliable backup service. RDS databases can be extended to the cloud with read-replicas, and over time, you can make that read-replica the master. You can import VMs to EC2 and manage them through Systems Manager. Slowly, you can move to managed services. You do need a reliable connection. Ideally a DirectConnect with a redundant line in case the DirectConnect fails. And the BGP protocol can magically route traffic over this redundant setup. Pretty cool. Although, honestly, I’ve never touched any of these techs in real life, so what do I know?

3. RPO and RTO are a fact of life. Deal with it. I never really knew the difference between a Multi-AZ RDS or an RDS with read-replicas. Well a Multi-AZ RDS is for high-availability. An entire AZ can go down and you will lose no data or availability. Read replicas on the other hand, are asynchronous. They sync data from the master to the replica. If the master dies, you can promote a read replica to become the master. But… there’s a but. You might lose data. There is a last point at which you received data. And then a bit later you crashed. Then you realise you crashed and you take time to recover. The RPO is the time between the last data receive and the crash. And the RTO is the time between the crash and the recovery. This is relevant for read-replicas, but also for snapshots, and basically anything that can crash. So yes, also your Multi-AZ RDS database, when you corrupt the data because of a bug in your code, or in case of region outages. These are very unlikely events, but not impossible. Remember when S3 was out last year? Reality is that disasters do occur. That’s ok. You just need to have that conversation with business about what their expectations are, and what your plan is for disaster recovery.