My top picks of re:Invent 2023
Tags: aws, database, cloud, news
Published: 2023-12-03 04:56:18

[re:Invent 2023](https://reinvent.awsevents.com/) has been wrapped up. Before we start preparing for the 2024 edition 😅 let me recap the announcements I was most excited about. 

Here is my favorite list, in no particular order

## 1. Amazon ElastiCache Serverless

As architects, developers, we love to get into database scalability discussions and right-sizing our database clusters. I know I do, specially when it comes to Redis, one of my favorite databases! But, with the recently announced [serverless option for ElastiCache](https://aws.amazon.com/about-aws/whats-new/2023/11/amazon-elasticache-serverless/), we may not have to do all that. 

I know, sad but true, because you can now set up a new cache in under a minute without worrying about choosing a cache node type, number of shards, number of replicas, or node placements across AZs. 

> By the way, this is applicable for both [Redis](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/WhatIs.deployment.html) and [Memcached](https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/WhatIs.deployment-mem.html) engines supported by ElastiCache.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v7smspzi7tktxwttujso.png)

## 2. Amazon EKS Pod Identity

Who likes configuring [IRSA in EKS](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html)? I am not a huge fan! Thank god for [Amazon EKS Pod Identity](https://aws.amazon.com/about-aws/whats-new/2023/11/amazon-eks-pod-identity) that simplifies this experience!

I have already taken this for a spin (detailed blog post coming soon) and [you should too]()! Here is the [announcement](https://aws.amazon.com/about-aws/whats-new/2023/11/amazon-eks-pod-identity/), [AWS News blog](https://aws.amazon.com/blogs/aws/amazon-eks-pod-identity-simplifies-iam-permissions-for-applications-on-amazon-eks-clusters/) and most importantly, the [documentation](https://docs.aws.amazon.com/eks/latest/userguide/pod-identities.html).

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xux7p43y5rj422n64i4d.png)

## 3. Knowledge Bases in Amazon Bedrock

RAG (Retrieval Augmented Generation) is a powerful technique, but it takes significant effort to build data pipelines to support continuous ingestion from data sources to your vector databases. [Knowledge Bases in Amazon Bedrock](https://aws.amazon.com/about-aws/whats-new/2023/11/amazon-bedrock-knowledge-bases/) delivers a *fully managed* RAG experience. You can pick Amazon OpenSearch Serverless as the default vector database or choose from other available options. 

I took it for a spin and it was so much easier to have my datasets ready for QnA - configure the Knowledge Base, load data in Amazon S3, sync it, and you're ready to go.

Check out [this great blog post](https://aws.amazon.com/blogs/aws/knowledge-bases-now-delivers-fully-managed-rag-experience-in-amazon-bedrock/) and [refer to the documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base.html) once you're ready to give it a shot.

![](https://d2908q01vomqb2.cloudfront.net/da4b9237bacccdf19c0760cab7aec4a8359010b0/2023/11/21/2023-kb-01.png)

> P.S. More content coming soon, especially around how you can use this from your Go applications and some open-source integrations as well. Stay tuned!

There were a bunch of Vector database engines announced this re:Invent!

## 4. Vector search for Amazon MemoryDB for Redis (preview)

Now you can use familiar Redis APIs for machine learning and generative AI use cases such as retrieval-augmented generation, anomaly detection, document retrieval, and real-time recommendations. Amazon MemoryDB for Redis now [let's you store, index, and search vectors](https://aws.amazon.com/about-aws/whats-new/2023/11/vector-search-amazon-memorydb-redis-preview/). 

At the time of writing, it's in preview but I encourage you to try it out. Check out the documentation [for details](https://docs.aws.amazon.com/memorydb/latest/devguide/vector-search-overview.html) including the supported [vector search commands](https://docs.aws.amazon.com/memorydb/latest/devguide/vector-search-commands.html).

> Don't forget to activate vector search while creating a new MemoryDB cluster!

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yt90s1k2fhd4vsuaqbcs.png)

## 5. Vector engine for Amazon OpenSearch Serverless is now GA

Underneath the hood, [vector database capability](https://aws.amazon.com/about-aws/whats-new/2023/11/amazon-opensearch-serverless-vector-engine/) uses [k-NN search in OpenSearch](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/knn.html). As I mentioned earlier, OpenSearch Serverless is already one of the vector database options in the Knowledge Base for Amazon Bedrock. Now you can create a specialized vector engine–based collection. 

The cool thing to note is that Amazon OpenSearch supported about 20 million vector embeddings during preview. Now (post GA), this limit has been raised to support a **billion** vector scale 🤯

I also recommend checking out [this blog post](https://aws.amazon.com/blogs/big-data/amazon-opensearch-services-vector-database-capabilities-explained/) that goes into vector database capabilities in-depth.

![](https://d2908q01vomqb2.cloudfront.net/b6692ea5df920cad691c20319a6fffd7a4a766b8/2023/06/19/BDB-3474-Semantic-Search.png)


## 6. Vector search for Amazon DocumentDB

[Vector search is now built into DocumentDB](https://aws.amazon.com/about-aws/whats-new/2023/11/vector-search-amazon-documentdb/) as well. You can set up, operate and scale databases for ML/AI applications without separate infrastructure. [Take a look at the documentation](https://docs.aws.amazon.com/documentdb/latest/developerguide/vector-search.html) to dive in deeper before creating your first vector collection!

![](https://d2908q01vomqb2.cloudfront.net/da4b9237bacccdf19c0760cab7aec4a8359010b0/2023/11/12/2023-vector-search-for-documentdb.png)

## 7. Agents for Amazon Bedrock, now GA too!

[Agents in Amazon Bedrock](https://aws.amazon.com/about-aws/whats-new/2023/11/boost-generative-ai-development-agents-bedrock/) aid with development of GenAI apps by providing a platform to orchestrate multi-step tasks. The instructions we provide is used to create an orchestration plan which is then carried out by invoking APIs and accessing knowledge bases (using RAG).

Amazon Bedrock makes it easy to get started with Agents by providing a working draft of an existing agent. Head over to the AWS console to try it out and select the **Test** button and enter a sample user request. To dive in deeper, [refer to the documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/agents.html), including [how to use the Agents API](https://docs.aws.amazon.com/bedrock/latest/userguide/agents-api.html).


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mx2mvy07bnxxae6z0e83.png)

## 8. Amazon DynamoDB zero-ETL integration with Amazon OpenSearch Service

Ok, you love Amazon DynamoDB and have tons of data in it. How do you make it easily searchable? I'm sure you've found different ways of making that happen. But **zero-ETL** integration simplifies data synchronization from Amazon DynamoDB to Amazon OpenSearch Service - no need to write custom ETL pipelines!

Read up on the announcement, check out [the blog post](https://aws.amazon.com/blogs/aws/amazon-dynamodb-zero-etl-integration-with-amazon-opensearch-service-is-now-generally-available/) and [follow along in the documentation](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/OpenSearchIngestionForDynamoDB.html) when you are ready to take it for a spin.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/z2tnw609fpidq9r6zdp5.png)

## 9. Amazon Q - A Generative AI–powered assistant (preview)

[Amazon Q](https://aws.amazon.com/about-aws/whats-new/2023/11/aws-amazon-q-preview/) is being integrated into services such as [Amazon CodeCatalyst](https://aws.amazon.com/about-aws/whats-new/2023/11/feature-development-capability-amazon-q-preview-codecatalyst/) and [Amazon QuickSight](https://aws.amazon.com/about-aws/whats-new/2023/11/amazon-q-quicksight-data-exploration-generative-bi-capabilities-preview/). It's also available in the [AWS console and your IDE](https://aws.amazon.com/blogs/aws/amazon-q-brings-generative-ai-powered-assistance-to-it-pros-and-developers-preview/). It's set of capabilities are built to support developers and IT professionals alike - be it helping you [optimize EC2 instance type selection](https://aws.amazon.com/about-aws/whats-new/2023/11/amazon-q-optimal-ec2-instance-selection-workload/), [build the next serverless app](https://aws.amazon.com/blogs/aws/amazon-q-brings-generative-ai-powered-assistance-to-it-pros-and-developers-preview/) or [upgrade your Java apps](https://aws.amazon.com/blogs/aws/upgrade-your-java-applications-with-amazon-q-code-transformation-preview/)!

![](https://d2908q01vomqb2.cloudfront.net/da4b9237bacccdf19c0760cab7aec4a8359010b0/2023/11/26/2023-q-dev-rev-3-2-1.png)

## 10. AWS SDK for Rust

(last but definitely, not the least)

Ok, I may not be actively using Rust as I was a while back, but I still have a soft corner for it. And that's why I am [happy to see the GA announcement for AWS SDK for Rust](https://aws.amazon.com/about-aws/whats-new/2023/11/aws-sdk-rust/) which means you can use it for production workloads! 

To be perfectly honest, I haven't taken it for a (re)spin yet, but when I do, I will be using [the getting started guide](https://docs.aws.amazon.com/sdk-for-rust/latest/dg/getting-started.html) to refresh my memory.

The AWS SDK for Rust contains one crate for each AWS service - you can [check them out here](https://awslabs.github.io/aws-sdk-rust/).

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7ye5w9rabn5fhq5rr105.png)

That's it for my short (and completely biased) top picks for this year. I couldn't make it to re:Invent this time (last minute emergency), but I hope to see you all in 2024. 

Happy building!