VishaalY/solutions-architect-hf-dataset huggingface dataset:
Question: Why is my Amazon DynamoDB maximum latency metric high when the average latency is normal?

When I review the Amazon CloudWatch metrics for my Amazon DynamoDB workloads, the maximum latency metric is high. But, average latency is normal.

ResolutionWhen you analyze the CloudWatch metric SuccessfulRequestLatency, it's a best practice to check the average latency. Maximum latency doesn't give a picture of overall latency on your DynamoDB table. Instead, it shows the maximum time taken by a single request in that period. For example, if you have 100 requests on a DynamoDB table at one time, even if 99 requests take 10 ms and a single request takes 100 ms, then the maximum latency metric is 100 ms.DynamoDB is a mass-scaled distribution system, with thousands of nodes in the backend fleet. So, a DynamoDB table might have multiple partitions in the tablespace, and each partition has multiple copies in the backend fleet. When you make an API call to DynamoDB, the DynamoDB service endpoint receives your call, and then routes it to one of the back end nodes for processing. When the call is successfully processed, DyanamoDB routes the results back to your client.In most cases, the API call is successfully processed in a single attempt, and you observe small latency on the client side. But, sometimes the first attempt fails if the back end node is experiencing:A busy periodFailoverPartition splitConnectivity issuesIn cases like these, the first attempt fails within a timeout on the server side (5000 ms). Then, the server automatically retries the API call on another node, often multiple times. The server returns the result back to your client when the API call is successfully processed. When this happens, you observe elevated latency for that particular request.So, a high maximum latency metric is generally not a cause for concern. If the DynamoDB service observes a consistently high latency from one node, then the service automatically removes that component from the back end fleet. You might observe an elevated level of latency for a certain percentage of API calls when the previously mentioned localized failure occurs on the service side. This is reflected in a high level of maximum SuccessfulRequestLatency in the CloudWatch metrics for the related DynamoDB tables. For this reason, localized failures can increase your maximum latency, but you do not need to take any action to control this failure.But, you can configure your application to react quickly by failing fast with exponential backoff retry. This means that the new request hits the new node, and you get faster results. For more information, see Tuning AWS Java SDK HTTP request settings for latency-aware Amazon DynamoDB applications.

Related informationHow can I troubleshoot high latency on an Amazon DynamoDB table?Latency metrics logging
Question: How do I recreate a default subnet for my default Amazon VPC?

I deleted the default subnet for my default Amazon Virtual Private Cloud (Amazon VPC). How do I recreate a default subnet for my default Amazon VPC?ResolutionYou can create a default subnet in any Availability Zone that doesn't have a default subnet. For more information, see Create a default subnet.Note: Default subnets can be created only in a default Amazon VPC. If you deleted your default Amazon VPC, you can create a new default Amazon VPC that contains a default subnet in each Availability Zone in the Region.

Related informationDefault VPCsDelete your default subnets and default VPCWhat is Amazon VPC?
Question: How can I resolve the "SYNTAX_ERROR: line 1:8: SELECT * not allowed in queries without FROM clause" error in Amazon Athena?

When I run an Amazon Athena query, I get the error "SYNTAX_ERROR: line 1:8: SELECT * not allowed in queries without FROM clause".

ResolutionCheck the table's permissionsIf the table is registered with AWS Lake Formation, then be sure that the user or role that's querying the table has SELECT permissions for the table. For more information, see Using Athena to query data registered with Lake Formation and Permissions example scenario.Check the table's classificationIf you created the table using the AWS Glue crawler, then be sure that the following are true:The table's classification isn't UNKNOWN.The table has defined columns.If the classification is UNKNOWN, then there's a problem with the table schema. Fix the schema and run the crawler again. For more information, see Adding classifiers to a crawler in AWS Glue.

Related informationSHOW CREATE TABLEUsing AWS Glue jobs for ETL with Athena
Question: How do I troubleshoot issues configuring authentication in my Application Load Balancer?

I receive an error when I configure authentication in my Application Load Balancer.

ResolutionMisconfigurations with the identity provider (IdP) or Application Load Balancer can cause errors when configuring authentication for the Application Load Balancer. Follow these steps to troubleshoot authentication errors.redirect_mismatchIf you're using Amazon Cognito, then set the callback URL to https://<domain>/oauth2/idpresponse. If you're using a different IdP, then set the redirect URI to https://<domain>/oauth2/idpresponse.Note: Replace <domain> with the domain used to access the Application Load Balancer.HTTP 401: UnauthorizedConfigure the following identically on your Application Load Balancer and IdP:IssuerAuthorization endpointToken endpointClient ID/Client SecretAlso, set Action on unauthenticated request to either Allow or Authenticate (client reattempt), based on your use case.HTTP 500: Internal Server ErrorComplete the following steps if you receive an "HTTP 500: Internal Server Error" error:Add an outbound rule to allow traffic to the IdP endpoints over HTTPS (port 443).Configure the network access control list rules on each Application Load Balancer subnet to allow traffic to and from the IdP endpoints.For egress rules, specify: Destination IP - Identity provider, Destination port -443 Allow.For ingress rules, specify: Source IP - Identity provider, Destination port 1024-65535 Allow.Configure the route table to include a route for the Application Load Balancer to access the IdP endpoints.For public Application Load Balancers and public endpoints, configure an internet gateway route for the route table.For private Application Load Balancers and private endpoints, configure a network address translation (NAT) gateway for the route table. Or, configure a NAT instance route for the IdP.For other scenarios, configure the route tables of the Application Load Balancer subnets with appropriate route entry to route connectivity to the IdP endpoints.Select a valid OAuth2 Grant type. Application Load Balancers support the Authorization code grant to obtain an access token. If an incorrect grant is configured at the IdP, then the Application Load Balancer generates an error.Additional HTTP error codesFor troubleshooting additional HTTP error codes generated by Application Load Balancers, see The load balancer generates an HTTP error.

Related informationSimplify login with Application Load Balancer built-in authenticationAuthenticate users using an Application Load BalancerConfiguring a user pool app client
Question: How can I forward the host header with private integration for an API Gateway REST API?

I want to use the host header value for my Amazon API Gateway endpoint or custom domain. How can I forward the host header with private integration for an API Gateway REST API?Short descriptionAmazon API Gateway overwrites the host value that it receives in the original request to the integration endpoint. Amazon API Gateway assigns the host value depending on the integration request endpoint URL.To use the host header value of an API Gateway endpoint in the backend, use the custom header. You can use the custom header in the integration request that contains the value of the request host header.

ResolutionCreate an API Gateway REST API1.    Open the API Gateway console.2.    Choose Create API.-or-(If this is your first time using API Gateway) A page that introduces you to the features of the service appears. In REST API, choose Build. When the Create Example API popup appears, choose OK.3.    For Choose an API type, in the REST API pane, choose Build.4.    In Create new API, choose New API.5.    In Settings, do the following:For API name, enter a name that describes your API's purpose. For example: ForwardHostHeader.(Optional) For Description, enter a short description of your API's purpose. for example, Forward host header for private integration.For Endpoint Type, choose either Regional, Edge optimized, or Private.6.    Choose Create API.Configure your API's method request and Integration request1.    Open the API Gateway console.2.    Follow the instructions to create an API with private integration using the API Gateway console.3.    In Method Request, add HTTP Request Headers as host.4.    In Integration Request, expand HTTP Headers, and choose Add header.5.    Enter a name for the custom header. For example, my_host.6.    In Mapped from, enter the method request. For example, method.request.header.host, and then choose the check mark icon.Deploy your API to a new stage1.    Open the API Gateway console.2.    Follow the instructions to Deploy a REST API to a stage. For more information, see Setting up a stage using the API Gateway console.3.    In the Stage Editor pane, copy the Invoke URL to your clipboard. You can use this URL to invoke your API.REST API invoke URL example:https://1a2bc3d456.execute-api.us-east-1.amazonaws.com/stage4.    If you use an Application Load Balancer for your backend, you can test this configuration by adding a rule. Choose "http header" as a condition and specify "my_host" as the same header and value as the API Gateway host value.5.    Paste the API Gateway URL into a browser window to invoke your API. The "my_host" header configuration in API Gateway forwards the API Gateway host value to the Application Load Balancer. The Application Load Balancer matches the condition set and then returns the response as configured.

Related informationSet up request and response data mappings using the API Gateway consoleSet up an API using the API Gateway consoleAmazon API Gateway important notesHow do I integrate an API Gateway REST API with an Application Load Balancer?
Question: How do I troubleshoot consistent and intermittent DNS-related errors in AWS Lambda?

I see DNS-related errors in my AWS Lambda function when I attempt to reach a host.

ResolutionDNS-related errors in AWS Lambda, like Temporary failure in name resolution, getaddrinfo, ENOTFOUND, and ETIMEDOUT, can be consistent or intermittent. The troubleshooting process for consistent errors is different from the intermittent error troubleshooting process.Troubleshooting consistent DNS-related errorsMisconfigured DNS servers, incorrect permissions, and network connectivity issues can cause consistent DNS-related errors. Take the following steps to troubleshoot consistent DNS-related errors:When you're accessing public resources with a virtual private cloud (VPC)-configured Lambda function, verify that the Lambda function has internet access.When you're accessing private resources, verify that the correct VPC and DNS servers are configured for the Lambda function.When you're using custom DHCP option sets for a VPC, verify the DNS query response through Amazon Route 53 Resolver Query Logs.Run your Amazon Elastic Compute Cloud (Amazon EC2) instance with the same network configuration as AWS Lambda. Then verify that you can resolve and reach the endpoint.Verify that the hostname is correct for ENOTFOUND errors.Try to access the target using an IP address instead of the hostname. This way, you can confirm whether the issue is caused by the hostname resolution or a missing endpoint route.Make sure that the Lambda function's security group and network access control list (network ACL) allow inbound traffic to the necessary ports and IP ranges.Troubleshooting intermittent DNS-related errorsNetwork congestion, high traffic, and other external factors can cause intermittent DNS-related errors. Take the following steps to troubleshoot intermittent DNS-related errors:Implement retries and exponential backoff for the Lambda function to handle intermittent DNS errors.Consider using runtime-specific DNS libraries that allows local caching of DNS queries. This helps in scenarios where your DNS servers reach the per-second packet limit. This can also happen with custom DNS servers.Make sure that the Lambda function has enough resources to perform its tasks.Monitor the Lambda errors in Amazon CloudWatch and then set up CloudWatch alarms based on the error messages to check for patterns.

Related informationHow does DNS work and how do I troubleshoot partial or intermittent DNS failures?
Question: How can I troubleshoot the "Runtime.InvalidEntrypoint" error for my Lambda function that's from a Docker image?

I received a "Runtime.InvalidEntrypoint" error when trying to create an AWS Lambda function from a Docker image.

ResolutionWhen the entry in the Lambda function configuration is incorrect, you receive the "Runtime.InvalidEntrypoint". To resolve this issue, check the following configuration settings for your Lambda function.Verify that the ENTRYPOINT and CMD to your container image include the absolute path as the location and doesn't contain a symlink as the ENTRYPOINT. For more information, see Container: Error occurs on runtime InvalidEntrypoint.Make sure that the ENTRYPOINT is correct in the Lambda function's configuration. Also, the ENTRYPOINT must match the command in the container's Dockerfile that's used for the runtime.Verify that the container image is pushed to the correct Amazon Elastic Container Registry (Amazon ECR).Make sure that the Docker image's architecture and the Lambda function are same. Lambda supports arm64 and x86_64 instruction set architectures.When the Lambda deployment package doesn't have the correct security permissions, you receive a "permission denied" error along with the "Runtime.InvalidEntrypoint" error occurs. In Unix permissions symbolic notation (from Wikipedia), 644 is the correct permissions for all files within a Lambda deployment package. For folders within a deployment package, the correct permissions setting is 755. For more information, see How do I troubleshoot "permission denied" or "unable to import module" errors when uploading a Lambda deployment package?Related informationHow do I create a Lambda layer using a simulated Lambda environment with Docker?Troubleshoot container image issues in LambdaHow do I use container images with Lambda?
Question: How do I update all Amazon EMR nodes after the bootstrap phase?

I want to update all nodes on an Amazon EMR cluster after the BOOTSTRAPPING state is complete.

Short descriptionThe bootstrap phase occurs before Amazon EMR installs and configures applications such as Apache Hadoop and Apache Spark. To make additional changes on all cluster nodes after Amazon EMR installs and configures the applications, run a bootstrap action that downloads and runs another script.Resolution1.    Create a bash script that specifies the changes that you want to make on all cluster nodes. Refer to the following examples.Example 1: This script waits for a configuration file ( /etc/hadoop/conf/hadoop-env.sh) to become available, and then performs additional work.#!/bin/bash## This is an example of script_b.sh#while [ ! -f /etc/hadoop/conf/hadoop-env.sh ]do  sleep 1done## Now the file is available, do your work here#exit 0Example 2: This script waits for the EMR cluster to enter the WAITING state before implementing additional customization. This allows the EMR cluster to finish installing and configuring Hadoop and other applications.#!/bin/bash## This is an example of script_b.sh## Wait for EMR provisioning to become successful.#while [[ $(sed '/localInstance {/{:1; /}/!{N; b1}; /nodeProvision/p}; d' /emr/instance-controller/lib/info/job-flow-state.txt | sed '/nodeProvisionCheckinRecord {/{:1; /}/!{N; b1}; /status/p}; d' | awk '/SUCCESSFUL/' | xargs) != "status: SUCCESSFUL" ]];do  sleep 1done## Now the EMR cluster is ready. Do your work here.#exit 02.    Launch an EMR cluster and add a bootstrap action similar to the following. This script downloads the script that you created in the previous step ( script_b.sh) and then runs it in the background.#!/bin/bashaws s3 cp s3://doc-example-bucket/script_b.sh .chmod +x script_b.shnohup ./script_b.sh &>/dev/null &Related informationUnderstanding the cluster lifecycle
Question: How can I find out which S3 buckets allow access from the internet?

I want to know which of my Amazon Simple Storage Service (Amazon S3) buckets are publicly accessible from the internet.

ResolutionTo check whether S3 buckets are publicly accessible, use any of the following methods:Use the Amazon S3 console to configure public access settings.Use AWS Trusted Advisor to check Amazon S3 bucket permissions.Use Access Analyzer to set up alerts for buckets that are publicly accessible.If you have a large number of S3 buckets in your AWS account, then AWS Config can provide a more efficient method. AWS Config rules allow you to quickly identify which buckets allow public read or write access. Also, you can set up AWS Config to notify you if any S3 buckets become publicly accessible after your initial review.To create AWS Config rules that flag which S3 buckets are publicly accessible, follow these steps:Note: Before you use AWS Config to analyze your S3 buckets, be sure to set up AWS Config on your AWS account.Open the AWS Config console and set the Region selector to an AWS Region that supports AWS Config rules.Note: AWS Config performs the compliance check for buckets in the corresponding AWS Region. If you have buckets in multiple Regions, then set up AWS Config rules in each Region.In the navigation pane, choose Rules.Select + Add rule.In the search bar, enter s3-bucket-public-read-prohibited. Then, choose the s3-bucket-public-read-prohibited rule. This rule flags buckets that allow public read access as Noncompliant.Select Save.Choose + Add rule.In the search bar, enter s3-bucket-public-write-prohibited. Then, choose the s3-bucket-public-write-prohibited rule. This rule flags buckets that allow public write access as Noncompliant.Select Save.It might take several minutes for AWS Config to complete the evaluation of your S3 buckets based on the new rules. After the AWS Config evaluation is complete, open the Rules page from the AWS Config console. Then, open each rule to see which S3 buckets are flagged as noncompliant. Noncompliant buckets are those that allow either public write or read access from the internet.To set up notifications from AWS Config when an S3 bucket becomes noncompliant, see Notifications that AWS Config sends to an Amazon SNS topic.For more information on setting permissions for S3 buckets, see Identity and access management in Amazon S3.

Related informationEvaluating resources with AWS Config rules
Question: How do I check the version number for an Amazon Aurora for MySQL or PostgreSQL DB instance?

I want to check the version number for my Amazon Aurora MySQL-Compatible Edition or Amazon Aurora PostgreSQL-Compatible Edition DB instance.

ResolutionUse the version number to cross-reference the database engine updates to see what features and enhancements are available for that particular Amazon Aurora version.1.    Log in to your Aurora instance using a client. For MySQL, use MySQL Workbench. For PostgreSQL, use psql or SQL Workbench/J.2.    Run a command similar to the following:select AURORA_VERSION();select @@aurora_version;show variables like '%version';3.    After you get your Aurora version number, check the Database engine updates for Aurora MySQL or Aurora PostgreSQL updates. These will show new features and improvements.Note: Aurora releases updates regularly. Updates are cluster-wide, and they require a simultaneous restart of all the instances in the cluster, which can cause brief interruptions in connectivity. If you use Aurora for MySQL, then under certain conditions the zero downtime patch feature can prevent interruptions in connectivity on the writer node. If you use Aurora for PostgreSQL, a reboot is required. To plan for these restarts, view or change your maintenance window settings from the Amazon Relational Database Service (Amazon RDS) console. For more information about changing your maintenance window, see Adjusting the preferred DB instance maintenance window.

Related informationThe Amazon RDS maintenance windowMySQL bugs fixed by Aurora MySQL database engine updatesAmazon Aurora PostgreSQL releases and engine versions
Question: Why can’t I install SSM Agent on my Amazon EC2 Linux instance?

When I try to install SSM Agent on my Amazon Elastic Compute Cloud (Amazon EC2) Linux instance, installation fails. How can I troubleshoot this issue?Short descriptionSSM Agent is pre-installed on most Amazon Web Services (AWS) provided Amazon Machine Images (AMIs), including the following:Amazon LinuxAmazon Linux 2Amazon Linux 2 ECS-optimized base AMIsUbuntu Server 16.04, 18.04, and 20.04However, to manage instances based on RedHat, SUSE, or CentOS AMIs, you must install SSM Agent manually.

ResolutionVerify the following to troubleshoot SSM Agent installation failures:Unsupported operating system versionSSM Agent isn’t available for all operating system (OS) versions. SSM Agent installation fails if you run an unsupported version of an OS. Verify the list of supported operating systems to confirm whether SSM Agent is available for your OS version.Package download failureDuring the manual installation process for SSM Agent, the SSM Agent package downloads and installs from an Amazon Simple Storage Service (Amazon S3) repository. If the instance can’t connect to the S3 bucket to download the package, SSM Agent installation fails.Verify that your Amazon EC2 instance has access to the S3 repository to download the SSM Agent package:If your instance is in a private subnet with a network address translation (NAT) gateway, then see NAT gateways.If your instance is in a private subnet with a NAT instance, then see NAT instances.If your instance is in a public subnet with an internet gateway, then see Enable internet access.If your instance is in a private or public subnet with an Amazon S3 virtual private cloud (VPC) endpoint, then see Endpoints for Amazon S3.The package download can also fail under the following circumstances:The Domain Name System (DNS) servers configured within the OS can’t resolve the Amazon S3 endpoint URLs.DNS resolution is deactivated for the VPC.Run the following command to verify that the /etc/resolv.conf file includes the correct IP address for your DNS server. Then, review the output and confirm that the nameserver IP matches the IP address for your DNS server.$ cat /etc/resolv.confFor more information, see Why can’t I connect to an S3 bucket using a gateway VPC endpoint?Missing public key for SSM Agent packageSSM Agent package files are cryptographically signed. To verify that the agent package is original, you can use a public key to verify the installer package signature using either RPM or GPG. RPM packages already include the required signature for RPM verification. However, if you use GPG to verify the installer package, you must manually import the public key. Otherwise, installation fails with the error “Public key for amazon-ssm-agent.rpm is not installed”.For more information, see Verifying the signature of the SSM Agent.

Related informationHow do I install AWS Systems Manager Agent (SSM Agent) on an Amazon EC2 Linux instance at launch?Automating updates to SSM AgentInstalling and configuring SSM Agent on EC2 instances for Linux