VishaalY/solutions-architect-hf-dataset huggingface dataset:
Question: How do I prevent "rate exceeded" ThrottlingException errors when I run monitoring scripts in Amazon EMR?

To monitor my Amazon EMR clusters, I run scripts that make API calls. The scripts return errors similar to the following:"Rate exceeded (Service: AmazonElasticMapReduce; Status Code: 400; Error Code: ThrottlingException; Request ID: e2b6191c-gkl5-269r-u735-cryyz251a837)"How do I prevent "rate exceeded" errors?Short descriptionAmazon EMR throttles API calls to maintain system stability. Throttling exceptions usually occur when you run monitoring scripts at regular intervals to check clusters for a parameter. Here's an example: calling DescribeCluster every 60 seconds to check if the cluster has reached the WAITING state. The more clusters that you have and the more monitoring scripts that you run, the more likely you are to get throttling errors.

ResolutionTo prevent throttling errors:Reduce the frequency of the API calls.Stagger the intervals of the API calls so that they don't all run at the same time.Implement exponential backoff when making API calls.Consider moving to an event-based architecture.To understand the source of throttling errors, use AWS CloudTrail to track Event History. CloudTrail can help identify event details such as the following:Frequent API callsRate exceeded errors and their related API callsWhether API calls are triggered by users or automation

Related informationCommon errorsManaging and monitoring API throttling in your workloadsHow CloudTrail works
Question: How do I configure and attach an internet gateway for use with Elastic Load Balancing?

How do I configure and attach an internet gateway for use with Elastic Load Balancing?ResolutionFirst, create an internet gateway and attach it to your Amazon Virtual Private Cloud (Amazon VPC).Next, configure the internet gateway to route traffic to and from the internet:Add an entry to the route table for your subnet that points to your internet gateway.Confirm that your Amazon VPC's security groups and network access control list (ACL) allow traffic to and from the internet. For example configurations for network ACLs, see Network ACL rules  .Confirm that any instances that route traffic through the internet gateway have a public IP address or an attached Elastic IP address.If you're creating a new load balancer, choose an internet-facing load balancer.

Related informationInternet-facing Classic Load Balancers
Question: How do I configure routing for my Direct Connect private virtual interface?

I created a private virtual interface (VIF) in AWS Direct Connect. How do I check if I'm routing properly over my Direct Connect connection?ResolutionAfter you create your private virtual interface, do the following to verify that routing is set up correctly.Verify that the virtual private gateway associated with your private virtual interface is attached to the correct Amazon Virtual Private Cloud (Amazon VPC):Sign in to the Direct Connect console.In the navigation pane, choose Virtual Interfaces.Choose the virtual interface (VIF), and then choose View details.For private VIFs attached to a virtual gateway (VGW), in General configuration choose the VGW ID.If the virtual gateway isn't attached to your VPC, follow the instructions to attach it.For private VIFs attached to a Direct Connect gateway, in General configuration choose the gateway ID.In Gateway associations, verify the Direct Connect gateway is attached to your virtual gateway.Confirm that the allowed prefixes contains the VPC CIDR.Verify that you're advertising and receiving the correct routes through Border Gateway Protocol (BGP). For more information, see Routing policies and BGP communities.Be sure that you're advertising routes to AWS that cover the networks that are communicating with your VPC.Be sure that you're receiving the VPC CIDR route from AWS.Verify that you've enabled route propagation to your subnet route tables. This step propagates the routes learned through VPN connections and Direct Connect virtual interfaces to your VPC route tables. Any changes to the routes are updated dynamically, and you don't need to manually enter or update routes.Verify that your security groups allow traffic from your local network.Sign in to the VPC console.In the navigation pane under Security, choose Security Groups.In the content pane, select the security group that's associated with your instances.Choose the Inbound Rules view.Be sure that there are rules permitting traffic from your local network over the desired ports.Choose the Outbound Rules view.Be sure that there are rules permitting traffic to your local network over the desired ports.Verify that your network access control lists (ACLs) allow traffic from your local network.Sign in to the VPC console.In the navigation pane under Security, choose Network ACLs.In the content pane, select the network ACL that's associated with your VPC and subnets.Choose the Inbound Rules view.Be sure that there are rules permitting traffic from your local network over the desired ports.Choose the Outbound Rules view.Be sure that there are rules permitting traffic to your local network over the desired ports.Verify that your Direct Connect private virtual interface is traversable using the ping utility. Security groups, network ACLs, and on-premises security allow for bidirectional connectivity tests using ping.

Related informationWhich type of virtual interface should I use to connect different resources in AWS?My virtual interface BGP status For Direct Connect is down in the AWS console. What should I do?
Question: How can I add certificates for multiple domains to a load balancer using AWS Certificate Manager?

I want to upload multiple certificates for different domains using Elastic Load Balancing (ELB).

Short descriptionAs of April 2018, Classic Load Balancer doesn't support adding multiple certificates.To add multiple certificates for different domains to a load balancer, do one of the following:Use a Subject Alternative Name (SAN) certificate to validate multiple domains behind the load balancer, including wildcard domains, with AWS Certificate Manager (ACM).Use either an Application Load Balancer (ALB) or Network Load Balancer (NLB), which supports multiple certificates and smart certificate selection using Server Name Indication (SNI).Note: ACM certificates can't be downloaded, and are used only with AWS services integrated with ACM.

ResolutionTo use a Classic Load Balancer, follow these steps to create a SAN certificate using ACM.Open the ACM console.Note: If you've never created a certificate, choose Get started.Follow the instructions for Requesting a public certificate.In the ACM console, verify that the Status of the certificate request has changed from Pending validation to Issued.Attach the certificate to a load balancer. For instructions, see Replace the SSL certificate for your Classic Load Balancer.To add multiple certificates with an ALB, see Application Load Balancers now support multiple TLS certificates with smart selection using SNI.To add multiple certificates with an NLB, see Elastic Load Balancing: Network Load Balancers now support multiple TLS certificates using Server Name Indication (SNI).Note: The ALB and NLB certificate quota limit is 25 (excluding default certificates). This limit can be increased. For more information, see Quotas for your Application Load Balancers and Quotas for your Network Load Balancers.

Related informationAWS Certificate Manager FAQsEmail validationDNS validation
Question: How can I resolve DNS resolution or SSL certificate mismatch errors for my API Gateway custom domain name?

I configured a custom domain name for my Amazon API Gateway API. I am unable to connect to the domain name and receive DNS resolution or SSL certificate mismatch errors. How can I resolve this?Short descriptionThere are two types of custom domain names that you can create for API Gateway APIs: Regional or (for REST APIs only) edge-optimized.

ResolutionBefore creating a custom domain name for your API, you must do one of the following:Request an SSL/TLS certificate from AWS Certificate Manager (ACM).-or-Import an SSL/TLS certificate into ACM.For more information, see Getting certificates ready in AWS Certificate Manager.After you have your SSL/TLS certificate, you can follow the instructions to set up a custom domain name for my API Gateway API.To connect to a custom domain name for API Gateway APIs, you must configure Amazon Route 53 to route traffic to an API Gateway endpoint.If the DNS records for the custom domain name aren't mapped to the correct API Gateway domain name, then the SSL connection fails. This is because the default *.execute-api.<region>.amazonaws.com certificate is returned instead of the SSL/TLS certificate.To confirm that the DNS mapping is correct, run the following command from the client:$ nslookup <customdomainname>The output should be the API Gateway domain name. Make sure that the domain name matches the API Gateway domain name. If a Route 53 alias record is used for DNS mapping, then the output is the IP address. Make sure that the IP address matches the API Gateway domain name IP address.Note:When configuring Route 53, you must create either a public hosted zone or a private hosted zone. For internet-facing applications with resources that you want to make available to users, choose a public hosted zone. For more information, see Working with hosted zones.Route 53 uses records to determine where traffic is routed for your domain. Alias records provide easier DNS queries to AWS resources, while CNAME (non-alias) records can redirect DNS queries outside of AWS resources. For more information, see Choosing between alias and non-alias records.

Related informationMigrating a custom domain name to a different API endpoint
Question: How do I troubleshoot subscription filter policy issues in Amazon SNS?

My Amazon Simple Notification Service (Amazon SNS) subscription filter policy isn't working. How do I troubleshoot the issue?ResolutionImportant: Additions or changes to a subscription filter policy require up to 15 minutes to take effect.Verify that message attributes are included in the messages published to your Amazon SNS topicSubscription filter policies can filter message attributes only, not the message body. If the MessageAttributeValue is left empty on a message, then the filter policy rejects the message.To see if your filter policy rejected messages because they didn't include attributes, review the following metric in your Amazon CloudWatch metrics for Amazon SNS:NumberOfNotificationsFilteredOut-NoMessageAttributesFor more information, see Amazon SNS message filtering. For a tutorial on how to send messages with attributes, see To publish messages to Amazon SNS topics using the AWS Management Console.Verify that the messages published to your Amazon SNS topic meet the required filter policy constraintsFor a complete list of restraints, see Filter policy constraints.Verify that your subscription filter policy's attributes are configured correctlyAfter you define a subscription filter policy's attributes, the subscription endpoint receives only messages that include those defined attributes. For more information see, Attribute string value matching and Attribute numeric value matching.To see the messages that your filter policy rejected because of mismatching or incorrectly formatted attributes, review the following CloudWatch metrics for Amazon SNS:NumberOfNotificationsFilteredOutNumberOfNotificationsFilteredOut-InvalidAttributesNote: The NumberOfNotificationsFilteredOut metric shows messages that your filter policy rejected because the message attributes didn’t match the policy attributes. The NumberOfNotificationsFilteredOut-InvalidAttributes metric shows messages that your filter policy rejected because the message attributes weren't in a valid format.
Question: How can I access an Amazon S3 bucket from an application running on an Elastic Beanstalk instance?

I want to access an Amazon Simple Storage Service (Amazon S3) bucket from an application running on an AWS Elastic Beanstalk instance.

Short descriptionTo access an S3 bucket from Elastic Beanstalk, verify that your AWS Identity and Access Management (IAM) instance profile is attached to an Amazon Elastic Compute Cloud (Amazon EC2) instance. The instance must have the correct permissions for Amazon S3. Then, confirm that your S3 bucket policy doesn't deny access to the role attached to your instance profile.

ResolutionValidate permissions for your instance profileOpen the Elastic Beanstalk console.Select your environment.From the navigation menu, choose Configuration.In the Configuration overview section, from the Category column, for Security, choose Modify.From the IAM instance profile menu, note the name of your instance profile.Open the IAM console.In the navigation pane, choose Roles.In the search box, enter the name of your instance profile from step 5.Verify that the role from step 8 has the required Amazon S3 permissions for the bucket that you want to access. For more information, see Identity and access management in Amazon S3 and Actions, resources, and condition keys for Amazon S3.Validate permissions for your S3 bucketOpen the Amazon S3 console.From the list of buckets, choose the bucket with the bucket policy that you want to change.Choose the Permissions tab.Choose Bucket Policy.Search for "Effect": "Deny" statements.In your bucket policy, edit or remove any "Effect": "Deny" statements that are denying the IAM instance profile access to your role. For more information, see Adding a bucket policy using the Amazon S3 console.Note: Be careful not to remove any necessary deny statements to align to the security best practice of principle of least privilege. For more information, see Amazon S3 security.Access your S3 bucketYou can now access your S3 bucket, and then use your S3 bucket to do the following tasks:Manage application versions.Note: Set the S3 URL when you create or update an application.Read or write application files or photos.Note: For more information, see Developing with Amazon S3 using the AWS SDKs, and explorers.

Related informationBuckets overviewElastic Beanstalk instance profileBucket policy examplesStoring private keys securely in Amazon S3
Question: How do I recreate a "Deleted" Amazon SNS topic subscription for an Amazon SQS queue in another AWS account?

My Amazon Simple Queue Service (Amazon SQS) queue was subscribed to an Amazon Simple Notification Service (Amazon SNS) topic in a different AWS account. I deleted the cross-account subscription, and the topic subscription is now in "Deleted" status. How do I recreate a deleted Amazon SNS topic subscription for an Amazon SQS queue in another account?Short descriptionIf you call the Amazon SNS Unsubscribe API from an account that doesn't own the subscription, the subscription enters Deleted status for 72 hours. While the SNS topic subscription is in Deleted status, the account that owns the subscription can't resubscribe the same endpoint to the topic.After 72 hours, Amazon SNS clears the Deleted subscription and the account that owns the subscription can resubscribe the same endpoint to the topic.If you don't want to wait 72 hours to resubscribe, you can manually recreate the subscription by doing any of the following.Note: It's a best practice to run Subscribe and Unsubscribe API calls from the same AWS account. When you call the Subscribe API, the AWS account that you use to make the call becomes the subscription owner.

ResolutionImportant: The following procedures apply to HTTP and HTTPS endpoint subscribers. It doesn't apply to AWS Lambda function subscribers.Send an HTTP GET method request to the SubscribeURL in the UnsubscribeConfirmation message you received1.    In the UnsubscribeConfirmation message sent to the SQS queue after you deleted the subscription, find the SubscribeURL. Then, copy and paste the URL to a text document.2.    Send an HTTP GET method request to the SubscribeURL.HTTP GET method request examplecurl -X GET "https://sns.us-west-2.amazonaws.com/?Action=ConfirmSubscription&TopicArn=arn:aws:sns:us-west-2:123456789012:MyTopic&Token=<token>"Call the Amazon SNS Subscribe API from the AWS account that owns the SNS topic, then confirm the subscription1.    Call the Amazon SNS Subscribe API from the AWS account that owns the SNS topic.2.    Have an AWS user with permissions to read messages from the SQS queue confirm the subscription.Create a new Amazon SNS topic to replace the current topic, then subscribe to the new topic1.    Create a new SNS topic to replace the current topic.2.    Subscribe the SQS queue to the new topic.

Related informationDeleting an Amazon SNS subscription and topicSending Amazon SNS messages to an Amazon SQS queue in a different accountFanout to Amazon SQS queues
Question: Why am I receiving an error when copying my ElastiCache snapshot?

I'm trying to copy my Amazon ElastiCache for Redis snapshot or backup to an Amazon Simple Storage Service (Amazon S3) bucket. The copy operation is stuck or failing. How can I troubleshoot this?Short descriptionElastiCache snapshot copy operations might fail or get stuck for one of the following reasons:The destination S3 bucket is not in the same Region as the snapshot.S3 doesn't have the necessary permission to access ElastiCache.The user doesn't have the proper Identify and Access Management (IAM) policy set to access an S3 bucket.The cluster is using data tiering.

ResolutionMake sure that you followed the steps for exporting your backup.Make sure that ElastiCache has the necessary permissions to access your S3 bucket. You can review error messages to rule out S3 permission issues.Make sure that you're not using data tiering in your cluster. Data tiering is turned on by default if you're using a combination of Redis engine version 6.2n or later with the rdg cache node family.Related informationcopy-snapshot (AWS CLI Command reference)
Question: Why isn't my SC1 or ST1 EBS volume achieving the rated throughput performance?

My ST1 or SC1 Amazon Elastic Block Store (Amazon EBS) volume isn't reaching the throughput performance listed in the AWS documentation. Why is this?Short descriptionWhen using HDD Amazon EBS volumes, such as SC1 and ST1, keep the following:These volumes always use a 1024 KiB I/O token regardless of the actual I/O size used by the workload on the instance. Even if the actual I/O size of the application workload is set to 16 KiB, the volume still uses the entire 1024 KiB size of the I/O token. This wastes most of the token's space. To maximize efficiency, fill the entire 1024KiB.When the I/O size of a sequential workload is greater than 32 KiB, Amazon EBS always merges the I/Os into a single I/O operation of 1024 KiB. This merging fills the entire token size.When the I/O size is smaller than 32 KiB, or if the workload is random, Amazon EBS doesn't merge the I/Os to 1024 KiB. However, Amazon EBS still uses the entire 1024 KiB token size. This leaves most of the space inside the token empty. Since the I/Os aren't merged, the instance uses more IOPS sending the same amount of data to the volume. This causes a reduction in the burst balance even though the throughput is below its baseline value.

ResolutionTo enable your ST1 and SC1 EBS volumes to reach their maximum rated throughput, do the following:Set up your application to use an I/O size greater than 32 KiB.Verify that your application uses sequential workload.ExampleWhen calculating throughput, use the following formula:Throughput = I/Osize * IOPSIf the I/O size is smaller than 32 KiB, the volume hits its IOPS limit, throttling the throughput. When this happens, the volume never achieves it's rated throughput performance.For example, setting the I/O size to 16 KiB and sending 3 MiB/s of data results in the following:3MiB/s/16KiB = 192 IOPSSetting the I/O size to 32 KiB with a sequential/contiguous workload causes Amazon EBS to merge to 1024 KiB. Amazon only sends 3 IOPS, in this case, as shown in the following calculation:3MiB/s/1024Kib = 3 IOPSAssume you're using 0.5 TiB (500 GiB) of an ST1 volume. This volume ideally provides a baseline throughput performance of 20 MiB/s and can burst up to 125 MiB/s.If the volume is bursting at 125 MiB/s, and the applications I/O size is 1024 KiB, then maximum theoretical IOPS equals 125 IOPS.Throughput / IOsize = 125MiB/s / 1024 KiB = 125 IOPS.However, if the application uses an I/O size of 16 KiB, then sending 3 MiB of data uses 192 IOPS. The application can't push 192 IOPS because the volume only achieves a theoretical maximum of 125 IOPS. In this case, the volume throttles IOPS to 125. In this scenario, the actual throughput is as follows:Actual throughput = 16 KiB * 125 = 1.95 MiB/sAs shown in the preceding calculations, IOPS is throttled to 125 IOPS, so Amazon EBS throttles the throughput to 1.95 MiB/s. This throttling occurs even though the burstable theoretical throughput for the volume is 125 IOPS given that the application used 1024 KiB I/O size.

Related informationI/O characteristics and monitoringAmazon EBS volume types
Question: How do I push Systems Manager SSM Agent logs to CloudWatch?

I want to send AWS Systems Manager SSM Agent logs to Amazon CloudWatch Logs. How can I do that?ResolutionCreate a log group in CloudWatch LogsTo create a log group in CloudWatch Logs, follow these steps:Open the CloudWatch console, and then choose Log groups from the navigation pane.Choose Create log group.For Log group name, enter a name.Choose Create.Attach permissionsThe Amazon Elastic Compute Cloud (Amazon EC2) instance must include AWS Identity and Access Management (IAM) permissions to send the logs. You must attach the CloudWatchLogsFullAccess IAM role to the instance. For instructions, see Attach an IAM role to an instance.Note: You can include these permissions with already existing permissions. You can also further narrow the permissions based on your requirements.Configure SSM Agent to send logs to CloudWatch LogsFor instructions to configure SSM Agent to send logs to CloudWatch Logs, see Sending SSM Agent logs to CloudWatch Logs.

Related informationChecking SSM Agent status and starting the agentWhat is Amazon CloudWatch Logs?Getting started with CloudWatch Logs