VishaalY/solutions-architect-hf-dataset huggingface dataset:
Question: How do I create Amazon EC2 instances through CloudFormation when the IAM policy for RunInstances has tag-based restrictions?

I want to create Amazon Elastic Compute Cloud (Amazon EC2) instances through AWS CloudFormation. But my AWS Identity and Access Management (IAM) policy for RunInstances has tag-based restrictions.

Short descriptionYou can use a launch template to create Amazon EC2 instances through CloudFormation.The Tags property of the AWS::EC2::Instance resource doesn't extend to the volumes that are created through CloudFormation. If the IAM policy that's associated with the user or role has restrictions on volume tags, then you receive the following error:"You are not authorized to perform this operation."To pass the tags through CloudFormation to ec2:CreateVolume, you must define your tags in the AWS::EC2::LaunchTemplate resource in your CloudFormation template.Resolution1.    Define a launch template in the stack with the tags that the IAM policy requires. For example:RequiredTagsLaunchTemplate:    Type: 'AWS::EC2::LaunchTemplate'    Properties:      LaunchTemplateData:        TagSpecifications:          - ResourceType: volume            Tags:              - Key: Env                 Value: Dev2.    Attach your launch template to your EC2 instance resource. For example:Instance:    Type: 'AWS::EC2::Instance'    Properties:      LaunchTemplate:        LaunchTemplateId: !Ref RequiredTagsLaunchTemplate        Version: 1      InstanceType: r4.xlarge      .      .  RequiredTagsLaunchTemplate:    Type: 'AWS::EC2::LaunchTemplate'    Properties:      LaunchTemplateData:        TagSpecifications:          - ResourceType: volume            Tags:              - Key: Env                Value: Dev3.    Confirm that your launch template has all the necessary tags.Important: You must confirm that the role or user that creates the stack has the permissions to create and use a launch template without tagging restrictions. You can use the aws:CalledVia condition key to create a new statement that exempts CloudFormation API calls from tagging requirements.
Question: How can I allow outgoing traffic from the IP address in my Elastic Beanstalk environment?

I'm required to have a static IP address for the outgoing traffic of the instance in my AWS Elastic Beanstalk environment. I'm also required to integrate that IP address with an external partner application or API. How can I meet these requirements in Elastic Beanstalk?Short DescriptionA single-instance environment in Elastic Beanstalk comes with a static IP address that's known as an Elastic IP address. You use a single-instance environment in Elastic Beanstalk to allow traffic from the IP address to access an external partner application, API, or other entity.If you want to assign a static IP address in a load-balanced Elastic Beanstalk environment, then complete the steps in How do I assign a static source IP address for all instances in a load balanced Elastic Beanstalk environment?Note: Single-instance environments are generally better suited for applications with low-volume traffic.

ResolutionIf you're using a custom VPC instead of the default VPC when you launch your Elastic Beanstalk environment, then verify that your instance is in a public subnet and has an internet gateway with a route to the internet (0.0.0.0/0).After launching your environment, use the Elastic IP address attached to the Amazon Elastic Compute Cloud (Amazon EC2) instance to allow traffic from that IP address to access the external application, API, or other entity.Related InformationConfigurationChanging Environment Type
Question: How can I troubleshoot connect errors when connecting an Amazon OpenSearch Service domain to QuickSight?

I'm trying to connect my Amazon OpenSearch Service domain to Amazon QuickSight. However, I received an error. How can I resolve this?Short descriptionTo connect QuickSight to your Amazon OpenSearch domain, make sure that:Your domain is version 7.7 or higher.Your OpenSearch version is 1.0 or higher.Your OpenSearch Service domain is in the same AWS Region and account as QuickSight.Note: Amazon OpenSearch Service supports OpenSearch and legacy Elasticsearch OSS up to 7.10.

ResolutionFollow these troubleshooting steps for your use case.Unsupported engine version errorWhen connecting to an OpenSearch Service domain, you might receive the following error:"Your database uses an unsupported engine version. Please review supported versions."This error occurs when the compatibility mode is not enabled on the OpenSearch Service domain. OpenSearch version 1.0 introduces a domain setting named Enable compatibility mode. Some OpenSearch Service clients and plugins check the cluster version before connecting. Compatibility mode sets OpenSearch Service to report its version as 7.10 so that these clients continue to work. When compatible mode is enabled, QuickSight identifies the correct version and the connection is established successfully.To enable compatibility mode, run the following command:PUT /_cluster/settings{     "persistent": {          "compatibility.override_main_response_version": true     }}Cursor not enabled or unable to prepare this table errorWhen importing the OpenSearch Service domain data into SPICE, you might receive one of the following errors:Can't perform this Action. The domain X has not enabled cursor.-or-Unable to prepare this tableTo enable SQL cursor in your OpenSearch Service domain, run the following command:PUT _opendistro/_sql/settings{     "transient": {          "opendistro.sql.cursor.enabled": "true"     },     "persistent": {          "opendistro.sql.cursor.enabled": "true"     }}Connection error forbiddenIf you have fine-grained access control enabled on OpenSearch Dashboards in your OpenSearch Service domain, you might receive the following error:GENERAL_SQL_EXCEPTION. Connection error forbiddenThis error occurs because the QuickSight service role isn't mapped to an OpenSearch Service role.To resolve this error, follow the instructions to add permissions to a domain to allow access from QuickSight.

Related informationUsing Amazon OpenSearch Service with Amazon QuickSightSupported data sourcesConnecting to a VPC with Amazon QuickSightOperational best practices for Amazon OpenSearch Service
Question: How can I prevent my Amazon EBS volumes from being deleted when I terminate Amazon EC2 instances?

When I terminate an Amazon Elastic Compute Cloud (Amazon EC2) instance, the Amazon Elastic Block Store (Amazon EBS) volume is deleted.

ResolutionWhen an instance terminates, the value of the DeleteOnTermination attribute for each attached EBS volume determines whether to preserve or delete the volume. By default, the DeleteOnTermination attribute is set to True for the root volume. It is set to False for all other volume types.To preserve the root volume when an instance terminates, change the DeleteOnTermination attribute for the root volume to False.Note: You don't incur billing costs for changing these values. However, charges are still incurred for the resources on EBS volumes that you choose to preserve, according to EBS pricing.If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.New instancesYou can set the DeleteOnTermination attribute to False when you launch a new instance.For instructions using the Amazon EC2 console, see Change the root volume to persist at launch using the console.To use the AWS CLI or AWS Tools for Windows PowerShell, see Change the root volume to persist at launch using the command line.Running instancesIf the instance is already running, you can set DeleteOnTermination to False using the command line.

Related informationAmazon EBS volumesWhat happens when you terminate an instance
Question: How do I troubleshoot custom launch template issues with managed node groups in Amazon EKS?

I want to troubleshoot errors when using custom launch template with managed node groups in my Amazon Elastic Kubernetes Service (Amazon EKS) cluster.

Short descriptionYou might get one of the following errors when using a custom launch template with managed node groups in your Amazon EKS cluster:"Client.InternalError: Client error on launch""SourceEc2LaunchTemplateNotFound or The Amazon EC2 Launch Template lt-xxxxxxxxxxxxxxxxx version x was not found""Node group entered a DEGRADED status"ResolutionClient.InternalError: Client error on launchThis error occurs when using encrypted Amazon Elastic Block Store (Amazon EBS) volumes in the custom launch template with incorrect permissions. To check the encryption status and AWS Identity and Access Management (IAM) permissions or policies, verify the following:The AWS Key Management Service (AWS KMS) key used to encrypt the volume is activated.The AWS KMS key has the correct key policies. To allow more IAM roles to work with encrypted Amazon EBS volumes, you can modify the key policies. For more information, see Allows key users to use the KMS key.SourceEc2LaunchTemplateNotFound or The Amazon EC2 Launch Template lt-xxxxxxxxxxxxxxxxx version x was not foundThis error occurs when you manually change the custom launch template version through the Auto Scaling group of the node instead of Amazon EKS. To resolve this issue, you must update the launch template version using Amazon EKS.To update your EC2 launch template from the managed node group, complete the following steps:Open the Amazon EKS console.In the navigation pane, choose Clusters.In Cluster name, choose the cluster that contains the node group to update.Choose the Compute tab.For Node groups, choose your node and then choose Change version.Select the version to apply to your node group. Make sure that the update strategy is set to Rolling Update.Choose Update.For more information, see Updating a managed node group.Node group entered a DEGRADED status after creating a new launch templateNode group can enter a DEGRADED status with an error similar to the following:"The Amazon EC2 Launch Template : lt-xxxxxxxxxxxxxxxxx has a new version associated with your Autoscaling group, which is not managed by Amazon EKS. Expected Launch Template version : x".This issue happens when the Amazon EC2 launch template version for your managed node group doesn't match the version that Amazon EKS created. Existing node groups that don't use a custom launch template can't be updated directly. To resolve this, create a launch template and version with your preferred settings, and then create the node group using that launch template. If the new node group is launched from your custom template, then you can create new versions of the template. You can use this template without putting the node group in a DEGRADED status.

Related informationLaunch template supportLaunch template configuration basics
Question: How do I troubleshoot memory and CPU issues in Elastic Beanstalk?

I want to know why my AWS Elastic Beanstalk CPU and memory usage is too high, and I want to troubleshoot issues.

Short descriptionTo gain insight into your Elastic Beanstalk CPU and memory usage and troubleshoot issues, do the following:Configure the environment to collect metrics on memory and CPU usageAnalyze log files to troubleshoot memory and CPU usage issues

ResolutionConfigure your environment to collect metricsTo gain insight into memory and CPU usage, do the following:Check the health of your Amazon Elastic Compute Cloud (Amazon EC2) instances with the Elastic Beanstalk health agent.Monitor your instances using Amazon CloudWatch.Turn on enhanced health reporting and monitoring, and then review the Environment health page for Amazon EC2 instance metrics.Turn on CloudWatch memory usage metrics in Amazon Linux 2 or Windows environments.Configure advanced health rules for your Elastic Beanstalk environment (such as CPU load averages).Analyze your log files to troubleshoot your memory and CPU issueAfter gathering data from your metrics, do the following:Review the log files from the instances in your Elastic Beanstalk environment.Analyze key log files (such as /var/log/messages), and then search for messages about high CPU or out-of-memory (OOM) events.Use your log file results to figure out why the process is consuming a high amount of CPU or memory.Note: For example, you might find a large number of requests in Apache access logs or NGINX access logs.If the issue is ongoing, then do the following:Connect to your Linux instance using SSH.Use the Linux top or free command to view details on CPU and memory consumption.Configure the ATOP monitoring tool. Review CPU and memory utilization stats.CPU-based command: atop -CMemory-based command: atop -mIf you don't find any errors, then the traffic load to your environment might have increased. To handle increased traffic, try the following:Scale up by increasing the instance type.Scale out to decrease the load per instance by increasing the size of the environment's Auto Scaling group.
Question: How can I resolve node label and YARN ResourceManager failures in Amazon EMR?

I enabled node labels on an Amazon EMR cluster. Then, YARN ResourceManager failed.

Short descriptionThis issue affects Amazon EMR release versions 5.19.0 - 5.21.0. In these versions, Amazon EMR stores node label files in HDFS:DEFAULT_DIR_NAME = "node-labels"MIRROR_FILENAME = "nodelabel.mirror"EDITLOG_FILENAME = "nodelabel.editlog"Amazon EMR stores these files at the following location in yarn-site.xml on all nodes: yarn.node-labels.fs-store.root-dir: '/apps/yarn/nodelabels'. The issue happens when these files become corrupted when you lose all nodes that contain the file's blocks during a resize operation. ResourceManager then restarts, gets stuck in a restart loop, and then CommonNodeLabelsManager throws an exception.To find the exception, search for "org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager" in /var/log/hadoop-yarn/yarn-yarn-resourcemanager-*.log.To resolve this error, delete the node label files. Then, restart ResourceManager to recreate the files.Resolution1.    Check file system health and locate the blocks:hdfs fsck /apps/yarn/nodelabels/ -locations -blocks -files2.    Remove the files:hdfs dfs -rm -skipTrash /apps/yarn/nodelabels/*3.    Restart ResourceManager:sudo stop hadoop-yarn-resourcemanagersudo start hadoop-yarn-resourcemanager4.    When ResourceManager restarts, it recreates the node label files. This resolves the restart loop. However, you can't submit YARN applications yet. Before you can submit YARN applications, manually add node label entries:yarn rmadmin -addToClusterNodeLabels "CORE(exclusive=false)"5.    List the labels to confirm that ResourceManager recreated them:yarn cluster --list-node-labels

Related informationUnderstand node types: master, core, and task nodes
Question: How can I troubleshoot stage failures in Spark jobs on Amazon EMR?

I want to troubleshoot stage failures in Apache Spark applications on Amazon EMR.

Short descriptionIn Spark, stage failures happen when there's a problem with processing a Spark task. These failures can be caused by hardware issues, incorrect Spark configurations, or code problems. When a stage failure occurs, the Spark driver logs report an exception similar to the following:org.apache.spark.SparkException: Job aborted due to stage failure: Task XXX in stage YYY failed 4 times, most recent failure: Lost task XXX in stage YYY (TID ZZZ, ip-xxx-xx-x-xxx.compute.internal, executor NNN): ExecutorLostFailure (executor NNN exited caused by one of the running tasks) Reason: ...

ResolutionFind the reason codeFor Spark jobs submitted with --deploy-mode client, the reason code is in the exception that's displayed in the terminal.For Spark jobs submitted with --deploy-mode cluster, run the following command on the master node to find stage failures in the YARN application logs. Replace application_id with the ID of your Spark application (for example, application_1572839353552_0008).yarn logs -applicationId application_id | grep  "Job aborted due to stage failure" -A 10You can also get this information from YARN ResourceManager in the application master container.Resolve the root causeAfter you find the exception, use one of the following articles to resolve the root cause:How do I resolve "Container killed on request. Exit code is 137" errors in Spark on Amazon EMR?How do I resolve "no space left on device" stage failures in Spark on Amazon EMR?How do I resolve ExecutorLostFailure "Slave lost" errors in Spark on Amazon EMR?
Question: How do I resolve the error "Exception in thread "main" java.lang.RuntimeException: Read throughput should not be less than 1" when moving data from DynamoDB to S3?

I created an AWS Data Pipeline to export data from Amazon DynamoDB to Amazon Simple Storage Service (Amazon S3). When I try to export the data, I get the "Exception in thread "main" java.lang.RuntimeException: Read throughput should not be less than 1" error.

ResolutionIf you're creating a new pipeline to export data from DynamoDB table, then create a pipeline using the Export DynamoDB table to S3 Data Pipeline template. For more information on creating the pipeline, see Part two: Export data from DynamoDB.If you're using an existing pipeline to export data from DynamoDB to S3, then you must use the latest EMR-DynamoDB connector for the pipeline. You might get the error when you export data with an older version of the EMR-DynamoDB connector.Be sure that the Amazon EMR version for the DynamoDB table is 5.24.0 or later. DynamoDB tables configured for On-Demand Capacity are supported only when using Amazon EMR release version 5.24.0 or later. For more information, see Export data from DynamoDB - Before you begin.To update your existing pipeline to use the latest EMR-DynamoDB connector and Amazon EMR version 5.24.0 or later, do the following:1.    Open the AWS Data Pipeline console.2.    Select the pipeline, and then choose Actions.3.    Choose Edit.4.    Choose EmrActivity in the left pane.5.    Expand the Activities pane, and then find the EmrActivity object.6.    In the Step field, enter the following JAR name:s3://dynamodb-dpl-#{myDDBRegion}/emr-ddb-storage-handler/4.11.0/emr-dynamodb-tools-4.11.0-SNAPSHOT-jar-with-dependencies.jar7.    Choose EmrCluster in the left pane.8.    Expand the Resources pane.9.    For Release Label, enter emr-5.24.0 or any of the later EMR release versions.10.    Choose Save.

Related informationHow can I use Data Pipeline to back up a DynamoDB table to an S3 bucket that's in a different account?
Question: How do I send feedback to AWS?

I have some suggestions to help improve an AWS document or service. ResolutionTo submit feedback about an AWS console or specific AWS service, open the service's console, and then choose Feedback. Or, use AWS re:Post. To submit general feedback about AWS re:Post, choose Feedback on any re:Post page. To submit feedback on an AWS Knowledge Center article in re:Post, comment on the article.To submit general feedback about AWS Documentation, choose Feedback on the AWS Documentation homepage. To submit feedback on a specific page, choose Provide feedback. You can also choose either Yes or No for Did this page help you? After submitting your vote, you have the option to enter detailed feedback.

Related informationHow do I get help with my AWS account and resources?
Question: I received GuardDuty UnauthorizedAccess brute force finding type alerts for my Amazon EC2 instance. What should I do?

Amazon GuardDuty detected alerts for the UnauthorizedAccess:EC2/RDPBruteForce or UnauthorizedAccess:EC2/SSHBruteForce finding types for my Amazon Elastic Compute Cloud (Amazon EC2) instance.

Short descriptionBrute force attacks can indicate unauthorized access to your AWS resources. For more information, see Finding types.

ResolutionFollow these instructions to check the GuardDuty finding type description, finding IDs, and detector IDs for more details about the brute force attack.Note: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.Check the GuardDuty finding type descriptionFollow the instructions to view and analyze your GuardDuty findings.In the finding details pane, note the title of the finding type similar to the following:"198.51.100.0 is performing RDP brute force attacks against i-99999999. Brute force attacks are used to gain unauthorized access to your instance by guessing the RDP password."In this example, the description indicates which Amazon EC2 instance is impacted, the direction of the brute force attack, and the IP address.Check the GuardDuty finding IDs and detector IDs1.    Open the GuardDuty console.2.    In the navigation pane, choose Findings.3.    In Finding type, choose the UnauthorizedAccess finding type.4.    In the finding type details pane, choose the Finding ID.5.    In Findings JSON, note the GuardDuty finding and detector IDs.6.    Run this AWS CLI command:Note: Replace your-detector-id and your-findings-id with your GuardDuty detector and finding IDs.aws guardduty get-findings --detector-id your-detector-id --finding-ids your-findings-id --query 'Findings[].Service.Action.NetworkConnectionAction.ConnectionDirection'You receive an output similar to the following:[    "INBOUND"]7.    Run this AWS CLI command:aws guardduty get-findings --detector-id your-detector-id --finding-ids your-findings-id --query 'Findings[].Service.Action.NetworkConnectionAction.RemoteIpDetails.IpAddressV4'You receive an output similar to the following:[    "198.51.100.0"]In this example, the Amazon EC2 instance security group allows SSH/RDP traffic and is open to the public.To mitigate the issue, you can restrict SSH/RDP traffic for only a set of IP addresses authorized to access the Amazon EC2 instance.To restrict SSH traffic, see Adding a rule for inbound SSH traffic to a Linux instance.To restrict RDP traffic, see Adding a rule for inbound RDP traffic to a Windows instance.

Related informationHow to use Amazon GuardDuty and AWS Web Application Firewall to automatically block suspicious hostsHow do I use GuardDuty to identify SSH brute force attacks on Linux instances?How do I set up a trusted IP address list for GuardDuty?
Question: How do I troubleshoot namespaces in a terminated state in my Amazon EKS cluster?

I tried to delete a namespace in my Amazon Elastic Kubernetes Service (Amazon EKS) cluster. However, the namespace is stuck in the "Terminating" status.

Short descriptionTo delete a namespace, Kubernetes must first delete all the resources in the namespace. Then, it must check registered API services for the status. A namespace gets stuck in Terminating status for the following reasons:The namespace contains resources that Kubernetes can't delete.An API service has a False status.Resolution1.    Save a JSON file like in the following example:kubectl get namespace TERMINATING_NAMESPACE -o json > tempfile.jsonNote: Replace TERMINATING_NAMESPACE with the name of your stuck namespace.2.    Remove the finalizers array block from the spec section of the JSON file:"spec": {        "finalizers": [            "kubernetes"        ]    }After you remove the finalizers array block, the spec section of the JSON file looks like this:"spec" : {    }3.    To apply the changes, run the following command:kubectl replace --raw "/api/v1/namespaces/TERMINATING_NAMESPACE/finalize" -f ./tempfile.jsonNote: Replace TERMINATING_NAMESPACE with the name of your stuck namespace.4.    Verify that the terminating namespace is removed:kubectl get namespacesRepeat these steps for any remaining namespaces that are stuck in the Terminating status.

Related informationWhat is Amazon EKS?Namespaces (on the Kubernetes site)