VishaalY/solutions-architect-hf-dataset huggingface dataset:
Question: How do I use the validation feature in AWS DMS?

How do I use the validation feature in AWS Database Migration Service (AWS DMS)?ResolutionBefore you enable and use the validation feature, review the limitations for validating AWS DMS tasks.Enable validation on a new taskOpen the AWS DMS console, and then choose Database migration tasks from the navigation pane.Choose Create task.Enter the details for your Task configuration and Task settings.From the Task settings section, select Enable validation and Enable CloudWatch logs.Configure the Advanced task settings, if needed.Choose Create task.Enable validation on an existing taskOpen the AWS DMS console, and then choose Database migration tasks from the navigation pane.Select your task, and then check that the status of the task is Failed or Stopped.Choose Actions, and then choose Modify.From the Task settings section, select Enable validation.Choose Save.View the results of the validationOpen the AWS DMS console, and then choose Database migration tasks from the navigation pane.Choose the name of your task.From the Table statistics section, review the results. For more information, see Replication task statistics.Revalidate tables in a taskOpen the AWS DMS console, and then choose Database migration tasks from the navigation pane.Choose the name of your task.From the Table statistics section, select the table, and then choose Validate again.To review the validation metrics, choose Database migration tasks from the navigation pane.Choose the name of the task, and then review the CloudWatch metrics section.

Related informationValidating AWS DMS tasksData validation task settings
Question: How can I see the Amazon S3 source file for a row in an Athena table?

I'm querying a table in Amazon Athena. I want to find out which Amazon Simple Storage Service (Amazon S3) file is the source for each row in the output, or which rows correspond to a specific file.

Short descriptionTo find out which Amazon S3 file contains data that is returned by a specific row in an Athena table, run a SELECT $path query.To find out which Athena table rows are associated with a specific Amazon S3 file, run a SELECT query with the WHERE $path condition.These queries are useful for investigating unexpected data and building queries that require information about the source data.

ResolutionBefore you run the following queries, be sure that you have the following:An Athena table. If you don't have a table, run a CREATE TABLE statement.An AWS Identity and Access Management (IAM) user or role that has permissions to run Athena queries.To find the S3 file that's associated with a row of an Athena table:1.    Run a SELECT query against your table to return the data that you want:SELECT * FROM "my_database"."my_table" WHERE year=2019;2.    To find the Amazon S3 source file for the data, run a query similar to the following:SELECT "$path" FROM "my_database"."my_table" WHERE year=2019;The query returns the Amazon S3 path for the data:s3://awsexamplebucket/datasets_mytable/year=2019/data_file1.jsonTo return Athena table rows that originate from a specific Amazon S3 file:1.    Confirm the name and location of the Amazon S3 object that you want to retrieve rows for. If you don't know the object name and location, run the ls command to list objects under a specific S3 bucket or prefix. In the following example, replace s3://awsexamplebucket/my_table/my_partition/ with your partition path.aws s3 ls s3://awsexamplebucket/my_table/my_partition/2.    In Athena, run a SELECT query against your table to return data that's associated with the Amazon S3 file:SELECT *,"$path" FROM "my_database"."my_table" WHERE regexp_like("$path", 's3://awsexamplebucket/my_table/my_partition/file-01.csv')The query returns data that matches the Amazon S3 path:id    name    year    $path3     John    1999    's3://awsexamplebucket/my_table/my_partition/file-01.csv'4     Jane    2000    's3://awsexamplebucket/my_table/my_partition/file-01.csv'This query also supports some wildcard functionality. For example, to return all data that contains part of a file name, run a query like this:SELECT *,"$path" FROM "my_database"."my_table" WHERE regexp_like("$path", '\-02')This query returns all rows with file names that contain -02:id    name    year    $path13    Mia     2009    's3://awsexamplebucket/my_table/my_partition/file-02.csv'14    Mary    2010    's3://awsexamplebucket/my_table/my_partition/file-02.csv'15    Max     2011    's3://awsexamplebucket/my_table/my_partition/file-02.csv'16    Merriam 2012    's3://awsexamplebucket/my_table/my_partition/file-02.csv'Related informationParquet SerDeConnecting to data sources
Question: How can I troubleshoot or update EC2 instances in my EC2 Auto Scaling group by temporarily removing them?

I want to temporarily remove Amazon Elastic Compute Cloud (Amazon EC2) instances from my Amazon EC2 Auto Scaling group so that I can troubleshoot or update them.

ResolutionYou can't stop, start, or reboot EC2 instances that are in the InService state in an Auto Scaling group without first terminating the instance. Use one of the following methods to troubleshoot or perform updates that require stopping, starting, or rebooting the instance:Temporarily remove an instanceYou can temporarily remove an instance by setting it to the Standby state. To do this, see the following:Temporarily remove an instance (console)Temporarily remove an instance AWS Command Line Interface (AWS CLI))After initiating the Standby state, perform troubleshooting or updating tasks. When you are done troubleshooting, return the instance to the InService state.Suspend the Terminate and HealthCheck processesOpen the EC2 console and then choose Auto Scaling groups from the navigation pane.Choose the Auto Scaling group where the affected instance is located.Select Advanced configurations, Edit.Under Suspended processes, select the check boxes next to Terminate and HealthCheck, and then choose Update.Important: Suspend both the Terminate and HealthCheck processes. If HealthCheck isn't selected, then the instance shows as unhealthy when the instance is stopped or rebooted. And, when you resume the Terminate process, Auto Scaling replaces the instance because of the failed health check.Complete troubleshooting or updating tasks.On the Auto Scaling Groups page, select the Auto Scaling group. Then, scroll down to Advanced configurations and choose EditFor Suspended processes, uncheck the Terminate and HealthCheck processes, and then choose Update.Follow these instructions to suspend the Terminate and HealthCheck processes using the AWS CLI:Note: If you receive errors when running AWS CLI commands, make sure that you're using the most recent version of the AWS CLI.1. Use the aws autoscaling suspend-processes command to suspend the Auto Scaling processes. In the following example command, replace [X] with the name of your Auto Scaling group.aws autoscaling suspend-processes --auto-scaling-group-name [X] --scaling-processes Terminate HealthCheck2. Perform your troubleshooting or updating tasks.3. Use the aws autoscaling resume-processes command to resume the processes. In the following example command, replace [X] value with the name of your Auto Scaling group.aws autoscaling resume-processes --auto-scaling-group-name [X] --scaling-processes Terminate HealthCheck

Related informationSuspend and resume a process for an Auto Scaling group
Question: How do I resolve the "maximum number of failed attempts" error when I try to verify my AWS account by phone?

When I try to phone-verify my account, I get an error similar to the following: “You have exceeded the maximum number of failed attempts at PIN verification that we allow. Your registration cannot proceed at this time.” How do I resolve this issue?ResolutionAWS Support can help you to manually activate your account:Sign in to your AWS account using the email address and password that you specified when creating your account.Open the AWS Support console, and then choose Create case.Choose Account and Billing Support.For Type, select Account.For Category, select Activation.In the Case description section, provide a date and time when you can be reached.In the Contact options section, select Chat for Contact methods.Choose Submit.AWS Support will contact you and attempt to manually activate your account.

Related informationHow do I create and activate a new AWS account?
Question: How can I deregister unused AMIs and delete snapshots from my account?

Unused AMIs or snapshots associated with my AWS account are causing an increase in Amazon Elastic Block Store (Amazon EBS) snapshot charges. I want to deregister unused AMIs or delete snapshots from my AWS account.

ResolutionNote: You can't deregister AMIs shared with you by an external AWS account. And, you can't delete EBS backup snapshots.If you try to deregister a shared AMI, you receive an error similar to the followingFailed to deregister the AMI ami-xxxxx. API error: "Not authorized for image:ami- xxxxx"You aren't billed for snapshots that another AWS account owns and shares with your account. You're billed only when you copy the shared snapshot to your account. You're also billed for EBS volumes that you create from the shared snapshot.To identify rising costs of EBS snapshots, review the list of AMIs associated with your account in that AWS Region:Open the Amazon Elastic Compute Cloud (Amazon EC2) console, and then select AMIs.From the dropdown list, select Owned by me.Choose the gear symbol in the upper-right corner.Turn on the Product code option, and then select Confirm. This option filters for AMIs owned by the account and denotes AWS Marketplace AMIs in the Product code column.Select the AMIs that you no longer need.Select Actions, Deregister AMI.Note: Deregistering an EBS-backed AMI doesn't affect any snapshots created for the instance's volumes when the AMI was created.
Question: How do I resolve the "The provided key element does not match the schema" error when importing DynamoDB tables using Hive on Amazon EMR?

When I try to import Amazon DynamoDB tables into Amazon EMR using Hive, I get the error "The provided key element does not match the schema (Service: AmazonDynamoDBv2; Status Code: 400; Error Code."ResolutionThis error usually happens when you have an incorrect schema, corrupt data, or mismatched data. If you still get the error message after ruling out these common causes, then check the Hive application logs. If you turned on logging, then you can find the logs on Amazon Simple Storage Service (Amazon S3) in the location that looks similar to this:s3://example-log-location/example-cluster-id/node/example-ec2-master-instance-id/applications/hiveOtherwise, you can find the logs in the /mnt/var/log/hive directory on the master node of the EMR cluster. You can connect to the master node, and then check for logs. The logs look similar to the following:Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"countryasin":"LOCATION '${INPUT}';","hts_type":null,"hts_code":null}at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:565)at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:86)... 17 moreCaused by: java.lang.RuntimeException: com.amazonaws.services.dynamodbv2.model.AmazonDynamoDBException: The provided key element does not match the schema (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException; Request ID: 0FF3KB36M2SJD8E79BUPOUP943VV4KQNSO5AEMVJF66Q9ASUAAJG)The row that's mentioned in the error message ({"countryasin":"LOCATION '${INPUT}';","hts_type":null,"hts_code":null}) is part of the Hive script. This Hive script is in the same Amazon Simple Storage Service (Amazon S3) location as the input files. The import job is sending the Hive script to the DynamoDB table as data, as well as using it in the import job. To resolve this problem, move the Hive script to a different Amazon S3 location.

Related informationOptimizing performance for Amazon EMR operations in DynamoDBDynamoDBMapper classView log files
Question: How do I attach a security group to my Elastic Load Balancer?

How do I configure and attach a security group to my Elastic Load Balancing (ELB) load balancer?ResolutionIf you're using a Classic Load Balancer, follow instructions at Manage security groups using the console or Manage security groups using the AWS CLI.Note: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.If you're using an Application Load Balancer, follow the instructions at Security groups for your Application Load Balancer.If you're using a Network Load Balancer, update the security groups for your target instances because Network Load Balancers don't have associated security groups.If your target type is an IP and the target group protocol is TCP/TLS/UDP/TCP_UDP - TCP/TLS, then the protocol defaults to load balancer private IP as the source IP. This means that it's a best practice to allowlist load balancer private IPs on your target security group. UDP/TCP_UDP defaults to preserve client IP addresses. This means that it's a best practice to allowlist client IPs in the target security group.Note: If preserve client IP isn't activated and the target security groups are allow-listing load balancer private IPs, then you are allowing all incoming traffic to access your service. If your service is designed to be access-restricted to specific CIDR ranges, then use network access control list (network ACL) to allow-list specific CIDRs and deny the rest. Or, you can activate client IP preservation and set up restrictions on the target security group as discussed later.If your target type is an instance and the target group protocol is TCP/ TLS/ UDP/TCP_UDP, then the default behavior of the Network Load Balancer is to preserve the client IP address. If the client IP preservation setting remains at the default value, then it's a best practice to allowlist client IP addresses on your target security group.As needed, you can change the default client IP preservation behavior for TCP/TLS target groups by setting a target group attribute "preserve_client_ip.enabled." It's not possible to change this behavior for UDP/TCP_UDP protocol target groups. Depending on whether client IP preservation is active or not active (based on your configuration choices), it's a best practice to adjust the IP CIDRs allow-listed on the target security groups. If the client IP preservation is activated, then it's a best practice to allowlist client IP addresses. If it's not active, then it's a best practice to allowlist load balancer private IP addresses. For more information on client IP preservation behavior of the Network Load Balancer, see Target groups for your Network Load Balancers.Note: Make sure that you associate at least one security group with each Classic or Application Load Balancer, and that the security group allows connections between the load balancer and associated backend instances.

Related informationMonitor your Classic Load BalancerMonitor your Application Load Balancers
Question: Why do I get the AmazonS3Exception "Access Denied with Status Code: 403" in Amazon Athena when I query a bucket in another account?

I'm using Amazon Athena to query objects in an Amazon Simple Storage Service (Amazon S3) bucket that's in a different account. Some of the objects in the bucket are owned by a third account. I get the AmazonS3Exception "Access Denied with Status Code: 403" when I run the query.

Short descriptionThis error commonly occurs when you try to query logs written by another AWS service, such as AWS CloudTrail, Amazon CloudFront, and Amazon Virtual Private Cloud (Amazon VPC). These services log events to Amazon S3. The bucket owner has full access to the S3 objects. The second account doesn't own the bucket or the objects. That's why the second account gets an access denied error when querying an Athena table that references these S3 objects.

ResolutionIt's not possible to transfer ownership of Amazon S3 objects. Instead, use one of the following options:In the Athena account, assume an AWS Identity and Access Management (IAM) role that has access to both the bucket and the objects. For more information, see Tutorial: Delegate access across AWS Accounts using IAM roles.Follow the instructions at How can I copy S3 objects from another AWS account? to copy the objects to a bucket in the Athena account. Then, delete any redundant or unnecessary objects from the destination bucket to avoid unnecessary charges.

Related informationWhy can't I access an object that was uploaded to my Amazon S3 bucket by another AWS account?How do I transfer the ownership of Amazon S3 objects to a different AWS account?Why do I get the "Access Denied" error when I run a query in Amazon Athena?
Question: Why can't I buy the Amazon EC2 Reserved Instance that I need through the EC2 console?

I tried to buy a particular Amazon EC2 Reserved Instance (RI) through the AWS Management Console. But I can't see any RIs, or I receive an error after choosing Search.

ResolutionHere are some common reasons why you might not see the RIs that you're looking for:AWS can't guarantee the capacity that you're looking for. Purchase capacity in a different Availability Zone in the same Region instead. Also, be sure that the RI that you're trying to purchase is for an instance configuration that AWS offers.You've reached your limit for RIs in a Region. For information about viewing and increasing your RI limit, see Amazon EC2 service quotas.You're purchasing a No Upfront RI with a new AWS account. AWS restricts the ability for new accounts to purchase No Upfront RIs. The restriction on purchasing No Upfront RIs is lifted automatically, usually after about a month. For more information, see Reserved Instances payment options.AWS Support can't manually update these limits or add No Upfront RIs to your account on your behalf.Note: AWS Support can't purchase RIs on your behalf.

Related informationAmazon EC2 Reserved InstancesHow to purchase Reserved InstancesBuy Reserved Instances
Question: How do I create Route 53 traffic policy records using the AWS CLI?

I want to create traffic policy records in Amazon Route 53 to route DNS traffic flow to multiple resources.

Short descriptionYou can use the AWS Command Line Interface (AWS CLI) to create Route 53 traffic policy records. Before you begin, install and configure the AWS CLI.If you want to use the Route 53 console to create a traffic policy, then see Creating a traffic policy.

ResolutionCreate a traffic policy with the AWS CLICreate a JSON file that defines your traffic policy configuration. For basic syntax, endpoints, and rules, refer to Traffic policy document format.To create your traffic policy in the AWS CLI, run the create-traffic-policy command with your own parameters:$ aws route53 create-traffic-policy --name POLICY_NAME --document file://JSON_FILENote: Replace POLICY_NAME with your policy name. Replace JSON_FILE with the full path of your JSON file.The following parameters are required:--name--documentThe following parameters are optional:--comment--cli-input-json--generate-cli-skeletonIn the output, note the traffic policy ID and version number:{  "TrafficPolicy": {    "Document": JSON_DOCUMENT,    "Version": VERSION_NUMBER,    "Type": "POLICY_TYPE",    "Id": "TRAFFIC_POLICY_ID",    "Name": "POLICY_NAME"  }}To create a traffic policy record, run the create-traffic-policy-instance API call.$ aws route53 create-traffic-policy-instance --hosted-zone-id VALUE --name VALUE --ttl VALUE --traffic-policy-id VALUE --traffic-policy-version VALUENote: Replace each VALUE with your relevant value for that parameter.Example traffic policyThe following example traffic policy creates a weighted record that points to two endpoints.This policy specifies the following values:The current policy format version (AWSPolicyFormatVersion)The record type (RecordType)Note: Configure this value based on your endpoint type. In this example, the record type is A.Two endpoints (EndPointReference) that point to IP addresses (Type)Weighted rules (RuleType) and a different weight for each endpoint (Weight)The traffic policy's starting pointNote: In the following example, StartRule specifies the policy starts with a rule instead of an endpoint (StartEndpoint).The health check setting (EvaluateTargetHealth)Note: You can configure this setting with HealthCheck. The following example does not perform a health check.{  "AWSPolicyFormatVersion": "2015-10-01",  "RecordType": "A",  "Endpoints": {    "endpoint-1": {      "Type": "value",      "Value": "192.0.1.1"      },    "endpoint-2": {      "Type": "value",      "Value": "192.0.1.2"      }    },  "Rules": {    "weighted-rule-name": {      "RuleType": "weighted",      "Items": [        {"Weight": "30",        "EvaluateTargetHealth": true,        "EndpointReference": "endpoint-1"        },        {"Weight": "20",        "EvaluateTargetHealth": false,        "EndpointReference": "endpoint-2"        }      ]    }  },  "StartRule": "weighted-rule-name"}Related InformationSupported DNS record types
Question: "I activated GuardDuty in my environment, but GuardDuty didn't generate any finding types"

I activated an Amazon GuardDuty account but I haven't received any finding types. How can I troubleshoot this?Short descriptionActivating GuardDuty immediately begins to monitor for security threats. If GuardDuty discovers a security issue, then a finding type is generated. If GuardDuty doesn't detect security threats, then finding types aren't generated.

ResolutionTo troubleshoot why GuardDuty hasn't generated any finding types, check the following configurations:The data sourcesThe GuardDuty statusThe trusted IP listsData sourcesGuardDuty uses its data sources to detect unauthorized and unexpected activity with resource types for some AWS services. The data sources include:AWS CloudTrail management event logs.Virtual Private Cloud (Amazon VPC) flow logs.DNS logs.CloudTrail data events for Amazon Simple Storage Service (Amazon S3)Kubernetes audit logAmazon Elastic Block Store (Amazon EBS) volume dataIt's a best practice to activate GuardDuty Kubernetes Protection, Amazon S3 protection, and Malware Protection which aren't activated by default.Note: GuardDuty only processes DNS logs if you use the default VPC DNS resolver. All other types of DNS resolvers won't generated DNS based findings.GuardDuty statusGuardDuty must be activated for finding types to generate. If GuardDuty is suspended or disabled, then no finding types are generated. It's a best practice to  activate GuardDuty in all supported AWS Regions. This allows GuardDuty to generate finding types for unauthorized or unusual activity even in Regions that you aren't actively using.Trusted IP listsYou can add IP addresses that you trust to communicate in your AWS environment to trusted IP lists. Trusted IP lists prevent GuardDuty from generating finding types for events that occurred from trusted IP addresses.It's a best practice to use a suppression rule instead of a trusted IP list to for awareness of detected issues in your environment. The suppression rule reduces the notifications from finding types. A suppression rule automatically archives new findings generated by GuardDuty that match specific criteria. You can review suppressed findings from the GuardDuty console by changing the Findings view dropdown menu from Current to Archived.To create GuardDuty findings for testing, do one of the following:Create sample findings from the GuardDuty console or API.Generate common GuardDuty findings automatically using the guardduty_tester.sh script.For more information, see How do I set up a trusted IP address list for GuardDuty?Related informationGetting started with GuardDutyWhy did GuardDuty send me alert findings for a trusted IP list address?