VishaalY/solutions-architect-hf-dataset huggingface dataset:
Question: How do I troubleshoot issues when bringing my custom container to Amazon SageMaker Studio?

I want to troubleshoot issues when building a custom Amazon SageMaker Studio.

ResolutionA SageMaker image is a file that identifies kernels, language packages, and other dependencies that are required to run a Jupyter notebook in SageMaker Studio. These images are used to create an environment from where you can run your Jupyter notebooks. If the built-in images provided by SageMaker don't support your use case, you can bring a custom image to use in SageMaker Studio.You might get errors when using your customer image in SageMaker Studio. These errors might result mostly from an incorrect configuration that you set up during the container image build process. Therefore, be sure that the custom image is compatible with SageMaker Studio.To do so, check the following when you build the Dockerfile:You have set the DefaultUID and DefaultGID values accurately. SageMaker Studio only supports a specific combination of DefaultUID and DefaultGID. Be sure that DefaultUID and DefaultGID are set to 1000 and 100, respectively for a non-privileged user. Be sure that the values are both set to 0 for the root user.The 'opt/.sagemakerinternal' and 'opt/ml' directories aren't used. These directories are reserved.When you create the configuration file app-image-config-input.json, be sure that the Name value for KernelSpec in this file matches the Name value for the associated image.Example Dockerfile that installs Python packages and sets the scope to non-privileged users:FROM public.ecr.aws/amazonlinux/amazonlinux:2ARG NB_USER="sagemaker-user"ARG NB_UID="1000"ARG NB_GID="100"RUN \    yum install --assumeyes python3 shadow-utils && \    useradd --create-home --shell /bin/bash --gid "${NB_GID}" --uid ${NB_UID} ${NB_USER} && \    yum clean all && \    python3 -m pip install ipykernel && \    python3 -m ipykernel installUSER ${NB_UID}You can view the error messages in Amazon CloudWatch logs when either of the following happens:Image version creation fails.You get an error when launching the image in SageMaker Studio.You can find these messages in the log group /aws/sagemaker/studio and log stream $domainID/$userProfileName/KernelGateway/$appName.Note: This article assumes that the AWS Identity and Access Management (IAM) user or role has the AmazonSageMakerFullAccess policy attached to it.

Related informationCustom SageMaker image specifications
Question: How do I deploy Lambda functions with external libraries using AWS Cloud9?

I want to add external libraries to an AWS Lambda function that I developed using AWS Cloud9. How do I do that?ResolutionNote: The following procedure uses the Python runtime as an example. A similar procedure can be used for Node.js Lambda functions.After you've created a Lambda function in your AWS Cloud9 environment, do the following:Import the external libraries' modules to your Lambda function's parent folder in your AWS Cloud9 environment1.    Open the AWS Cloud9 console.2.    Open your AWS Cloud9 environment.3.    In the left navigation pane, choose Environment. The Environment window opens.4.    Open the context (right-click) menu for your function's parent folder.Note: Your function's parent folder is listed with the same name as your function.5.    Choose Open Terminal Here. A new terminal session starts.6.    Import an external library module by running the following pip command in the new terminal:Important: Replace ModuleName with the name of the module that you want to import.python -m pip install --target=./ ModuleNameNote: To make sure that the correct versions of Python packages are installed, it's a best practice to specify a requirements.txt file in the pip command. For more information, see the Requirements file format section of the pip documentation.7.    Repeat step six for each module that you want to import. The imported modules appear in your function's parent folder.Deploy your Lambda function using the AWS ToolkitFollow the instructions in Deploying a serverless application in the AWS Cloud9 User Guide. After deploying your function, you can run your code using the additional modules that you imported.For more information, see Working with AWS serverless applications using the AWS Toolkit.

Related informationWorking with AWS Lambda functions using the AWS ToolkitLocally debug a serverless application from an IDE
Question: How do I save my work in Amazon Lightsail?

I want to create a backup of my Amazon Lightsail instance to save my work.

ResolutionCreate a snapshot of your Lightsail instance to use as a backup. Lightsail snapshots preserve the state of your instance at the time the snapshot was taken.After you create a snapshot, you can delete the running Lightsail instance. If you need that instance again, you can create a new Lightsail instance from that snapshot.Note: Snapshots are billed at a lower rate than running Lightsail instances.

Related informationSnapshots in Amazon LightsailAmazon Lightsail pricing
Question: How do I resolve the "cannot restore index [.kibana]" error in Amazon OpenSearch Service?

When I try to restore indices from manual snapshots in Amazon OpenSearch Service, the restoration fails with an error message. How do I resolve this?ResolutionAmazon OpenSearch Service monitors the .kibana index and recreates the index when it's deleted. This behavior can cause the restoration to fail with the following error message:{    "error": {        "root_cause": [{            "type": "snapshot_restore_exception",            "reason": "[repository-name:snapshot-name/1A2B34aZQFWQpFOYYJfxmQ] cannot restore index [.kibana] because an open index with same name already exists in the cluster. Either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name"        }],        "type": "snapshot_restore_exception",        "reason": "[repository-name:snapshot-name/1A2B34aZQFWQpFOYYJfxmQ] cannot restore index [.kibana] because an open index with same name already exists in the cluster. Either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name"    },    "status": 500}To resolve this issue, perform the following steps:1.    Restore the indices and rename the .kibana index, like this:# restore indices.   $ curl -XPOST -H 'Content-Type: application/json' 'https://your-domain-end-point/_snapshot/your-repository-name/your-snapshot-name/_restore' -d'{    "indices": "*",    "ignore_unavailable": true,    "include_global_state": true,    "rename_pattern": ".kibana",    "rename_replacement": "restored_.kibana"}In this example, the .kibana index is renamed to "restored_.kibana".2.    Use the _reindex API operation to rename "restored_.kibana" back to ".kibana", like this:# reindex restored_.kibana to .kibana $ curl -XPOST -H 'Content-Type: application/json' 'https://your-domain-end-point/_reindex' -d'{    "source": {        "index": "restored_.kibana"    },    "dest": {        "index": ".kibana"    }}You can now restore your indices from a manual snapshot.
Question: What are some of the best practices to create a composite key for my Amazon DynamoDB table?

I want to learn the best practices to create a composite key for my Amazon DynamoDB table.

ResolutionWhen you create a composite key for your DynamoDB table, it's a best practice to do the following:Choose a unique partition key with high cardinality: The partition key for a DynamoDB table must be unique. Choosing a partition key with low cardinality might affect the way data is stored and distributed over different partitions in the table. Therefore, be sure to choose a partition key with a high cardinality. For more information on data distribution in DynamoDB, see Partitions and data distributions.Choose a sort key that compliments access patterns: Choose a sort key that gathers related information together in one place so that the information can be queried efficiently. Carefully choosing the sort key attribute is important because it can significantly improve the selectivity of the items retrieved by a query. Also, a composite sort key allows you to define a hierarchical relationship in your data so that you can query at any level of the hierarchyFor example, a composite sort key must look similar to the following:[building]#[department]#[team]#[manager]In this case, you can select employeeID as the partition key so that you can have unique combinations all the way up to manager.For more information, see Best practices for using sort keys to organize data. The combination of partition key and sort key or composite sort key creates a new unique composite key that allows for effective querying and distribution of data over partitions in DynamoDB.Consider the following when you design your composite key:The combination of the partition key and sort key must form a unique value.You can have repeating partition keys with different sort keys.

Related informationChoosing the right DynamoDB partition key
Question: How do I troubleshoot Classic Load Balancer capacity issues in ELB?

My Amazon CloudWatch metric SurgeQueueLength for my Classic Load Balancer has an increased maximum statistic. Clients also receive HTTP 503 Service Unavailable or HTTP 504 Gateway Timeout errors when they try to connect to my Classic Load Balancer. How do I troubleshoot these Elastic Load Balancing capacity issues?Short descriptionThe Classic Load Balancer metric SurgeQueueLength measures the total number of requests queued by your Classic Load Balancer. An increased maximum statistic for SurgeQueueLength indicates that backend systems can't process incoming requests as fast as the requests are received. Possible reasons for a high SurgeQueueLength metric include:Overloaded Amazon Elastic Compute Cloud (Amazon EC2) instances behind the Classic Load Balancer that are unable to process all incoming requestsApplication dependency issues due to external resource performance issuesMaximum allowable connection limits for instancesWhen requests exceed the maximum SurgeQueueLength, the SpilloverCount metric starts to measure rejected requests. The maximum SurgeQueueLength is 1024.

ResolutionConfigure Auto Scaling groups with your Classic Load Balancer to scale your instances based on demand.Configure CloudWatch to monitor your Classic Load Balancer.Turn on detailed monitoring for instances behind your Classic Load Balancer to monitor the CPUUtilization metric. You can also get statistics for a specific instance. If CPU utilization spikes, your instances are too busy processing existing requests to accept new requests. Consider scaling out your instances or scaling up to an instance type with more processing power.Troubleshoot HTTP errors for your Classic Load Balancer. Load-related issues typically return 503 or 504 HTTP errors.If your Amazon EC2 instances run Apache, then see How do I tune Apache web server's memory allocation to prevent out of memory errors in an EC2 Linux instance?For performance issues related to normal traffic increases over time, consider increasing your instance capacity.

Related informationCloudWatch metrics for your Classic Load Balancer
Question: Why is the TIMESTAMP result empty when I query a table in Amazon Athena?

When I query a column of TIMESTAMP data in my Amazon Athena table, I get empty results or the query fails. The data exists in the input file.

Short descriptionThe TIMESTAMP data might be in the wrong format. Athena requires the Java TIMESTAMP format:YYYY-MM-DD HH:MM:SS.fffffffffTo change TIMESTAMP data to the correct format:Define the column as STRING.Use Presto's date and time functions to read the column as DATE or TIMESTAMP in your query.

ResolutionDefine the column as STRINGRun a command similar to the following:CREATE EXTERNAL TABLE 'test' ('ts' STRING)ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'LOCATION 's3://doc-example-bucket/path-to-data/'Use Presto's date and time functions to read the column as DATE or TIMESTAMP in your queryIf your input data is in the following format:20200921T01500520200923T095605Then use the date_parse function as shown in the following example:SELECT date_parse(ts,'%Y%m%dT%h%i%s') FROM timestamptestcsv3Note: The format in the date_parse(string,format) function must be the TIMESTAMP format that's used in your data.If your input data is in ISO 8601 format, as in the following:2020-11-14T02:34:48ZThen use the from_iso8601_timestamp() function to read the ts field as TIMESTAMP. Example:SELECT from_iso8601_timestamp(ts) FROM timestamptestcsv3Related informationData types in Amazon Athena
Question: How do I fix an Amazon WorkDocs Drive for Windows that is stuck or crashing after logging in?

After logging in, my Amazon WorkDocs Drive client for Windows is stuck at the authentication screen or is shutting down. How can I fix this?Short descriptionThe Message Queuing service is failing or not in the Started status. The WorkDocs Drive client requires the Message Queuing service to write to the System Journal file.

ResolutionFor Windows 10 OS1.    Open a PowerShell session as Administrator.2.    Run the Enable-WindowsOptionalFeature -Online -FeatureName MSMQ-ADIntegration -All command.For Windows ServerFirst, uninstall the Message Queueing service from the Windows operating system.1.    Open a PowerShell session as Administrator.2.    Run the Import-Module ServerManager command.3.    Run the Uninstall-WindowsFeature -Name MSMQ -Restart command. The Windows operating system restarts.After the Windows operating system restarts, reinstall the Messaging Queueing service.1.    Open a PowerShell session as Administrator.2.    Run the Import-Module ServerManager command.3.    Run the Install-WindowsFeature -Name MSMQ command.You can now log in to the WorkDocs Drive client.

Related informationTroubleshooting Amazon WorkDocs issues
Question: How can I publish a message to an Amazon SNS topic using a Lambda function?

I want to publish a message to an Amazon Simple Notification Service (Amazon SNS) topic from an AWS Lambda function.

ResolutionFollow these steps to send a message to an Amazon SNS topic using a Lambda function.Note: The example in this article uses a Python runtime, but you can use your preferred Lambda runtimes.1.    Create an Amazon SNS topic.2.    Create a Lambda function.3.    From the Lambda function, make sure that the Lambda execution role has permissions to publish the SNS message similar to the following:Note: Replace your-region, your-account-number, and your-topic-name with your variables.{  "Version": "2012-10-17",  "Statement": [    {      "Sid": "PublishSNSMessage",      "Effect": "Allow",      "Action": "sns:Publish",      "Resource": "arn:aws:sns:<your-region>:<your-account-number>:<your-topic-name>"    }  ]}4.     Choose the Code tab. Then, in Code source, copy and paste the following code for the Publish Boto3 action into the code editor:import jsonimport boto3client = boto3.client('sns')def lambda_handler(event, context):   response = client.publish(TopicArn='arn:aws:sns:<region>:<account-number>:<topic-name>',Message="Test message")   print("Message published")   return(response)5.    Test your Lambda function.If your Lambda function is connected to an Amazon Virtual Private Cloud (Amazon VPC), then do one of the following:If your Lambda function requires internet access, make sure that you grant internet access. For more information, see How do I give internet access to a Lambda function that's connected to an Amazon VPC?-or-For private networks, create an Amazon VPC endpoint for Amazon SNS in the same VPC. For more information, see Creating an Amazon VPC endpoint for Amazon SNS.

Related informationWhy do I get an authorization error when I try to subscribe my Lambda function to my Amazon SNS topic?
Question: Why does the HTTPS connection between my CloudFront distribution and load balancer fail?

I have HTTPS and HTTP listeners configured on my Classic Load Balancer or Application Load Balancer as the origin for my Amazon CloudFront distribution. The HTTPS communication between CloudFront and my load balancer fails. I want to resolve the HTTPS communication issues.

ResolutionThe HTTPS communication failure might be caused by issues with the associated SSL certificate, security groups, or network access control list (ACL). Be sure that your distribution and load balancer meet the following security requirements:You must have a valid SSL certificate installed on the load balancer. If you're still getting HTTPS errors after installing an SSL certificate, troubleshoot the SSL connection between CloudFront and the custom origin server.If your CloudFront distribution connects to your load balancer on port 443, then the security groups associated with your load balancer must allow traffic on port 443 from CloudFront IP addresses. For more information on updating security groups, see Configure security groups for your Classic Load Balancer or Security groups for your Application Load Balancer. To allow only CloudFront IP addresses in security group attached to your distribution, use the AWS managed prefix list. This list contains all CloudFront IP addresses and is updated automatically if there are any changes in IP addresses. For more information, see Limit access to your origins using the AWS managed prefix list for Amazon CloudFront.The network ACLs associated with your load balancer's Amazon Virtual Private Cloud (Amazon VPC) must allow traffic from CloudFront on HTTPS ports (typically port 443).Note: Application Load Balancers support multiple TLS certificates with smart selection using Server Name Indication (SNI). If your CloudFront distribution caches based on the host header, then verify that the Application Load Balancer has a TLS certificate configured with the same name. Otherwise, the Application Load Balancer offers its default certificate, which might not match the SNI associated with the ClientHello message from CloudFront.

Related informationRequiring HTTPS for communication between CloudFront and your custom origin
Question: How do I use IAM Identity Center permission sets?

I want to use AWS IAM Identity Center (successor to AWS Single Sign-On) permission sets to provide users and groups access to an AWS account.

ResolutionUse IAM Identity Center to create a permission set for a user or group. Then, confirm the permissions from the user portal. In the following example, a federated user receives the ViewOnlyAccess permission set.Create a ViewOnlyAccess permission setOpen the IAM Identity Center console.In the navigation pane, choose AWS accounts.Select the AWS account that you want to create the permission set for. Then, choose Assign users or groups.Select the users that you want to create the permission set for. Then, choose Next.In Assign permission sets, choose Create permission set. For more information, see Create a permission set.Choose Create.In Permission set, choose ViewOnlyAccess, and then choose Finish.When you successfully create your permission set, you receive the following message:"We have successfully configured your AWS account. Your users can access this AWS account with the permissions you assigned".Verify that the user has ViewOnlyAccess permissionChoose Proceed to AWS accounts.In the navigation pane, choose Dashboard.In AWS access portal URL, choose your access portal URL.In the user portal login page, log in to your user portal with your user name and password for the IAM Identity Center user.Choose the AWS Account icon.Choose the account dropdown list. Note that the account has ViewOnlyAcccess permission.Choose Management console for the ViewOnlyAccess set.You are signed in to the AWS Management Console with ViewOnlyAcccess permission.Remove or delete permission setsYou can remove a permission set from an AWS account or completely delete it. Before you delete a permission set, you must remove it from all accounts. For more information, see Delete permission sets.

Related informationSingle sign-on access to AWS accounts
Question: How do I use external Python libraries in my AWS Glue 1.0 or 0.9 ETL job?

I want to use an external Python library in my AWS Glue 1.0 or 0.9 extract, transform, and load (ETL) job.

Short descriptionTo use an external library in an Apache Spark ETL job, do the following:1.    Package the library files in a .zip file (unless the library is contained in a single .py file).2.    Upload the package to Amazon Simple Storage Service (Amazon S3).3.    Use the library in a job or JobRun.

ResolutionThe following is an example of how to use an external library in a Spark ETL AWS Glue 1.0 or 0.9 ETL job.Important: If you want to use an external library in your AWS Glue 2.0 job, then see How do I use external Python libraries in my AWS Glue 2.0 ETL job? If you want to use an external library in a Python shell job, then follow the steps at Providing your own Python library.1.    Create a Python 2 or Python 3 library for boto3. Be sure that the AWS Glue version that you're using supports the Python version that you choose for the library. AWS Glue version 1.0 supports Python 2 and Python 3, and AWS Glue version 0.9 supports only Python 2.Note: Libraries and extension modules for Spark jobs must be written in Python. Libraries, such as pandas, that are written in C aren't supported in Glue 0.9 or 1.0. If you need to use a Library written in C, then upgrade AWS Glue to at least version 2.0 and use the --additional-python-modules option. For more information, see How do I use external Python libraries in my AWS Glue 2.0 ETL job?2.    Launch an Amazon Elastic Compute Cloud (Amazon EC2) Linux instance.3.    Connect to the Linux instance using SSH.4.    Run the following commands to install Python and Boto3. For more information, see Boto3 documentation for Quickstart.sudo yum groupinstall "Development Tools"sudo yum -y install openssl-develwget https://www.python.org/ftp/python/3.6.9/Python-3.6.9.tgztar xvf Python-3.6.9.tgzcd Python-3.6.9/./configure --enable-optimizationssudo make installsudo pip install boto35.    Confirm the location of the Python site-packages directory:python -m siteYou receive an output similar to the following:/usr/lib/python3.6/site-packages6.    Package the external library files in a .zip file unless the library is contained in a single .py file. The .zip file must include an __init__.py file, and the package directory must be at the root of the archive. The __init__.py file can be empty. For more information, see Python documentation for Packages.Example:cd /usr/lib/python3.6/site-packagessudo zip -r -X "/home/ec2-user/site-packages.zip" *7.    Upload the package to Amazon S3:aws s3 cp /home/ec2-user/site-packages.zip s3://awsexamplebucket/8.    Use the library in a job or JobRun.To use an external library in a development endpoint, do the following:1.    Package the library and upload the file to Amazon S3, as explained previously.2.    Create the development endpoint. For Python library path, enter the Amazon S3 path for the package. For more information, see Loading Python libraries in a development endpoint.

Related informationUsing Python libraries with AWS Glue