Repository: aws/aws-cli

Issue:
s3 cp "unknown encoding: idna"
I am using the msi installed aws cli on Windows (from [Install the AWS CLI on Windows](https://docs.aws.amazon.com/cli/latest/userguide/install-windows.html)) that reports the following versions:
`
PS > aws --version                                                                                                                                                                                                       aws-cli/1.16.180 Python/3.6.0 Windows/10 botocore/1.12.170    
`

I have a folder called `build` that I am trying to upload to a s3 bucket using the following command:
`
PS > aws s3 cp --recursive build s3://my-s3-bucket/
`

This results in most of the files copied successfully, but one or two (the exact situation varies between executions) fail with the following error:
`
upload failed: build\myfile.ext to s3://my-s3-bucket/myfile.ext An HTTP Client raised and unhandled exception: unknown encoding: idna
`

Uploading the files one by one like this succeeds:
`
PS > aws s3 cp build\myfile.ext s3://my-s3-bucket/
`

Also, adding the `--debug` flag seems to make it succeed.

There is a [pull request #4073](https://github.com/aws/aws-cli/pull/4073) in this same repository that seems related. However, it is already merged, but I still encounter the problem.

There is also [some discussion in Stack Overflow](https://stackoverflow.com/questions/53144254/aws-cli-upload-failed-unknown-encoding-idna/54337242). The workaround of installing the aws cli with pip instead of msi is not easily applicable for me, due to conflicting dependency versions with other pip packages that I need.

The other fix from the mentioned Stack Overflow discussion, adding `import encodings.idna` to file `C:\Program Files\Amazon\AWSCLI\bin\aws-scipt.py`, seems to fix the issue for me.


Discussion:
Confirmed bug.  What happened here was that linked PR updated the `__main__.py` file, but that file is only executed when running the `aws.cmd` file, which is a legacy file that's kept around for backwards compatibility.  The newer `aws.exe` uses the `aws-script.py`, which does not have the fix from #4073 (it's an autogenerated file as part of our release process).

We should consolidate all these various entry point code paths, but I think for now, the safest option to ensure this import always happen is to just add this in `awscli.clidriver` because that file *always* gets executed for every CLI command.

Resolving patch:
diff --git a/awscli/__main__.py b/awscli/__main__.py
--- a/awscli/__main__.py
+++ b/awscli/__main__.py
@@ -11,17 +11,6 @@
 # ANY KIND, either express or implied. See the License for the specific
 # language governing permissions and limitations under the License.
 
-# Don't remove this line.  The idna encoding
-# is used by getaddrinfo when dealing with unicode hostnames,
-# and in some cases, there appears to be a race condition
-# where threads will get a LookupError on getaddrinfo() saying
-# that the encoding doesn't exist.  Using the idna encoding before
-# running any CLI code (and any threads it may create) ensures that
-# the encodings.idna is imported and registered in the codecs registry,
-# which will stop the LookupErrors from happening.
-# See: https://bugs.python.org/issue29288
-u''.encode('idna')
-
 
 import sys
 
diff --git a/awscli/clidriver.py b/awscli/clidriver.py
--- a/awscli/clidriver.py
+++ b/awscli/clidriver.py
@@ -52,6 +52,16 @@
 LOG_FORMAT = (
     '%(asctime)s - %(threadName)s - %(name)s - %(levelname)s - %(message)s')
 HISTORY_RECORDER = get_global_history_recorder()
+# Don't remove this line.  The idna encoding
+# is used by getaddrinfo when dealing with unicode hostnames,
+# and in some cases, there appears to be a race condition
+# where threads will get a LookupError on getaddrinfo() saying
+# that the encoding doesn't exist.  Using the idna encoding before
+# running any CLI code (and any threads it may create) ensures that
+# the encodings.idna is imported and registered in the codecs registry,
+# which will stop the LookupErrors from happening.
+# See: https://bugs.python.org/issue29288
+u''.encode('idna')
 
 
 def main():


Test patch: