Repository: getmoto/moto

Issue:
s3.copy_object() doesn't increment the VersionId correctly
If you update the file at an S3 key using `copy_object()` in a version-controlled bucket, moto does not assign the key a new VersionId.

Here’s a brief summary of repro steps, with a full runnable test case below:

* Create a version-controlled S3 bucket
* Upload two files to the bucket (say `key1` and `key2`). Check they’re both assigned a VersionID `0`
* Use the `copy_object` API to copy the contents of `key1` into `key2`
* Check that the VersionId of `key2` is updated accordingly

I’d expect the VersionId to change for the new version (this is the behaviour in Amazon S3 proper); what actually happens is the VersionID is created at `0` and stays at `0`.

### Test environment

I’m using Python 3.6.1, with moto 1.1.23 and boto 1.4.7 installed in a virtualenv using pip.

<details>
<summary>Output of "pip freeze"</summary>

<br/>

```
asn1crypto==0.23.0
aws-xray-sdk==0.92.2
boto==2.48.0
boto3==1.4.7
botocore==1.7.30
certifi==2017.7.27.1
cffi==1.11.2
chardet==3.0.4
cookies==2.2.1
cryptography==2.1.1
docker==2.5.1
docker-pycreds==0.2.1
docutils==0.14
idna==2.6
Jinja2==2.9.6
jmespath==0.9.3
jsonpickle==0.9.5
MarkupSafe==1.0
mock==2.0.0
moto==1.1.23
pbr==3.1.1
pyaml==17.10.0
pycparser==2.18
python-dateutil==2.6.1
pytz==2017.2
PyYAML==3.12
requests==2.18.4
s3transfer==0.1.11
six==1.11.0
urllib3==1.22
websocket-client==0.44.0
Werkzeug==0.12.2
wrapt==1.10.11
xmltodict==0.11.0
```

</details>

### Test case

This is a complete, runnable test case:

```python
import boto3
import moto

moto.mock_s3().start()

client = boto3.client('s3')

BUCKET = 'mygreatbukkit'
SRC_KEY = 'file_src.txt'
DST_KEY = 'file_dst.txt'

# Create the bucket, and enable versioning
client.create_bucket(
    Bucket=BUCKET,
    CreateBucketConfiguration={'LocationConstraint': 'eu-west-1'}
)
client.put_bucket_versioning(
    Bucket=BUCKET, VersioningConfiguration={'Status': 'Enabled'}
)

# Then create both keys, and inspect their VersionId
for Key in [SRC_KEY, DST_KEY]:
    client.put_object(
        Bucket=BUCKET, Key=Key, Body=b'I live at %s' % Key.encode()
    )
    resp = client.get_object(Bucket=BUCKET, Key=Key)
    print((Key, resp['VersionId']))

# Now copy the object from one bucket to the other, and check that S3
# assigns it a new VersionId
client.copy_object(
    CopySource={'Bucket': BUCKET, 'Key': SRC_KEY},
    Bucket=BUCKET,
    Key=DST_KEY
)
resp = client.get_object(Bucket=BUCKET, Key=DST_KEY)
print((DST_KEY, resp['VersionId']))
```

This is the output when running under moto; notice that VersionId stays the same:

```
('file_src.txt', '0')
('file_dst.txt', '0')
('file_dst.txt', '0')
```

This is the output if you remove `moto.mock_s3().start()`, and run it against S3 directly:

```
('file_src.txt', '00ehtAiYbwaQu5qz9EAT.jL4J4mxbaS9')
('file_dst.txt', '2sCyWtdg6yfMKBELO0PxLq11xLWD.EcB')
('file_dst.txt', 'W2UQ7ec4PUK30T.28D0j1ZE50NzSuDAs')
```

Notice that the VersionId has changed after calling `copy_object()`.


Discussion:
Yep confirmed bug, am making a PR as we speak.

Resolving patch:
diff --git a/moto/s3/models.py b/moto/s3/models.py
--- a/moto/s3/models.py
+++ b/moto/s3/models.py
@@ -81,6 +81,9 @@ def append_to_value(self, value):
     def restore(self, days):
         self._expiry = datetime.datetime.utcnow() + datetime.timedelta(days)
 
+    def increment_version(self):
+        self._version_id += 1
+
     @property
     def etag(self):
         if self._etag is None:
@@ -323,19 +326,10 @@ class CorsRule(BaseModel):
 
     def __init__(self, allowed_methods, allowed_origins, allowed_headers=None, expose_headers=None,
                  max_age_seconds=None):
-        # Python 2 and 3 have different string types for handling unicodes. Python 2 wants `basestring`,
-        # whereas Python 3 is OK with str. This causes issues with the XML parser, which returns
-        # unicode strings in Python 2. So, need to do this to make it work in both Python 2 and 3:
-        import sys
-        if sys.version_info >= (3, 0):
-            str_type = str
-        else:
-            str_type = basestring  # noqa
-
-        self.allowed_methods = [allowed_methods] if isinstance(allowed_methods, str_type) else allowed_methods
-        self.allowed_origins = [allowed_origins] if isinstance(allowed_origins, str_type) else allowed_origins
-        self.allowed_headers = [allowed_headers] if isinstance(allowed_headers, str_type) else allowed_headers
-        self.exposed_headers = [expose_headers] if isinstance(expose_headers, str_type) else expose_headers
+        self.allowed_methods = [allowed_methods] if isinstance(allowed_methods, six.string_types) else allowed_methods
+        self.allowed_origins = [allowed_origins] if isinstance(allowed_origins, six.string_types) else allowed_origins
+        self.allowed_headers = [allowed_headers] if isinstance(allowed_headers, six.string_types) else allowed_headers
+        self.exposed_headers = [expose_headers] if isinstance(expose_headers, six.string_types) else expose_headers
         self.max_age_seconds = max_age_seconds
 
 
@@ -389,25 +383,16 @@ def set_cors(self, rules):
         if len(rules) > 100:
             raise MalformedXML()
 
-        # Python 2 and 3 have different string types for handling unicodes. Python 2 wants `basestring`,
-        # whereas Python 3 is OK with str. This causes issues with the XML parser, which returns
-        # unicode strings in Python 2. So, need to do this to make it work in both Python 2 and 3:
-        import sys
-        if sys.version_info >= (3, 0):
-            str_type = str
-        else:
-            str_type = basestring  # noqa
-
         for rule in rules:
-            assert isinstance(rule["AllowedMethod"], list) or isinstance(rule["AllowedMethod"], str_type)
-            assert isinstance(rule["AllowedOrigin"], list) or isinstance(rule["AllowedOrigin"], str_type)
+            assert isinstance(rule["AllowedMethod"], list) or isinstance(rule["AllowedMethod"], six.string_types)
+            assert isinstance(rule["AllowedOrigin"], list) or isinstance(rule["AllowedOrigin"], six.string_types)
             assert isinstance(rule.get("AllowedHeader", []), list) or isinstance(rule.get("AllowedHeader", ""),
-                                                                                 str_type)
+                                                                                 six.string_types)
             assert isinstance(rule.get("ExposedHeader", []), list) or isinstance(rule.get("ExposedHeader", ""),
-                                                                                 str_type)
-            assert isinstance(rule.get("MaxAgeSeconds", "0"), str_type)
+                                                                                 six.string_types)
+            assert isinstance(rule.get("MaxAgeSeconds", "0"), six.string_types)
 
-            if isinstance(rule["AllowedMethod"], str_type):
+            if isinstance(rule["AllowedMethod"], six.string_types):
                 methods = [rule["AllowedMethod"]]
             else:
                 methods = rule["AllowedMethod"]
@@ -745,6 +730,10 @@ def copy_key(self, src_bucket_name, src_key_name, dest_bucket_name,
         if dest_key_name != src_key_name:
             key = key.copy(dest_key_name)
         dest_bucket.keys[dest_key_name] = key
+
+        # By this point, the destination key must exist, or KeyError
+        if dest_bucket.is_versioned:
+            dest_bucket.keys[dest_key_name].increment_version()
         if storage is not None:
             key.set_storage_class(storage)
         if acl is not None:


Test patch:
diff --git a/tests/test_s3/test_s3.py b/tests/test_s3/test_s3.py
--- a/tests/test_s3/test_s3.py
+++ b/tests/test_s3/test_s3.py
@@ -1364,6 +1364,29 @@ def test_boto3_head_object_with_versioning():
     old_head_object['ContentLength'].should.equal(len(old_content))
 
 
+@mock_s3
+def test_boto3_copy_object_with_versioning():
+    client = boto3.client('s3', region_name='us-east-1')
+
+    client.create_bucket(Bucket='blah', CreateBucketConfiguration={'LocationConstraint': 'eu-west-1'})
+    client.put_bucket_versioning(Bucket='blah', VersioningConfiguration={'Status': 'Enabled'})
+
+    client.put_object(Bucket='blah', Key='test1', Body=b'test1')
+    client.put_object(Bucket='blah', Key='test2', Body=b'test2')
+
+    obj1_version = client.get_object(Bucket='blah', Key='test1')['VersionId']
+    obj2_version = client.get_object(Bucket='blah', Key='test2')['VersionId']
+
+    # Versions should be the same
+    obj1_version.should.equal(obj2_version)
+
+    client.copy_object(CopySource={'Bucket': 'blah', 'Key': 'test1'}, Bucket='blah', Key='test2')
+    obj2_version_new = client.get_object(Bucket='blah', Key='test2')['VersionId']
+
+    # Version should be different to previous version
+    obj2_version_new.should_not.equal(obj2_version)
+
+
 @mock_s3
 def test_boto3_head_object_if_modified_since():
     s3 = boto3.client('s3', region_name='us-east-1')