Repository: googleapis/google-cloud-python

Issue:
Pubsub message timestamp broken
The message timestamp for pubsub is fully broken - it is looking for the timestamp in a message attribute, but per the docs, it is a primary field of the message resource.

[https://github.com/GoogleCloudPlatform/gcloud-python/blob/master/gcloud/pubsub/message.py#L51](https://github.com/GoogleCloudPlatform/gcloud-python/blob/a5aeb33cc3cfc16b5daf8d4fe092fddd9fe39e43/gcloud/pubsub/message.py#L51)

the docs - clearly reference in source comments point to where the timestamp actually is:
https://cloud.google.com/pubsub/reference/rest/v1/PubsubMessage

**In API explorer, here is the returned value:**

```
cache-control:  private
content-encoding:  gzip
content-length:  322
content-type:  application/json; charset=UTF-8
date:  Thu, 17 Mar 2016 01:20:32 GMT
server:  ESF
vary:  Origin, X-Origin, Referer

{
 "receivedMessages": [
  {
   "ackId": "Pn4qKUVBXkASTCEcRElTK0MLKlgRTgQhIT4wPkVTRFAGFixdRkhRNxkIaFEOT14jPzUgKEUXC1MTUVx2BFgQajNcdQdRDRh3fml3aFhCAwBNWX5VWzM9SmJddgRYChl1eGB3aVIbASrfoMQWZhk9XBJLLA",
   "message": {
    "data": "dGhpcyBpcyB0aGUgbWVzc2FnZV9wYXlsb2FkMg==",
    "messageId": "13397075123880",
    "publishTime": "2016-03-17T01:13:41.166Z"
   }
  }
 ]
}
```



Discussion:
Thanks @ptone, you are our best bug-filer! Do you have a stacktrace / code snippet to trigger the error?

FWIW, the `Topic.publish` wrapper actually adds a timestamp into the message attributes, IFF `timestamp_messages` is set to `True` on the topic:  that is the value which `Message.timestamp` expects to consume.  The need for such a field was discussed in #797, which quoted the pubsub docs as saying:

> Although Pub/Sub usually delivers messages in order of publication, this is not guaranteed; it is possible for subscriptions to receive messages out of order. For this reason, we suggest that you include sequence information in the message payload or attribute so that subscribers that need in-order messaging can implement logic to do so.

Support for generating the attiribute was added to `Topic` in #809, and the property which parses it out was added to `Message` in #815.

If I had been aware of the `PubsubMessage.publishTime` field, I would have just recommended using it, rather than adding an extra attribute.

@tseaver Does this mean a change is on the way?

I can appreciate the good will of trying to capture a best practice into the class definition.  But I think these sort of sweetners should be called out explicitly in general, and in this case there is an unfortunate semantic conflict with the resource originated `publishTime`

I think there are going to be times you need to preserve strict order from the publisher environment, but I would have rather named the convention for the attribute something like "user_timestamp" - there is nothing about this provided by the service.  If you have well coordinated clocks among a set of publishers, the user_timestamp will be more reliable than publishTime - which is going to be an internal service-side time, and is really best effort based on arrival time.  It is poorly named, as it is really internalPublishTime between one part of PubSub and another.

I don't have any particularly strong opinion about how to "fix" the semantic conflict here - but I do hope we can see the resource property exposed in the Message class.

> Does this mean a change is on the way?

I'm not sure.  The simplest change would be to expose a read-only `Message.service_timestamp` property.  We would then need to be careful to document the differences between the two.  Renaming the current `Message.timestamp` would break any existing consumers:  I'm not sure how free we feel to make such changes.

Do you have anything roughly approximating this:
https://docs.djangoproject.com/en/1.9/internals/release-process/#internal-release-deprecation-policy

I would rename .timestamp to .user_timestamp, keep the old property wrapped in a deprecation warning, and change the docs for anyone coming in for the first time.

Then you can make clear in the docs the difference between the two timestamps

While we are `0.x.y` we don't really have issues deprecating features.


Resolving patch:
diff --git a/gcloud/pubsub/message.py b/gcloud/pubsub/message.py
--- a/gcloud/pubsub/message.py
+++ b/gcloud/pubsub/message.py
@@ -35,6 +35,8 @@ class Message(object):
     :param attributes: Extra metadata associated by the publisher with the
                        message.
     """
+    _service_timesstamp = None
+
     def __init__(self, data, message_id, attributes=None):
         self.data = data
         self.message_id = message_id
@@ -64,6 +66,15 @@ def timestamp(self):
             raise ValueError('No timestamp')
         return _rfc3339_to_datetime(stamp)
 
+    @property
+    def service_timestamp(self):
+        """Return server-set timestamp.
+
+        :rtype: string
+        :returns: timestamp (in UTC timezone) in RFC 3339 format
+        """
+        return self._service_timesstamp
+
     @classmethod
     def from_api_repr(cls, api_repr):
         """Factory:  construct message from API representation.
@@ -72,5 +83,8 @@ def from_api_repr(cls, api_repr):
         :param api_repr: The API representation of the message
         """
         data = base64.b64decode(api_repr.get('data', b''))
-        return cls(data=data, message_id=api_repr['messageId'],
-                   attributes=api_repr.get('attributes'))
+        instance = cls(
+            data=data, message_id=api_repr['messageId'],
+            attributes=api_repr.get('attributes'))
+        instance._service_timesstamp = api_repr.get('publishTimestamp')
+        return instance


Test patch:
diff --git a/gcloud/pubsub/test_message.py b/gcloud/pubsub/test_message.py
--- a/gcloud/pubsub/test_message.py
+++ b/gcloud/pubsub/test_message.py
@@ -31,6 +31,7 @@ def test_ctor_no_attributes(self):
         self.assertEqual(message.data, DATA)
         self.assertEqual(message.message_id, MESSAGE_ID)
         self.assertEqual(message.attributes, {})
+        self.assertEqual(message.service_timestamp, None)
 
     def test_ctor_w_attributes(self):
         DATA = b'DEADBEEF'
@@ -41,6 +42,7 @@ def test_ctor_w_attributes(self):
         self.assertEqual(message.data, DATA)
         self.assertEqual(message.message_id, MESSAGE_ID)
         self.assertEqual(message.attributes, ATTRS)
+        self.assertEqual(message.service_timestamp, None)
 
     def test_timestamp_no_attributes(self):
         DATA = b'DEADBEEF'
@@ -85,17 +87,24 @@ def test_from_api_repr_missing_data(self):
         self.assertEqual(message.data, b'')
         self.assertEqual(message.message_id, MESSAGE_ID)
         self.assertEqual(message.attributes, {})
+        self.assertEqual(message.service_timestamp, None)
 
     def test_from_api_repr_no_attributes(self):
         from base64 import b64encode as b64
         DATA = b'DEADBEEF'
         B64_DATA = b64(DATA)
         MESSAGE_ID = '12345'
-        api_repr = {'data': B64_DATA, 'messageId': MESSAGE_ID}
+        TIMESTAMP = '2016-03-18-19:38:22.001393427Z'
+        api_repr = {
+            'data': B64_DATA,
+            'messageId': MESSAGE_ID,
+            'publishTimestamp': TIMESTAMP,
+        }
         message = self._getTargetClass().from_api_repr(api_repr)
         self.assertEqual(message.data, DATA)
         self.assertEqual(message.message_id, MESSAGE_ID)
         self.assertEqual(message.attributes, {})
+        self.assertEqual(message.service_timestamp, TIMESTAMP)
 
     def test_from_api_repr_w_attributes(self):
         from base64 import b64encode as b64
@@ -103,10 +112,15 @@ def test_from_api_repr_w_attributes(self):
         B64_DATA = b64(DATA)
         MESSAGE_ID = '12345'
         ATTRS = {'a': 'b'}
-        api_repr = {'data': B64_DATA,
-                    'messageId': MESSAGE_ID,
-                    'attributes': ATTRS}
+        TIMESTAMP = '2016-03-18-19:38:22.001393427Z'
+        api_repr = {
+            'data': B64_DATA,
+            'messageId': MESSAGE_ID,
+            'publishTimestamp': TIMESTAMP,
+            'attributes': ATTRS,
+        }
         message = self._getTargetClass().from_api_repr(api_repr)
         self.assertEqual(message.data, DATA)
         self.assertEqual(message.message_id, MESSAGE_ID)
+        self.assertEqual(message.service_timestamp, TIMESTAMP)
         self.assertEqual(message.attributes, ATTRS)