Repository: googleapis/google-cloud-python

Issue:
Datastore Key's reference is missing location identifier in app argument in to_legacy_urlsafe()
@dhermes I put this as a new issue because I thought people might have a problem with this too. Please feel free to edit/remove this issue if it is not appropriate to put it here.

I was running `pip install --upgrade google-cloud-datastore` to get latest version that has the `key.to_legacy_urlsafe()` implemented in [#3491](https://github.com/GoogleCloudPlatform/google-cloud-python/pull/3491). I tested the method and I found out that the urlsafe generated from the method is still not the same as datastore's urlsafe.

I have implemented my own function to generate the urlsafe key before the update was released and so far it worked properly for me (I can share my code if anybody wants). So I compared my function with the to_legacy_urlsafe and I found the problem:


```
reference = _app_engine_key_pb2.Reference(
        app=self.project,
        path=_to_legacy_path(self._path),  # Avoid the copy.
        name_space=self.namespace,
        )
```

In the app argument, the location identifier is missing unlike in my function where I concatenated the "s~" at the beginning of the string. I think this is the cause of the problem why it was not getting the urlsafe correct, although I don't understand why it needs the location identifier in front of the project name in the first place.

Do you think I should send a PR to fix this?


Discussion:
@michaelenglo I'll let @jonparrott handle this. I voiced this concern when writing the original PR and he said we shouldn't worry about it.

I would suggest that you rely on `from_legacy_urlsafe` much more than `to_legacy_urlsafe`, i.e. these are really meant as a way to go **FROM** legacy **TO** "modern", not the other direction.
As for sending a PR, there is no "right" way to fix this. The `app` may begin with `dev~`, `s~`, `e~` and possibly other strings I am unaware of. There is no way to determine which of these is appropriate based on the project string.
Because there's no "right" way to fix this, I think we should just document that the string returned by our `to_legacy_urlsafe` is equivalent, but not identical to the key returned by `ndb`.
I am a bit concerned about this because urlsafe play an important role in our app. Currently in the implementation of our Document Control app, every document file has its information (in datastore), and its blob (in cloud storage). The way we name each document is by its urlsafe key (since we thought it's always unique, we decided to use it instead of creating app-specific identifier). Now, If we cannot guarantee that both services in Std and Flex generate the same urlsafe key consistently, then there is a chance that in the future we can have corrupted files lost in the database.

Is using urlsafe as unique name for objects something that we should avoid? If yes, then what is the proper way for mapping entities to blob objects in cloud storage?

And I wonder why if NDB has a method to create a ursafe key that is identical from the one in Datastore, what prevents the Client Library from having one?
@michaelenglo You can get around this by just using the same identifier here that is using in App Engine standard. It's fairly like your application uses `s~` as the project ID. You can deserialize a protobuf from a `urlsafe` value to find out.
> And I wonder why if NDB has a method to create a ursafe key that is identical from the one in Datastore, what prevents the Client Library from having one?

`ndb` runs in App Engine, so it has access to the application ID **according to App Engine**, which contains the prefix (`s~`, `e~`, etc.). If you manually add the correct prefix, you'll get the same values (see above).
Hey @michaelenglo,
I just wanted to follow up on this item. I think we are probably leaning toward leaving this as-is (except maybe adding some documentation), but I wanted to give you an opportunity to follow up on it first.

Resolving patch:
diff --git a/datastore/google/cloud/datastore/key.py b/datastore/google/cloud/datastore/key.py
--- a/datastore/google/cloud/datastore/key.py
+++ b/datastore/google/cloud/datastore/key.py
@@ -307,6 +307,11 @@ def to_legacy_urlsafe(self):
         argument to ``ndb.Key(urlsafe=...)``. The base64 encoded values
         will have padding removed.
 
+        .. note::
+
+            The string returned by ``to_legacy_urlsafe`` is equivalent, but
+            not identical, to the string returned by ``ndb``.
+
         :rtype: bytes
         :returns: A bytestring containing the key encoded as URL-safe base64.
         """


Test patch: