Repository: googleapis/google-cloud-python

Issue:
Provide a method to convert ndb Key's urlsafe format to/from datastore.Key
In GAE standard, ndb has the concept of the urlsafe string version of a Key. 
e.g. `url_string = sandy_key.urlsafe()` produces a result like `agVoZWxsb3IPCxIHQWNjb3VudBiZiwIM`.

However, I cannot find a way to use this format with the google.cloud.datastore library for code running outside GAE standard. Please add a method to the Key class to create urlsafe strings and a method to generate Keys from urlsafe strings. Alternatively, provide some documentation on how to decode/encode them with the existing library.

NB, the [console support page](https://support.google.com/cloud/answer/6365503?hl=en) says "The encoding method is available in any Cloud Datastore client library."



Discussion:
Hi @ndenny,
Thanks for reporting. We will look into this.
For [reference][1], `urlsafe` just takes the key, turns it into raw bytes as a serialized protobuf, then urlsafe-base64 encodes those raw bytes. However, the protobuf used by App Engine is different than the one used by the Cloud Datastore API (I think).

@jonparrott Can you weigh in?

---

If you'd like a "band-aid" for now:

```python
>>> import base64
>>> from google.cloud import datastore
>>>
>>> client = datastore.Client()
>>> key = client.key('Foo', 'bar')
>>> key_pb_bytes = key.to_protobuf().SerializeToString()
>>> urlsafe = base64.urlsafe_b64encode(key_pb_bytes).rstrip(b'=')
```

[1]: https://github.com/GoogleCloudPlatform/datastore-ndb-python/blob/caac0c3e7dd4d9b2c6b32dfc5d59386dd02e6b57/ndb/key.py#L553-L562
There's two separate requests here:

1. We need to add `Key.urlsafe()` and `Key.from_urlsafe()` that operates with our current proto format.
2. We should make sure the first method accepts App Engine standard's protobuf format as well.
Go to the [source](https://github.com/GoogleCloudPlatform/datastore-ndb-python/blob/caac0c3e7dd4d9b2c6b32dfc5d59386dd02e6b57/ndb/key.py#L553-L562) and you will find this:

```python
def _DecodeUrlSafe(urlsafe):
  """Decode a url-safe base64-encoded string.
  This returns the decoded string.
  """
  if not isinstance(urlsafe, basestring):
    raise TypeError('urlsafe must be a string; received %r' % urlsafe)
  if isinstance(urlsafe, unicode):
    urlsafe = urlsafe.encode('utf8')
  mod = len(urlsafe) % 4
  if mod:
    urlsafe += '=' * (4 - mod)
  #This is 3-4x faster than urlsafe_b64decode()
  return base64.b64decode(urlsafe.replace('-', '+').replace('_', '/'))
```

I just copied and pasted this snippet in my code and it works now. Hope this helps.
@michaelenglo Thanks for providing the snippet, however the issue is that the raw bytes are different for an entity in App Engine and one using the Google Cloud Datastore API.
@dhermes That is true, in fact, When I try to print it to the console, it gives me non-textual characters. I prematurely said it worked because I didn't get an exception for the first time when running it lol (been stuck on this for 2 hours now).

By any chance, have we found any solution to this already?
@michaelenglo Feel free to provide some code. The issue though is that the two different APIs (the native RPC API in App Engine and the external Cloud Datastore API) use different protobuf definitions for entities, keys, etc.

I'm [pretty sure][1] the GAE protos are in `google.appengine.datastore.entity_pb` (I'm verifying right now) and the Cloud Datastore protos are [here][2].

[1]: https://github.com/GoogleCloudPlatform/datastore-ndb-python/blob/caac0c3e7dd4d9b2c6b32dfc5d59386dd02e6b57/ndb/google_imports.py#L49
[2]: https://github.com/googleapis/googleapis/tree/ee1b1532f5cec7d349408ae01e90d780aa761f6e/google/datastore/v1
Here is a confirmation of the difference

```python
>>> import base64
>>> from google.appengine.datastore import entity_pb
>>> from google.appengine.ext import ndb
>>>
>>> key = ndb.Key('MyModel', 5910974510923776)
>>> urlsafe = key.urlsafe()
>>> urlsafe
'ahNkZXZ-c3R1ZHlidWRkeXgtaHJkchQLEgdNeU1vZGVsGICAgICAgMAKDA'
>>> urlsafe += '=='  # Needs padding
>>> raw_bytes = base64.urlsafe_b64decode(urlsafe)
>>> raw_bytes
'j\x13dev~studybuddyx-hrdr\x14\x0b\x12\x07MyModel\x18\x80\x80\x80\x80\x80\x80\xc0\n\x0c'
>>>
>>> ref = entity_pb.Reference(raw_bytes)
>>> print(ref)
app: "dev~studybuddyx-hrd"
path <
  Element {
    type: "MyModel"
    id: 0x15000000000000
  }
>
```

vs.

```python
>>> from google.cloud import datastore
>>> client = datastore.Client(project='studybuddyx-hrd')
>>>
>>> key = client.key('MyModel', 5910974510923776)
>>>
>>> key_pb = key.to_protobuf()
>>> key_pb
partition_id {
  project_id: "studybuddyx-hrd"
}
path {
  kind: "MyModel"
  id: 5910974510923776
}
>>>
>>> key_pb_bytes = key_pb.SerializeToString()
>>> key_pb_bytes
'\n\x11\x12\x0fstudybuddyx-hrd\x12\x12\n\x07MyModel\x10\x80\x80\x80\x80\x80\x80\xc0\n'
```
```
>>> print(ref)
app: "dev~studybuddyx-hrd"
path <
  Element {
    type: "MyModel"
    id: 0x15000000000000
  }
>
```
What kind of data format is this? Is this something that has a publicly available parser?

My only goal here is just to extract the ID from the key. And by the way, I don't understand why the data formatting must be different between AE Standard Env. API and the external GC client library?
> What kind of data format is this?

This is just a pretty-printed representation of an `entity_pb.Reference` object, which is a Python wrapper for a protobuf.

> Is this something that has a publicly available parser?

The `ref` instance is already parsed. I recommend firing up the `dev_appserver` and then using the Interactive Console to play around.

> My only goal here is just to extract the ID from the key.

The rules for deserializing a protobuf are public (i.e. not a Google secret), though I'm not sure what they are.

> And by the way, I don't understand why the data formatting must be different between AE Standard Env. API and the external GC client library?

I can't help with that. Our library is just a wrapper for the existing cloud APIs. As a historical note, the Datastore within App Engine has existed since 2008, but the Cloud Datastore API came along in 2012 (and the current revision came along in 2015 or 2016).

Okay, so I get that. Now the question will be how to actually convert the raw binary data after it has been decoded?

When I called the _DecodeUrlSafe(urlsafe) when urlsafe="agxzfm9lLXNlcnZpY2VyGwsSDkRyYXdpbmdWZXJzaW9uGICAgID4woQKDA", and print the returned value, I get this:

js~oe-servicerDrawingVersionÇÇÇÇ°┬ä

How to convert this value into a protobuf? Is there an API call for this in the Client Library?

If this is possible, then I can call google.cloud.datastore.helpers.entity_from_protobuf(pb) to get the entity.
You'll want to print the `repr` of the value. That already is a protobuf (as raw bytes). See my snippet above:

```python
from google.appengine.datastore import entity_pb
ref = entity_pb.Reference(raw_bytes)
```
So I did some more digging and it turns out the `Reference` protobuf in App Engine is `proto1` (i.e. first generation of protobuf), so the modern `proto2/proto3` tooling can't convert it. I got [very close][1] to converting using standard tools, but some small differences in the protocols prevent it.

You can **definitely** include the `ProtocolBuffer` module in your application (in any location):

```
google-cloud-sdk/platform/google_appengine/google/net/proto/ProtocolBuffer.py
```

and then just define your own local `Reference` and `Path` classes from

```
google-cloud-sdk/platform/google_appengine/google/appengine/datastore/entity_pb.py
```

I.e. you can use these files outside of App Engine. I have done so in the gist I linked to and it works just fine.

[1]: https://gist.github.com/dhermes/9d658dc68c55f042f9766490721ac16a
Scratch that, I just had the wrong proto2 definition. Now you can convert with [`onestore_v3_pb2.py`][1]

See the README in that [gist][2] for an example.

[1]: https://gist.githubusercontent.com/dhermes/9d658dc68c55f042f9766490721ac16a/raw/2ef08924e13280081ac06f1339d46adaf549fd4e/onestore_v3_pb2.py
[2]: https://gist.github.com/dhermes/9d658dc68c55f042f9766490721ac16a
I am going to send a PR ASAP.

Resolving patch:
diff --git a/datastore/google/cloud/datastore/_app_engine_key_pb2.py b/datastore/google/cloud/datastore/_app_engine_key_pb2.py
new file mode 100644
--- /dev/null
+++ b/datastore/google/cloud/datastore/_app_engine_key_pb2.py
@@ -0,0 +1,184 @@
+# Generated by the protocol buffer compiler.  DO NOT EDIT!
+# source: _app_engine_key.proto
+
+import sys
+_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
+from google.protobuf import descriptor as _descriptor
+from google.protobuf import message as _message
+from google.protobuf import reflection as _reflection
+from google.protobuf import symbol_database as _symbol_database
+from google.protobuf import descriptor_pb2
+# @@protoc_insertion_point(imports)
+
+_sym_db = _symbol_database.Default()
+
+
+
+
+DESCRIPTOR = _descriptor.FileDescriptor(
+  name='_app_engine_key.proto',
+  package='',
+  syntax='proto2',
+  serialized_pb=_b('\n\x15_app_engine_key.proto\"V\n\tReference\x12\x0b\n\x03\x61pp\x18\r \x02(\t\x12\x12\n\nname_space\x18\x14 \x01(\t\x12\x13\n\x04path\x18\x0e \x02(\x0b\x32\x05.Path\x12\x13\n\x0b\x64\x61tabase_id\x18\x17 \x01(\t\"Y\n\x04Path\x12\x1e\n\x07\x65lement\x18\x01 \x03(\n2\r.Path.Element\x1a\x31\n\x07\x45lement\x12\x0c\n\x04type\x18\x02 \x02(\t\x12\n\n\x02id\x18\x03 \x01(\x03\x12\x0c\n\x04name\x18\x04 \x01(\t')
+)
+_sym_db.RegisterFileDescriptor(DESCRIPTOR)
+
+
+
+
+_REFERENCE = _descriptor.Descriptor(
+  name='Reference',
+  full_name='Reference',
+  filename=None,
+  file=DESCRIPTOR,
+  containing_type=None,
+  fields=[
+    _descriptor.FieldDescriptor(
+      name='app', full_name='Reference.app', index=0,
+      number=13, type=9, cpp_type=9, label=2,
+      has_default_value=False, default_value=_b("").decode('utf-8'),
+      message_type=None, enum_type=None, containing_type=None,
+      is_extension=False, extension_scope=None,
+      options=None),
+    _descriptor.FieldDescriptor(
+      name='name_space', full_name='Reference.name_space', index=1,
+      number=20, type=9, cpp_type=9, label=1,
+      has_default_value=False, default_value=_b("").decode('utf-8'),
+      message_type=None, enum_type=None, containing_type=None,
+      is_extension=False, extension_scope=None,
+      options=None),
+    _descriptor.FieldDescriptor(
+      name='path', full_name='Reference.path', index=2,
+      number=14, type=11, cpp_type=10, label=2,
+      has_default_value=False, default_value=None,
+      message_type=None, enum_type=None, containing_type=None,
+      is_extension=False, extension_scope=None,
+      options=None),
+    _descriptor.FieldDescriptor(
+      name='database_id', full_name='Reference.database_id', index=3,
+      number=23, type=9, cpp_type=9, label=1,
+      has_default_value=False, default_value=_b("").decode('utf-8'),
+      message_type=None, enum_type=None, containing_type=None,
+      is_extension=False, extension_scope=None,
+      options=None),
+  ],
+  extensions=[
+  ],
+  nested_types=[],
+  enum_types=[
+  ],
+  options=None,
+  is_extendable=False,
+  syntax='proto2',
+  extension_ranges=[],
+  oneofs=[
+  ],
+  serialized_start=25,
+  serialized_end=111,
+)
+
+
+_PATH_ELEMENT = _descriptor.Descriptor(
+  name='Element',
+  full_name='Path.Element',
+  filename=None,
+  file=DESCRIPTOR,
+  containing_type=None,
+  fields=[
+    _descriptor.FieldDescriptor(
+      name='type', full_name='Path.Element.type', index=0,
+      number=2, type=9, cpp_type=9, label=2,
+      has_default_value=False, default_value=_b("").decode('utf-8'),
+      message_type=None, enum_type=None, containing_type=None,
+      is_extension=False, extension_scope=None,
+      options=None),
+    _descriptor.FieldDescriptor(
+      name='id', full_name='Path.Element.id', index=1,
+      number=3, type=3, cpp_type=2, label=1,
+      has_default_value=False, default_value=0,
+      message_type=None, enum_type=None, containing_type=None,
+      is_extension=False, extension_scope=None,
+      options=None),
+    _descriptor.FieldDescriptor(
+      name='name', full_name='Path.Element.name', index=2,
+      number=4, type=9, cpp_type=9, label=1,
+      has_default_value=False, default_value=_b("").decode('utf-8'),
+      message_type=None, enum_type=None, containing_type=None,
+      is_extension=False, extension_scope=None,
+      options=None),
+  ],
+  extensions=[
+  ],
+  nested_types=[],
+  enum_types=[
+  ],
+  options=None,
+  is_extendable=False,
+  syntax='proto2',
+  extension_ranges=[],
+  oneofs=[
+  ],
+  serialized_start=153,
+  serialized_end=202,
+)
+
+_PATH = _descriptor.Descriptor(
+  name='Path',
+  full_name='Path',
+  filename=None,
+  file=DESCRIPTOR,
+  containing_type=None,
+  fields=[
+    _descriptor.FieldDescriptor(
+      name='element', full_name='Path.element', index=0,
+      number=1, type=10, cpp_type=10, label=3,
+      has_default_value=False, default_value=[],
+      message_type=None, enum_type=None, containing_type=None,
+      is_extension=False, extension_scope=None,
+      options=None),
+  ],
+  extensions=[
+  ],
+  nested_types=[_PATH_ELEMENT, ],
+  enum_types=[
+  ],
+  options=None,
+  is_extendable=False,
+  syntax='proto2',
+  extension_ranges=[],
+  oneofs=[
+  ],
+  serialized_start=113,
+  serialized_end=202,
+)
+
+_REFERENCE.fields_by_name['path'].message_type = _PATH
+_PATH_ELEMENT.containing_type = _PATH
+_PATH.fields_by_name['element'].message_type = _PATH_ELEMENT
+DESCRIPTOR.message_types_by_name['Reference'] = _REFERENCE
+DESCRIPTOR.message_types_by_name['Path'] = _PATH
+
+Reference = _reflection.GeneratedProtocolMessageType('Reference', (_message.Message,), dict(
+  DESCRIPTOR = _REFERENCE,
+  __module__ = '_app_engine_key_pb2'
+  # @@protoc_insertion_point(class_scope:Reference)
+  ))
+_sym_db.RegisterMessage(Reference)
+
+Path = _reflection.GeneratedProtocolMessageType('Path', (_message.Message,), dict(
+
+  Element = _reflection.GeneratedProtocolMessageType('Element', (_message.Message,), dict(
+    DESCRIPTOR = _PATH_ELEMENT,
+    __module__ = '_app_engine_key_pb2'
+    # @@protoc_insertion_point(class_scope:Path.Element)
+    ))
+  ,
+  DESCRIPTOR = _PATH,
+  __module__ = '_app_engine_key_pb2'
+  # @@protoc_insertion_point(class_scope:Path)
+  ))
+_sym_db.RegisterMessage(Path)
+_sym_db.RegisterMessage(Path.Element)
+
+
+# @@protoc_insertion_point(module_scope)
diff --git a/datastore/google/cloud/datastore/key.py b/datastore/google/cloud/datastore/key.py
--- a/datastore/google/cloud/datastore/key.py
+++ b/datastore/google/cloud/datastore/key.py
@@ -14,11 +14,28 @@
 
 """Create / interact with Google Cloud Datastore keys."""
 
+import base64
 import copy
 import six
 
 from google.cloud.proto.datastore.v1 import entity_pb2 as _entity_pb2
 
+from google.cloud._helpers import _to_bytes
+from google.cloud.datastore import _app_engine_key_pb2
+
+
+_DATABASE_ID_TEMPLATE = (
+    'Received non-empty database ID: {!r}.\n'
+    'urlsafe strings are not expected to encode a Reference that '
+    'contains a database ID.')
+_BAD_ELEMENT_TEMPLATE = (
+    'At most one of ID and name can be set on an element. Received '
+    'id = {!r} and name = {!r}.')
+_EMPTY_ELEMENT = (
+    'Exactly one of ID and name must be set on an element. '
+    'Encountered an element with neither set that was not the last '
+    'element of a path.')
+
 
 class Key(object):
     """An immutable representation of a datastore Key.
@@ -79,7 +96,7 @@ class Key(object):
 
     * namespace (string): A namespace identifier for the key.
     * project (string): The project associated with the key.
-    * parent (:class:`google.cloud.datastore.key.Key`): The parent of the key.
+    * parent (:class:`~google.cloud.datastore.key.Key`): The parent of the key.
 
     The project argument is required unless it has been set implicitly.
     """
@@ -281,6 +298,53 @@ def to_protobuf(self):
 
         return key
 
+    def to_legacy_urlsafe(self):
+        """Convert to a base64 encode urlsafe string for App Engine.
+
+        This is intended to work with the "legacy" representation of a
+        datastore "Key" used within Google App Engine (a so-called
+        "Reference"). The returned string can be used as the ``urlsafe``
+        argument to ``ndb.Key(urlsafe=...)``.
+
+        :rtype: bytes
+        :returns: A bytestring containing the key encoded as URL-safe base64.
+        """
+        reference = _app_engine_key_pb2.Reference(
+            app=self.project,
+            path=_to_legacy_path(self._path),  # Avoid the copy.
+            name_space=self.namespace,
+        )
+        raw_bytes = reference.SerializeToString()
+        return base64.urlsafe_b64encode(raw_bytes)
+
+    @classmethod
+    def from_legacy_urlsafe(cls, urlsafe):
+        """Convert urlsafe string to :class:`~google.cloud.datastore.key.Key`.
+
+        This is intended to work with the "legacy" representation of a
+        datastore "Key" used within Google App Engine (a so-called
+        "Reference"). This assumes that ``urlsafe`` was created within an App
+        Engine app via something like ``ndb.Key(...).urlsafe()``.
+
+        :type urlsafe: bytes or unicode
+        :param urlsafe: The base64 encoded (ASCII) string corresponding to a
+                        datastore "Key" / "Reference".
+
+        :rtype: :class:`~google.cloud.datastore.key.Key`.
+        :returns: The key corresponding to ``urlsafe``.
+        """
+        urlsafe = _to_bytes(urlsafe, encoding='ascii')
+        raw_bytes = base64.urlsafe_b64decode(urlsafe)
+
+        reference = _app_engine_key_pb2.Reference()
+        reference.ParseFromString(raw_bytes)
+
+        project = _clean_app(reference.app)
+        namespace = _get_empty(reference.name_space, u'')
+        _check_database_id(reference.database_id)
+        flat_path = _get_flat_path(reference.path)
+        return cls(*flat_path, project=project, namespace=namespace)
+
     @property
     def is_partial(self):
         """Boolean indicating if the key has an ID (or name).
@@ -427,3 +491,142 @@ def _validate_project(project, parent):
             raise ValueError("A Key must have a project set.")
 
     return project
+
+
+def _clean_app(app_str):
+    """Clean a legacy (i.e. from App Engine) app string.
+
+    :type app_str: str
+    :param app_str: The ``app`` value stored in a "Reference" pb.
+
+    :rtype: str
+    :returns: The cleaned value.
+    """
+    parts = app_str.split('~', 1)
+    return parts[-1]
+
+
+def _get_empty(value, empty_value):
+    """Check if a protobuf field is "empty".
+
+    :type value: object
+    :param value: A basic field from a protobuf.
+
+    :type empty_value: object
+    :param empty_value: The "empty" value for the same type as
+                        ``value``.
+    """
+    if value == empty_value:
+        return None
+    else:
+        return value
+
+
+def _check_database_id(database_id):
+    """Make sure a "Reference" database ID is empty.
+
+    :type database_id: unicode
+    :param database_id: The ``database_id`` field from a "Reference" protobuf.
+
+    :raises: :exc:`ValueError` if the ``database_id`` is not empty.
+    """
+    if database_id != u'':
+        msg = _DATABASE_ID_TEMPLATE.format(database_id)
+        raise ValueError(msg)
+
+
+def _add_id_or_name(flat_path, element_pb, empty_allowed):
+    """Add the ID or name from an element to a list.
+
+    :type flat_path: list
+    :param flat_path: List of accumulated path parts.
+
+    :type element_pb: :class:`._app_engine_key_pb2.Path.Element`
+    :param element_pb: The element containing ID or name.
+
+    :type empty_allowed: bool
+    :param empty_allowed: Indicates if neither ID or name need be set. If
+                          :data:`False`, then **exactly** one of them must be.
+
+    :raises: :exc:`ValueError` if 0 or 2 of ID/name are set (unless
+             ``empty_allowed=True`` and 0 are set).
+    """
+    id_ = element_pb.id
+    name = element_pb.name
+    # NOTE: Below 0 and the empty string are the "null" values for their
+    #       respective types, indicating that the value is unset.
+    if id_ == 0:
+        if name == u'':
+            if not empty_allowed:
+                raise ValueError(_EMPTY_ELEMENT)
+        else:
+            flat_path.append(name)
+    else:
+        if name == u'':
+            flat_path.append(id_)
+        else:
+            msg = _BAD_ELEMENT_TEMPLATE.format(id_, name)
+            raise ValueError(msg)
+
+
+def _get_flat_path(path_pb):
+    """Convert a legacy "Path" protobuf to a flat path.
+
+    For example
+
+        Element {
+          type: "parent"
+          id: 59
+        }
+        Element {
+          type: "child"
+          name: "naem"
+        }
+
+    would convert to ``('parent', 59, 'child', 'naem')``.
+
+    :type path_pb: :class:`._app_engine_key_pb2.Path`
+    :param path_pb: Legacy protobuf "Path" object (from a "Reference").
+
+    :rtype: tuple
+    :returns: The path parts from ``path_pb``.
+    """
+    num_elts = len(path_pb.element)
+    last_index = num_elts - 1
+
+    result = []
+    for index, element in enumerate(path_pb.element):
+        result.append(element.type)
+        _add_id_or_name(result, element, index == last_index)
+
+    return tuple(result)
+
+
+def _to_legacy_path(dict_path):
+    """Convert a tuple of ints and strings in a legacy "Path".
+
+    .. note:
+
+        This assumes, but does not verify, that each entry in
+        ``dict_path`` is valid (i.e. doesn't have more than one
+        key out of "name" / "id").
+
+    :type dict_path: lsit
+    :param dict_path: The "structured" path for a key, i.e. it
+                      is a list of dictionaries, each of which has
+                      "kind" and one of "name" / "id" as keys.
+
+    :rtype: :class:`._app_engine_key_pb2.Path`
+    :returns: The legacy path corresponding to ``dict_path``.
+    """
+    elements = []
+    for part in dict_path:
+        element_kwargs = {'type': part['kind']}
+        if 'id' in part:
+            element_kwargs['id'] = part['id']
+        elif 'name' in part:
+            element_kwargs['name'] = part['name']
+        element = _app_engine_key_pb2.Path.Element(**element_kwargs)
+        elements.append(element)
+
+    return _app_engine_key_pb2.Path(element=elements)


Test patch:
diff --git a/datastore/tests/unit/test_key.py b/datastore/tests/unit/test_key.py
--- a/datastore/tests/unit/test_key.py
+++ b/datastore/tests/unit/test_key.py
@@ -18,6 +18,20 @@
 class TestKey(unittest.TestCase):
 
     _DEFAULT_PROJECT = 'PROJECT'
+    # NOTE: This comes directly from a running (in the dev appserver)
+    #       App Engine app. Created via:
+    #
+    #           from google.appengine.ext import ndb
+    #           key = ndb.Key(
+    #               'Parent', 59, 'Child', 'Feather',
+    #               namespace='space', app='s~sample-app')
+    #           urlsafe = key.urlsafe()
+    _URLSAFE_EXAMPLE = (
+        b'agxzfnNhbXBsZS1hcHByHgsSBlBhcmVudBg7DAsSBUNoaWxkIgdGZ'
+        b'WF0aGVyDKIBBXNwYWNl')
+    _URLSAFE_APP = 's~sample-app'
+    _URLSAFE_NAMESPACE = 'space'
+    _URLSAFE_FLAT_PATH = ('Parent', 59, 'Child', 'Feather')
 
     @staticmethod
     def _get_target_class():
@@ -372,6 +386,27 @@ def test_to_protobuf_w_no_kind(self):
         # Unset values are False-y.
         self.assertEqual(pb.path[0].kind, '')
 
+    def test_to_legacy_urlsafe(self):
+        key = self._make_one(
+            *self._URLSAFE_FLAT_PATH,
+            project=self._URLSAFE_APP,
+            namespace=self._URLSAFE_NAMESPACE)
+        # NOTE: ``key.project`` is somewhat "invalid" but that is OK.
+        urlsafe = key.to_legacy_urlsafe()
+        self.assertEqual(urlsafe, self._URLSAFE_EXAMPLE)
+
+    def test_from_legacy_urlsafe(self):
+        klass = self._get_target_class()
+        key = klass.from_legacy_urlsafe(self._URLSAFE_EXAMPLE)
+
+        self.assertEqual('s~' + key.project, self._URLSAFE_APP)
+        self.assertEqual(key.namespace, self._URLSAFE_NAMESPACE)
+        self.assertEqual(key.flat_path, self._URLSAFE_FLAT_PATH)
+        # Also make sure we didn't accidentally set the parent.
+        self.assertIsNone(key._parent)
+        self.assertIsNotNone(key.parent)
+        self.assertIs(key._parent, key.parent)
+
     def test_is_partial_no_name_or_id(self):
         key = self._make_one('KIND', project=self._DEFAULT_PROJECT)
         self.assertTrue(key.is_partial)
@@ -431,3 +466,245 @@ def test_parent_multiple_calls(self):
         self.assertEqual(parent.path, _PARENT_PATH)
         new_parent = key.parent
         self.assertIs(parent, new_parent)
+
+
+class Test__clean_app(unittest.TestCase):
+
+    PROJECT = 'my-prahjekt'
+
+    @staticmethod
+    def _call_fut(app_str):
+        from google.cloud.datastore.key import _clean_app
+
+        return _clean_app(app_str)
+
+    def test_already_clean(self):
+        app_str = self.PROJECT
+        self.assertEqual(self._call_fut(app_str), self.PROJECT)
+
+    def test_standard(self):
+        app_str = 's~' + self.PROJECT
+        self.assertEqual(self._call_fut(app_str), self.PROJECT)
+
+    def test_european(self):
+        app_str = 'e~' + self.PROJECT
+        self.assertEqual(self._call_fut(app_str), self.PROJECT)
+
+    def test_dev_server(self):
+        app_str = 'dev~' + self.PROJECT
+        self.assertEqual(self._call_fut(app_str), self.PROJECT)
+
+
+class Test__get_empty(unittest.TestCase):
+
+    @staticmethod
+    def _call_fut(value, empty_value):
+        from google.cloud.datastore.key import _get_empty
+
+        return _get_empty(value, empty_value)
+
+    def test_unset(self):
+        for empty_value in (u'', 0, 0.0, []):
+            ret_val = self._call_fut(empty_value, empty_value)
+            self.assertIsNone(ret_val)
+
+    def test_actually_set(self):
+        value_pairs = (
+            (u'hello', u''),
+            (10, 0),
+            (3.14, 0.0),
+            (['stuff', 'here'], []),
+        )
+        for value, empty_value in value_pairs:
+            ret_val = self._call_fut(value, empty_value)
+            self.assertIs(ret_val, value)
+
+
+class Test__check_database_id(unittest.TestCase):
+
+    @staticmethod
+    def _call_fut(database_id):
+        from google.cloud.datastore.key import _check_database_id
+
+        return _check_database_id(database_id)
+
+    def test_empty_value(self):
+        ret_val = self._call_fut(u'')
+        # Really we are just happy there was no exception.
+        self.assertIsNone(ret_val)
+
+    def test_failure(self):
+        with self.assertRaises(ValueError):
+            self._call_fut(u'some-database-id')
+
+
+class Test__add_id_or_name(unittest.TestCase):
+
+    @staticmethod
+    def _call_fut(flat_path, element_pb, empty_allowed):
+        from google.cloud.datastore.key import _add_id_or_name
+
+        return _add_id_or_name(flat_path, element_pb, empty_allowed)
+
+    def test_add_id(self):
+        flat_path = []
+        id_ = 123
+        element_pb = _make_element_pb(id=id_)
+
+        ret_val = self._call_fut(flat_path, element_pb, False)
+        self.assertIsNone(ret_val)
+        self.assertEqual(flat_path, [id_])
+        ret_val = self._call_fut(flat_path, element_pb, True)
+        self.assertIsNone(ret_val)
+        self.assertEqual(flat_path, [id_, id_])
+
+    def test_add_name(self):
+        flat_path = []
+        name = 'moon-shadow'
+        element_pb = _make_element_pb(name=name)
+
+        ret_val = self._call_fut(flat_path, element_pb, False)
+        self.assertIsNone(ret_val)
+        self.assertEqual(flat_path, [name])
+        ret_val = self._call_fut(flat_path, element_pb, True)
+        self.assertIsNone(ret_val)
+        self.assertEqual(flat_path, [name, name])
+
+    def test_both_present(self):
+        element_pb = _make_element_pb(id=17, name='seventeen')
+        flat_path = []
+        with self.assertRaises(ValueError):
+            self._call_fut(flat_path, element_pb, False)
+        with self.assertRaises(ValueError):
+            self._call_fut(flat_path, element_pb, True)
+
+        self.assertEqual(flat_path, [])
+
+    def test_both_empty_failure(self):
+        element_pb = _make_element_pb()
+        flat_path = []
+        with self.assertRaises(ValueError):
+            self._call_fut(flat_path, element_pb, False)
+
+        self.assertEqual(flat_path, [])
+
+    def test_both_empty_allowed(self):
+        element_pb = _make_element_pb()
+        flat_path = []
+        ret_val = self._call_fut(flat_path, element_pb, True)
+        self.assertIsNone(ret_val)
+        self.assertEqual(flat_path, [])
+
+
+class Test__get_flat_path(unittest.TestCase):
+
+    @staticmethod
+    def _call_fut(path_pb):
+        from google.cloud.datastore.key import _get_flat_path
+
+        return _get_flat_path(path_pb)
+
+    def test_one_pair(self):
+        kind = 'Widget'
+        name = 'Scooter'
+        element_pb = _make_element_pb(type=kind, name=name)
+        path_pb = _make_path_pb(element_pb)
+        flat_path = self._call_fut(path_pb)
+        self.assertEqual(flat_path, (kind, name))
+
+    def test_two_pairs(self):
+        kind1 = 'parent'
+        id1 = 59
+        element_pb1 = _make_element_pb(type=kind1, id=id1)
+
+        kind2 = 'child'
+        name2 = 'naem'
+        element_pb2 = _make_element_pb(type=kind2, name=name2)
+
+        path_pb = _make_path_pb(element_pb1, element_pb2)
+        flat_path = self._call_fut(path_pb)
+        self.assertEqual(flat_path, (kind1, id1, kind2, name2))
+
+    def test_partial_key(self):
+        kind1 = 'grandparent'
+        name1 = 'cats'
+        element_pb1 = _make_element_pb(type=kind1, name=name1)
+
+        kind2 = 'parent'
+        id2 = 1337
+        element_pb2 = _make_element_pb(type=kind2, id=id2)
+
+        kind3 = 'child'
+        element_pb3 = _make_element_pb(type=kind3)
+
+        path_pb = _make_path_pb(element_pb1, element_pb2, element_pb3)
+        flat_path = self._call_fut(path_pb)
+        self.assertEqual(flat_path, (kind1, name1, kind2, id2, kind3))
+
+
+class Test__to_legacy_path(unittest.TestCase):
+
+    @staticmethod
+    def _call_fut(dict_path):
+        from google.cloud.datastore.key import _to_legacy_path
+
+        return _to_legacy_path(dict_path)
+
+    def test_one_pair(self):
+        kind = 'Widget'
+        name = 'Scooter'
+        dict_path = [{'kind': kind, 'name': name}]
+        path_pb = self._call_fut(dict_path)
+
+        element_pb = _make_element_pb(type=kind, name=name)
+        expected_pb = _make_path_pb(element_pb)
+        self.assertEqual(path_pb, expected_pb)
+
+    def test_two_pairs(self):
+        kind1 = 'parent'
+        id1 = 59
+
+        kind2 = 'child'
+        name2 = 'naem'
+
+        dict_path = [{'kind': kind1, 'id': id1}, {'kind': kind2, 'name': name2}]
+        path_pb = self._call_fut(dict_path)
+
+        element_pb1 = _make_element_pb(type=kind1, id=id1)
+        element_pb2 = _make_element_pb(type=kind2, name=name2)
+        expected_pb = _make_path_pb(element_pb1, element_pb2)
+        self.assertEqual(path_pb, expected_pb)
+
+    def test_partial_key(self):
+        kind1 = 'grandparent'
+        name1 = 'cats'
+
+        kind2 = 'parent'
+        id2 = 1337
+
+        kind3 = 'child'
+
+        dict_path = [
+            {'kind': kind1, 'name': name1},
+            {'kind': kind2, 'id': id2},
+            {'kind': kind3},
+        ]
+        path_pb = self._call_fut(dict_path)
+
+        element_pb1 = _make_element_pb(type=kind1, name=name1)
+        element_pb2 = _make_element_pb(type=kind2, id=id2)
+        element_pb3 = _make_element_pb(type=kind3)
+        expected_pb = _make_path_pb(element_pb1, element_pb2, element_pb3)
+        self.assertEqual(path_pb, expected_pb)
+
+
+def _make_element_pb(**kwargs):
+    from google.cloud.datastore import _app_engine_key_pb2
+
+    return _app_engine_key_pb2.Path.Element(**kwargs)
+
+
+def _make_path_pb(*element_pbs):
+    from google.cloud.datastore import _app_engine_key_pb2
+
+    return _app_engine_key_pb2.Path(element=element_pbs)