Repository: googleapis/google-cloud-python

Issue:
Bigtable HappyBase enable/disable table should be a no-op
Currently methods which aren't available in Bigtable for HappyBase throw an NotImplementedError. https://github.com/GoogleCloudPlatform/gcloud-python/blob/master/gcloud/bigtable/happybase/connection.py#L392

This will make it more difficult to port applications to and from HBase. These operations should be no-ops (possibly logging a warning when run).



Discussion:
@dhermes I'll defer to your judgement here:  I'm not sure what would be worse for the developer porting an application:  having unexpected `NotImplementedError` exceptions raised, or quasi-silent failures.  Maybe we need to have an environment variable which toggles the exceptions on (or off)?

@tswast I feel like not raising the exception would be misleading for the majority of people. Can you elaborate why you'd prefer the silent failure?

No-op is what the Java client does.

This is especially important for third-party tools that build on top of HBase to work on Cloud Bigtable with minimal modification.

I'm not sure if there are any similar tools that build on HappyBase that would have similar concerns, but ideally it should be as easy as possible to port an application to and from Cloud Bigtable and HappyBase.

@mbrukman Do you have an opinion about this?

I'm not sure where a missing feature of the backend API falls into "easy as possible". As for minimal modification, they could simply remove calls to disable and enable table

Let's first agree in principle on the purpose of having a compatibility library such as the HBase client for Java and the HappyBase client for Python:
- our goal is to be compatible with code that uses HBase — this gets us instant integrations that work with HBase and user code that works with HBase becomes trivially portable
- as such, we should aim to have Bigtable become a drop-in replacement for HBase: code that works with HBase should be working with Bigtable with a tiny amount of configuration changes + code to connect to Bigtable
- we should avoid having to tell users to special-case Bigtable in their code that works with HBase, it should be transparent to the code as to whether it's working with HBase or Bigtable
- in essence, our library should _hide_ the differences between HBase and Bigtable, and thus absorb complexity where possible, rather than exposing the differences and forcing customers to write 2 backend implementations, one for HBase and a separate one for Bigtable

Assuming we agree on the above, I think this argues that we should **not** throw exceptions, but either implement what we need to do, and make unnecessary functions no-ops whenever possible. Of course, each function/method should be considered on its own merits, what its intended semantics are, etc., but the goal here is not to reimplement the HBase _semantics_ 1:1, but make it easy for code to use HBase and Bigtable transparently, with little to no work from the user.

AIUI, in HBase, before deleting a table or modifying some settings, you have to first disable it. Thus, all HBase-compliant code will _always_ call `disable_table()` prior to calling `delete_table()`.

I would argue that in this case, we should:
- convert calls to `disable_table()` and `enable_table()` to be no-ops rather than throw exceptions
- e.g., always return true to `is_enabled()`
- drop the warning from `drop_table()` being called with `True` for dropping table first
- document the difference that Bigtable tables have no concept of `disabled` state, because we can modify settings and delete tables without the prerequisite of disabling first

Note that this means that we would fail a test of the following type:

``` python
...
table.enable()
assertTrue(table.is_enabled())  # succeeds
table.disable()
assertFalse(table.is_enabled())  # fails
```

but I would argue this is a corner case and not something anyone should be doing in practice, outside of HBase tests themselves.

Other functions/methods should be considered on a case-by-case basis.

Let's consider the opposite: what do we gain from throwing exceptions? I am not sure.

What is the side effect of throwing exceptions?

Typically, this means that portability is hindered from HBase to Bigtable, leading to several options:
- users will either give up, or
- if they want to persevere, they will write abstraction libraries that separate HBase functions from Bigtable functions, or
- users fork gcloud-python and are unable to upgrade since they made local modifications

None of these are good outcomes, from my perspective, as they are providing a sub-par user experience, and we would be the cause of all of them.

Thus, I would advocate that we try to adapt our API to be _functionally_ matching the intended semantics of HappyBase rather than trying to implement them _strictly_ and throw exceptions where we are incompatible, because that breaks the spirit of the point of having a compatibility library in the first place.

Thoughts?

Seems reasonable enough. I'll send out a PR.

@mbrukman What about `Connection.compact_table(...)`, `Table.regions()` and `Table.counter_set(...)`?


Resolving patch:
diff --git a/gcloud/bigtable/happybase/connection.py b/gcloud/bigtable/happybase/connection.py
--- a/gcloud/bigtable/happybase/connection.py
+++ b/gcloud/bigtable/happybase/connection.py
@@ -49,9 +49,16 @@
 
 _LEGACY_ARGS = frozenset(('host', 'port', 'compat', 'transport', 'protocol'))
 _WARN = warnings.warn
+_BASE_DISABLE = 'Cloud Bigtable has no concept of enabled / disabled tables.'
 _DISABLE_DELETE_MSG = ('The disable argument should not be used in '
-                       'delete_table(). Cloud Bigtable has no concept '
-                       'of enabled / disabled tables.')
+                       'delete_table(). ') + _BASE_DISABLE
+_ENABLE_TMPL = 'Connection.enable_table(%r) was called, but ' + _BASE_DISABLE
+_DISABLE_TMPL = 'Connection.disable_table(%r) was called, but ' + _BASE_DISABLE
+_IS_ENABLED_TMPL = ('Connection.is_table_enabled(%r) was called, but ' +
+                    _BASE_DISABLE)
+_COMPACT_TMPL = ('Connection.compact_table(%r, major=%r) was called, but the '
+                 'Cloud Bigtable API handles table compactions automatically '
+                 'and does not expose an API for it.')
 
 
 def _get_instance(timeout=None):
@@ -378,61 +385,70 @@ def delete_table(self, name, disable=False):
         name = self._table_name(name)
         _LowLevelTable(name, self._instance).delete()
 
-    def enable_table(self, name):
+    @staticmethod
+    def enable_table(name):
         """Enable the specified table.
 
         .. warning::
 
             Cloud Bigtable has no concept of enabled / disabled tables so this
-            method does not work. It is provided simply for compatibility.
+            method does nothing. It is provided simply for compatibility.
 
-        :raises: :class:`NotImplementedError <exceptions.NotImplementedError>`
-                 always
+        :type name: str
+        :param name: The name of the table to be enabled.
         """
-        raise NotImplementedError('The Cloud Bigtable API has no concept of '
-                                  'enabled or disabled tables.')
+        _WARN(_ENABLE_TMPL % (name,))
 
-    def disable_table(self, name):
+    @staticmethod
+    def disable_table(name):
         """Disable the specified table.
 
         .. warning::
 
             Cloud Bigtable has no concept of enabled / disabled tables so this
-            method does not work. It is provided simply for compatibility.
+            method does nothing. It is provided simply for compatibility.
 
-        :raises: :class:`NotImplementedError <exceptions.NotImplementedError>`
-                 always
+        :type name: str
+        :param name: The name of the table to be disabled.
         """
-        raise NotImplementedError('The Cloud Bigtable API has no concept of '
-                                  'enabled or disabled tables.')
+        _WARN(_DISABLE_TMPL % (name,))
 
-    def is_table_enabled(self, name):
+    @staticmethod
+    def is_table_enabled(name):
         """Return whether the specified table is enabled.
 
         .. warning::
 
             Cloud Bigtable has no concept of enabled / disabled tables so this
-            method does not work. It is provided simply for compatibility.
+            method always returns :data:`True`. It is provided simply for
+            compatibility.
 
-        :raises: :class:`NotImplementedError <exceptions.NotImplementedError>`
-                 always
+        :type name: str
+        :param name: The name of the table to check enabled / disabled status.
+
+        :rtype: bool
+        :returns: The value :data:`True` always.
         """
-        raise NotImplementedError('The Cloud Bigtable API has no concept of '
-                                  'enabled or disabled tables.')
+        _WARN(_IS_ENABLED_TMPL % (name,))
+        return True
 
-    def compact_table(self, name, major=False):
+    @staticmethod
+    def compact_table(name, major=False):
         """Compact the specified table.
 
         .. warning::
 
-            Cloud Bigtable does not support compacting a table, so this
-            method does not work. It is provided simply for compatibility.
+            Cloud Bigtable supports table compactions, it just doesn't expose
+            an API for that feature, so this method does nothing. It is
+            provided simply for compatibility.
+
+        :type name: str
+        :param name: The name of the table to compact.
 
-        :raises: :class:`NotImplementedError <exceptions.NotImplementedError>`
-                 always
+        :type major: bool
+        :param major: Whether to perform a major compaction.
         """
-        raise NotImplementedError('The Cloud Bigtable API does not support '
-                                  'compacting a table.')
+        _WARN(_COMPACT_TMPL % (name, major))
 
 
 def _parse_family_option(option):


Test patch:
diff --git a/gcloud/bigtable/happybase/test_connection.py b/gcloud/bigtable/happybase/test_connection.py
--- a/gcloud/bigtable/happybase/test_connection.py
+++ b/gcloud/bigtable/happybase/test_connection.py
@@ -488,37 +488,81 @@ def mock_warn(msg):
         self.assertEqual(warned, [MUT._DISABLE_DELETE_MSG])
 
     def test_enable_table(self):
+        from gcloud._testing import _Monkey
+        from gcloud.bigtable.happybase import connection as MUT
+
         instance = _Instance()  # Avoid implicit environ check.
         connection = self._makeOne(autoconnect=False, instance=instance)
 
         name = 'table-name'
-        with self.assertRaises(NotImplementedError):
+
+        warned = []
+
+        def mock_warn(msg):
+            warned.append(msg)
+
+        with _Monkey(MUT, _WARN=mock_warn):
             connection.enable_table(name)
 
+        self.assertEqual(warned, [MUT._ENABLE_TMPL % (name,)])
+
     def test_disable_table(self):
+        from gcloud._testing import _Monkey
+        from gcloud.bigtable.happybase import connection as MUT
+
         instance = _Instance()  # Avoid implicit environ check.
         connection = self._makeOne(autoconnect=False, instance=instance)
 
         name = 'table-name'
-        with self.assertRaises(NotImplementedError):
+
+        warned = []
+
+        def mock_warn(msg):
+            warned.append(msg)
+
+        with _Monkey(MUT, _WARN=mock_warn):
             connection.disable_table(name)
 
+        self.assertEqual(warned, [MUT._DISABLE_TMPL % (name,)])
+
     def test_is_table_enabled(self):
+        from gcloud._testing import _Monkey
+        from gcloud.bigtable.happybase import connection as MUT
+
         instance = _Instance()  # Avoid implicit environ check.
         connection = self._makeOne(autoconnect=False, instance=instance)
 
         name = 'table-name'
-        with self.assertRaises(NotImplementedError):
-            connection.is_table_enabled(name)
+
+        warned = []
+
+        def mock_warn(msg):
+            warned.append(msg)
+
+        with _Monkey(MUT, _WARN=mock_warn):
+            result = connection.is_table_enabled(name)
+
+        self.assertTrue(result)
+        self.assertEqual(warned, [MUT._IS_ENABLED_TMPL % (name,)])
 
     def test_compact_table(self):
+        from gcloud._testing import _Monkey
+        from gcloud.bigtable.happybase import connection as MUT
+
         instance = _Instance()  # Avoid implicit environ check.
         connection = self._makeOne(autoconnect=False, instance=instance)
 
         name = 'table-name'
-        major = True
-        with self.assertRaises(NotImplementedError):
-            connection.compact_table(name, major=major)
+
+        warned = []
+
+        def mock_warn(msg):
+            warned.append(msg)
+
+        with _Monkey(MUT, _WARN=mock_warn):
+            connection.compact_table(name)
+
+        self.assertEqual(warned, [MUT._COMPACT_TMPL % (name, False)])
 
 
 class Test__parse_family_option(unittest2.TestCase):