Repository: googleapis/google-cloud-python

Issue:
[bigtable] InvalidChunk
1. OS type and version
Ubuntu 14.04

2. Python version and virtual environment information `python --version`
Python 2.7.6

3. google-cloud-python version `pip show google-cloud`, `pip show google-<service>` or `pip freeze`

Name: google-cloud-bigtable
Version: 0.22.0

4. Stacktrace if available
```
Traceback (most recent call last):
  File "bt.py", line 13, in <module>
    row = bt_table.read_row('mykey')
  File "/home/jungho.ahn/tmp/test/local/lib/python2.7/site-packages/google/cloud/bigtable/table.py", line 234, in read_row
    rows_data.consume_all()
  File "/home/jungho.ahn/tmp/test/local/lib/python2.7/site-packages/google/cloud/bigtable/row_data.py", line 323, in consume_all
    self.consume_next()
  File "/home/jungho.ahn/tmp/test/local/lib/python2.7/site-packages/google/cloud/bigtable/row_data.py", line 275, in consume_next
    self._validate_chunk(chunk)
  File "/home/jungho.ahn/tmp/test/local/lib/python2.7/site-packages/google/cloud/bigtable/row_data.py", line 390, in _validate_chunk
    self._validate_chunk_row_in_progress(chunk)
  File "/home/jungho.ahn/tmp/test/local/lib/python2.7/site-packages/google/cloud/bigtable/row_data.py", line 370, in _validate_chunk_row_in_progress
    _raise_if(not chunk.timestamp_micros or not chunk.value)
  File "/home/jungho.ahn/tmp/test/local/lib/python2.7/site-packages/google/cloud/bigtable/row_data.py", line 441, in _raise_if
    raise InvalidChunk(*args)
google.cloud.bigtable.row_data.InvalidChunk
```
5. Steps to reproduce
See the code example
6. Code example
```
from google.cloud import bigtable

bt_client = bigtable.client.Client(project='myproject', read_only=True)
bt_instance = bt_client.instance(instance_id='myinstance')
bt_table = bt_instance.table(table_id='mytable')

row = bt_table.read_row('mykey')
print row.cells
```

This exception is raised all the time. Did I use incorrectly?

Thanks,


Discussion:
It's tough to say without actually seeing the failed chunk. Can you run this with a debugger? It may be easiest to just dump it in a script and run `python -m pdb script_name.py` or to run it in IPython and use the `%debug` magic after it fails.
```
(Pdb) print chunk
timestamp_micros: 1485886500000000
(Pdb) print not chunk.value
True
```

Actually the value is an empty string.
Gotcha, ISTM the chunk is equivalent to:

```python
>>> from google.cloud.bigtable._generated import bigtable_pb2
>>> Chunk = bigtable_pb2.ReadRowsResponse.CellChunk
>>> chunk = Chunk(timestamp_micros=1485886500000000)
>>> chunk
timestamp_micros: 1485886500000000

>>> 
```
@tseaver The culprit is exactly as described by @junghoahnsc. I don't quite recognize the validation since it was re-worked for `v2` while I was on "paternity leave". Is there a reason we don't allow empty cells on read?

**UPDATE**: I just verified that writing an empty cell works:

```python
>>> row = table.row(b'row-key')
>>> row.set_cell(u'col-fam-id', b'col-name', b'') 
>>> row.commit()
```

but then reading this row doesn't fail either (because the chunk type must be different):

```python
>>> partial_result = table.read_row(b'row-key')
>>> cells = partial_result[u'col-fam-id'][b'col-name']
>>> cells[0].value   
''
>>> cells[0].timestamp
datetime.datetime(2017, 2, 2, 22, 5, 4, 333000, tzinfo=<UTC>)
```

@junghoahnsc The state of the chunks is also relevant here, so I need a little more info to reproduce.

[1]: https://github.com/GoogleCloudPlatform/google-cloud-python/blob/7e1632598365af1c03709bb5f5ff29f4d8a98847/bigtable/google/cloud/bigtable/row_data.py#L370
Too much water under the bridge since June :).  I think we need to ask @garye, @mbrukman, or @sduskis.  The [context](https://github.com/GoogleCloudPlatform/google-cloud-python/blob/7e1632598365af1c03709bb5f5ff29f4d8a98847/bigtable/google/cloud/bigtable/row_data.py#L365-L377) is:

- The state machine is in the "row in progress" state (no current partial cell)
- It is validating  the chunk which would trigger a new cell (is doesn't reset or commit the row).
- It is asserting that there must be both a timestamp and a value set on that chunk.
Hi, I am trying to run this step by step:
The PartialRowData object's state is  "row in progress"
The chunk it currently processing has only "qualifier" and "timestamp_micros". "reset_now" is False, no "commit_row" field, "value" is empty.


The error is triggered in _validate_chunk_row_in_progress method:
        if not chunk.HasField('commit_row') and not chunk.reset_row:
            _raise_if(not chunk.timestamp_micros or not chunk.value)
Yeah I think the chunk and timestamp check there is invalid. 
Is there any workaround until it is fixed?
@dhermes do you need any more information from our side? Actually our bigtable was written by Go/Java apps and now we're trying to read it in Python.
Is there any way you can validate the same read using Go or Java? Just to gain confidence that the fix I suggested will work... And/or can you just remove those checks from your local copy and verify?
A timestamp of 0 is valid.  I think that there needs to be a chunk.value.
Timestamp of 0 is fine and empty value for chunk.value is fine, but I think "not chunk.value" is true if it's an empty string which is wrong. 
empty values are not fine in these cases.  An empty cell isn't a thing in BT.  If there's no reset, there needs to be a value, IIRC.
@junghoahnsc @yuanbaisc Can you consume the response using the [low-level API][1]?

```python
>>> # NOT: partial_result = table.read_row(b'row-key')
>>> from google.cloud.bigtable.table import _create_row_request
>>> request_pb = _create_row_request(table.name, row_key=b'row-key')
>>> response_iterator = client._data_stub.ReadRows(request_pb)
>>> all_responses = list(response_iterator)
```

**UPDATE**: If there is private data returned, you can redact it or feel free to email me (email in my GitHub profile)

[1]: https://github.com/GoogleCloudPlatform/google-cloud-python/blob/7e1632598365af1c03709bb5f5ff29f4d8a98847/bigtable/google/cloud/bigtable/table.py#L229-L230
@garye there is no issue in reading in Go/Java.
@dhermes Here is the output of `all_responses`. I trimmed it since it's private data. Plz let me know if you need full output.

```
[chunks {
  row_key: "166657957cd9d730abe8f5f73bcfecda"
  family_name {
    value: "cluster"
  }
  qualifier {
    value: "concept"
  }
  timestamp_micros: 1485887340000000
}
chunks {
  timestamp_micros: 1485886500000000
}
...
chunks {
  qualifier {
    value: "debug"
  }
  timestamp_micros: 1485887340000000
  value: "\022\332\222\001x..."
}
chunks {
  timestamp_micros: 1485886500000000
  value: "\022\242\222\001x\234..."
}
....
chunks {
  timestamp_micros: 1485879798000000
  value: "\n2\n\006OTR517\022(\n\022\tj\274t\223..."
}
chunks {
  timestamp_micros: 1485877620000000
  value: "\n2\n\006OTR517\022(\n..."
}
chunks {
  qualifier {
    value: "metastory"
  }
  timestamp_micros: 1485887340000000
}
chunks {
  timestamp_micros: 1485886500000000
}
...
chunks {
  qualifier {
    value: "snapgraph"
  }
  timestamp_micros: 1485887340000000
}
chunks {
  timestamp_micros: 1485886500000000
}
...
chunks {
  timestamp_micros: 1485877620000000
  commit_row: true
}
]
```
Regarding my earlier comment, @garye and I spoke offline.  While there's no such thing as a Bigtable cell with an value, someone can set a filter that strips the value; that's useful in cases where all you want to do is check for existence of a row.  

The check for value should be removed as well as @garye suggested.  I authored an implementation guide which may be incorrect.  Sorry about that.
IIUC, it looks like the current validation check is not correct. Is it easy to remove that?
Or should we use a low-level api until it is fixed?

Resolving patch:
diff --git a/bigtable/google/cloud/bigtable/row_data.py b/bigtable/google/cloud/bigtable/row_data.py
--- a/bigtable/google/cloud/bigtable/row_data.py
+++ b/bigtable/google/cloud/bigtable/row_data.py
@@ -366,8 +366,6 @@ def _validate_chunk_row_in_progress(self, chunk):
         """Helper for :meth:`_validate_chunk`"""
         assert self.state == self.ROW_IN_PROGRESS
         self._validate_chunk_status(chunk)
-        if not chunk.HasField('commit_row') and not chunk.reset_row:
-            _raise_if(not chunk.timestamp_micros or not chunk.value)
         _raise_if(chunk.row_key and
                   chunk.row_key != self._row.row_key)
         _raise_if(chunk.HasField('family_name') and


Test patch:
diff --git a/bigtable/unit_tests/test_row_data.py b/bigtable/unit_tests/test_row_data.py
--- a/bigtable/unit_tests/test_row_data.py
+++ b/bigtable/unit_tests/test_row_data.py
@@ -416,20 +416,6 @@ def test_invalid_empty_chunk(self):
         with self.assertRaises(InvalidChunk):
             prd.consume_next()
 
-    def test_invalid_empty_second_chunk(self):
-        from google.cloud.bigtable.row_data import InvalidChunk
-
-        chunks = _generate_cell_chunks(['', ''])
-        first = chunks[0]
-        first.row_key = b'RK'
-        first.family_name.value = 'A'
-        first.qualifier.value = b'C'
-        response = _ReadRowsResponseV2(chunks)
-        iterator = _MockCancellableIterator(response)
-        prd = self._make_one(iterator)
-        with self.assertRaises(InvalidChunk):
-            prd.consume_next()
-
 
 class TestPartialRowsData_JSON_acceptance_tests(unittest.TestCase):