zopefoundation/ZODB — pull request #9 by ossmkitty (2013-05-16 16:38:19+00:00)
Title: UnicodeDecodeError with bushy blob layout and non-ASCII OID

When we updated from ZODB 4.0.0a4 to 4.0.0b2 running under Python 2.7 some of our unit tests started breaking with tracebacks like this:

> Module ZODB.Connection, line 569, in commit
> self._commit(transaction)
> Module ZODB.Connection, line 625, in _commit
> self._store_objects(ObjectWriter(obj), transaction)
> Module ZODB.Connection, line 677, in _store_objects
> '', transaction)
> Module ZODB.DemoStorage, line 305, in storeBlob
> oid, oldserial, data, blobfilename, '', transaction)
> Module ZODB.blob, line 691, in storeBlob
> self._blob_storeblob(oid, serial, blobfilename)
> Module ZODB.blob, line 676, in _blob_storeblob
> self.fshelper.getPathForOID(oid, create=True)
> Module ZODB.blob, line 391, in getPathForOID
> path = self.layout.oid_to_path(oid)
> Module ZODB.blob, line 555, in oid_to_path
> for byte in oid.decode():
>UnicodeDecodeError: 'ascii' codec can't decode byte 0xcb in position 1: ordinal not in range(128)

The OID in question looked something like `b'>\xf1<0\xe9Q\x99\xf0'`. This seems to have changed in e7d8ca7229998146f79e1f90302ae6486bc95a60.

The attached pull request adds a test for this case and changes `ZODB.blob:BushyLayout.oid_to_path` to not be dependent on the default character encoding and to produce the correct results in this case under all tox environments (though it's probably not as efficient as it could be?).