mbr/simplekv — pull request #12 by jvrsantacruz (2013-12-30 21:11:53+00:00)
Title: Python3

This pull requests adds Python3 compatibility to the library.

### Moved from `str` to `bytes`
The `bytes` type replaces `str` and `io.BytesIO` does for `StringIO.StringIO`.
The library currently allows to store binary files (using the `put_file` method) and save raw strings (compressed data, pickle...) with no encoding.

Actually, there should be already quite a lot of code that stores `unicode` strings like this:

```python
store.put('key', data.encode('utf-8'))
store.get('key').decode('utf-8')
```
Using `bytes`, this is still valid while doesn't have problems with binary files.

### Unicode as value

From what I read in #11, there was thought of moving the library into `unicode`.
That would be very nice indeed, but it would also introduce backwards incompatibility and does not solve some use cases, like binary files as commented before. The simple `data.encoding('utf-8')` usage would break, or it should detect and handle the two different types accordignly.

In order to use `unicode` as value in the library, I think it would be easier and safer to add some sort of new `StringStore` wrapper over Store, with an `encoding` optional argument added to all the `put` method.

This is a quick and dirty sketch of the idea, allowing multiple encodings by using the first bytes of the value.

```python
StringStore(Store):
def put(key, value, encoding='utf-8'):
self._put(key, value, encoding)

def _put(key, value, encoding):
prefix = bytes(len(encoding)) + bytes(encoding)
self._put_file(key, value, BytesIO(prefix + data.encode(encoding)))

def _get(key, value):
buf = BytesIO()

self._get_file(key, buf)

encoding = self._get_encoding(buf)

value = buf.getvalue()[buf.tell():]

return value

def _get_encoding(content):
codec_name_length = buf.read(1)
encoding = buf.read(codec_name_length)
return encoding
```

### Testing

I added [tox](http://tox.readthedocs.org/en/latest/) for multiversion testing. It currently tests against `python 2.6`, `python 2.7` and `python 3.3`, but it seems to be running fine on `python 3.4 pre` and `pypy`.

To run the tests just:

```
pip install tox
tox
```

All tests passes, but the ones for the `S3` and `GAE` backends, which are being skipped :(

### Changes

- Added tox for multitesting support.
- Added a `_compat` module to help with imports.
- Fixed a bunch of different Warnings in the tests (unclosed files mostly).
- Fixed some problems skipping class-based-tests.
- Modified `HMACReader` to be used as a context manager along the `with` statement.
- Changed file handling to use context managers and `with`.

I realize that those are quite a lot of changes for a single pull request, but it was what it took to get all the tests passing and my own projects (using the `db/sql` backend) to work.

The original branch is divided in three different incremental branches:

1. [jvrsantacruz:fix_file_closing](https://github.com/jvrsantacruz/simplekv/tree/fix_file_closing)
2. [jvrsantacruz:multiversion_testing](https://github.com/jvrsantacruz/simplekv/tree/multiversion_testing)
3. [jvrsantacruz:python3](https://github.com/jvrsantacruz/simplekv/tree/python3)

I can perform a different pull request for each of those if you feel like it; I'm not doing it now because I feared that would result confusing.

BTW, congrats for your library, it's being quite useful :)