deanmalmgren/textract — pull request #39 by deanmalmgren (2014-08-11 14:18:46+00:00)
Title: standardize encoding of textract.process?

*I'm not really sure this is needed but I thought I'd use this issue as an opportunity to get feedback from others. All thoughts welcome!*

Some parsers return ascii-encoded text like the [`odt_parser`](https://github.com/deanmalmgren/textract/blob/master/textract/parsers/odt_parser.py#L13) whereas others are encoded as utf8 like the [`html_parser`](https://github.com/deanmalmgren/textract/blob/master/textract/parsers/html_parser.py#L35) whereas others don't specify an encoding at all like the [`json_parser`](https://github.com/deanmalmgren/textract/blob/master/textract/parsers/json_parser.py#L4-L11). Perhaps this matters. Perhaps it doesn't. But this seems like the type of thing that should maybe be standardized.

If we do standardize the encoding of parser methods, we should probably give end users the ability to specify the encoding that they want rather than enforcing a particular encoding on everyone. This would involve supporting a `-e/--encoding` option in the command line tool and supporting an `encoding` kwarg to the `textract.parsers.*_parser.extract` functions.

To make it as easy as possible to continue to add parsers, it might make sense to switch to using classes for parsers instead of functions so that we can use class inheritance for standardizing the encoding with something like this:

```python
# textract/parsers/base.py
class BaseParser(object):
def __call__(self, filename, encoding='utf-8', **kwargs):
"""This makes it possible to do things like:
parser = BaseParser()
output = parser(filename, **kwargs)
"""
return self.encode(self.extract(filename, **kwargs), encoding)
def extract(self, filename, **kwargs):
raise NotImplementedError('must be overridden by child classes')
def encode(self, output, encoding):
return output.encode(encoding)

class ShellParser(BaseParser):
def run(self, command):
# this would be where we could put the current textract.shell.run implementation
pass

# textract/parsers/txt.py
from .base import BaseParser
class Parser(BaseParser):
def extract(self, filename, **kwargs):
with open(filename) as stream:
return stream.read()
```

I'm also open to decorator implementations but, IMHO, decorators can be extremely confusing very quickly.