xiaoxu193/PyTeaser — pull request #26 by tiangolo (2014-05-19 21:56:52+00:00)
Title: Fixed Python unicode support.

Fixes issue #25

Allows Unicode support.

For example:

import pyteaser
pyteaser.Summarize(u'Título', u'Artículo. Contenido.')

Results in:

[u' Contenido.', u'Art\xedculo.']

Instead of:

Traceback (most recent call last):
File "", line 1, in
File "C:\Users\LVELEZ\Anaconda\lib\site-packages\pyteaser.py", line 86, in Summarize
sentences = split_sentences(text)
File "C:\Users\LVELEZ\Anaconda\lib\site-packages\pyteaser.py", line 215, in split_sentences
s_iter = [''.join(map(str,y)).lstrip() for y in s_iter]
UnicodeEncodeError: 'ascii' codec can't encode character u'\xed' in position 3:
ordinal not in range(128)
>>> pyteaser.Summarize(u'Título', u'Artículo. Contenido.')
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\LVELEZ\Anaconda\lib\site-packages\pyteaser.py", line 86, in Summarize
sentences = split_sentences(text)
File "C:\Users\LVELEZ\Anaconda\lib\site-packages\pyteaser.py", line 215, in split_sentences
s_iter = [''.join(map(str,y)).lstrip() for y in s_iter]
UnicodeEncodeError: 'ascii' codec can't encode character u'\xed' in position 3:ordinal not in range(128)