sferik/twitter — pull request #481 by eric-vautour (2013-11-12 19:52:19+00:00)
Title: Added detection of split token

Looks like when we call ssl_client.readpartial(1024), it may split the token used by the BufferedTokenizer. Thus we get entries like:

@tokenizer.extract(something-before\r)
@tokenizer.extract(\nsomethin-after)

And thus BufferedTokenizer will not parse on the \r\n, and will result in invalid JSON
{somestuff}\r
\n{somotherstuff}

Results in JSON.parse('{somestuff}{someotherstuff}')