dtao/lazy.js — pull request #70 by FrozenCow (2014-02-23 17:33:30+00:00)
Title: Made SplitStreamSequence work across multiple chunks for strings

(Hi again, sorry for the previous pull request, but apparently I still stumbled across a different problem ;) )
In my use-case of Lazy I'm reading a file that needs to be split into large pieces. The chunks that the stream spits out are smaller than the pieces I need. At the moment Lazy.js splits on each chunk separately, which results in Lazy spitting out the chunks as-is, instead of actual splitted pieces.

This also goes for line-splitting. There can be multiple lines inside one chunk. Most of those will be correct, but the lines that span across 2 chunks will be incorrect as far as I've seen.

This patch contains a solution for this by concatenating chunks and looking up the divider inside that concatenation. It also includes a test that fails with the earlier code and passes with the code in this patch.

However, some other tests fail. This is because I have only implemented a solution for string-dividers, not regexp-dividers! I need some hints as to how to solve this for regexp-dividers, since those don't have a constant length. Maybe for regexp dividers it should just fallback to the existing implementation, but that's somewhat confusing as you don't expect the output to be like this.