bergie/noflo — pull request #58 by kenhkan (2012-12-10 04:41:47+00:00)
Title: Buffer all IPs and flush on disconnect

Phew. This is a big PR. So here we go:

This may be familiar to you:
```
class SomeComponent ...
...
@inPorts.in.on "beginGroup", (group) ->
@groups.push(group)
@inPorts.in.on "data", (data) ->
# Do something here
@inPorts.in.on "endGroup", (group) ->
@groups.pop()
@inPorts.in.on "disconnect", () ->
# Do some more things here
...
```

This is quite common in the existing libraries and I find myself doing this over and over again. Not only that, when I need to manipulate the data in batch, like examining the groups and deciding to send the data forward or not, I need to do something like this:
```
class SomeComponent ...
...
@inPorts.in.on "beginGroup", (group) ->
@groups.push(group)
@inPorts.in.on "data", (data) ->
@data.push(data)
@inPorts.in.on "endGroup", (group) ->
@groups.pop()
@inPorts.in.on "disconnect", () ->
# Do some tests/filters/manipulation/whatever here
for group in @groups
@outPorts.out.beginGroup(group)
for datum in @data
@outPorts.out.send(datum)
for group in @groups
@outPorts.out.endGroup()
@outPorts.out.disconnect()
...
```

Doing either the first example or the second example repeatedly is not fun, nor is it correct: you lose the structure of groupings. Instead of having something like an XML-like tree structure, which groups are supposed to provide, by the time you access it, it's linear.

Hence this PR. It expands the role of the port to allow *each* port to have a buffer of incoming/outgoing connection. It flushes itself every time a connection is disconnected. The primary benefit is that until you disconnect, you know that you still have the data at bay and you can manipulate it. So for example:
```
class SomeComponent ...
...
@inPorts.in.on "beginGroup", (group) ->
@outPorts.out.beginGroup(group)
@inPorts.in.on "data", (data) ->
groups = @outPorts.out.getGroups()
# Only grouped data are forwarded
if groups.length > 0
@outPorts.out.send(data)
@inPorts.in.on "endGroup", (group) ->
@outPorts.out.endGroup()
@inPorts.in.on "disconnect", () ->
buffer = @outPorts.out.getBuffer()
# Do something with the buffer
@outPorts.out.setBuffer(buffer)
@outPorts.out.disconnect()
...
```

This is a trivial example but you can see that I can choose to manipulate all the "sent" groups/data *before* they actually get sent. Great for batch operations.

Here are the side benefits:
1. Now a downstream process would not be aware that an IP is coming its way *until* the connection between its upstream processes disconnect. This is a problem right now because it forwards groups/data immediately so in a synchronous graph, all processes would receive a `beginGroup` before they get a `data`. This is not how the mental model of FBP should work (at least not mine :p)
2. There is no initial downstream connection problem. See #40.
3. You force the programmer to write correct FBP programs. I mean, you *have* to disconnect for your stuff to be sent.

Now this was done a while back. I have been testing it in production so it was before the style change and me knowing that there was a test directory. I will throw in tests for this but I need to know if this is something from which noflo can benefit. I imagined it could be a separate `class BufferedPort extends Port` but I think it's not necessary because it's backward-compatible.

Let me know if I can clarify any aspect of this since this is a pretty big change in the codebase.