coderplay/dragon — Issue #1: Serializable DAG utilities is needed
Author: coderplay
Created: 2012-01-23 13:08:18+00:00
State: open

Every stream computing workflow can be represented by a DAG. These kind of DAG jobs submitted by Hadoop dargon users should be serialized into a file on ApplicationMaster's local filesystem or a distributed filesystem, thus NodeManager can download it and launch worker's containers according to the serialized DAG. Each worker with the same kind acts as a vertex of the DAG. Its input stream comes from another kind of workers(vertexs) and its output is transferred into vertexs downstream. The stream between two vertexs is represented by edge.