dynamic framework for processing high volume data streams via processing pipelines