Idempotent pipeline processing (with debugging)

Still some debug stuff left in there ... will clean up