added more field validation, now generating id's per item, separated pipelines