Creates end to end test of pipeline including upload.

Also moved test data folders into more convenient locations.
ah! that would probably be obvious if I were looking at the code and not the diff