copy everything to s3 and kafka

not quite ready to merge, needs testing and verification.