convert to new Hadoop API; allow non-s3 inputs