temporary delete provided in pom to circumvent a bug in IntelliJ
temporary extend SparkRunnable for the same bug
add files to accommodate acceptance test

add new parameter distinct to many functions of PageRDD to avoid crawling duplicate link