move all integration test to example module, environment are configured such that integration test and execution without --fullrun parameter will use preview mode: join and slice are limited to the first few pages
fix a bug in join related to https://issues.scala-lang.org/browse/SI-8985
change example's url
all integration test use less var
fix a bug still caused by https://issues.scala-lang.org/browse/SI-8985#add-comment
all actions now use Expr[String] for interpolatable parameters
flatten & flattenTemp add indexKey into KeySet
fix a bug where null value will cause MapFunction.flattenKey to fail
Various Integration test changes
add parameter pageIndexKey to join
add option to use distinct and join in fetch instead of groupByKey
cleanup
flattenPages now add indexKey into KeySet
add VWR example
accumulable renamed to metrics
add a new metric on total number of pages
PageSchemaRDD.fetch add a new parameter: exclude to handle ergodic pagination/crawling
withPage in fetch is persisted before used twice
fix a typo where pagecount accumulator still use the old name
revert fetch as it turns out to be slower