added code for sampling and index page detection