move mapreduce algorithms into index implementation for further optimisations