[SPARK-4902][CORE] gap-sampling performance optimization

jira: [SPARK-4902](https://issues.apache.org/jira/browse/SPARK-4902)
cc @mengxr