Break redirect/display hook logic out of bp_core_new_subnav_item()

By moving some of the critical logic into bp_core_maybe_hook_new_subnav_screen_function(),
it becomes possible to write unit tests for it.

See #5720

git-svn-id: https://buddypress.svn.wordpress.org/trunk@8544 cdf35c40-ae34-48e0-9cc9-0c9da1808c22