Don't join against the group members table when fetching total group counts.

The join doesn't take place in the 'paged' count, which leads to inconsistent
query results when groups have no members.

This reverts changes made in [4064].

Props Mamaduka.
Fixes #5584.

git-svn-id: https://buddypress.svn.wordpress.org/trunk@9230 cdf35c40-ae34-48e0-9cc9-0c9da1808c22