Show all invited users and infinite scroll active

Closes #4239
Do we need this check in two places?