Fix load user list too slow

tower: https://tower.im/projects/8f19f0bf0e754f0b82ef2c24bc230973/todos/be53cf36a1784470b78a1c1c78c265c7/

Change-Id: Ied99b9779582ad8c79a651d0c97c950bf7569fff