Remove AllowUsers

Too specific. Thanks to @azet for the feedback!