Make recent edits by admins configurable

Backported from 6c660660