fix check in memory limit in drr's GUI (and fix
error message in custodian-limit-memory)

closes PR 13778