Pass `--pattern` from rake task (#1589 fixed up)

This is #1589 rebased and with some fixes applied.