> If there is no -of the user would like a binary in the program's directory

Absolutely, this is wrong. If there is no `-of` and no `--build-only`, the user just wants to run the program, and `rdmd` needs to build to the temporary directory.