Only attempt to add module if -m is not empty