Only delete bower components if the dir exists