There are a few things that goats dislike the smell of. Some of these include lavender, sage, hydrangea, cayenne pepper, and even animal dung and peppermint oil. To keep your goats away from these items, you can try to cover them up or spray them with repellent.