Plants that have a strong smell irritate the groundhog the most. In fact, groundhogs also hate the smell of herbs like basil, mint, lemon balm, sage, thyme, rosemary, oregano, and chives.