Preferably, don't feed your own pets outside. If you must feed pets outside, remove any uneaten food quickly. Remember gardens attract wildlife such as raccoons. Consider ways to make your garden less appealing such as low voltage electric fence. Check with your county extension or experienced gardeners for other suggestions.
Many animals become very aggressive if they believe their young are in danger.
When you're sure that there are no young or that the young ones are gone, watch the entrance at dusk and block it up after the animals leave for the night. But remember, some animals have more than one entrance. Another good idea is to mount a flap of wood or heavy gauge wire on a hinge over the hole so that the animal can push it out to leave, but can't push it back in to re-enter. If the animal refuses to leave, call your local animal control officer. They can set live traps for the animals.