If your goat cheese has gone bad, you can easily identify it. The goat cheese will have a sour smell and a bitter taste. It is best not to consume it at this point.