Spoiled caramel may have an off smell, sour taste, or a change in texture. It might not be as smooth and creamy as it should be, and could even have black or brownish flecks in it. If you notice any of these signs, it's best to discard the caramel.