You can feel bad after consuming horseradish if mold is present in the product. In other cases, rotten horseradish will taste bad but should not cause serious digestive problems.