Before you can decide whether it is safe to eat the brown spots on cauliflower, you should understand what they are. They are the end product of bacterial infections. Luckily, these spots are not harmful, and you can safely eat them. If you see these brown spots on your cauliflower, you can cut them off with a knife. It is important not to consume them if they are already infected, as they could be a sign of a more serious problem.