Absolutely, especially if your gutters are rusty or the water has been collecting in the gutter trough for a while. Bacteria can grow in these cases and can transfer to your hands if you clean your gutters.
No matter how or when you decide to provide maintenance to your gutters, make sure to wear gloves while you do so, or wash your hands afterward.