You' re dealing with caustic chemicals here. If you don't have rubber gloves but insist on going ahead with the cleanup, at least give your hands a protective coating of petroleum jelly or a non-disappearing lotion. Keep in mind that some cleaning chemicals can scratch or dull jewelry as well.
Preventive maintenance is the best strategy.
If you can spend a couple of minutes each week doing a quick wipe-off with gentler cleansers, you'll save yourself a harder scrubbing later on. This way you'll be able to pick up any drips as you go along. Fill that bucket with good clean water, and use the sponge to wet down the walls after cleaning them. It's tempting to use the shower head to rinse the walls, but unless it's one of those detachable types you'll probably just succeed in getting yourself wet. Make a pass at the mildew, but don't be surprised if it doesn't come right off.