I'll look like a geek. You'll look like a real bicylists, and car drivers will probably respect you more and give you more room on the road because of it. They're too hot. Because they're so light and have lots of openings for air to pass through, helmets aren't any hotter than having your head exposed to the sun while riding. I don't need one if I don't ride on busy roads.
Falls are a much bigger problem, and they can happen on any road or path.
I never fall off my bike. Maybe not yet, but everybody falls sometime, and it only takes once to do permanent danage. They're funny looking. Helmets today come in lots of cool colors and designs. Find one you like so you'll wear it.