Photos are made by light, so you must have light in the areas you want to come out. Usually faces are very important to your photographs. There needs to be light on faces to expose them properly. Try to find an angle where there is direct light falling ON people's faces. There is much less light under wide-brimmed hats than outside them.
People looking down are usually in their own shadows.
Move so that there is direct light falling on faces in your shots, or at least strong reflections from something like a bright wall. If you take pictures early or late in the day when the sun is lower, your chances are better of getting more light directly on faces. Many newer cameras provide fill-in flash for situations like this. Check your camera manual to see how to set your camera to flash on every shot when you are in a back-lit situation. You cannot always solve the problem but at least be aware of it.