There is nothing wrong with white flour, there just isn't anything right with it. Whole wheat by itself is NOT a complete food. However, by mixing the whole grains together a synergistic effect is created, and a nearly complete food.
Some people are allergic to one or more of the grains.
Leave them out, there are plenty of others. Try using Grain Mix Flour in recipes you already use. Many times the difference in taste and texture is small, but the difference in health can be very big. It is fun to eat healthy. Eating doesn't have to be the same old thing.