One of them is that people may chose to deal with women in a biased and offensive manner. Such discrimination is simply ignorant behavior, which is shows incredibly poor taste. But both freedom of speech and freedom of association guarantee that people have the right to be wrong. To be offensive. To be prejudiced.
Freedom of association requires the right to say 'no' and to refuse to associate.
It is an inescapable part of forming preferences and tastes. Everyone reaches their own conclusions about other people. You friends into your home and bar those who are unpleasant. In the same manner, you have the right to hire whomever you consider appropriate. The decision may be biased.