Not really! In this situation, you have to ask yourself if you really would want them to be there on your special day. If the person has some kind of bias or prejudice that makes them an inappropriate guest, feel free to not invite them!