Don't take it personally. Smile and laugh. Also, remember that if you do come out, think about how you do it and of the impression that you will make on the marijuana reform movement. This isn't just about you. If it were, you wouldn't have a chance.
Gays have the same problems with some of our more colorful brethren.
However, the gay rights movement was not started by people who look like me. It was started by drag queens rioting when they got fed up with police harassment. People have the right to be themselves. The marijuana reform movement can learn a lot from the gay rights movement. On the other hand, if you are not a marijuana user and you are just coming out as an anti-prohibitionist, then say that you are not a user.