Don't return extra fields for joins resolved through other managers.

This *might* be a fix for #84. I had a query that required me to do `taggedmodel__tags__name__in` in my filters, which would trigger this `extra_filters()` bug in #84. My fix simply says, "Is the first piece of this filter me?" If the answer is no, then `extra_filters()` is entirely skipped.

I'm not well versed in the ORM internals so I can't be 100% sure this is a firm fix. What I can say is that it's likely a safer fix than entirely removing `extra_filters()`. I also don't have a test for this; sorry about that!