Solution: Polymorphic Votes. Added in user and votes association.