It is easy to be misled by the public record, the public rhetoric in which these ideas are generally framed. So in The Federalist Papers, which people read, which were of course written for the public kind of like a propaganda document, The Federalist Papers do discuss the rights of minorities and the need to protect their rights. Say, for example, take this pen. It's my property. Well the answer is that the pen doesn't have any rights.
Property has no rights.
Maybe I have a right to property. But the pen itself has no rights. So rights of property is an extremely misleading phrase. What it refers to, and sort of masks, is the rights of persons to own property. Furthermore, it's worth noting that that particular right is not like other rights.