No, SCOTUS has affirmed "marriage" in the general sense as being a fundamental right 14 times. There's not a lot of qualification in the recognition of the right.

e.g. Maynard v. Hill (1888)

"Marriage is something more than a mere contract, though founded upon the agreement of the parties. When once formed, a relation is created between the parties which they cannot change, and the rights and obligations of which depend not upon their agreement, but upon the law, statutory or common. It is an institution of society, regulated and controlled by public authority. Legislation, therefore, affecting this institution and annulling the relation between the parties is not within the prohibition of the Constitution of the United States against the impairment of contracts by state legislation."

Meyer v. Nebraska (1923)

"While this Court has not attempted to define with exactness the liberty thus guaranteed, the term has received much consideration and some of the included things have been definitely stated. Without doubt, it denotes not merely freedom from bodily restraint, but also the right of the individual to contract, to engage in any of the common occupations of life, to acquire useful knowledge, to marry, establish a home and bring up children, to worship God according to the dictates of his own conscience, and generally to enjoy those privileges long recognized at common law as essential to the orderly pursuit of happiness by free men."

Skinner v. Oklahoma (1942)

"But the instant legislation runs afoul of the equal protection clause, though we give Oklahoma that large deference which the rule of the foregoing cases requires. We are dealing here with legislation which involves one of the basic civil rights of man. Marriage and procreation are fundamental to the very existence and survival of the race. The power to sterilize, if exercised, may have subtle, far-reaching and devastating effects. In evil or reckless hands, it can cause races or types which are inimical to the dominant group to wither and disappear. There is no redemption for the individual whom the law touches. Any experiment which the State conducts is to his irreparable injury. He is forever deprived of a basic liberty."

Griswold v. Connecticut (1965)

"We deal with a right of privacy older than the Bill of Rights -- older than our political parties, older than our school system. Marriage is a coming together for better or for worse, hopefully enduring, and intimate to the degree of being sacred. It is an association that promotes a way of life, not causes; a harmony in living, not political faiths; a bilateral loyalty, not commercial or social projects. Yet it is an association for as noble a purpose as any involved in our prior decisions."

etc.

So it's actually a modern assertion that gay-marriage is not allowed by law as "marriage" in the general sense has been recognized and reinforced numerous times, without specificity as to the nature of the parties involved. It's the modern dawning realization by parties wishing to marry and being denied, and by parties wishing to deny them, that the scope of who can marry is largely unconstrained.

Persons against gay-marriage are actually in a battle to contract the existing right, while persons for gay-marriage are looking for explicit confirmation that they are included in the right. Given the general trend of rights expansion over American history, it's unlikely that an argument to contract an existing right will win without a general rewrite of several Constitutional Amendments (privacy, due process, etc.) explicitly naming the minority class that they wish to specifically oppress.