There are alleles for black, brown, agouti, gray, albino, and others. The twist here is that the same allele can be dominant or recessive depending on context. This means that agouti is dominant to black, and black is dominant to albino. If the black allele is in the presence of an agouti allele, the mouse will be agouti because black is recessive to agouti. If that same black allele is paired with an albino allele, the mouse will be black since black is dominant to albino.
Polygenic traits are the result of the interaction of several genes.
Each of the contributing genes can also have multiple alleles. Alleles are different versions or forms of ONE gene. Each person can have only two versions of the trait in their genome at one time, but in the population there are 3 forms. In fruit flies, there are many different versions of eye color in the population, but one fly can have only 2 alleles at a time. Polygenic traits are those where more than one GENE contributes to the phenotype.