According to Merriam-Webster, ˈdau̇-i-jər   

  
 == Plural? == 

 Queen Dowagers looks wrong.  Should it not be Queens Dowager, similar to courts martial?  The argument would be that they are queens first and dowager is a modifier so the plural should go with queen.  Dowager Queens is used as well, so there is precedent to put the plural on queen.  I looked for a definitive reference and found none, so it may be too obscure a point to worry about.