Yellow mustard seeds are the mildest, while brown and black seeds have a more pungent flavor. They also differ in color and size, with yellow seeds being the smallest and lightest in color.