You would however notice their color if set in white gold or platinum. As one gets further down the alphabet, the tinting gets stronger and the value lower. That is, until you get to the extreme. As the color becomes richer, you have a fancy colored diamond, rather than an off colored one. Then the value starts going up again.
Color grading is done by placing a diamond next to a set of previously graded gems.
The color is compared to the graded gems to see which it comes closest to matching. While this low tech approach is accurate, it is also expensive and time consuming. A compromise is often made on smaller gems, by grading batches within a range, rather than coming up with a specific grade. You will usually find diamonds under a carat graded as GH, or IJ, meaning that they are in that range. This information is meaningful and saves you quite a bit of money.