When astronemers talk about colored dwarfs, they are usually referring to how bright the object is. Therefore a white dwarf is very bright, and a brown dwarf is very dark. Even though both of these objects are called dwarfs, the name is misleading since they are very different objects. White dwarfs are really small bright stars, with very high surface temperatures, but low luminosities. Even though they have about the same mass as our own sun, they are roughly 40 times smaller.
Brown dwarfs are really large planets like Jupiter.
Since they aren't stars they don't shine so astronomers refer to them as being brown, since they are very dark. Here are some additional comments provided by Phil Plait. I have a special page on my web site that talks all about star colors. Hot stars are bluer, while cooler stars are red. A red dwarf is a tiny red star, pretty much like the Sun but much cooler and smaller.