Use the server modes values when building the mode_colors array