More modes typically mean lower bandwidth. The reason is dispersion. As a pulse of light travels through the fiber, it spreads out in time. While there are several reasons for such dispersion, two are of principal concern. The first is modal dispersion, which is caused by different path lengths followed by light rays as they bounce down the fiber.
Some rays follow a more direct route than others.
By limiting the number of wavelengths of light, you limit the material dispersion. Dispersion limits the bandwidth of the fiber. At high data rates, dispersion will allow pulses to overlap so that the receiver can no longer distinguish where one pulse begins and another ends.