Lowering or raising the voltage affects the brightness, but only because it also affects the current: a greater voltage indicates a higher current in an LED.