The researchers contend that shortly after exercise, rather than before night, is the ideal time to consume milk to encourage muscle building and weight loss. You should, however, modify your intake accordingly.