Vegans used to be extremely rare, but today almost everyone knows at least one vegan person. It is perfectly consistent, so long as the central purpose behind the choice is to increase one's devotion to Krishna. Scripturally, this is not accurate. Devotion is not dependent on any material cause or circumstance. Nothing is needed to be Krishna conscious except love for Krishna.
Milk is said to contain subtle mental nutrients.
However, there is a gulf of difference between modern milk and ideal milk. They are milked for 10 months of the year, and for seven of those months, they are also pregnant. The cow is then put back into intensive milk production. The largest outbreak of salmonella poisoning ever recorded in the United States came from pasteurized milk.