Avoid giving your kitten cow′s milk, goat′s milk … or any milk a person might drink. The only kinds of milk that are healthy for kittens are their mom′s milk or a kitten milk replacer. Other kinds don′t have the right nutritional balance for cats — and may even make them sick.