You can, but it won’t be the same. Instead of an iced latte type of drink you will end up with coffee flavored milk.  This is great for recipes (see question above) but may not be quite what you had in mind for drinking.  You can read how to make cold brew in milk here and give it a try. You won’t know if you like it until you do.