You can have a milk allergy or lactose intolerance and still consume casein protein safely. Look for casein protein powders that are isolated and do not contain milk sugar or other dairy components. As long as you’re not allergic to casein itself, isolated casein protein should not cause an allergic reaction.