The first thing to do in cleaning your coffee maker is to unplug it, this will prevent any electric or water hazard. Then pour some warm soapy water into the accessible parts of the machine that has the milk residue, ensure you do this with care, so you do not damage your machine. You can then add some distilled vinegar to all the affected parts of the coffee maker.