Puppies should be off milk replacer by five weeks of age. They should start consuming semi-solid or solid food. But, you can still provide milk replacer as an additional supplement now and then.