A drinking tube is the most practical way to provide water. They are easy to clean and less messy than water provided in a bowl. Since mice like to dig, water in a bowl often becomes full of bedding. A pelleted diet formulated especially for mice can be purchased at most pet stores. In addition, supplements of raw vegetables such as carrots are suggested.
Mice also enjoy dandelion blossoms and grass.
Be sure that any fresh food from outdoors has been washed thoroughly and is free of pesticides. The same article says whole-grain breads that have been moistened with water are good for mice. Rodents have teeth that continuously grow. They require something to gnaw on to prevent the teeth from becoming too long. There are chew sticks or blocks for rodents available in most pet stores.