Yes, you can, as long as you are using full-fat can coconut products, the popsicles will be creamy. Avoid low-fat coconut milk carton that makes the popsicles super icy.