Yes, you can! Keep in mind that liquid expands when frozen, so leave some space in the container to account for that.