Yes, you can. But it is not recommended because, the herbs and spices will loose the unique flavor that is infused only in cold water. Also, hot water can infuse a very strong flavor that may not appeal to most palettes.