Yes, you can put hot water in an infuser if it is made of heat-resistant materials such as plastic or stainless steel. Hot water will release more flavor from the ingredients than cold water, so you may want to experiment with different temperatures depending on your preference.