No, you don't. Using coffee ice cubes will keep your drink from getting watered down though compared to regular ice cubes. Which means bigger, more authentic flavor for longer.