Honestly, the longer the better.  Since the fruit will add flavor to the drink, it’s best to let it soak for a minimum of 4 to 24 hours.  Cover the pitcher and place it in the fridge.  If you live where it’s cold, cover it and place it in your garage.