Yes, if your kitchen is not too cold during winter and if you will be using it up within a few days. If not, store the herb-infused olive oil in the fridge to make sure no harmful bacteria can grow on them which can cause food poisoning or other infections.