Close the prepared stmt if we don't use the LRU