The pickle should be stored in borosilicate glass, ceramic or ceramic bottles. If you have pickles at home, you also know that storing them in plastic bottles is the worst thing you can do with pickles. They should be stored in borosilicate glass, ceramic or ceramic bottles.