They seem to leak charge a little easier. On a Leyden jar they sometimes produce a very weak spark if the ball is too big in proportion to the jar. Another way to produce a conductive sphere out of a plastic ball is to cover it with foil tape. Not all plastics are created equal. The dielectric constant is a reference for how good an insulator something is.
Different plastics have different dielectric constants.
Breakdown voltage is the voltage required for electricity to puncture a given thickness of a given material. I recently made a Leyden jar out of a plastic soda bottle and was disappointed to find that its capacitance was much lower than expected. When it comes to building up a charge on plastic, these two references help explain why things behave the way they do. Take a look at a variety of plastic containers and try to find the letters that identify what kind of plastic they are made from. You may see PTFE, PVC, HDPE, PP, PETE, or something else.