Your coffee is leaking around the rubber seal. With use the seal eventually will become loose and let some coffee leak around it. You wrote that you store the plunger inside the main part. The AeroPress instructions advise against storing the seal inside the chamber because holding the seal compressed inside the chamber eventually will reduce the size of the seal, enabling coffee to leak around it. When your seal becomes loose, you can buy a new seal on this website. The seal can easily be replaced on the end of the plunger.