To clean a pencil holder, empty out the contents and wipe it with a damp cloth or use a mild soap solution if needed. Make sure to dry it thoroughly before placing your pens and pencils back in.