It could be happening due to any of the following reasons:
Using the wrong hole, which doesn’t accommodate the pencil adequately (if applicable)
Holding or pressing the pencil with too much force
Holding or pressing a stubborn pencil too lightly
The graphite core is set off-centered in the wood
Worn-out internal gear of the sharpener