Several factors are at work here.
If the needle is too thick for the fabric, it produces a larger hole, causing the fabric to draw up at the stitching line. If the needle is too thin for the fabric, the needle has difficulty piercing the fabric, resulting in frequent thread breakage. To confuse matters even further, needles are shaped differently to accommodate different fabric properties. Letters which follow the needle number indicate the needle type. And no doubt by the time you read this, another category may be available.