Mice do, in fact, have backbones. It's a common myth that they don't. What they also have are sloping, flexible collarbones, which makes it incredibly easy for them to sneak into holes the size of their heads. They also have very flexible ribs, which makes wriggling into tight places a cinch.