Most music is written in 4's. Human brains seem to like the repetition of rhythms in groups of 4. They can change anytime at all but to be able to be enjoyed by the masses, a certain regularity and even predictability must be included. Most people prefer to listen to music they can tap along to. Most of us seem to have a deep-rooted understanding of rhythm, something that goes way back.
Strange time signatures thwart that foot tapping.
Symetry seems to be the prerequisite for that to happen naturally. This is the way I count it when I have to. Often it's done to follow a lyric or grab the attention. This gets trickier to figure out and count, but once you hear it, it becomes straight forward. The time aspect of music is also the part that is more difficult to teach.