However, I work in New York City and come into contact with a number of people of varying ethnicity every day. One man here at work has a very thick Hispanic accent, which I find very difficult to understand. However, most of the other people here understand him perfectly. On the other hand, bring in someone with a thick Irish or British accent, and I can understand them clearly, while others cannot catch even a word. From college, I have come to understand Asian accents better, but even Asians who speak English very well can sometimes confuse me.
Some people can deal with different accents better than others.
People with hearing problems often have more difficulty than others, for example. Perhaps that is the kind of situation you are running into. They are about rhythm and timing. People will stand in rows, or move forming different shapes and everyone stomps their feet, claps and smacks their thighs, or spins around, doing dance moves, creating a harmony of sound with their own bodies. It's pretty cool to watch, too.