Yes, a dirty speaker can get spoilt easily. A dirty, sticky speaker will have bacteria and dust all over it. So, if you’re not cleaning your speaker every time you use it then there is a good chance that it will be spoilt before too long.