I believe that the semantic meaning of <address> is the contact information of the nearest <body> or <article> tag.  In other words, it's the contact of the page's author, not just any contact.  If it isn't the appropriate tag semantically, I would prefer not using it.