It is not impolite to invite yourself to someone's home, depending on who you are inviting and why you are doing so. You must carefully gauge this and time the request with deft diplomacy.
Before you ask, think about your personal or business privacy concerns.