He cannot bar you from the apartment, but may be able to bar you from the grounds (e.g. if this is a building with substantial common area, rather than, say, a 2- or 3-family home where you basically go from the street to the apartment). The landord has no control over who the renter(s) have in their home; in renting, they acquire the right to possession of the unit, which includes the right to determine guests and visitors (subject, though, to any limitations in a lease, such as on the maximum # of guest overnight stays per year). However, the landord retains control over common areas, and can bar individuals from them as long as the landlord is not discriminating (e.g. against a racial group) in doing so. In some situations, unfortunately, this may mean that you effectively cannot visit your kids or wife if that means transversing common areas under the control of the landlord.