We might be empty nesters sooner than we thought.