Prevent null pointer exceptions when onConfigurationChanged is called on a destroyed PullToRefreshAttacher