Only remove set fragment to null in fragments if position allows for it