Remove Reference layers and update map center