If new pane is created, make it the active pane

This is for #9255 

cc @JeffryBooher 
this is fine. still not sure how we would get into this state though :)