Fix race condition / issues are pried-open windows

cc: @jwngr