Remove follow_mouse option && guru-mode dont focus windows with mouse