Only move selector when over rose or outside the UI elements.