On windows, you can press the Windows key and A to switch between Borderless window mode and Full screen.