Remove x11 dependency if use_x11 is not set