Try adjusting the fit of your headset. Adjust its position on your face by moving it up and down or left and right, and adjust the straps so it feels snug.
If your headset supports it, you can also adjust its calibration settings. If the headset has a knob to adjust calibration, use that. If it doesn’t, go to Settings > Mixed reality > Visual quality and adjust the calibration there. For more information on calibration for your specific device, check with your headset manufacturer.