Add to position instead of setting it

- Seems to work now, yay!