Moved all logic to Update from FixedUpdate.

Whoops, forgot that I had everything in FixedUpdate() Hopefully that
should fix some of the no clipping problems.