Changed colliders from spheres to capsule colliders. Now working much better :)