If the hand has no fingertips, use the palm center as the pointer.
Track the correct arm more accurately.
Increase the delay between sending arm images slightly.