testing formula for finding rotation matrix