The kick of the gun is the opposite reaction as a result of propelling the bullet out of the gun. The reason the gun doesn't rocket through your arm is due to conservation of momentum.
P stands for momentum, m for mass and v for velocity.
The gun has a higher mass than the bullet, and since these two values have to remain equal, the velocity of the gun in the opposite direction in relation to the bullet has to be significantly less than that of the high velocity bullet. In rocketry, the high velocity gasses escaping from the rocket nozzle act in the same way a bullet does, propelling the rocket in the opposite direction.