Original: Momentum is a conserved quantity (law of conservation of linear momentum), meaning that if a closed system is not affected by external forces, its total momentum can not change.
Simplified: Using vector addition to add the momentum of the bullet to the momentum of the gun (equal in size but opposite in direction) gives a total system momentum of zero. The momentum of the gun-bullet system has been conserved.