The armory is located below the first floor of Vault 34 at the end of a hallway. You can only access the armory after retrieving two passwords from two Vault technicians Ghouls found in the underwater areas of the Vault. Those passwords are used to first drain a flooded area and then gain access to the Overseer's rooms. The Overseer still dwells there as a Feral Ghoul and killing them will allow you to finally grab the password that unlocks the armory.