Spend the night there. Spend your last Kip in one of the noodlesoup shops, then head to the Lao Immigration and walk 500 meters to the Vietnamese border. Traffic on either side of the border is still very erratic, so start early in case you have to wait somewhere on the way. Or you can take an international bus from Vientiane to Vinh.
Vietnamese visas are available at the embassy in Vientiane.
The official bank rate is about 4300 Kip to the Dollar. You get 5500 Kip from unlicensed moneychangers. They are everywhere on Thanon Phothisalat in Luang Prabang and on Talaat Sao and around the GPO in Vientiane. Spend the night there. Spend your last Kip in one of the noodlesoup shops, then head to the Lao Immigration and walk 500 meters to the Vietnamese border.