The U.N. refugee agency says the death toll has risen to 91 from two boats that capsized off the coast of Yemen while crossing from the Horn of Africa .