The church was built on the north bank of the Neman River around 1400 for Franciscan friars and foreign merchants. It was ordered and funded by Vytautas the Great as a commendation to the Blessed Virgin Mary for saving his life after a major defeat in the Battle of the Vorskla River.