It was built between 1801 and 1809 to designs by Chrystian Piotr Aigner on the foundations of a Baroque garden pavilion which had been destroyed by Russian troops in 1794. Its masonry was partly new but partly also reused pieces of stone from historic buildings and sites in Italy, Spain and Poland collected by Izabela Czartoryska for her lapidarium. It was designed to expand the display space in the nearby Temple of the Sibyl, displaying items on the exterior as well as the interior.