Soswaewon was originally constructed by Yang San-bo (양산보) from the end of 1520 through the middle of 1530. Yang gave up his government position after his teacher, Jo Gwang-jo (1482 - 1519) was exiled and then killed in the purge of 1519 called "Gimyo sahwa" (기묘사화). So Yang made this garden to seclude himself from social position. The part of the garden was burnt down during the Japanese invasions of Korea (1592–1598), but the buildings were reconstructed by his descendants, and the present Sosweawon was renovated by Yang Taek-ji, Yang san-bo's descendant in the fifth generation.