A river pilot, who had traveled to Asia, built this house influenced by Japanese architecture