A river pilot, who had traveled to Asia, built these houses influenced by Japanese architecture