We had a wonderful stay in The Captain's House. The house is tastefully decorated, very comfortable and has great outdoor spaces.
The bedrooms are slightly small (compared to the size of the house) but the beds are comfy.
The location is very good, right in the heart of Oia. You should not expect the famous caldera view (as this is 50 meters further down the road), but we knew this.
We were a bit unlucky that on our first night a neighboring restaurant hosted a party with loud music until 4am. But this was a one-off event (Panos checked for us), and the we slept very well the following nights.
Panos' restaurant recommendations on the island were very good. Do go for lunch to To Psaraki (on the far south of the island), and 1800 just down the road from the house is great for dinner.