I would rate this 7 stars just for the amazing view and relative calm! We went in early July when Oia is overflowing with tourists. Yet there was very little noise from the crowds up above. We found our stay peaceful and relaxing. So much so that the night we arrived we slept 10 to 12 hours straight!
Hospitality is quite good with a porter helping with the bags while checking in and out.
If you have bad knees or difficulty climbing, then this unfortunately is not the place for you. There is fair amount of steps to climb to reach the main streets of Oia. However even that climb is beautiful thanks to the views of the caldera and the blue-white buildings all around.