I had a lovely stay in Dunedin at the Stables Heritage Accommodation. It was close to town (although being Dunedin the return walk is uphill!), clean and tidy, and comfortable.