You can walk to Ruwanwelisaya while staying at Rajarata Lodge - you’ll be about 2 mi away.