Basically, you can backpack anywhere in Sri Lanka. There is no "no-go zone". It is a safe country with many great hostels and a fun backpacker scene. It is easy to meet fellow travelers, too.