The burning sensation after handling chillies comes from capsaicin. To get rid of the chili oil, rub some vegetable or olive oil on your hands. Then wash your hands with mild dish soap and you are all set.