Organize pingdom code into seperate subdirectory

This will break existing imports.