Completing each Papyrus Puzzle gifts you 500 XP, 300 coins and most importantly, rare or legendary weapons or armour which will be at a level similar to your character level.