The cross section makes a gentle V centered in the rotten area. Naturally you have to remove only a chips of the sound wood at the both ends and more in the middle area. The same method goes with your mast if it has been made out of solid lumber but the work is slow. We have used West and SP -epoxies with good results. Regular varnishing of the masts is the best preventive measure to avoid the damages inside the structure.
Wooden boats have aluminium masts.
How much you can spend for a used mast is hard to tell. It depends on the size and condition on the mast.