Parallelize sub-makes in ext/ to improve clean build speed

With this patch, make -j8 improved from 45 seconds to 30 seconds on my
Core i7 four physical core machine.