Update openspecfun to master and install it properly

The new openspecfun no longer uses headers from openlibm
sources, but relies on them being installed to $(includedir)
instead (cf. 682fb0a). Also make the install commands more
standard and similar to openlibm's.
Replace .dylib with $(SHLIB_EXT)

For consistency within the file.
Merge pull request #6238 from nalimilan/openspecfun

Update openspecfun and install it properly