Avoid urljoin by catching the ultra common # ref

PR for #158.  Cut about half a second off the test suite for me, but has pretty substantial savings in my application (~30sec/150k rows).