Location Value Error when trying to post through a session

I am using a 3rd party library that uses requests, and when I try to post using a session I get this error:

File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/pacer_lib/scraper.py", line 42, in init
self.refresh_login()
File "/usr/local/lib/python2.7/dist-packages/pacer_lib/scraper.py", line 74, in refresh_login
response = self.br.post(login_url, data=payload)
File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 500, in post
return self.request('POST', url, data=data, json=json, _kwargs)
File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 457, in request
resp = self.send(prep, *send_kwargs)
File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 595, in send
history = [resp for resp in gen] if allow_redirects else []
File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 189, in resolve_redirects
allow_redirects=False,
File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 569, in send
r = adapter.send(request, *_kwargs)
File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 329, in send
conn = self.get_connection(request.url, proxies)
File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 243, in get_connection
conn = self.poolmanager.connection_from_url(url)
File "/usr/lib/python2.7/dist-packages/urllib3/poolmanager.py", line 131, in connection_from_url
return self.connection_from_host(u.host, port=u.port, scheme=u.scheme)
File "/usr/lib/python2.7/dist-packages/urllib3/poolmanager.py", line 102, in connection_from_host
raise LocationValueError("No host specified.")
urllib3.exceptions.LocationValueError: No host specified.

here is the relevant source from pacer_lib

def refresh_login(self):
        """
        Logs in to the PACER system using the login and password provided at
        the initialization of `search_agent()`. This will create a Requests
        session that will allow you to query the PACER system. If 
        _auto_login_ =False, `refresh_login()` must be called before you can
        query the case_locator. This function will raise an error if you 
        supply an invalid login or password.

```
    Returns nothing.
    """
    #SETTINGS (determined from the form from PACER's '/login.pl')
    payload = {'loginid':self.username, 'passwd':self.password}
    login_url = 'https://pacer.login.uscourts.gov/cgi-bin/check-pacer-passwd.pl'
    self.br = requests.Session()
    response = self.br.post(login_url, data=payload)
```

I have tried just posting without a session and it works (at least I get a 200), but every time I try to use a session I get that same error. Any help would be greatly appreciated


In this instance the problem seems to be related to a redirect occurring. Are you sure that nothing is being set on the session here, anywhere in the code?


It doesn't look like it is, as far as I can tell. This method that is creating the session is invoked in the constructor of a class, and the session 'br' is being initialized as an empty string. Is it possible that the library contains a bad link that doesn't work the way it was intended anymore?


I sincerely doubt it. Requests uses a `Session` internally when you use a `requests.*` method, and that includes following redirects. There is essentially no difference between `requests.post(x)` and `s = requests.Session(); s.post(x)`.

Because I don't have a PACER login I can't validate the problem, of course, but one option would be to set `allow_redirects=False` and then manually check the Location header in the response to see what it says.


So the location I get is "http:///cgi-bin/links.pl", which I tried to use directly in a post and got that it is an invalid URL (obvious, but still figured I'd try). It looks like the pacer.login.uscourts.gov part is being taken out


Hrm...yeah, that URL is invalid. Is that the `Location` header value directly?


Yeah. Here's the code I'm running
'''
s = requests.Session()
print s.headers
r = s.post('https://pacer.login.uscourts.gov/cgi-bin/check-pacer-passwd.pl', data = payload)
print r.headers
'''
and the print headers prints this:
{'Content-Length': '208', 'X-Content-Type-Options': 'nosniff', 'Set-Cookie': 'PacerSession="janfFdEMIJ14lGFtPtR0S5itQ0LhmKdvtcpM3zde72I2wDcyVyNktJvvSZU92AK7OOUFKkpLzHLKoQIA9fvzaH0UY3m3VFtTIvBJoIM26XaMV6SJSFo4JFRpJEZQ7TIs"; path=/; domain=.uscourts.gov; secure, PacerClientCode=""; path=/; domain=.uscourts.gov; secure, PacerPref="receipt=Y"; path=/; domain=.uscourts.gov; secure', 'Keep-Alive': 'timeout=15, max=100', 'Server': 'Apache', 'Connection': 'Keep-Alive', 'Location': 'http:///cgi-bin/links.pl', 'Date': 'Wed, 01 Jun 2016 00:15:43 GMT', 'X-Frame-Options': 'SAMEORIGIN', 'Content-Type': 'text/html; charset=iso-8859-1'}


Wow. Ok, so that means that PACER is producing the invalid header, presumably by having something in their server stack screw up the location header they _wanted_ to produce (which is probably `/cgi-bin/links.pl`). I don't know that we can really do much about this: the URL is severely under-specified.


Fair enough. Could I try passing the location header myself, or is that not how it works? I don't really understand http (which is why I appreciate so much what you guys do). Is this the kind of thing I could raise with pacer's IT team?


Sadly you can't pass the header yourself, it's returned from the server. I think it's exactly the kind of thing you could raise with pacer's IT team. =)

What you can do is set `allow_redirects=False` and then fixup the URL yourself, manually following the redirects until they terminate. That might not be a good idea if you're not confident with HTTP though.


Awesome, thank you so much for the help. One final question before I get out of your hair, if I were able to login via another url using a Session, I should still be able to access the various pages within pacer that I want to, correct?


Absolutely!

You may also find that you can use pacer immediately after the exception: the login still worked and the cookies should still be stored.