Debugging traces

Couldn't find anywhere to ask this question so forgive me for opening up an issue.
Is there a way in Requests to get a trace of the over the wire http for a request response.
In httplib2 your could do it by setting httplib2.debuglevel

I hope requests has something better. such as a trace attribute on the response object.


I'm not certain there is but the tests print debugging information occasionally. The clue may lie in there. I know urllib3 does do logging but I'm not sure how we control that honestly. 


urllib3 logs to its `__name__` logger namespace (that's usually "urllib3", or "requests.packages.urllib3" for requests). There is a handy helper for getting it to print: `urllib3.add_stderr_logger()`

Or for requests, you'd do: `requests.packages.urllib3.add_stderr_logger()`


Thanks thats helpful to know. I think it would be a good idea, in keeping with the Requests motto of Human Friendly Http, to add an optional parameter that automatically setup the capture of the http over the wire. I have found that in a production environment its a really useful capability when you are interfacing to external services that have "bugs". Usually the easiest way to get them to acknowledge the bug is to provide a trace of the http packets involved. One approach I have used with httplib2 is to redirect the debug stderr to a string variable and then expose it as an attribute that also gets attached to any exceptions. That way whatever facility it trapping exceptions can extract the trace. 

I guess if I start using Requests, that it is something I would look into doing.


@SmithSamuelM you should probably wait until the great refactor (#970) is over before you look into adding it, but at that, I think the logging you're looking for belongs in (if it doesn't already exist in) urllib3.


Good, that will make it easy. The idea is to do something like this

response=requests.get(...,  trace=True)
response.trace
     Would have the over the wire packet flow


Not sure @kennethreitz will approve of the extra parameter. At best you might be able to add that as a configuration parameter. Even so, as I already said, the majority of the trace work only makes sense if it is performed in urllib3. If it doesn't exist there, you'll be hard pressed to see it introduced in requests.


Ya well, I guess I would have to make the case. A config parameter is fine. The idea is to make it easy to turn on and off from the requests api not have to import urlib3 and then do extra configuration just to get a trace. 
I mean if configurating stuff at the urllib3 layer was the preferred path then why use requests at all. Just configure urllib3 and do it the hard way?


Of course inside the api the configuration would be performed at the urllib3 layer but the complexity is not exposed to the requests user


Thanks for your helpful comments


Tried the 

``` python
requests.packages.urllib3.add_stderr_logger()
```

All it logged was this

2012-12-20 09:30:12,493 INFO Starting new HTTPS connection (1): example.com
2012-12-20 09:30:12,806 DEBUG "POST /soap/default HTTP/1.1" 500 572

Not even close to the level of detail that httplib2 logs


I'm sure @shazow would be delighted to have extra logging in urllib3. =D


This feature should be implemented in urllib3 not requests. Discussion is in shazow/urllib3/issues/107


Looking at httplib2 a bit, this is most definitely something for urllib3 and not us. I'm closing it, but someone else can feel free to re-open later.