Pluggable Request objects?

It would be super cool if it were possible to plug a custom Request object over a Session object. However it looks as though the Request object is hardcoded into the request method, e.g. https://github.com/kennethreitz/requests/blob/1a87f15e6f8d22be6855424b46b80f66d178fe40/requests/sessions.py#L264

Any thoughts on this?


Like changing the Session class's `__init__(self)` to `__init__(self, req=None)` then in the request method only create a new Request() if self.req is None?


What would your custom request object do?


@alanhamlett I think he means customizing the the Request class itself and using that instead of the one that comes with requests.


@sigmavirus24 exactly what I was saying.

The primary issue I see today is that if I want to either manipulate a Request instance or subclass the Request object and do my own thing I lose the functionality of the `Session.request` method. It would seem pretty terrible to have to reimplement that method in an ad hoc way just for things like timeouts on the request.

The broader point I'm driving at is `Session.request` explicitly instantiates `Request` which means a caller can't control the behavior of that object. It would really be a shame to have to implement a custom `Session.request` with what would effectively be almost an entire repetition of the request method's logic save for one line's difference.

Unless I'm missing something, this seems to be encouraging some non-idiomatic solutions so perhaps it would be nice to allow a caller to tell Session which Request object to use?


Apologies, I meant "allow_redirects" where I said "timeout" in my previous comment. Needed some :coffee:!


All right let me follow up here and say that I think I was misunderstanding the design intent of Request, PreparedRequest, Session, and the adaptor design:

I think the blog post announcement for v1.0.0 is fairly confusing given that it leads with an example of the Request object being dispatched via send with a note stating that such an object is fully mutable. This led me to believe that the "normal" codepath for a standard requests' call looked exactly like this. However in reality there is a bunch of logic happening in the request method. If you call `Session.send` directly, you lose that functionality. That's a big ouch if your intent is to preserve the standard `requests.request` API.

Later on in that post custom adaptors are mentioned, e.g. `HTTPAdapter`. After going about it in the opposite way, e.g. creating my own `Request` object and doing some fancy manipulation over this object with the expectation of dispatching the `PreparedRequest` it would generate via `Session.send` I found that, oops, I lost the `Session.request` method in the process, e.g. things like `allow_redirects` are explicitly handled in the scope of `Session.reqiest`. So it seems the actual design intent is that I would instead create an adaptor and then overload the `send` method wherein I could do manipulation over the `PreparedRequest` object.

So correct me if I'm wrong, but the fact of mutability over the `Request` object is probably best exploited by creating a custom `HTTPAdapter` if my intent is to preserve cool methods such as `Session.request`? However showing me a simple example as is done in the blog post caused me to assume I could use it directly by subclassing or some such. Maybe this bears a little extra explanation so as to avoid confusion?


Basically, the manual creation of Request objects is a new feature that wasn't present before. When you use a normal request method, as before, like `requests.get`, all it is going is:
- Creating a Request object with the parameters you passed in.
- Creating a new session
- Preparing that Request and sending it through the session

Now, if you are unhappy with some aspect of the generated PreparedRequest, you can change it to behave _exactly_ as you like. 


@kennethreitz yes but there's a caveat to that, unless I'm misunderstanding: if you want the wrapper around `send`, i.e. `request` and its helpers, then you have to write your own `Session.send` in order to manipulate the `Request`/`PreparedRequest`, yes?


No, you simply send the modified request through the normal session.send

```
s = requests.Session()
r = requests.Request('POST', 'http://httpbin.org/post', data={'bad': 'data'}).prepare()
r.body = 'good-data'

>>> s.send(r)
<Response [200]>
>>> _.json()['body']
u'good-data'

```


@kennethreitz where do you have access to `Session.request` then? The request has to be sent via `s.send` which eschews all the cool logic happening in `s.request`, right? I can't bootstrap the `Session` with my `Request` instance so I don't see how I can ever use `s.request`.


@maxcountryman, that's the cool part. All s.request does is create a Request object, which you already did :)


All of the logic is now contained in `Request` and `session.send`


Everything else is sugar


@kennethreitz that "sugar" is really handy to have around and it's also expected by anyone who uses the requests standard API, e.g. via `requests.request`. So for instance, say you want to pass `allow_redirects`, unfortunately if I have to invoke `s.send` directly this means I need to write my own wrapper around `s.send` which would basically only change one line of code in `s.request`, i.e. the `req = Request()` line. Am I totally off the mark here?


I don't think I follow at all. Did you see the code example above?


@kennethreitz how can you pass `allow_redirects` to `s.send`? Unless I'm totally missing something, the answer is you can't, because `s.request` does more than just construct a `Request` object. It does neat things like extrapolate on that object to follow redirects, it handles cookies, etc. If I want all that, i.e. if I want to write a custom interface around a Request object that allows the caller to pass in the exact same params they could to `requests.request` then I have to reimplement `s.request` because there's just one line of code there that is fixed and I can't touch, namely `req = Request()`. If all `s.request` did was construct a `Request` object then that would be fine, but from what I can see there's a bunch of stuff happening in its scope which I am in need of. :)


@kennethreitz: He wants to use what is essentially a subclass of the `Request` class, which presumably implements different logic for either `Request.prepare` or one of the data fields.

We can solve this, however, by using MAGIC!

``` python
import requests

class MagicRequest(requests.Request):
    def prepare(self):
        self.headers['Awesome'] = 'Hell Yeah'
        return super(MagicRequest, self).prepare()

requests.Request = MagicRequest
```

Done. Hell yeah. :sunglasses:


@maxcountryman It's a parameter of send.

```
s.send(r, allow_redirects=False)
```


@kennethreitz I get a keyword error on that...

``` python
>>> s.send(r.prepare(), allow_redirects=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Python/2.7/site-packages/requests/sessions.py", line 374, in send
    r = adapter.send(request, **kwargs)
TypeError: send() got an unexpected keyword argument 'allow_redirects'
```


The handling of `allow_redirects` seems to happen in `s.request` as far as I can tell...


hmm, that shouldn't be happening


@kennethreitz heh, yeah it's quite unfortunate. :p


@kennethreitz: Nope.

``` python
class Session(SessionRedirectMixin):
# ...snip...
    def send(self, request, **kwargs):
        adapter = self.get_adapter(url=request.url)
        r = adapter.send(request, **kwargs)
        return r

# ...snip...
class HTTPAdapter(BaseAdapter):
    def send(self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None):
# ...snip...
```


FYI:

``` python
>>> requests.__version__
'1.1.0'
```


Yeah I think `s.request` does more than you think...and I really wish it didn't. :(


Because otherwise the description of the design changes in the API are absolutely awesome!


Well let's fix this :)


Awesome! Thanks for bearing with me @kennethreitz. :) I may have a chance to make a pull request later on if someone doesn't beat me to it.


So I guess a question worth asking is: is the preferred solution to modify the `HTTPAdapter` object so that it handles things like `allow_redirects` or to "plug-in" a custom `Request` object? (I'm assuming the latter, based on the discussion here.)


@maxcountryman, @kennethreitz and I discussed this on IRC (https://botbot.me/freenode/python-requests/1756572/ from the highlighted line, down) and there are some changes that need to be made (which should have taken place during the refactor) which are going to be made now. So you might want to hold off on working on adapters and custom Request objects just yet.