geek4377/scrapy — comment on commit ae6ba09f5db37e91f2e65bc56d33df2000285a8d
File:  line 0
By geek4377:

Our use case for doing this was to use a different proxy for each
request (using some way of rotation). If we want a fixed proxy per
session then indeed it doesn't make sense to add the unnecessary
overhead on each request.

On Wed, Nov 23, 2011 at 1:22 AM, Pablo Hoffman

wrote:
> Parsing proxy url on *every* request adds some considerable overhead that I'm not sure it's worth. The idea is that if 'proxy' key was already present in request.meta, it was assumed to be have been populated properly - hence the if 'proxy' in request.meta: return.
>
> ---
> Reply to this email directly or view it on GitHub:
> https://github.com/geek4377/scrapy/commit/ae6ba09f5db37e91f2e65bc56d33df2000285a8d#commitcomment-738853
>