scrapinghub/splash — pull request #40 by kmike (2013-09-12 23:35:26+00:00)
Title: [WIP] use single instances of QNetworkAccessManager and QNetworkDiskCache

It seems that after many false starts I finally came up with a solution for #39 :)

The main stumbling block was per-request proxy profile support: QNetworkProxyFactory.createRequest effectively receives only an URL that is to be proxied (this is hardcoded in C++ code, see http://qt.gitorious.org/qt/qt/source/67376be28ca51930ff0f4fad2dd58f53968655a9:src/network/access/qnetworkaccessmanager.cpp#L718 ), but we need it to make a decision based on request to splash itself. So the question was how to pass information to QNetworkProxyFactory.createRequest.

What didn't work:

* simple hacks like setting cache to None - anyways, it is documented that multiple caches sharing the same folder are not supported by QT;
* subclassing QNetworkRequest to carry extra information, or just setting some attribute on request - custom request gets converted to standard request somewhere in C++ land;
* using QNetworkRequest.setAttribute - this worked only for the first, "main" request; requests created automatically (e.g. to images, css and js files) don't copy this attribute
* shared state for QNetworkAccessManager - it looks wrong.

What worked is to use QNetworkRequest.setOriginatingObject and access web_page attributes via this originatingObject - as always, it seems obvious in retrospect :)

There are some consequences of a single QNetworkAccessManager, e.g. 6 connections per domain/port become a global limit.