Package: scrapy-splash
This library providesScrapyand JavaScript integration usingSplash.
The license is BSD 3-clause.InstallationInstall scrapy-splash using pip:$ pip install scrapy-splashScrapy-Splash usesSplashHTTP API, so you also need a Splash instance.
Usually to install & run Splash, something like this is enough:$ docker run -p 8050:8050 scrapinghub/splashCheck Splashinstall docsfor more info.ConfigurationAdd the Splash server address tosettings.pyof your Scrapy project
like this:SPLASH_URL = 'http://192.168.59.103:8050'Enable the Splash middleware by adding it toDOWNLOADER_MIDDLEWARESin yoursettings.pyfile and changing HttpCompressionMiddleware
priority:DOWNLOADER_MIDDLEWARES = {
    'scrapy_splash.SplashCookiesMiddleware': 723,
    'scrapy_splash.SplashMiddleware': 725,
    'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware': 810,
}Order723is just beforeHttpProxyMiddleware(750) in default
scrapy settings.HttpCompressionMiddleware priority should be changed in order to allow
advanced response processing; seehttps://github.com/scrapy/scrapy/issues/1895for details.EnableSplashDeduplicateArgsMiddlewareby adding it toSPIDER_MIDDLEWARESin yoursettings.py:SPIDER_MIDDLEWARES = {
    'scrapy_splash.SplashDeduplicateArgsMiddleware': 100,
}This middleware is needed to supportcache_argsfeature; it allows
to save disk space by not storing duplicate Splash arguments multiple
times in a disk request queue. If Splash 2.1+ is used the middleware
also allows to save network traffic by not sending these duplicate
arguments to Splash server multiple times.Set a customDUPEFILTER_CLASS:DUPEFILTER_CLASS = 'scrapy_splash.SplashAwareDupeFilter'If you use Scrapy HTTP cache then a custom cache storage backend
is required. scrapy-splash provides a subclass ofscrapy.contrib.httpcache.FilesystemCacheStorage:HTTPCACHE_STORAGE = 'scrapy_splash.SplashAwareFSCacheStorage'If you use other cache storage then it is necesary to subclass it and
replace allscrapy.util.request.request_fingerprintcalls withscrapy_splash.splash_request_fingerprint.NoteSteps (4) and (5) are necessary because Scrapy doesn’t provide a way
to override request fingerprints calculation algorithm globally; this
could change in future.There are also some additional options available.
Put them into yoursettings.pyif you want to change the defaults:SPLASH_COOKIES_DEBUGisFalseby default.
Set toTrueto enable debugging cookies in theSplashCookiesMiddleware.
This option is similar toCOOKIES_DEBUGfor the built-in scarpy cookies middleware: it logs sent and received cookies
for all requests.SPLASH_LOG_400isTrueby default - it instructs to log all 400 errors
from Splash. They are important because they show errors occurred
when executing the Splash script. Set it toFalseto disable this logging.SPLASH_SLOT_POLICYisscrapy_splash.SlotPolicy.PER_DOMAIN(as object, not just a string) by default.
It specifies how concurrency & politeness are maintained for Splash requests,
and specify the default value forslot_policyargument forSplashRequest, which is described below.UsageRequestsThe easiest way to render requests with Splash is to
usescrapy_splash.SplashRequest:yield SplashRequest(url, self.parse_result,
    args={
        # optional; parameters passed to Splash HTTP API
        'wait': 0.5,

        # 'url' is prefilled from request url
        # 'http_method' is set to 'POST' for POST requests
        # 'body' is set to request body for POST requests
    },
    endpoint='render.json', # optional; default is render.html
    splash_url='<url>',     # optional; overrides SPLASH_URL
    slot_policy=scrapy_splash.SlotPolicy.PER_DOMAIN,  # optional
)Alternatively, you can use regular scrapy.Request and'splash'Requestmetakey:yield scrapy.Request(url, self.parse_result, meta={
    'splash': {
        'args': {
            # set rendering arguments here
            'html': 1,
            'png': 1,

            # 'url' is prefilled from request url
            # 'http_method' is set to 'POST' for POST requests
            # 'body' is set to request body for POST requests
        },

        # optional parameters
        'endpoint': 'render.json',  # optional; default is render.json
        'splash_url': '<url>',      # optional; overrides SPLASH_URL
        'slot_policy': scrapy_splash.SlotPolicy.PER_DOMAIN,
        'splash_headers': {},       # optional; a dict with headers sent to Splash
        'dont_process_response': True, # optional, default is False
        'dont_send_headers': True,  # optional, default is False
        'magic_response': False,    # optional, default is True
    }
})Userequest.meta['splash']API in middlewares or when scrapy.Request
subclasses are used (there is alsoSplashFormRequestdescribed below).
For example,meta['splash']allows to create a middleware which enables
Splash for all outgoing requests by default.SplashRequestis a convenient utility to fillrequest.meta['splash'];
it should be easier to use in most cases. For eachrequest.meta['splash']key there is a correspondingSplashRequestkeyword argument: for example,
to setmeta['splash']['args']useSplashRequest(...,args=myargs).meta['splash']['args']contains arguments sent to Splash.
scrapy-splash adds some default keys/values toargs:‘url’ is set to request.url;‘http_method’ is set to ‘POST’ for POST requests;‘body’ is set to to request.body for POST requests.You can override default values by setting them explicitly.Note that by default Scrapy escapes URL fragments using AJAX escaping scheme.
If you want to pass a URL with a fragment to Splash then seturlinargsdict manually. This is handled automatically if you useSplashRequest, but you need to keep that in mind if you use rawmeta['splash']API.Splash 1.8+ is required to handle POST requests; in earlier Splash versions
‘http_method’ and ‘body’ arguments are ignored. If you work with/executeendpoint and want to support POST requests you have to handlehttp_methodandbodyarguments in your Lua script manually.meta['splash']['cache_args']is a list of argument names to cache
on Splash side. These arguments are sent to Splash only once, then cached
values are used; it allows to save network traffic and decreases request
queue disk memory usage. Usecache_argsonly for large arguments
which don’t change with each request;lua_sourceis a good candidate
(if you don’t use string formatting to build it). Splash 2.1+ is required
for this feature to work.meta['splash']['endpoint']is the Splash endpoint to use.
In case of SplashRequestrender.htmlis used by default. If you’re using raw scrapy.Request thenrender.jsonis a default (for historical reasons). It is better to always pass endpoint
explicitly.See SplashHTTP API docsfor a full list of available endpoints
and parameters.meta['splash']['splash_url']overrides the Splash URL set
insettings.py.meta['splash']['splash_headers']allows to add or change headers
which are sent to Splash server. Note that this optionis notfor
setting headers which are sent to the remote website.meta['splash']['slot_policy']customize how
concurrency & politeness are maintained for Splash requests.Currently there are 3 policies available:scrapy_splash.SlotPolicy.PER_DOMAIN(default) - send Splash requests to
downloader slots based on URL being rendered. It is useful if you want
to maintain per-domain politeness & concurrency settings.scrapy_splash.SlotPolicy.SINGLE_SLOT- send all Splash requests to
a single downloader slot. It is useful if you want to throttle requests
to Splash.scrapy_splash.SlotPolicy.SCRAPY_DEFAULT- don’t do anything with slots.
It is similar toSINGLE_SLOTpolicy, but can be different if you access
other services on the same address as Splash.meta['splash']['dont_process_response']- when set to True,
SplashMiddleware won’t change the response to a custom scrapy.Response
subclass. By default for Splash requests one of SplashResponse,
SplashTextResponse or SplashJsonResponse is passed to the callback.meta['splash']['dont_send_headers']: by default scrapy-splash passes
request headers to Splash in ‘headers’ JSON POST field. For all render.xxx
endpoints it means Scrapy header options are respected by default
(http://splash.readthedocs.org/en/stable/api.html#arg-headers). In Lua
scripts you can useheadersargument ofsplash:goto apply the
passed headers:splash:go{url, headers=splash.args.headers}.Set ‘dont_send_headers’ to True if you don’t want to passheadersto Splash.meta['splash']['http_status_from_error_code']- set response.status
to HTTP error code whenassert(splash:go(..))fails; it requiresmeta['splash']['magic_response']=True.http_status_from_error_codeoption is False by default if you use raw meta API;
SplashRequest sets it to True by default.meta['splash']['magic_response']- when set to True and a JSON
response is received from Splash, several attributes of the response
(headers, body, url, status code) are filled using data returned in JSON:response.headers are filled from ‘headers’ keys;response.url is set to the value of ‘url’ key;response.body is set to the value of ‘html’ key,
or to base64-decoded value of ‘body’ key;response.status is set to the value of ‘http_status’ key.
Whenmeta['splash']['http_status_from_error_code']is True
andassert(splash:go(..))fails with an HTTP error
response.status is also set to HTTP error code.Original URL, status and headers are available asresponse.real_url,response.splash_response_statusandresponse.splash_response_headers.This option is set to True by default if you use SplashRequest.render.jsonandexecuteendpoints may not have all the necessary
keys/values in the response.
For non-JSON endpoints, only url is filled, regardless of themagic_responsesetting.Usescrapy_splash.SplashFormRequestif you want to make aFormRequestvia splash. It accepts the same arguments asSplashRequest,
and alsoformdata, likeFormRequestfrom scrapy:>>> SplashFormRequest('http://example.com', formdata={'foo': 'bar'})
<POST http://example.com>SplashFormRequest.from_responseis also supported, and works as described
inscrapy documentation.Responsesscrapy-splash returns Response subclasses for Splash requests:SplashResponse is returned for binary Splash responses - e.g. for
/render.png responses;SplashTextResponse is returned when the result is text - e.g. for
/render.html responses;SplashJsonResponse is returned when the result is a JSON object - e.g.
for /render.json responses or /execute responses when script returns
a Lua table.To use standard Response classes setmeta['splash']['dont_process_response']=Trueor passdont_process_response=Trueargument to SplashRequest.All these responses setresponse.urlto the URL of the original request
(i.e. to the URL of a website you want to render), not to the URL of the
requested Splash endpoint. “True” URL is still available asresponse.real_url.SplashJsonResponse provide extra features:response.dataattribute contains response data decoded from JSON;
you can access it likeresponse.data['html'].If Splash session handling is configured, you can access current cookies
asresponse.cookiejar; it is a CookieJar instance.If Scrapy-Splash response magic is enabled in request (default),
several response attributes (headers, body, url, status code)
are set automatically from original response body:response.headers are filled from ‘headers’ keys;response.url is set to the value of ‘url’ key;response.body is set to the value of ‘html’ key,
or to base64-decoded value of ‘body’ key;response.status is set from the value of ‘http_status’ key.Whenresponse.bodyis updated in SplashJsonResponse
(either from ‘html’ or from ‘body’ keys) familiarresponse.cssandresponse.xpathmethods are available.To turn off special handling of JSON result keys either setmeta['splash']['magic_response']=Falseor passmagic_response=Falseargument to SplashRequest.Session HandlingSplash itself is stateless - each request starts from a clean state.
In order to support sessions the following is required:client (Scrapy) must send current cookies to Splash;Splash script should make requests using these cookies and update
them from HTTP response headers or JavaScript code;updated cookies should be sent back to the client;client should merge current cookies wiht the updated cookies.For (2) and (3) Splash providessplash:get_cookies()andsplash:init_cookies()methods which can be used in Splash Lua scripts.scrapy-splash provides helpers for (1) and (4): to send current cookies
in ‘cookies’ field and merge cookies back from ‘cookies’ response field
setrequest.meta['splash']['session_id']to the session
identifier. If you only want a single session use the samesession_idfor
all request; any value like ‘1’ or ‘foo’ is fine.For scrapy-splash session handling to work you must use/executeendpoint
and a Lua script which accepts ‘cookies’ argument and returns ‘cookies’
field in the result:function main(splash)
    splash:init_cookies(splash.args.cookies)

    -- ... your script

    return {
        cookies = splash:get_cookies(),
        -- ... other results, e.g. html
    }
endSplashRequest setssession_idautomatically for/executeendpoint,
i.e. cookie handling is enabled by default if you use SplashRequest,/executeendpoint and a compatible Lua rendering script.If you want to start from the same set of cookies, but then ‘fork’ sessions
setrequest.meta['splash']['new_session_id']in addition tosession_id. Request cookies will be fetched from cookiejarsession_id,
but response cookies will be merged back to thenew_session_idcookiejar.Standard Scrapycookiesargument can be used withSplashRequestto add cookies to the current Splash cookiejar.ExamplesGet HTML contents:import scrapy
from scrapy_splash import SplashRequest

class MySpider(scrapy.Spider):
    start_urls = ["http://example.com", "http://example.com/foo"]

    def start_requests(self):
        for url in self.start_urls:
            yield SplashRequest(url, self.parse, args={'wait': 0.5})

    def parse(self, response):
        # response.body is a result of render.html call; it
        # contains HTML processed by a browser.
        # ...Get HTML contents and a screenshot:import json
import base64
import scrapy
from scrapy_splash import SplashRequest

class MySpider(scrapy.Spider):

    # ...
        splash_args = {
            'html': 1,
            'png': 1,
            'width': 600,
            'render_all': 1,
        }
        yield SplashRequest(url, self.parse_result, endpoint='render.json',
                            args=splash_args)

    # ...
    def parse_result(self, response):
        # magic responses are turned ON by default,
        # so the result under 'html' key is available as response.body
        html = response.body

        # you can also query the html result as usual
        title = response.css('title').extract_first()

        # full decoded JSON data is available as response.data:
        png_bytes = base64.b64decode(response.data['png'])

        # ...Run a simpleSplash Lua Script:import json
import base64
from scrapy_splash import SplashRequest


class MySpider(scrapy.Spider):

    # ...
        script = """
        function main(splash)
            assert(splash:go(splash.args.url))
            return splash:evaljs("document.title")
        end
        """
        yield SplashRequest(url, self.parse_result, endpoint='execute',
                            args={'lua_source': script})

    # ...
    def parse_result(self, response):
        doc_title = response.text
        # ...More complexSplash Lua Scriptexample - get a screenshot of an HTML
element by its CSS selector (it requires Splash 2.1+).
Note how are arguments passed to the script:import json
import base64
from scrapy_splash import SplashRequest

script = """
-- Arguments:
-- * url - URL to render;
-- * css - CSS selector to render;
-- * pad - screenshot padding size.

-- this function adds padding around region
function pad(r, pad)
  return {r[1]-pad, r[2]-pad, r[3]+pad, r[4]+pad}
end

-- main script
function main(splash)

  -- this function returns element bounding box
  local get_bbox = splash:jsfunc([[
    function(css) {
      var el = document.querySelector(css);
      var r = el.getBoundingClientRect();
      return [r.left, r.top, r.right, r.bottom];
    }
  ]])

  assert(splash:go(splash.args.url))
  assert(splash:wait(0.5))

  -- don't crop image by a viewport
  splash:set_viewport_full()

  local region = pad(get_bbox(splash.args.css), splash.args.pad)
  return splash:png{region=region}
end
"""

class MySpider(scrapy.Spider):


    # ...
        yield SplashRequest(url, self.parse_element_screenshot,
            endpoint='execute',
            args={
                'lua_source': script,
                'pad': 32,
                'css': 'a.title'
            }
         )

    # ...
    def parse_element_screenshot(self, response):
        image_data = response.body  # binary image data in PNG format
        # ...Use a Lua script to get an HTML response with cookies, headers, body
and method set to correct values;lua_sourceargument value is cached
on Splash server and is not sent with each request (it requires Splash 2.1+):import scrapy
from scrapy_splash import SplashRequest

script = """
function main(splash)
  splash:init_cookies(splash.args.cookies)
  assert(splash:go{
    splash.args.url,
    headers=splash.args.headers,
    http_method=splash.args.http_method,
    body=splash.args.body,
    })
  assert(splash:wait(0.5))

  local entries = splash:history()
  local last_response = entries[#entries].response
  return {
    url = splash:url(),
    headers = last_response.headers,
    http_status = last_response.status,
    cookies = splash:get_cookies(),
    html = splash:html(),
  }
end
"""

class MySpider(scrapy.Spider):


    # ...
        yield SplashRequest(url, self.parse_result,
            endpoint='execute',
            cache_args=['lua_source'],
            args={'lua_source': script},
            headers={'X-My-Header': 'value'},
        )

    def parse_result(self, response):
        # here response.body contains result HTML;
        # response.headers are filled with headers from last
        # web page loaded to Splash;
        # cookies from all responses and from JavaScript are collected
        # and put into Set-Cookie response header, so that Scrapy
        # can remember them.HTTP Basic AuthIf you need to use HTTP Basic Authentication to access Splash, use theSPLASH_USERandSPLASH_PASSoptional settings:SPLASH_USER = 'user'
SPLASH_PASS = 'userpass'Another option ismeta['splash']['splash_headers']: it allows to set
custom headers which are sent to Splash server; add Authorization header
tosplash_headersif you want to change credentials per-request:import scrapy
from w3lib.http import basic_auth_header

class MySpider(scrapy.Spider):
    # ...
    def start_requests(self):
        auth = basic_auth_header('user', 'userpass')
        yield SplashRequest(url, self.parse,
                            splash_headers={'Authorization': auth})WARNING:Don’t useHttpAuthMiddleware(i.e.http_user/http_passspider attributes) for Splash
authentication: if you occasionally send a non-Splash request from your spider,
you may expose Splash credentials to a remote website, as HttpAuthMiddleware
sets credentials for all requests unconditionally.Why not use the Splash HTTP API directly?The obvious alternative to scrapy-splash would be to send requests directly
to the SplashHTTP API. Take a look at the example below and make
sure to read the observations after it:import json

import scrapy
from scrapy.http.headers import Headers

RENDER_HTML_URL = "http://127.0.0.1:8050/render.html"

class MySpider(scrapy.Spider):
    start_urls = ["http://example.com", "http://example.com/foo"]

    def start_requests(self):
        for url in self.start_urls:
            body = json.dumps({"url": url, "wait": 0.5}, sort_keys=True)
            headers = Headers({'Content-Type': 'application/json'})
            yield scrapy.Request(RENDER_HTML_URL, self.parse, method="POST",
                                 body=body, headers=headers)

    def parse(self, response):
        # response.body is a result of render.html call; it
        # contains HTML processed by a browser.
        # ...It works and is easy enough, but there are some issues that you should be
aware of:There is a bit of boilerplate.As seen by Scrapy, we’re sending requests toRENDER_HTML_URLinstead
of the target URLs. It affects concurrency and politeness settings:CONCURRENT_REQUESTS_PER_DOMAIN,DOWNLOAD_DELAY, etc could behave
in unexpected ways since delays and concurrency settings are no longer
per-domain.As seen by Scrapy, response.url is an URL of the Splash server.
scrapy-splash fixes it to be an URL of a requested page.
“Real” URL is still available asresponse.real_url. scrapy-splash also
allows to handleresponse.statusandresponse.headerstransparently
on Scrapy side.Some options depend on each other - for example, if you usetimeoutSplash option then you may want to setdownload_timeoutscrapy.Request meta key as well.It is easy to get it subtly wrong - e.g. if you won’t usesort_keys=Trueargument when preparing JSON body then binary POST body
content could vary even if all keys and values are the same, and it means
dupefilter and cache will work incorrectly.Default Scrapy duplication filter doesn’t take Splash specifics in
account. For example, if an URL is sent in a JSON POST request body
Scrapy will compute request fingerprint without canonicalizing this URL.Splash Bad Request (HTTP 400) errors are hard to debug because by default
response content is not displayed by Scrapy. SplashMiddleware logs content
of HTTP 400 Splash responses by default (it can be turned off by settingSPLASH_LOG_400 = Falseoption).Cookie handling is tedious to implement, and you can’t use Scrapy
built-in Cookie middleware to handle cookies when working with Splash.Large Splash arguments which don’t change with every request
(e.g.lua_source) may take a lot of space when saved to Scrapy disk
request queues.scrapy-splashprovides a way to store such static
parameters only once.Splash 2.1+ provides a way to save network traffic by caching large
static arguments on server, but it requires client support: client should
send propersave_argsandload_argsvalues and handle HTTP 498
responses.scrapy-splash utlities allow to handle such edge cases and reduce
the boilerplate.Getting helpfor problems with rendering pages read “Splash FAQ” pagefor Scrapy-related bugs take a look at “reporting Scrapy bugs” pageBest approach to get any other help is to ask a question onStack OverflowContributingSource code and bug tracker are on github:https://github.com/scrapy-plugins/scrapy-splashTo run tests, install “tox” Python package and then runtoxcommand
from the source checkout.To run integration tests, start Splash and set SPLASH_URL env variable
to Splash address before runningtoxcommand:docker run -d --rm -p8050:8050 scrapinghub/splash:3.0
SPLASH_URL=http://127.0.0.1:8050 tox -e py36Changes0.9.0 (2023-02-03)Removed official support for Python 2.7, 3.4, 3.5 and 3.6, and added official
support for Python 3.9, 3.10 and 3.11.DeprecatedSplashJsonResponse.body_as_unicode(), to be replaced bySplashJsonResponse.text.Removed calls to obsoleteto_native_str, removed in Scrapy 2.8.0.8.0 (2021-10-05)Security bug fix:If you useHttpAuthMiddleware(i.e. thehttp_userandhttp_passspider attributes) for Splash authentication, any non-Splash request will
expose your credentials to the request target. This includesrobots.txtrequests sent by Scrapy when theROBOTSTXT_OBEYsetting is set toTrue.Use the newSPLASH_USERandSPLASH_PASSsettings instead to set
your Splash authentication credentials safely.Responses now expose the HTTP status code and headers from Splash asresponse.splash_response_statusandresponse.splash_response_headers(#158)Themetaargument passed to thescrapy_splash.request.SplashRequestconstructor is no longer modified (#164)Website responses with 400 or 498 as HTTP status code are no longer
handled as the equivalent Splash responses (#158)Cookies are no longer sent to Splash itself (#156)scrapy_splash.utils.dict_hashnow also works withobj=None(225793b)Our test suite now includes integration tests (#156) and tests can be run
in parallel (6fb8c41)There’s a new ‘Getting help’ section in theREADME.rstfile (#161,
#162), the documentation aboutSPLASH_SLOT_POLICYhas been improved
(#157) and a typo as been fixed (#121)Made some internal improvements (ee5000d,25de545,2aaa79d)0.7.2 (2017-03-30)fixed issue with response type detection.0.7.1 (2016-12-20)Scrapy 1.0.x support is back;README updates.0.7 (2016-05-16)SPLASH_COOKIES_DEBUGsetting allows to log cookies
sent and received to/from Splash incookiesrequest/response fields.
It is similar to Scrapy’s builtinCOOKIES_DEBUG, but works for
Splash requests;README cleanup.0.6.1 (2016-04-29)Warning about HTTP methods is no longer logged for non-Splash requests.0.6 (2016-04-20)SplashAwareDupeFilterandsplash_request_fingerprintare improved:
they now canonicalize URLs and take URL fragments in account;cache_argsvalue fingerprints are now calculated faster.0.5 (2016-04-18)cache_argsSplashRequest argument andrequest.meta['splash']['cache_args']key allow to save network traffic
and disk storage by not storing duplicate Splash arguments in disk request
queues and not sending them to Splash multiple times. This feature requires
Splash 2.1+.To upgrade from v0.4 enableSplashDeduplicateArgsMiddlewarein settings.py:SPIDER_MIDDLEWARES = {
    'scrapy_splash.SplashDeduplicateArgsMiddleware': 100,
}0.4 (2016-04-14)SplashFormRequest class is added; it is a variant of FormRequest which uses
Splash;Splash parameters are no longer stored in request.meta twice; this change
should decrease disk queues data size;SplashMiddleware now increases request priority when rescheduling the request;
this should decrease disk queue data size and help with stale cookie
problems.0.3 (2016-04-11)Package is renamed fromscrapyjstoscrapy-splash.An easiest way to upgrade is to replacescrapyjsimports withscrapy_splashand updatesettings.pywith new defaults
(check the README).There are many new helpers to handle JavaScript rendering transparently;
the recommended way is now to usescrapy_splash.SplashRequestinstead
ofrequest.meta['splash']. Please make sure to read the README if
you’re upgrading from scrapyjs - you may be able to drop some code from your
project, especially if you want to access response html, handle cookies
and headers.new SplashRequest class; it can be used as a replacement for scrapy.Request
to provide a better integration with Splash;added support for POST requests;SplashResponse, SplashTextResponse and SplashJsonResponse allow to
handle Splash responses transparently, taking care of response.url,
response.body, response.headers and response.status. SplashJsonResponse
allows to access decoded response JSON data asresponse.data.cookie handling improvements: it is possible to handle Scrapy and Splash
cookies transparently; current cookiejar is exposed as response.cookiejar;headers are passed to Splash by default;URLs with fragments are handled automatically when using SplashRequest;logging is improved:SplashRequest.__repr__shows both requested URL
and Splash URL;in case of Splash HTTP 400 errors the response is logged by default;an issue with dupefilters is fixed: previously the order of keys in
JSON request body could vary, making requests appear as non-duplicates;it is now possible to pass custom headers to Splash server itself;test coverage reports are enabled.0.2 (2016-03-26)Scrapy 1.0 and 1.1 support;Python 3 support;documentation improvements;project is moved tohttps://github.com/scrapy-plugins/scrapy-splash.0.1.1 (2015-03-16)Fixed fingerprint calculation for non-string meta values.0.1 (2015-02-28)Initial release