Package: python-scrapyd-api
python-scrapyd-apiA Python wrapper for working withScrapyd'sAPI.Current released version: 2.1.2 (seehistory).Allows a Python application to talk to, and therefore control, theScrapydaemon:Scrapyd.Supports Python 2.6, 2.7, 3.3 & 3.4Free software: BSD licenseFull documentationOn thePython Package Index (PyPI)Scrapyd'sAPI DocumentationInstallEasiest installation is viapip:pipinstallpython-scrapyd-apiQuick UsagePlease refer to thefull documentationfor more detailed usage but to get you started:>>>fromscrapyd_apiimportScrapydAPI>>>scrapyd=ScrapydAPI('http://localhost:6800')Add a projectegg as a new version:>>>egg=open('some_egg.egg','rb')>>>scrapyd.add_version('project_name','version_name',egg)# Returns the number of spiders in the project.3>>>egg.close()Cancel a scheduled job:>>>scrapyd.cancel('project_name','14a6599ef67111e38a0e080027880ca6')# Returns the "previous state" of the job before it was cancelled: 'running' or 'pending'.'running'Delete a projectand all sibling versions:>>>scrapyd.delete_project('project_name')# Returns True if the request was met with an OK response.TrueDelete a versionof a project:>>>scrapyd.delete_version('project_name','version_name')# Returns True if the request was met with an OK response.TrueRequest statusof a job:>>>scrapyd.job_status('project_name','14a6599ef67111e38a0e080027880ca6')# Returns 'running', 'pending', 'finished' or '' for unknown state.'running'List all jobsregistered:>>>scrapyd.list_jobs('project_name')# Returns a dict of running, finished and pending job lists.{'pending':[{u'id':u'24c35...f12ae',u'spider':u'spider_name'},],'running':[{u'id':u'14a65...b27ce',u'spider':u'spider_name',u'start_time':u'2014-06-17 22:45:31.975358'},],'finished':[{u'id':u'34c23...b21ba',u'spider':u'spider_name',u'start_time':u'2014-06-17 22:45:31.975358',u'end_time':u'2014-06-23 14:01:18.209680'}]}List all projectsregistered:>>>scrapyd.list_projects()[u'ecom_project',u'estate_agent_project',u'car_project']List all spidersavailable to a given project:>>>scrapyd.list_spiders('project_name')[u'raw_spider',u'js_enhanced_spider',u'selenium_spider']List all versionsregistered to a given project:>>>scrapyd.list_versions('project_name'):[u'345',u'346',u'347',u'348']Schedule a jobto run with a specific spider:# Schedule a job to run with a specific spider.>>>scrapyd.schedule('project_name','spider_name')# Returns the Scrapyd job id.u'14a6599ef67111e38a0e080027880ca6'Schedule a jobto run while passing override settings:>>>settings={'DOWNLOAD_DELAY':2}>>>scrapyd.schedule('project_name','spider_name',settings=settings)u'25b6588ef67333e38a0e080027880de7'Schedule a jobto run while passing extra attributes to spider initialisation:>>>scrapyd.schedule('project_name','spider_name',extra_attribute='value')# NB: 'project', 'spider' and 'settings' are reserved kwargs for this# method and therefore these names should be avoided when trying to pass# extra attributes to the spider init.u'25b6588ef67333e38a0e080027880de7'Setting up the project to contribute codePlease seeCONTRIBUTING.md.  This will guide you through our pull request
guidelines, project setup and testing requirements.License2-clause BSD. See the fullLICENSE.History2.1.1 (2018-04-01)Base set of docs converted to markdown (README, AUTHORS, CONTRIBUTING, HISTORY)2.1.0 (2018-03-31)Introduces thetimeoutkeyword argument, which allows the caller to specify
a timeout after which requests to the scrapyd server give up. This works as
per the underlyingrequestslibrary, and raisesrequests.exceptions.Timeoutwhen the timeout is exceeded. See docs for usage.2.0.1 (2016-02-27)v2.0.0 shipped with docs which were slightly out of date for the cancel
endpoint, this release corrects that.2.0.0 (2016-02-27)Why Version 2? This package has been production ready and stable in use
for over a year now, so it's ready  to commit to a stable API /w semver.
Version 1 has deliberately been skipped to make it absolutely clear that
this release contains a breaking change:Breaking changes:The cancel job endpoint now returns the previous state of the successfully
cancelled spider rather than a simple boolean True/False. This change was
made because:
a) the boolean return was relatively useless and actually hiding data the
scrapyd API passes us as part of the cancel endpoint response.
b) before this change, the method would have returnedTrueonly if the
cancelled job was previously running, and this resulted in us incorrectly
reportingFalsewhen apendingjob was cancelled.
This may require no changes to your codebase but nevertheless it is a change
in a public API, thus the requirement for major version bumping.Other changes:The cancel job endpoint now accepts asignalkeyword argument which is
the termination signal Scrapyd uses to cancel the spider job. If not
specified, the value is not sent to the scrapyd endpoint at all, therefore
allows scrapyd control over which default signal gets used (currentlyTERM).0.2.0 (2015-01-14)Added the newjob_statusmethod which can retrieve the job status of a
specific job from a project. See docs for usage.Increased and improved test coverage.0.1.0 (2014-09-16)First release on PyPI.