Package: scrapy-mongoengine-item
BringingScrapyandMongoEnginetogether.scrapy-mongoengine-itemis an extension that allows you to defineScrapy itemsusing existingMongoEngine documents.Documentation is available onRead the Docs.PrerequisitesBothPython 2.7andPython 3.5/3.6are
supported. ForPython 3you needScrapy v1.1or above.Latest tested MongoEngine version isMongoEngine 0.17.0.InstallationInstall latest stable version from PyPI:pipinstallscrapy-mongoengine-itemor latest stable version from GitHub:pipinstallhttps://github.com/barseghyanartur/scrapy-mongoengine-item/archive/stable.tar.gzor latest stable version from BitBucket:pipinstallhttps://bitbucket.org/barseghyanartur/scrapy-mongoengine-item/get/stable.tar.gzIntroductionMongoEngineItemis a class of item that gets its fields definition from a
MongoEngine document, you simply create aMongoEngineItemand specify what
MongoEngine document it relates to.Besides of getting the document fields defined on your item,MongoEngineItemprovides a method to create and populate a MongoEngine document instance with
the item data.UsageMongoEngineItemworks as follows: you create a subclass and define itsmongoengine_documentattribute to be a valid MongoEngine document. With
this you will get an item with a field for each MongoEngine document field.In addition, you can define fields that aren’t present in the document and even
override fields that are present in the model defining them in the item.Let’s see some examples:Creating a MongoEngine document for the examples:frommongoengineimportfields,documentclassPerson(document.Document):name=fields.StringField(max_length=255)age=fields.IntField()Defining a basicMongoEngineItem:fromscrapy_mongoengine_itemimportMongoEngineItemclassPersonItem(MongoEngineItem):mongoengine_document=PersonMongoEngineItemworks just like Scrapy items:p=PersonItem()p['name']='John'p['age']=22To obtain the MongoEngine document from the item, we call the extra methodMongoEngineItem.save()of theMongoEngineItem:person=p.save()person.name# 'John'person.age# 22person.id# 1The document is already saved when we callMongoEngineItem.save(), we
can prevent this by calling it withcommit=False. We can usecommit=FalseinMongoEngineItem.save()method to obtain an unsaved
document:person=p.save(commit=False)person.name# 'John'person.age# 22person.id# NoneAs said before, we can add other fields to the item:importscrapyfromscrapy_mongoengine_itemimportMongoEngineItemclassPersonItem(MongoEngineItem):mongoengine_document=Personsex=scrapy.Field()p=PersonItem()p['name']='John'p['age']=22p['sex']='M'And we can override the fields of the document with your own:classPersonItem(MongoEngineItem):mongoengine_document=Personname=scrapy.Field(default='No Name')This is useful to provide properties to the field, like a default or any other
property that your project uses. Those additional fields won’t be taken into
account when doing aMongoEngineItem.save().DevelopmentTestingTo run tests in your working environment type:./runtests.pyTo test with all supported Python versions type:toxRunning MongoDBThe easiest way is to run it via Docker:dockerpullmongo:latestdockerrun-p27017:27017mongo:latestWriting documentationKeep the following hierarchy.=====
title
=====

header
======

sub-header
----------

sub-sub-header
~~~~~~~~~~~~~~

sub-sub-sub-header
^^^^^^^^^^^^^^^^^^

sub-sub-sub-sub-header
++++++++++++++++++++++

sub-sub-sub-sub-sub-header
**************************LicenseGPL-2.0-only OR LGPL-2.1-or-laterSupportFor any issues contact me at the e-mail given in theAuthorsection.AuthorArtur Barseghyan <artur.barseghyan@gmail.com>