Issue 1:
Hello everyone.

I have a question.

I have a simple scrapy project.

I run my project like:

scrapy crawl mp -a city="Toronto, ON"

In

def __init__(self, *args, **kwargs):

I get argument value like:

self.city = kwargs["city"]

In seetings.py I can add a line like:

FEED_FORMAT = "csv"
FEED_URI = "output/some_city.csv"

And information about need city will be saved in output/some_city.csv file.

Problem

But if I will be pass different cities like arguments to my spider, like:

scrapy crawl mp -a city="New York, NY"

then

scrapy crawl mp -a city="Toronto, ON"

then

scrapy crawl mp -a city="Chicago, IL"

etc

All information will be saved in output/some_city.csv file.

What I want

I want to find a decision, when I pass "Chicago, IL" - data will be saved in Chicago_IL.csv if I pass "Toronto, ON" in Toronto_ON.csv

I just need to change FEED_URI in my spider.

But how?

self.settings.attributes['FEED_URI']

I can get my FEED_URI in spider use this command, but how change this data?

I read this link but I wrote a solution and it does not work.

I can use -s but it not conveniently.

Do you know how I can resolve it?

Thank you very much.

Issue 2:
Description

The static variables set in scrapy.Items class are not registered in the internal _values dict and therefore I cannot access them via self['field_name'], i.e., the __getattr__ function.

Steps to Reproduce

In my items_error.py file I have this. (the actual code is larger but I isolate the code causing the error)

from scrapy import Item, Field

class BaseItem(Item):
url=Field()
id=Field()
type=Field()

def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
if not self['type']: #<--- error here. Raises KeyError
raise NotImplementedError('type of ad must be set as static class variable')
#use url hash as id to avoid duplicates
self['id']=hash(self.get('url'))

class ChildItem(BaseItem):
type='estate'
other_fields=Field()

When I ran it this raises:

C:\Users\ivangonzalez\Documents\Code\scrady>python
Python 3.7.6 | packaged by conda-forge | (default, Jan 7 2020, 21:48:41) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from scrady.items_error import *
>>> ChildItem()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\ivangonzalez\Documents\Code\scrady\scrady\items_error.py", line 10, in __init__
if not self['type']: #<--- error here. Raises KeyError
File "C:\Users\ivangonzalez\Anaconda3\lib\site-packages\scrapy\item.py", line 91, in __getitem__
return self._values[key]
KeyError: 'type'
>>> BaseItem()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\ivangonzalez\Documents\Code\scrady\scrady\items_error.py", line 10, in __init__
if not self['type']: #<--- error here. Raises KeyError
File "C:\Users\ivangonzalez\.virtualenvs\scrady-d4yjsu4b\lib\site-packages\scrapy\item.py", line 91, in __getitem__ return self._values[key]
KeyError: 'type'
>>>

Expected behavior: self['field_name'] should return the Field() attribute.

Actual behavior: static fields are not registered by scrapy.Items in scrapy.Items._values[key].

Reproduces how often: Ran in pipenv, so 100%.

Versions

Please paste here the output of executing scrapy version --verbose in the command line:

C:\Users\ivangonzalez\Documents\Code\scrady>scrapy version --verbose
Scrapy : 1.8.0
lxml : 4.4.2.0
libxml2 : 2.9.5
cssselect : 1.1.0
parsel : 1.5.2
w3lib : 1.21.0
Twisted : 19.10.0
Python : 3.7.6 | packaged by conda-forge | (default, Jan 7 2020, 21:48:41) [MSC v.1916 64 bit (AMD64)]
pyOpenSSL : 19.1.0 (OpenSSL 1.1.1d 10 Sep 2019)
cryptography : 2.8
Platform : Windows-10-10.0.18362-SP0

Additional context

In scrapy/item.py : ItemMeta class the attributes should be registered by this code:

class ItemMeta(ABCMeta):
"""Metaclass_ of :class:`Item` that handles field definitions.

.. _metaclass: https://realpython.com/python-metaclasses
"""

def __new__(mcs, class_name, bases, attrs):
classcell = attrs.pop('__classcell__', None)
new_bases = tuple(base._class for base in bases if hasattr(base, '_class'))
_class = super(ItemMeta, mcs).__new__(mcs, 'x_' + class_name, new_bases, attrs)

fields = getattr(_class, 'fields', {})
new_attrs = {}
for n in dir(_class):
v = getattr(_class, n)
if isinstance(v, Field):
fields[n] = v
elif n in attrs:
new_attrs[n] = attrs[n] #<--- here

new_attrs['fields'] = fields
new_attrs['_class'] = _class
if classcell is not None:
new_attrs['__classcell__'] = classcell
return super(ItemMeta, mcs).__new__(mcs, class_name, bases, new_attrs)

But they aren't. I don't know why.