Repository: scrapy/scrapy
Pull request: start_requests can return items
resolve https://github.com/scrapy/scrapy/issues/5289
based on code sample from https://github.com/scrapy/scrapy/issues/5289#issuecomment-1522396870

I used this сode sample to test this locally.
<details><summary>script.py</summary>

```python
import scrapy
from scrapy.crawler import CrawlerProcess

class QuotesToScrapeSpider(scrapy.Spider):
    def start_requests(self):
        yield scrapy.Request(url='http://quotes.toscrape.com/', callback=self.parse)
        yield {"quote": ['“It is possible to return items on start_requests. But I don\'t understand Why it\'s needed?“'], "author": ["Georgiy"], "tags": ["scrapy"]}

    def parse(self, response):
        for quote in response.css("div.quote"):
            yield {
                "quote": quote.css("span.text::text").getall(),
                "author": quote.css("small.author::text").getall(),
                "tags": quote.css("a.tag::text").getall()
            }
if __name__ == "__main__":
    p = CrawlerProcess(); p.crawl(QuotesToScrapeSpider); p.start()
```

</details>
At this moment I didn't found test where output(type) of  start_requests checked

Problem statement:
Better error (or support?) when yielding items from start requests
We have seen more than once users trying to yield items from the `start_requests` method.

We should consider providing a better error message in this scenario, the current one is not very helpful.

We could also consider supporting this use case instead.

Hints from issue discussion:
Example of the current situation:

<details>
<summary>
A minimal sample spider code.
</summary>

```python
#coding: utf8

import scrapy


class Test(scrapy.Spider):
    name = "test"

    def start_requests(self):
        yield {}
```
</details>

<details>
<summary>
The full error output using Scrapy 2.5.1 at Python 3.9
</summary>

```
$ scrapy runspider test.py 
2021-10-22 14:55:22 [scrapy.utils.log] INFO: Scrapy 2.5.1 started (bot: scrapybot)
2021-10-22 14:55:22 [scrapy.utils.log] INFO: Versions: lxml 4.6.3.0, libxml2 2.9.10, cssselect 1.1.0, parsel 1.6.0, w3lib 1.22.0, Twisted 21.7.0, Python 3.9.7 (default, Aug 31 2021, 13:28:12) - [GCC 11.1.0], pyOpenSSL 21.0.0 (OpenSSL 1.1.1l  24 Aug 2021), cryptography 35.0.0, Platform Linux-5.12.15-arch1-1-x86_64-with-glibc2.33
2021-10-22 14:55:22 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.epollreactor.EPollReactor
2021-10-22 14:55:22 [scrapy.crawler] INFO: Overridden settings:
{'EDITOR': 'vim', 'SPIDER_LOADER_WARN_ONLY': True}
2021-10-22 14:55:22 [scrapy.extensions.telnet] INFO: Telnet Password: 73c8531757965038
2021-10-22 14:55:22 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.memusage.MemoryUsage',
 'scrapy.extensions.logstats.LogStats']
2021-10-22 14:55:22 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
 'scrapy.downloadermiddlewares.stats.DownloaderStats']
2021-10-22 14:55:22 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware']
2021-10-22 14:55:22 [scrapy.middleware] INFO: Enabled item pipelines:
[]
2021-10-22 14:55:22 [scrapy.core.engine] INFO: Spider opened
2021-10-22 14:55:22 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2021-10-22 14:55:22 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2021-10-22 14:55:22 [scrapy.utils.signal] ERROR: Error caught on signal handler: <bound method RefererMiddleware.request_scheduled of <scrapy.spidermiddlewares.referer.RefererMiddleware object at 0x7f1e7455ce50>>
Traceback (most recent call last):
  File "/tmp/env/lib/python3.9/site-packages/scrapy/utils/signal.py", line 32, in send_catch_log
    response = robustApply(receiver, signal=signal, sender=sender, *arguments, **named)
  File "/tmp/env/lib/python3.9/site-packages/pydispatch/robustapply.py", line 55, in robustApply
    return receiver(*arguments, **named)
  File "/tmp/env/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 346, in request_scheduled
    redirected_urls = request.meta.get('redirect_urls', [])
AttributeError: 'dict' object has no attribute 'meta'
Unhandled Error
Traceback (most recent call last):
  File "/tmp/env/lib/python3.9/site-packages/scrapy/commands/runspider.py", line 56, in run
    self.crawler_process.start()
  File "/tmp/env/lib/python3.9/site-packages/scrapy/crawler.py", line 327, in start
    reactor.run(installSignalHandlers=False)  # blocking call
  File "/tmp/env/lib/python3.9/site-packages/twisted/internet/base.py", line 1318, in run
    self.mainLoop()
  File "/tmp/env/lib/python3.9/site-packages/twisted/internet/base.py", line 1328, in mainLoop
    reactorBaseSelf.runUntilCurrent()
--- <exception caught here> ---
  File "/tmp/env/lib/python3.9/site-packages/twisted/internet/base.py", line 994, in runUntilCurrent
    call.func(*call.args, **call.kw)
  File "/tmp/env/lib/python3.9/site-packages/scrapy/utils/reactor.py", line 50, in __call__
    return self._func(*self._a, **self._kw)
  File "/tmp/env/lib/python3.9/site-packages/scrapy/core/engine.py", line 137, in _next_request
    self.crawl(request, spider)
  File "/tmp/env/lib/python3.9/site-packages/scrapy/core/engine.py", line 218, in crawl
    self.schedule(request, spider)
  File "/tmp/env/lib/python3.9/site-packages/scrapy/core/engine.py", line 223, in schedule
    if not self.slot.scheduler.enqueue_request(request):
  File "/tmp/env/lib/python3.9/site-packages/scrapy/core/scheduler.py", line 78, in enqueue_request
    if not request.dont_filter and self.df.request_seen(request):
builtins.AttributeError: 'dict' object has no attribute 'dont_filter'

2021-10-22 14:55:22 [twisted] CRITICAL: Unhandled Error
Traceback (most recent call last):
  File "/tmp/env/lib/python3.9/site-packages/scrapy/commands/runspider.py", line 56, in run
    self.crawler_process.start()
  File "/tmp/env/lib/python3.9/site-packages/scrapy/crawler.py", line 327, in start
    reactor.run(installSignalHandlers=False)  # blocking call
  File "/tmp/env/lib/python3.9/site-packages/twisted/internet/base.py", line 1318, in run
    self.mainLoop()
  File "/tmp/env/lib/python3.9/site-packages/twisted/internet/base.py", line 1328, in mainLoop
    reactorBaseSelf.runUntilCurrent()
--- <exception caught here> ---
  File "/tmp/env/lib/python3.9/site-packages/twisted/internet/base.py", line 994, in runUntilCurrent
    call.func(*call.args, **call.kw)
  File "/tmp/env/lib/python3.9/site-packages/scrapy/utils/reactor.py", line 50, in __call__
    return self._func(*self._a, **self._kw)
  File "/tmp/env/lib/python3.9/site-packages/scrapy/core/engine.py", line 137, in _next_request
    self.crawl(request, spider)
  File "/tmp/env/lib/python3.9/site-packages/scrapy/core/engine.py", line 218, in crawl
    self.schedule(request, spider)
  File "/tmp/env/lib/python3.9/site-packages/scrapy/core/engine.py", line 223, in schedule
    if not self.slot.scheduler.enqueue_request(request):
  File "/tmp/env/lib/python3.9/site-packages/scrapy/core/scheduler.py", line 78, in enqueue_request
    if not request.dont_filter and self.df.request_seen(request):
builtins.AttributeError: 'dict' object has no attribute 'dont_filter'

2021-10-22 14:55:27 [scrapy.core.engine] INFO: Closing spider (finished)
2021-10-22 14:55:27 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'elapsed_time_seconds': 5.006256,
 'finish_reason': 'finished',
 'finish_time': datetime.datetime(2021, 10, 22, 13, 55, 27, 820470),
 'log_count/CRITICAL': 1,
 'log_count/ERROR': 1,
 'log_count/INFO': 10,
 'memusage/max': 57970688,
 'memusage/startup': 57970688,
 'start_time': datetime.datetime(2021, 10, 22, 13, 55, 22, 814214)}
2021-10-22 14:55:27 [scrapy.core.engine] INFO: Spider closed (finished)
```
</details>
On `scrapy.core.scraper.Scraper._process_spidermw_output` application call `engine.crawl` if spider output type is Request and `.itemproc.process_item` if spider output is item:
https://github.com/scrapy/scrapy/blob/8c8fb67057609d79a31e76587fc66b162d746906/scrapy/core/scraper.py#L290-L297

While on `scrapy.core.engine.ExecutionEngine._schedule_request` where it process results of `.start_requests` only Request objects is expected (we see only engine's call `self.crawl(request)`):
https://github.com/scrapy/scrapy/blob/8c8fb67057609d79a31e76587fc66b162d746906/scrapy/core/engine.py#L180-L195

On following code sample I've added.. code from `Scraper` that call item processing into.. subclass of `ExecutionEngine`

<details><summary>start_items.py</summary>

```python
import csv
from itemadapter import is_item
import logging
import scrapy
from scrapy.crawler import CrawlerProcess

logger = logging.getLogger('scrapy.core.engine')

class CsvWriterPipeline(object):

    def open_spider(self, spider):
        self.file = open('start_items.csv', 'w',encoding="utf8")
        self.csvwriter = csv.writer(self.file)
        self.csvwriter.writerow(["Author", "Quote", "Tags"])

    def process_item(self, item, spider):
        row = [item["author"], item["quote"], ",".join(item["tags"])]
        self.csvwriter.writerow(row); return item

    def close_spider(self, spider): self.file.close()

class ExecutionEngineStartItems(scrapy.core.engine.ExecutionEngine):
    def _next_request(self) -> None:
        if self.slot is None: return
        assert self.spider is not None  # typing
        if self.paused: return None
        while (not self._needs_backout() and self._next_request_from_scheduler() is not None): pass

        if self.slot.start_requests is not None and not self._needs_backout():
            try:
                request = next(self.slot.start_requests)
            except StopIteration:
                self.slot.start_requests = None
            except Exception:
                self.slot.start_requests = None
                logger.error(
                    "Error while obtaining start requests",
                    exc_info=True,
                    extra={"spider": self.spider},
                )
            else:
                if isinstance(request, scrapy.Request):
                    self.crawl(request)
                elif is_item(request):
                    self.scraper.slot.itemproc_size += 1
                    dfd = self.scraper.itemproc.process_item(request, self.crawler.spider)
                    dfd.addBoth(self.scraper._itemproc_finished, request, f'{self.crawler.spider.__class__.__name__}.start_requests', self.crawler.spider)
                    # return dfd

        if self.spider_is_idle() and self.slot.close_if_idle: self._spider_idle()

class QuotesToScrapeSpider(scrapy.Spider):
    name = "quotes"; custom_settings = {'ITEM_PIPELINES': {CsvWriterPipeline:1}}

    def start_requests(self):
        yield scrapy.Request(url='http://quotes.toscrape.com/', callback=self.parse)
        yield {"quote": ['“It is possible to return items on start_requests. But I don\'t understand Why it\'s needed?“'], "author": ["Georgiy"], "tags": ["scrapy"]}

    def parse(self, response):
        for quote in response.css("div.quote"):
            yield {
                "quote": quote.css("span.text::text").getall(),
                "author": quote.css("small.author::text").getall(),
                "tags": quote.css("a.tag::text").getall()
            }
        # if next_page := response.css("li.next a::attr(href)").get():
            # yield scrapy.Request(url=response.urljoin(next_page), callback=self.parse)

def _create_engine(self):
    engine = ExecutionEngineStartItems(self, lambda _: self.stop())
    return engine

scrapy.crawler.Crawler._create_engine = _create_engine

process = CrawlerProcess()
process.crawl(QuotesToScrapeSpider)
process.start()
```
</details>
<details><summary>log_output</summary>

```
2023-04-25 23:15:06 [scrapy.utils.log] INFO: Scrapy 2.6.2 started (bot: scrapybot)
2023-04-25 23:15:06 [scrapy.utils.log] INFO: Versions: lxml 4.9.1.0, libxml2 2.9.14, cssselect 1.1.0, parsel 1.6.0, w3lib 1.21.0, Twisted 22.2.0, Python 3.10.8 | packaged by conda-forge | (main, Nov 24 2022, 14:07:00) [MSC v.1916 64 bit (AMD64)], pyOpenSSL 22.0.0 (OpenSSL 1.1.1t  7 Feb 2023), cryptography 38.0.1, Platform Windows-10-10.0.22621-SP0
2023-04-25 23:15:06 [scrapy.crawler] INFO: Overridden settings:
{}
2023-04-25 23:15:06 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.selectreactor.SelectReactor
2023-04-25 23:15:06 [scrapy.extensions.telnet] INFO: Telnet Password: 9c940bb77bfd27d5
2023-04-25 23:15:06 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.logstats.LogStats']
2023-04-25 23:15:06 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
 'scrapy.downloadermiddlewares.stats.DownloaderStats']
2023-04-25 23:15:06 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware']
2023-04-25 23:15:06 [scrapy.middleware] INFO: Enabled item pipelines:
[<class '__main__.CsvWriterPipeline'>]
2023-04-25 23:15:06 [scrapy.core.engine] INFO: Spider opened
2023-04-25 23:15:07 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2023-04-25 23:15:07 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2023-04-25 23:15:07 [scrapy.core.scraper] DEBUG: Scraped from QuotesToScrapeSpider.start_requests
{'quote': ["“It is possible to return items on start_requests. But I don't understand Why it's needed?“"], 'author': ['Georgiy'], 'tags': ['scrapy']}
2023-04-25 23:15:07 [scrapy.core.engine] DEBUG: Crawled (200) <GET http://quotes.toscrape.com/> (referer: None)
2023-04-25 23:15:08 [scrapy.core.scraper] DEBUG: Scraped from <200 http://quotes.toscrape.com/>
{'quote': ['“The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.”'], 'author': ['Albert Einstein'], 'tags': ['change', 'deep-thoughts', 'thinking', 'world']}
2023-04-25 23:15:08 [scrapy.core.scraper] DEBUG: Scraped from <200 http://quotes.toscrape.com/>
{'quote': ['“It is our choices, Harry, that show what we truly are, far more than our abilities.”'], 'author': ['J.K. Rowling'], 'tags': ['abilities', 'choices']}
2023-04-25 23:15:08 [scrapy.core.scraper] DEBUG: Scraped from <200 http://quotes.toscrape.com/>
{'quote': ['“There are only two ways to live your life. One is as though nothing is a miracle. The other is as though everything is a miracle.”'], 'author': ['Albert Einstein'], 'tags': ['inspirational', 'life', 'live', 'miracle', 'miracles']}
2023-04-25 23:15:08 [scrapy.core.scraper] DEBUG: Scraped from <200 http://quotes.toscrape.com/>
{'quote': ['“The person, be it gentleman or lady, who has not pleasure in a good novel, must be intolerably stupid.”'], 'author': ['Jane Austen'], 'tags': ['aliteracy', 'books', 'classic', 'humor']}
2023-04-25 23:15:08 [scrapy.core.scraper] DEBUG: Scraped from <200 http://quotes.toscrape.com/>
{'quote': ["“Imperfection is beauty, madness is genius and it's better to be absolutely ridiculous than absolutely boring.”"], 'author': ['Marilyn Monroe'], 'tags': ['be-yourself', 'inspirational']}
2023-04-25 23:15:08 [scrapy.core.scraper] DEBUG: Scraped from <200 http://quotes.toscrape.com/>
{'quote': ['“Try not to become a man of success. Rather become a man of value.”'], 'author': ['Albert Einstein'], 'tags': ['adulthood', 'success', 'value']}
2023-04-25 23:15:08 [scrapy.core.scraper] DEBUG: Scraped from <200 http://quotes.toscrape.com/>
{'quote': ['“It is better to be hated for what you are than to be loved for what you are not.”'], 'author': ['André Gide'], 'tags': ['life', 'love']}
2023-04-25 23:15:08 [scrapy.core.scraper] DEBUG: Scraped from <200 http://quotes.toscrape.com/>
{'quote': ["“I have not failed. I've just found 10,000 ways that won't work.”"], 'author': ['Thomas A. Edison'], 'tags': ['edison', 'failure', 'inspirational', 'paraphrased']}
2023-04-25 23:15:08 [scrapy.core.scraper] DEBUG: Scraped from <200 http://quotes.toscrape.com/>
{'quote': ["“A woman is like a tea bag; you never know how strong it is until it's in hot water.”"], 'author': ['Eleanor Roosevelt'], 'tags': ['misattributed-eleanor-roosevelt']}
2023-04-25 23:15:08 [scrapy.core.scraper] DEBUG: Scraped from <200 http://quotes.toscrape.com/>
{'quote': ['“A day without sunshine is like, you know, night.”'], 'author': ['Steve Martin'], 'tags': ['humor', 'obvious', 'simile']}
2023-04-25 23:15:08 [scrapy.core.engine] INFO: Closing spider (finished)
2023-04-25 23:15:08 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 223,
 'downloader/request_count': 1,
 'downloader/request_method_count/GET': 1,
 'downloader/response_bytes': 2087,
 'downloader/response_count': 1,
 'downloader/response_status_count/200': 1,
 'elapsed_time_seconds': 0.991017,
 'finish_reason': 'finished',
 'finish_time': datetime.datetime(2023, 4, 25, 20, 15, 8, 28261),
 'httpcompression/response_bytes': 11053,
 'httpcompression/response_count': 1,
 'item_scraped_count': 11,
 'log_count/DEBUG': 13,
 'log_count/INFO': 10,
 'response_received_count': 1,
 'scheduler/dequeued': 1,
 'scheduler/dequeued/memory': 1,
 'scheduler/enqueued': 1,
 'scheduler/enqueued/memory': 1,
 'start_time': datetime.datetime(2023, 4, 25, 20, 15, 7, 37244)}
2023-04-25 23:15:08 [scrapy.core.engine] INFO: Spider closed (finished)
```
</details>

Commit message: start_requests: test_crawl updated

Patch to review:
diff --git a/scrapy/core/engine.py b/scrapy/core/engine.py
index 5318cbd64e7..ef1fd57baaa 100644
--- a/scrapy/core/engine.py
+++ b/scrapy/core/engine.py
@@ -24,6 +24,7 @@
     cast,
 )
 
+from itemadapter import is_item
 from twisted.internet.defer import Deferred, inlineCallbacks, succeed
 from twisted.internet.task import LoopingCall
 from twisted.python.failure import Failure
@@ -205,7 +206,19 @@ def _next_request(self) -> None:
                     extra={"spider": self.spider},
                 )
             else:
-                self.crawl(request)
+                if isinstance(request, Request):
+                    self.crawl(request)
+                elif is_item(request):
+                    self.scraper.slot.itemproc_size += 1
+                    dfd = self.scraper.itemproc.process_item(
+                        request, self.crawler.spider
+                    )
+                    dfd.addBoth(
+                        self.scraper._itemproc_finished,
+                        request,
+                        f"{self.crawler.spider.__class__.__name__}.start_requests",
+                        self.crawler.spider,
+                    )
 
         if self.spider_is_idle() and self.slot.close_if_idle:
             self._spider_idle()
diff --git a/tests/spiders.py b/tests/spiders.py
index ea419afbdac..5c64ba95c91 100644
--- a/tests/spiders.py
+++ b/tests/spiders.py
@@ -328,6 +328,11 @@ def parse(self, response):
         yield from super().parse(response)
 
 
+class StartRequestsItemSpider(FollowAllSpider):
+    def start_requests(self):
+        yield {"name": "test item"}
+
+
 class SingleRequestSpider(MetaSpider):
     seed = None
     callback_func = None
diff --git a/tests/test_crawl.py b/tests/test_crawl.py
index 6cde4ed8c50..bd5fd1fd7ee 100644
--- a/tests/test_crawl.py
+++ b/tests/test_crawl.py
@@ -49,6 +49,7 @@
     HeadersReceivedErrbackSpider,
     SimpleSpider,
     SingleRequestSpider,
+    StartRequestsItemSpider,
 )
 
 
@@ -184,6 +185,14 @@ def test_start_requests_bug_yielding(self):
         self.assertIsNotNone(record.exc_info)
         self.assertIs(record.exc_info[0], ZeroDivisionError)
 
+    @defer.inlineCallbacks
+    def test_start_requests_items(self):
+        with LogCapture("scrapy", level=logging.ERROR) as log:
+            crawler = get_crawler(StartRequestsItemSpider)
+            yield crawler.crawl(mockserver=self.mockserver)
+
+        self.assertEqual(len(log.records), 0)
+
     @defer.inlineCallbacks
     def test_start_requests_laziness(self):
         settings = {"CONCURRENT_REQUESTS": 1}
diff --git a/tests/test_spidermiddleware.py b/tests/test_spidermiddleware.py
index 38ca8d95026..726f7d57d0f 100644
--- a/tests/test_spidermiddleware.py
+++ b/tests/test_spidermiddleware.py
@@ -1,5 +1,5 @@
 import collections.abc
-from typing import Optional
+from typing import Optional, Union
 from unittest import mock
 
 from testfixtures import LogCapture
@@ -328,12 +328,13 @@ def process_start_requests(self, start_requests, spider):
 class ProcessStartRequestsSimple(BaseAsyncSpiderMiddlewareTestCase):
     """process_start_requests tests for simple start_requests"""
 
-    ITEM_TYPE = Request
+    ITEM_TYPE = Union[Request, dict]
     MW_SIMPLE = ProcessStartRequestsSimpleMiddleware
 
     def _start_requests(self):
-        for i in range(3):
+        for i in range(2):
             yield Request(f"https://example.com/{i}", dont_filter=True)
+        yield {"name": "test item"}
 
     @defer.inlineCallbacks
     def _get_middleware_result(self, *mw_classes, start_index: Optional[int] = None):


Code review comments:
diff_hunk: @@ -328,12 +328,13 @@ def process_start_requests(self, start_requests, spider):
 class ProcessStartRequestsSimple(BaseAsyncSpiderMiddlewareTestCase):
     """process_start_requests tests for simple start_requests"""
 
-    ITEM_TYPE = Request
+    ITEM_TYPE = Union[Request, dict]
path: tests/test_spidermiddleware.py
text: @user1:
Looks like this doesn't work on 3.8-3.9 because `isinstance()` can't use it. 

https://bugs.python.org/issue44529

@author:
Changing this to tuple `ITEM_TYPE = (Request, dict)` resolve issues with `isinstance()` but it cause failing of `checks (3.8 typing-tests)`

@user2:
Does it work if you change the typing tests to 3.9? If so, we can expedite https://github.com/scrapy/scrapy/issues/6466.

@user1:
I doubt that, `isinstance()` supporting a tuple is unique to it, you cannot use a tuple instead of a union.

@user1:
(I haven't looked at this code yet but I suspect we need to rewrite it, e.g. by rewriting the assert)

@author:
> I doubt that, isinstance() supporting a tuple is unique to it, you cannot use a tuple instead of a union.

It is directly mentioned on python docs that it support tuples as argument
https://docs.python.org/3.8/library/functions.html?highlight=tuple#isinstance
https://docs.python.org/3.9/library/functions.html?highlight=tuple#isinstance
https://docs.python.org/3.10/library/functions.html?highlight=tuple#isinstance
https://docs.python.org/3.11/library/functions.html?highlight=tuple#isinstance

built in `issubclass` - also supports tuple for its checks as well as `isinstance` so it is not unique.
As it is directly mentioned on docs - I think it's valid to use tuple here.

@user1:
I've meant that you can use a tuple for isinstance() but not for type hints. But I've now checked the actual failure, it's "Incompatible types in assignment (expression has type "Tuple[Type[Request], Type[Dict[Any, Any]]]", base class "BaseAsyncSpiderMiddlewareTestCase" defined the type as "type")" and it should be possible to solve it by updating the type hint of `BaseAsyncSpiderMiddlewareTestCase.ITEM_TYPE` to e.g. `Union[type, Union]` or something like that.