Repository: scrapy/scrapy
Pull request: Remove deprecated spider parameter
Closes #5588

Problem statement:
Scrapy warns about itself
I'm using Scrapy 2.6.2. Created a new project, run a spider, and there is 

```
2022-08-02 23:50:26 [py.warnings] WARNING: [redacted]/python3.8/site-packages/scrapy/core/engine.py:279: ScrapyDeprecationWarning: Passing a 'spider' argument to ExecutionEngine.download is deprecated
  return self.download(result, spider) if isinstance(result, Request) else result
```

message in logs. We also have the same warning in our testing suite, but I haven't realized it's not a test-only issue.

Hints from issue discussion:

Commit message: Fix CI error in 3.7 checks

Patch to review:
diff --git a/scrapy/core/engine.py b/scrapy/core/engine.py
index 6602f661d2e..edf9a3adea3 100644
--- a/scrapy/core/engine.py
+++ b/scrapy/core/engine.py
@@ -257,9 +257,7 @@ def _schedule_request(self, request: Request, spider: Spider) -> None:
 
     def download(self, request: Request, spider: Optional[Spider] = None) -> Deferred:
         """Return a Deferred which fires with a Response as result, only downloader middlewares are applied"""
-        if spider is None:
-            spider = self.spider
-        else:
+        if spider is not None:
             warnings.warn(
                 "Passing a 'spider' argument to ExecutionEngine.download is deprecated",
                 category=ScrapyDeprecationWarning,
@@ -267,9 +265,9 @@ def download(self, request: Request, spider: Optional[Spider] = None) -> Deferre
             )
             if spider is not self.spider:
                 logger.warning("The spider '%s' does not match the open spider", spider.name)
-        if spider is None:
+        if self.spider is None:
             raise RuntimeError(f"No open spider to crawl: {request}")
-        return self._download(request, spider).addBoth(self._downloaded, request, spider)
+        return self._download(request, spider).addBoth(self._downloaded, request, spider)  # type: ignore
 
     def _downloaded(
         self, result: Union[Response, Request], request: Request, spider: Spider


Code review comments:
diff_hunk: @@ -257,19 +257,17 @@ def _schedule_request(self, request: Request, spider: Spider) -> None:
 
     def download(self, request: Request, spider: Optional[Spider] = None) -> Deferred:
         """Return a Deferred which fires with a Response as result, only downloader middlewares are applied"""
-        if spider is None:
-            spider = self.spider
-        else:
+        if spider is not None:
             warnings.warn(
                 "Passing a 'spider' argument to ExecutionEngine.download is deprecated",
                 category=ScrapyDeprecationWarning,
                 stacklevel=2,
             )
             if spider is not self.spider:
                 logger.warning("The spider '%s' does not match the open spider", spider.name)
-        if spider is None:
+        if self.spider is None:
             raise RuntimeError(f"No open spider to crawl: {request}")
-        return self._download(request, spider).addBoth(self._downloaded, request, spider)
+        return self._download(request, spider).addBoth(self._downloaded, request, spider)  # type: ignore
path: scrapy/core/engine.py
text: @user1:
I see…

It looks like we should change the typing of `Spider` in those 2 methods to `Optional[Spider]`, instead of ignoring the typing issue.

It also seems like `_download` should get the `if spider is None: spider = self.spider` logic.

@author:
Yeah i think the same, done.