Repository: scrapy/scrapy
Pull request #6740 (2025-03-19 21:01:15)

feat: Add count to spider_exceptions stats
close: #6739

codecov[bot]: ## [Codecov](https://app.codecov.io/gh/scrapy/scrapy/pull/6740?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scrapy) Report
Attention: Patch coverage is `0%` with `1 line` in your changes missing coverage. Please review.
> Project coverage is 33.49%. Comparing base [(`8729247`)](https://app.codecov.io/gh/scrapy/scrapy/commit/872924721344502cca92a72ce68d5f78a9b9e5bc?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scrapy) to head [(`d85145b`)](https://app.codecov.io/gh/scrapy/scrapy/commit/d85145bf6bbaf28b9d9e59d468467fca9e267bb8?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scrapy).

| [Files with missing lines](https://app.codecov.io/gh/scrapy/scrapy/pull/6740?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scrapy) | Patch % | Lines |
|---|---|---|
| [scrapy/core/scraper.py](https://app.codecov.io/gh/scrapy/scrapy/pull/6740?src=pr&el=tree&filepath=scrapy%2Fcore%2Fscraper.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scrapy#diff-c2NyYXB5L2NvcmUvc2NyYXBlci5weQ==) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/scrapy/scrapy/pull/6740?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scrapy) |

<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff             @@
##           master    #6740       +/-   ##
===========================================
- Coverage   90.67%   33.49%   -57.19%     
===========================================
  Files         162      162               
  Lines       12129    12122        -7     
  Branches     1567     1567               
===========================================
- Hits        10998     4060     -6938     
- Misses        829     7924     +7095     
+ Partials      302      138      -164     
```

| [Files with missing lines](https://app.codecov.io/gh/scrapy/scrapy/pull/6740?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scrapy) | Coverage Δ | |
|---|---|---|
| [scrapy/core/scraper.py](https://app.codecov.io/gh/scrapy/scrapy/pull/6740?src=pr&el=tree&filepath=scrapy%2Fcore%2Fscraper.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scrapy#diff-c2NyYXB5L2NvcmUvc2NyYXBlci5weQ==) | `24.08% <0.00%> (-64.34%)` | :arrow_down: |

... and [145 files with indirect coverage changes](https://app.codecov.io/gh/scrapy/scrapy/pull/6740/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scrapy)

</details>
<details><summary>🚀 New features to boost your workflow: </summary>

- ❄ [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>

Patch:
diff --git a/scrapy/core/scraper.py b/scrapy/core/scraper.py
index b664b61f6..496adb500 100644
--- a/scrapy/core/scraper.py
+++ b/scrapy/core/scraper.py
@@ -246,6 +246,7 @@ class Scraper:
             spider=spider,
         )
         assert self.crawler.stats
+        self.crawler.stats.inc_value("spider_exceptions/count", spider=spider)
         self.crawler.stats.inc_value(
             f"spider_exceptions/{_failure.value.__class__.__name__}", spider=spider
         )


Test patch:
diff --git a/tests/test_closespider.py b/tests/test_closespider.py
index 476662789..4a17b254b 100644
--- a/tests/test_closespider.py
+++ b/tests/test_closespider.py
@@ -88,6 +88,7 @@ class TestCloseSpider(TestCase):
         assert reason == "closespider_errorcount"
         key = f"spider_exceptions/{crawler.spider.exception_cls.__name__}"
         errorcount = crawler.stats.get_value(key)
+        assert crawler.stats.get_value("spider_exceptions/count") >= close_on
         assert errorcount >= close_on
 
     @defer.inlineCallbacks