Repository: bookwyrm-social/bookwyrm

Issue:
OWASP Core Rule Set 913101
**Describe the bug**
BookWyrm's user agent is blocked by an OWASP-compliant web application firewall (WAF) for violating rule 913101. No other fediverse applications violate this rule.


**To Reproduce**
This issue is not reproducible between normal servers and clients.


**Expected behavior**
The WAF allows communication.


**Screenshots**
`python-requests/2.31.0 (BookWyrm/0.6.6; +https://bookwyrm.social/)`
```
[Thu Nov 09 04:13:56.824444 2023] [security2:error] [pid 2117:tid 140508772919040] [client 143.110.147.80:53962] [client 143.110.147.80] ModSecurity: Warning. Matched phrase "python-requests" at REQUEST_HEADERS:User-Agent. [file "/usr/apache/conf/waf/rules/REQUEST-913-SCANNER-DETECTION.conf"] [line "143"] [id "913101"] [msg "Found User-Agent associated with scripting/generic HTTP client"] [data "Matched Data: python-requests found within REQUEST_HEADERS:User-Agent: python-requests/2.31.0 (bookwyrm/0.6.6; +https://bookwyrm.social/)"] [severity "CRITICAL"] [ver "OWASP_CRS/3.3.3"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-reputation-scripting"] [tag "OWASP_CRS"] [tag "capec/1000/118/224/541/310"] [tag "PCI/6.5.10"] [tag "paranoia-level/2"] [hostname "muri.network"] [uri "/users/Yae"] [unique_id "ZUxchFymkmHm47qNPINTzgAAAKI"]
[Thu Nov 09 04:13:56.824875 2023] [security2:error] [pid 2117:tid 140508772919040] [client 143.110.147.80:53962] [client 143.110.147.80] ModSecurity: Access denied with code 403 (phase 2). Operator GE matched 5 at TX:anomaly_score. [file "/usr/apache/conf/waf/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "94"] [id "949110"] [msg "Inbound Anomaly Score Exceeded (Total Score: 5)"] [severity "CRITICAL"] [ver "OWASP_CRS/3.3.3"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [hostname "muri.network"] [uri "/users/Yae"] [unique_id "ZUxchFymkmHm47qNPINTzgAAAKI"]
[Thu Nov 09 04:13:56.825023 2023] [security2:error] [pid 2117:tid 140508772919040] [client 143.110.147.80:53962] [client 143.110.147.80] ModSecurity: Warning. Operator GE matched 5 at TX:inbound_anomaly_score. [file "/usr/apache/conf/waf/rules/RESPONSE-980-CORRELATION.conf"] [line "92"] [id "980130"] [msg "Inbound Anomaly Score Exceeded (Total Inbound Score: 5 - SQLI=0,XSS=0,RFI=0,LFI=0,RCE=0,PHPI=0,HTTP=0,SESS=0): individual paranoia level scores: 0, 5, 0, 0"] [ver "OWASP_CRS/3.3.3"] [tag "event-correlation"] [hostname "muri.network"] [uri "/users/Yae"] [unique_id "ZUxchFymkmHm47qNPINTzgAAAKI"]
```


**Instance**
bookwyrm.social and another all servers.


**Additional context**
The Bookwyrm server security staff may allow 913101 for communication between Bookwyrm servers.

This becomes a problem when Bookwyrm servers send communication requests to other fediverse application servers.

Script commands that violate 913101 are putting requests to most fediverse servers, and most fediverse servers do not violate 913101. 

Security staff on other fediverse servers are unlikely to exclude 913101 from their WAFs, which limits Bookwyrm's federated communications.

---




Discussion:
Rule 913101 appears to be "User-Agent associated with scripting or generic HTTP client", though I don't think it's right to say BookWyrm "violates" this rule, since it's simply a heuristic for identifying possible problematic agents. 

I'm guessing the problem is `python-requests/2.31.0` since the python requests library would be associated with lots of web crawlers etc.

We could adjust [the relevant setting](https://github.com/bookwyrm-social/bookwyrm/blob/ab9cea1742a799ea60d1c43cd6ccbac6c03e6b2d/bookwyrm/settings.py#L339):

```python
USER_AGENT = f"BookWyrm/{VERSION}; +https://{DOMAIN}/)"
```

@fofwisdom is there some kind of checking tool we can use to determine whether this would resolve the problem?
> though I don't think it's right to say BookWyrm "violates" this rule

Yes, you're right, these protective rules are called paranoia mode. It's like tapping on a stone bridge and trying to cross it.

There is testing documentation, but unfortunately, I have no experience using it. 

- https://coreruleset.org/docs/development/testing/

Thanks!


The sandbox is useful: https://coreruleset.org/docs/development/sandbox/

I can't get it to report rule 913101, but I _can_ get it to report a score above 10, and then report nothing if the `requests` info is removed from the User-Agent, so that seems like the issue. Given the app is actually BookWyrm and not `requests`, this actually seems like the correct value anyway.
> We could adjust [the relevant setting](https://github.com/bookwyrm-social/bookwyrm/blob/ab9cea1742a799ea60d1c43cd6ccbac6c03e6b2d/bookwyrm/settings.py#L339):
> 
> ```python
> USER_AGENT = f"BookWyrm/{VERSION}; +https://{DOMAIN}/)"
> ```

I'd suggest:

```python
USER_AGENT = f"BookWyrm (BookWyrm/{VERSION}; +https://{DOMAIN}/)"
```

to maintain compatibility with `BOOKWYRM_USER_AGENT` regex ([link](https://github.com/bookwyrm-social/bookwyrm/blob/06568aab88b3b099c42c8b7c9d426f523db4ffdb/bookwyrm/utils/regex.py#L12)).

-----

Additional context, since this is tagged _good first issue_: the regex above is used by `is_bookwyrm_request()` to detect whether an incoming request comes from a BookWyrm instance. Although the regex could be changed to match a USER_AGENT without brackets, this would not address the case of instances running older versions of BookWyrm, which would stop recognizing newer versions as their own.

Thanks!

Resolving patch:
diff --git a/bookwyrm/settings.py b/bookwyrm/settings.py
--- a/bookwyrm/settings.py
+++ b/bookwyrm/settings.py
@@ -347,8 +347,7 @@
 USE_TZ = True
 
 
-agent = requests.utils.default_user_agent()
-USER_AGENT = f"{agent} (BookWyrm/{VERSION}; +https://{DOMAIN}/)"
+USER_AGENT = f"BookWyrm (BookWyrm/{VERSION}; +https://{DOMAIN}/)"
 
 # Imagekit generated thumbnails
 ENABLE_THUMBNAIL_GENERATION = env.bool("ENABLE_THUMBNAIL_GENERATION", False)


Test patch: