Repository: secdev/scapy

Issue:
UDS Scanner - UDS_ServiceEnumerator tagging negative response as Timeout
### Brief description

![Screenshot from 2023-03-22 15-19-24](https://user-images.githubusercontent.com/115094878/226933609-883ff0b4-4660-449a-a47d-e0dfb4d71e26.png)


I have been running the UDS_ServiceEnumerator on an ECU and there are some SIDs whose scan results are always tagged as "Timeout" even though on wireshark, I can see the negative resp being given.

There are other SIDs which give the exactly same error response (as shown in the attached screenshot) but they are correctly tagged by the scanner as "NR: serviceNotSupported"
```

UDS_ServiceEnumerator_kwargs={ "scan_range": [0x40], "timeout": 10 }
                  
-------------+-----------------+
             | session1 | 
-------------+-----------------+
0x40-1: 0x40 | Timeout  | 
-------------+-----------------+
````

### Scapy version

43582d36fe202916fab4f36f0fd10d697eabd8a5

### Python version

3.10.6

### Operating system

5.15.0-67-generic #74-Ubuntu SMP Wed Feb 22 14:14:39 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

### Additional environment information

_No response_

### How to reproduce

Run UDS_ServiceEnumerator on a range of SIDs which are not supported.

### Actual result

_No response_

### Expected result

_No response_

### Related resources

_No response_
UDS Scanner - UDS_ServiceEnumerator tagging negative response as Timeout
### Brief description

![Screenshot from 2023-03-22 15-19-24](https://user-images.githubusercontent.com/115094878/226933609-883ff0b4-4660-449a-a47d-e0dfb4d71e26.png)


I have been running the UDS_ServiceEnumerator on an ECU and there are some SIDs whose scan results are always tagged as "Timeout" even though on wireshark, I can see the negative resp being given.

There are other SIDs which give the exactly same error response (as shown in the attached screenshot) but they are correctly tagged by the scanner as "NR: serviceNotSupported"
```

UDS_ServiceEnumerator_kwargs={ "scan_range": [0x40], "timeout": 10 }
                  
-------------+-----------------+
             | session1 | 
-------------+-----------------+
0x40-1: 0x40 | Timeout  | 
-------------+-----------------+
````

### Scapy version

43582d36fe202916fab4f36f0fd10d697eabd8a5

### Python version

3.10.6

### Operating system

5.15.0-67-generic #74-Ubuntu SMP Wed Feb 22 14:14:39 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

### Additional environment information

_No response_

### How to reproduce

Run UDS_ServiceEnumerator on a range of SIDs which are not supported.

### Actual result

_No response_

### Expected result

_No response_

### Related resources

_No response_


Discussion:
Hi, thanks for this issue. However, to diagnose your issue I would need a full code example, how you are starting the scanner.


Could you also send a full pcap file with the entire scan, so that I can diagnose the issue?
Hi... ofcourse... thank you for looking into it. 

pcap file is available at https://file.io/cFQBPTF8UvfM

stdout
```
python3 uds-scan2.py 
INFO: Routing activation successful! Target address set to: 0x3107
INFO: [i] Scan progress 0.00
INFO: Scan path [session1]
INFO: Target reset
INFO: Target reconnect
INFO: Routing activation successful! Target address set to: 0x3107
INFO: Execute <scapy.contrib.automotive.uds_scan.UDS_ServiceEnumerator object at 0x7f9b26b910> for path [session1]
INFO: [i] Scan progress 0.25
INFO: Scan path [session1]
INFO: Target reset
INFO: Target reconnect
INFO: Routing activation successful! Target address set to: 0x3107
INFO: Execute <scapy.contrib.automotive.uds_scan.UDS_ServiceEnumerator object at 0x7f9b26b910> for path [session1]
INFO: Finished iterator execution
INFO: [i] Scan progress 1.25
INFO: Scan path [session1]
INFO: Execute failure or scan completed. Exit scan!
INFO: Target reset
finished


============================================================
     Available services and negative response per state
------------------------------------------------------------


5 requests were sent, 0 answered, 5 unanswered
Statistics per state
-------------------+-----+----------+
                   | all | session1 | 
-------------------+-----+----------+
answertime_avg     | -   | -        | 
answertime_avg_nr  | -   | -        | 
answertime_avg_pr  | -   | -        | 
answertime_max     | -   | -        | 
answertime_max_nr  | -   | -        | 
answertime_max_pr  | -   | -        | 
answertime_min     | -   | -        | 
answertime_min_nr  | -   | -        | 
answertime_min_pr  | -   | -        | 
num_answered       | 0   | 0        | 
num_negative_resps | 0   | 0        | 
num_unanswered     | 5   | 5        | 
-------------------+-----+----------+


0 negative responses were received

These negative response codes were received 



-----------+----------+
           | session1 | 
-----------+----------+
0x40: 0x40 | Timeout  | 
0x41: 0x41 | Timeout  | 
0x42: 0x42 | Timeout  | 
0x43: 0x43 | Timeout  | 
-----------+----------+
```

actual code
```
from scapy.contrib.automotive.uds import *
from scapy.contrib.automotive.uds_scan import *
from scapy.contrib.automotive.doip import *


def reconnect():
    return UDS_DoIPSocket(ip="169.254.96.24",port=13400)

es = [UDS_ServiceEnumerator]

s = UDS_Scanner(reconnect(), reconnect_handler=reconnect,
                reset_handler=None, test_cases=es, debug=True,
                UDS_ServiceEnumerator_kwargs={
                    "scan_range": range(0x40, 0x44), "timeout": 10, "retry_if_none_received":True}
                )
try:
    s.scan()
except KeyboardInterrupt:
    pass

print("finished")

for t in s.configuration.test_cases:
     x = t.show(dump=True,filtered=False)
     print(x)

```

Thanks.. will have a look at it. However, scanning service 0x40 is not a usual case, since 0x40 is defined as Positive Response service ID, and using it for a request doesn't make much sense. 
understood.. what I want to do is just run a scan from 0x00 to 0xFF. Since the ecu is able to send a negative resp to it (which is tagged correctly by wireshark), I was expecting scapy to also tag it as such.
Otherwise it will be impossible to distinguish between the actual timeouts (where no response was sent/request was discarded) and these ones.
Hi, thanks for this issue. However, to diagnose your issue I would need a full code example, how you are starting the scanner.


Could you also send a full pcap file with the entire scan, so that I can diagnose the issue?
Hi... ofcourse... thank you for looking into it. 

pcap file is available at https://file.io/cFQBPTF8UvfM

stdout
```
python3 uds-scan2.py 
INFO: Routing activation successful! Target address set to: 0x3107
INFO: [i] Scan progress 0.00
INFO: Scan path [session1]
INFO: Target reset
INFO: Target reconnect
INFO: Routing activation successful! Target address set to: 0x3107
INFO: Execute <scapy.contrib.automotive.uds_scan.UDS_ServiceEnumerator object at 0x7f9b26b910> for path [session1]
INFO: [i] Scan progress 0.25
INFO: Scan path [session1]
INFO: Target reset
INFO: Target reconnect
INFO: Routing activation successful! Target address set to: 0x3107
INFO: Execute <scapy.contrib.automotive.uds_scan.UDS_ServiceEnumerator object at 0x7f9b26b910> for path [session1]
INFO: Finished iterator execution
INFO: [i] Scan progress 1.25
INFO: Scan path [session1]
INFO: Execute failure or scan completed. Exit scan!
INFO: Target reset
finished


============================================================
     Available services and negative response per state
------------------------------------------------------------


5 requests were sent, 0 answered, 5 unanswered
Statistics per state
-------------------+-----+----------+
                   | all | session1 | 
-------------------+-----+----------+
answertime_avg     | -   | -        | 
answertime_avg_nr  | -   | -        | 
answertime_avg_pr  | -   | -        | 
answertime_max     | -   | -        | 
answertime_max_nr  | -   | -        | 
answertime_max_pr  | -   | -        | 
answertime_min     | -   | -        | 
answertime_min_nr  | -   | -        | 
answertime_min_pr  | -   | -        | 
num_answered       | 0   | 0        | 
num_negative_resps | 0   | 0        | 
num_unanswered     | 5   | 5        | 
-------------------+-----+----------+


0 negative responses were received

These negative response codes were received 



-----------+----------+
           | session1 | 
-----------+----------+
0x40: 0x40 | Timeout  | 
0x41: 0x41 | Timeout  | 
0x42: 0x42 | Timeout  | 
0x43: 0x43 | Timeout  | 
-----------+----------+
```

actual code
```
from scapy.contrib.automotive.uds import *
from scapy.contrib.automotive.uds_scan import *
from scapy.contrib.automotive.doip import *


def reconnect():
    return UDS_DoIPSocket(ip="169.254.96.24",port=13400)

es = [UDS_ServiceEnumerator]

s = UDS_Scanner(reconnect(), reconnect_handler=reconnect,
                reset_handler=None, test_cases=es, debug=True,
                UDS_ServiceEnumerator_kwargs={
                    "scan_range": range(0x40, 0x44), "timeout": 10, "retry_if_none_received":True}
                )
try:
    s.scan()
except KeyboardInterrupt:
    pass

print("finished")

for t in s.configuration.test_cases:
     x = t.show(dump=True,filtered=False)
     print(x)

```

Thanks.. will have a look at it. However, scanning service 0x40 is not a usual case, since 0x40 is defined as Positive Response service ID, and using it for a request doesn't make much sense. 
understood.. what I want to do is just run a scan from 0x00 to 0xFF. Since the ecu is able to send a negative resp to it (which is tagged correctly by wireshark), I was expecting scapy to also tag it as such.
Otherwise it will be impossible to distinguish between the actual timeouts (where no response was sent/request was discarded) and these ones.

Resolving patch:
diff --git a/scapy/contrib/automotive/gm/gmlan.py b/scapy/contrib/automotive/gm/gmlan.py
--- a/scapy/contrib/automotive/gm/gmlan.py
+++ b/scapy/contrib/automotive/gm/gmlan.py
@@ -112,7 +112,7 @@ def answers(self, other):
 
     def hashret(self):
         if self.service == 0x7f:
-            return struct.pack('B', self.requestServiceId)
+            return struct.pack('B', self.requestServiceId & ~0x40)
         return struct.pack('B', self.service & ~0x40)
 
 
diff --git a/scapy/contrib/automotive/kwp.py b/scapy/contrib/automotive/kwp.py
--- a/scapy/contrib/automotive/kwp.py
+++ b/scapy/contrib/automotive/kwp.py
@@ -111,7 +111,7 @@ def answers(self, other):
     def hashret(self):
         # type: () -> bytes
         if self.service == 0x7f:
-            return struct.pack('B', self.requestServiceId)
+            return struct.pack('B', self.requestServiceId & ~0x40)
         else:
             return struct.pack('B', self.service & ~0x40)
 
diff --git a/scapy/contrib/automotive/uds.py b/scapy/contrib/automotive/uds.py
--- a/scapy/contrib/automotive/uds.py
+++ b/scapy/contrib/automotive/uds.py
@@ -115,7 +115,7 @@ def answers(self, other):
     def hashret(self):
         # type: () -> bytes
         if self.service == 0x7f:
-            return struct.pack('B', self.requestServiceId)
+            return struct.pack('B', self.requestServiceId & ~0x40)
         return struct.pack('B', self.service & ~0x40)
 
 


Test patch:
diff --git a/test/contrib/automotive/scanner/uds_scanner.uts b/test/contrib/automotive/scanner/uds_scanner.uts
--- a/test/contrib/automotive/scanner/uds_scanner.uts
+++ b/test/contrib/automotive/scanner/uds_scanner.uts
@@ -1070,6 +1070,26 @@ assert 0xff02 in ids
 assert 0xff03 in ids
 assert 0xffff in ids
 
+= UDS_ServiceEnumerator weird issue
+
+resps = [EcuResponse(None, [UDS()/UDS_NR(negativeResponseCode=0x13, requestServiceId=0x40)]),
+         EcuResponse(None, [UDS()/UDS_NR(negativeResponseCode="subFunctionNotSupported", requestServiceId=0x41)]),
+         EcuResponse(None, [UDS()/UDS_NR(negativeResponseCode="subFunctionNotSupported", requestServiceId=0x11)]),
+         EcuResponse(None, [UDS()/UDS_NR(negativeResponseCode="subFunctionNotSupported", requestServiceId=0x42)]),
+         EcuResponse(None, [UDS()/UDS_NR(negativeResponseCode="subFunctionNotSupported", requestServiceId=0x43)])]
+
+es = [UDS_ServiceEnumerator]
+
+scanner = executeScannerInVirtualEnvironment(
+    resps, es, UDS_ServiceEnumerator_kwargs={"scan_range": [0x11, 0x40, 0x41, 0x42]})
+
+assert scanner.scan_completed
+assert scanner.progress() > 0.95
+tc = scanner.configuration.test_cases[0]
+tc.show()
+
+assert len(tc.results_with_negative_response) == 4
+
 + Cleanup
 
 = Delete testsockets