Repository: secdev/scapy

Issue:
srp is missing response (Centos 5.11)
**Brief description**

Sending a simple TCP SYN packet results in no response according to scapy.

How is decided if a packet is a response to another?

**Environment**
- Python 3.6.1 (default, Jul 19 2019, 17:04:00)
- Scapy Version 2.4.3
- Linux hostname 2.6.18-398.el5 #1 SMP Tue Sep 16 20:50:52 EDT 2014 x86_64 x86_64 x86_64 GNU/Linux

**How to reproduce**
```
>>> tcp_packet = Ether(src="00:50:56:AF:A8:83", dst="00:50:56:AF:2F:87")/IP(src="101.100.100.66", dst="201.200.0.64")/TCP(sport=333, dport=12345, flags="S")
>>> ans, no_ans = srp(tcp_packet, iface="eth1", timeout=5)
Begin emission:
Finished sending 1 packets.

Received 0 packets, got 0 answers, remaining 1 packets
>>> ans.show()
>>> no_ans.show()
0000 Ether / IP / TCP 101.100.100.66:texar > 201.200.0.64:italk S
>>>
```

**Actual result**
- No Answer

**Expected result**
- Answer

**Related resources**
[scapy_srp_missing_response.pcap.zip](https://github.com/secdev/scapy/files/4127937/scapy_srp_missing_response.pcap.zip)



Discussion:
Did you retrieve the PCAP file from `eth1`? Could you activate `conf.debug_match = True`, then your packet, then share the output of `debug.recv.show() `?

Scapy identifies the second frame as an answer to the first one:
```
>>> l = rdpcap("scapy_srp_missing_response.pcap")
>>> l[1].answers(l[0]) 
```
The PCAP is indeed from eth1.

```
Python 3.6.1 (default, Jul 19 2019, 17:04:00)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from scapy.all import *
>>> conf.debug_match = True
>>> tcp_packet = Ether(src="00:50:56:AF:A8:83", dst="00:50:56:AF:2F:87")/IP(src="101.100.100.66", dst="201.200.0.64")/TCP(sport=333, dport=12345, flags="S")
>>> ans, no_ans = srp(tcp_packet, iface="eth1", timeout=5)
Begin emission:
Finished sending 1 packets.

Received 0 packets, got 0 answers, remaining 1 packets
>>> ans.show()
>>> no_ans.show()
0000 Ether / IP / TCP 101.100.100.66:texar > 201.200.0.64:italk S
>>> debug.recv.show()
>>> 
```
It looks like that the packet is not reached by Scapy. It should at least be in `debug.recv`. Do you have a similar behavior with the master branch in the interactive mode (i.e. using `./run_scapy`)?
The master branch fails differently:
```
>>> from scapy.all import *
>>> conf.debug_match = True
>>> tcp_packet = Ether(src="00:50:56:AF:A8:83", dst="00:50:56:AF:2F:87")/IP(src="101.100.100.66", dst="201.200.0.64")/TCP(sport=334, dport=12346, flags="S")
>>> ans, no_ans = srp(tcp_packet, iface="eth1", timeout=5)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/root/scapy/scapy/sendrecv.py", line 555, in srp
    filter=filter, nofilter=nofilter, type=type)
  File "/root/scapy/scapy/arch/linux.py", line 468, in __init__
    1
OSError: [Errno 92] Protocol not available
>>>
```
After commenting out the problematic setsockopt line that caused the `Protocol no available` error, I get almost the same results as before (Now it also prints this warning):
```
>>> from scapy.all import *
>>> conf.debug_match = True
>>> tcp_packet = Ether(src="00:50:56:AF:A8:83", dst="00:50:56:AF:2F:87")/IP(src="101.100.100.66", dst="201.200.0.64")/TCP(sport=334, dport=12346, flags="S")
>>> ans, no_ans = srp(tcp_packet, iface="eth1", timeout=5)
Begin emission:
Finished sending 1 packets.
WARNING: Socket <scapy.arch.linux.L2Socket object at 0x2b1f3913d5c0> failed with 'Buffer size too small (16 instead of at least 20 bytes)'. It was closed.

Received 0 packets, got 0 answers, remaining 1 packets
>>> ans.show()
>>> no_ans.show()
0000 Ether / IP / TCP 101.100.100.66:334 > 201.200.0.64:12346 S
>>> debug.recv.show()
>>>
```
Does this trigger the issue `s = L2Socket(iface=conf.iface, type=ETH_P_ALL); s.recv_raw(x=1024)`?
indeed (again, only after commenting out the unrelated bug in master)
Could you copy/paste the complete output? I am trying to identify the cause of the issue.
The traceback under 2.4.3:
```
>>> s = L2Socket(iface=conf.iface, type=ETH_P_ALL); s.recv_raw(x=1024)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "python3.6/site-packages/scapy/arch/linux.py", line 551, in recv_raw
    pkt, sa_ll = self._recv_raw(self.ins, x)
  File "python3.6/site-packages/scapy/arch/linux.py", line 537, in _recv_raw
    auxdata = tpacket_auxdata.from_buffer_copy(cmsg_data)
ValueError: Buffer size too small (16 instead of at least 20 bytes)
```

The traceback under master:
```
>>> s = L2Socket(iface=conf.iface, type=ETH_P_ALL); s.recv_raw(x=1024)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/root/scapy/scapy/arch/linux.py", line 503, in recv_raw
    pkt, sa_ll, ts = self._recv_raw(self.ins, x)
  File "/root/scapy/scapy/supersocket.py", line 97, in _recv_raw
    auxdata = tpacket_auxdata.from_buffer_copy(cmsg_data)
ValueError: Buffer size too small (16 instead of at least 20 bytes)
```
It seems that your issue is related to your kernel version and the support of VLAN in Scapy. According to http://manpages.ubuntu.com/manpages/bionic/man7/packet.7.html, your Linux 2.6.18 won't work. Could you share the output of these variables https://github.com/secdev/scapy/blob/master/scapy/supersocket.py#L90 ?
```
>>> from scapy.all import *
>>> s = L2Socket(iface=conf.iface, type=ETH_P_ALL); s.recv_raw(x=1024)
> /root/scapy/scapy/supersocket.py(93)_recv_raw()
-> if not pkt:
(Pdb) pkt
b'\xff\xff\xff\xff\xff\xff\x00PV\xafJ\xac\x08\x06\x00\x01\x08\x00\x06\x04\x00\x01\x00PV\xafJ\xac\n\x00\x00\x9e\x00\x00\x00\x00\x00\x00\n\x00\x01O\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
(Pdb) ancdata
[(263, 8, b'\x01\x00\x00\x00<\x00\x00\x00<\x00\x00\x00\x00\x00\x0e\x00')]
(Pdb) flags
0
(Pdb) sa_ll
('eth0', 2054, 1, 1, b'\x00PV\xafJ\xac')
(Pdb)
```
Thanks. Does replacing the line with `pkt, ancdata, flags, sa_ll = sock.recvmsg(x)` works?
Yes! And also solves the original flow!
Can you confirm that the only thing that you modified is removing `flags_len`?
Yes, exactly as you requested.
Thanks. I will try to provide a fix to master soon.
Thanks! Please take in mind also the not compatible setsockopt.
Yes, you have indeed raised two issues:
1. setsockopt
2. ancillary data

Does this patch work for the second one?
```
diff --git a/scapy/supersocket.py b/scapy/supersocket.py
index b8c4cd04..4e9ee239 100644
--- a/scapy/supersocket.py
+++ b/scapy/supersocket.py
@@ -94,7 +94,10 @@ class SuperSocket(six.with_metaclass(_SuperSocket_metaclass)):
                 # Check available ancillary data
                 if (cmsg_lvl == SOL_PACKET and cmsg_type == PACKET_AUXDATA):
                     # Parse AUXDATA
-                    auxdata = tpacket_auxdata.from_buffer_copy(cmsg_data)
+                    try:
+                        auxdata = tpacket_auxdata.from_buffer_copy(cmsg_data)
+                    except ValueError:
+                        return pkt, sa_ll, time.time()
                     if auxdata.tp_vlan_tci != 0 or \
                             auxdata.tp_status & TP_STATUS_VLAN_VALID:
                         # Insert VLAN tag
```
Here is the patch for the `setsockopt` issue:
```
diff --git a/scapy/arch/linux.py b/scapy/arch/linux.py
index a6a70b98..12058e60 100644
--- a/scapy/arch/linux.py
+++ b/scapy/arch/linux.py
@@ -461,12 +461,15 @@ class L2Socket(SuperSocket):
         )
         if not six.PY2:
             # Receive Auxiliary Data (VLAN tags)
-            self.ins.setsockopt(SOL_PACKET, PACKET_AUXDATA, 1)
-            self.ins.setsockopt(
-                socket.SOL_SOCKET,
-                SO_TIMESTAMPNS,
-                1
-            )
+            try:
+               self.ins.setsockopt(SOL_PACKET, PACKET_AUXDATA, 1)
+               self.ins.setsockopt(
+                   socket.SOL_SOCKET,
+                   SO_TIMESTAMPNS,
+                   1
+               )
+            except OSError:
+                warning("Your Linux Kernel does not support Auxiliary Data!")
         if isinstance(self, L2ListenSocket):
             self.outs = None
         else:
```
Once you confirm the fixes, I will provide a PR.
Both patches work!
```
>>> from scapy.all import *
>>> conf.debug_match = True
>>> tcp_packet = Ether(src="00:50:56:AF:A8:83", dst="00:50:56:AF:2F:87")/IP(src="101.100.100.66", dst="201.200.0.64")/TCP(sport=334, dport=12346, flags="S")
>>> ans, no_ans = srp(tcp_packet, iface="eth1", timeout=5)
WARNING: Your Linux Kernel does not support Auxiliary Data!
Begin emission:
Finished sending 1 packets.
*
Received 1 packets, got 1 answers, remaining 0 packets
>>> ans.show()
0000 Ether / IP / TCP 101.100.100.66:334 > 201.200.0.64:12346 S ==> Ether / IP / TCP 201.200.0.64:12346 > 101.100.100.66:334 RA / Padding
>>>
```

Resolving patch:
diff --git a/scapy/arch/linux.py b/scapy/arch/linux.py
--- a/scapy/arch/linux.py
+++ b/scapy/arch/linux.py
@@ -31,7 +31,7 @@
     SO_TIMESTAMPNS
 from scapy.supersocket import SuperSocket
 from scapy.error import warning, Scapy_Exception, \
-    ScapyInvalidPlatformException
+    ScapyInvalidPlatformException, log_runtime
 from scapy.arch.common import get_if, compile_filter
 import scapy.modules.six as six
 from scapy.modules.six.moves import range
@@ -461,12 +461,19 @@ def __init__(self, iface=None, type=ETH_P_ALL, promisc=None, filter=None,
         )
         if not six.PY2:
             # Receive Auxiliary Data (VLAN tags)
-            self.ins.setsockopt(SOL_PACKET, PACKET_AUXDATA, 1)
-            self.ins.setsockopt(
-                socket.SOL_SOCKET,
-                SO_TIMESTAMPNS,
-                1
-            )
+            try:
+                self.ins.setsockopt(SOL_PACKET, PACKET_AUXDATA, 1)
+                self.ins.setsockopt(
+                    socket.SOL_SOCKET,
+                    SO_TIMESTAMPNS,
+                    1
+                )
+                self.auxdata_available = True
+            except OSError:
+                # Note: Auxiliary Data is only supported since
+                #       Linux 2.6.21
+                msg = "Your Linux Kernel does not support Auxiliary Data!"
+                log_runtime.info(msg)
         if isinstance(self, L2ListenSocket):
             self.outs = None
         else:
diff --git a/scapy/supersocket.py b/scapy/supersocket.py
--- a/scapy/supersocket.py
+++ b/scapy/supersocket.py
@@ -61,6 +61,7 @@ class SuperSocket(six.with_metaclass(_SuperSocket_metaclass)):
     closed = 0
     nonblocking_socket = False
     read_allowed_exceptions = ()
+    auxdata_available = False
 
     def __init__(self, family=socket.AF_INET, type=socket.SOCK_STREAM, proto=0):  # noqa: E501
         self.ins = socket.socket(family, type, proto)
@@ -85,16 +86,26 @@ def _recv_raw(self, sock, x):
             """Internal function to receive a Packet,
             and process ancillary data.
             """
-            flags_len = socket.CMSG_LEN(4096)
             timestamp = None
+            if not self.auxdata_available:
+                pkt, _, _, sa_ll = sock.recvmsg(x)
+                return pkt, sa_ll, timestamp
+            flags_len = socket.CMSG_LEN(4096)
             pkt, ancdata, flags, sa_ll = sock.recvmsg(x, flags_len)
             if not pkt:
-                return pkt, sa_ll
+                return pkt, sa_ll, timestamp
             for cmsg_lvl, cmsg_type, cmsg_data in ancdata:
                 # Check available ancillary data
                 if (cmsg_lvl == SOL_PACKET and cmsg_type == PACKET_AUXDATA):
                     # Parse AUXDATA
-                    auxdata = tpacket_auxdata.from_buffer_copy(cmsg_data)
+                    try:
+                        auxdata = tpacket_auxdata.from_buffer_copy(cmsg_data)
+                    except ValueError:
+                        # Note: according to Python documentation, recvmsg()
+                        #       can return a truncated message. A ValueError
+                        #       exception likely indicates that Auxiliary
+                        #       Data is not supported by the Linux kernel.
+                        return pkt, sa_ll, timestamp
                     if auxdata.tp_vlan_tci != 0 or \
                             auxdata.tp_status & TP_STATUS_VLAN_VALID:
                         # Insert VLAN tag
@@ -213,13 +224,20 @@ def __init__(self, type=ETH_P_IP, filter=None, iface=None, promisc=None, nofilte
         if iface is not None:
             self.ins.bind((iface, type))
         if not six.PY2:
-            # Receive Auxiliary Data (VLAN tags)
-            self.ins.setsockopt(SOL_PACKET, PACKET_AUXDATA, 1)
-            self.ins.setsockopt(
-                socket.SOL_SOCKET,
-                SO_TIMESTAMPNS,
-                1
-            )
+            try:
+                # Receive Auxiliary Data (VLAN tags)
+                self.ins.setsockopt(SOL_PACKET, PACKET_AUXDATA, 1)
+                self.ins.setsockopt(
+                    socket.SOL_SOCKET,
+                    SO_TIMESTAMPNS,
+                    1
+                )
+                self.auxdata_available = True
+            except OSError:
+                # Note: Auxiliary Data is only supported since
+                #       Linux 2.6.21
+                msg = "Your Linux Kernel does not support Auxiliary Data!"
+                log_runtime.info(msg)
 
     def recv(self, x=MTU):
         pkt, sa_ll, ts = self._recv_raw(self.ins, x)


Test patch: