Repository: secdev/scapy

Issue:
100% CPU load with Pipetools on 2.4.5
#### Brief description

While sniffing for probe requests on a wireless interface on a Raspberry Pi 4 Model B Rev 1.4, one CPU core is constantly 100% busy.

#### Environment

-   Scapy version: 2.4.5 / master branch
-   Python version: 3.7.3
-   Operating System: Raspbian GNU/Linux 10 (buster)

#### How to reproduce

1. Put the wireless interface in monitor mode:
```bash
ip link set wlanX down
iw dev wlanX set type monitor
ip link set wlanX up
```

2. Start sniffing for probe requests:
```python
source = SniffSource(iface='wlanX', filter='type mgt subtype probe-req')
sink = ConsoleSink()
source > sink

p = PipeEngine(source)
p.start()
p.wait_and_stop()
```

#### Actual result

Now start top in another session and watch CPU load:
```
top - 20:50:42 up 6 days,  2:46,  2 users,  load average: 0.74, 0.24, 0.09
Tasks: 120 total,   1 running, 119 sleeping,   0 stopped,   0 zombie
%Cpu0  : 66.1 us, 33.6 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.3 si,  0.0 st
%Cpu1  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu2  :  3.0 us,  0.0 sy,  0.0 ni, 97.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu3  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :   7875.3 total,   7303.0 free,     97.1 used,    475.1 buff/cache
MiB Swap:    100.0 total,    100.0 free,      0.0 used.   7526.4 avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
26761 root      20   0   44948  27364   9028 S 100.0   0.3   1:21.48 scapy
...
```

#### Expected result

Expected CPU load to be far less than 100%.

#### Additional information
1. Downgrading to 2.4.4 solves the issue. CPU load is just a few percent, depending on how many frames are actually received by the frame filter
2. Both 2.4.5 and master branch expose this issue
3. Only dissecting the packets with scapy 2.4.5 (using pcapy to capture with frame filter) does not show the same CPU load. Tested with:
```python
import pcapy
from scapy.all import *
import sys

def pkt_callback(hdr, data):
    rt = RadioTap(data)
    rt.show()
    pr = rt.getlayer(Dot11ProbeReq)
    print(f"type:{rt.type} subtype:{rt.subtype} ts:{rt.time} signal:{rt.dBm_AntSignal}dBm rate:{rt.Rate} ant:{rt.Antenna} s_mac:{rt.addr2} BSSID:{pr.info.decode('utf-8')}")

def main():
    packets = pcapy.open_live(sys.argv[1], 1514, 1, 10)
    packets.setfilter('subtype probe-req')
    packets.loop(-1, pkt_callback)

if __name__ == '__main__':
    main()
```

100% CPU load with Pipetools on 2.4.5
#### Brief description

While sniffing for probe requests on a wireless interface on a Raspberry Pi 4 Model B Rev 1.4, one CPU core is constantly 100% busy.

#### Environment

-   Scapy version: 2.4.5 / master branch
-   Python version: 3.7.3
-   Operating System: Raspbian GNU/Linux 10 (buster)

#### How to reproduce

1. Put the wireless interface in monitor mode:
```bash
ip link set wlanX down
iw dev wlanX set type monitor
ip link set wlanX up
```

2. Start sniffing for probe requests:
```python
source = SniffSource(iface='wlanX', filter='type mgt subtype probe-req')
sink = ConsoleSink()
source > sink

p = PipeEngine(source)
p.start()
p.wait_and_stop()
```

#### Actual result

Now start top in another session and watch CPU load:
```
top - 20:50:42 up 6 days,  2:46,  2 users,  load average: 0.74, 0.24, 0.09
Tasks: 120 total,   1 running, 119 sleeping,   0 stopped,   0 zombie
%Cpu0  : 66.1 us, 33.6 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.3 si,  0.0 st
%Cpu1  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu2  :  3.0 us,  0.0 sy,  0.0 ni, 97.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu3  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :   7875.3 total,   7303.0 free,     97.1 used,    475.1 buff/cache
MiB Swap:    100.0 total,    100.0 free,      0.0 used.   7526.4 avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
26761 root      20   0   44948  27364   9028 S 100.0   0.3   1:21.48 scapy
...
```

#### Expected result

Expected CPU load to be far less than 100%.

#### Additional information
1. Downgrading to 2.4.4 solves the issue. CPU load is just a few percent, depending on how many frames are actually received by the frame filter
2. Both 2.4.5 and master branch expose this issue
3. Only dissecting the packets with scapy 2.4.5 (using pcapy to capture with frame filter) does not show the same CPU load. Tested with:
```python
import pcapy
from scapy.all import *
import sys

def pkt_callback(hdr, data):
    rt = RadioTap(data)
    rt.show()
    pr = rt.getlayer(Dot11ProbeReq)
    print(f"type:{rt.type} subtype:{rt.subtype} ts:{rt.time} signal:{rt.dBm_AntSignal}dBm rate:{rt.Rate} ant:{rt.Antenna} s_mac:{rt.addr2} BSSID:{pr.info.decode('utf-8')}")

def main():
    packets = pcapy.open_live(sys.argv[1], 1514, 1, 10)
    packets.setfilter('subtype probe-req')
    packets.loop(-1, pkt_callback)

if __name__ == '__main__':
    main()
```



Discussion:
The same thing happens when testing this locally on my laptop:

- Architecture: x86_64
- Python version: 3.9.4
- Operating System: Manjaro 21.0.3 / kernel 5.4.114-1-MANJARO

So it does not seem to be hardware or platform related.
Thanks for this report. I could confirm the bug on my system as well. I performed a git bisect test to identify the commit which caused the bug. 
@gpotter2 This bug is introduced in this commit: 880d0487c6885a5ddd0c5f747f6a9e0461a35278

I've tested this bug on 5.10.19-1-MANJARO. 
Thanks! Looking at the commit, it sets to_ms to 0 when calling open_pcap(). According to [man pcap_set_timeout](https://www.tcpdump.org/manpages/pcap_set_timeout.3pcap.html):

> _The behavior, if the timeout isn't specified, is undefined, as is
> the behavior if the timeout is set to zero or to a negative
> value.  We recommend always setting the timeout to a non-zero
> value unless immediate mode is set, in which case the timeout has
> no effect._

Immediate mode does not seem to be set anywhere.
That was reverted in a later commit: https://github.com/secdev/scapy/commit/2b96a8e3a48003c0b4a33a88a2ad570549058b6c not the issue here.
Thanks for the note polybassa I will have a look. TBH I'm rather intrigued, the commit you pointed out aimed at rewriting the behavior on Windows.. It shouldn't have changed anything on linux :/ I might have messed up something
Does this only happen when using `Pipetools` or also when using a plain `sniff` ?
If it's the former, try reverting this https://github.com/secdev/scapy/commit/880d0487c6885a5ddd0c5f747f6a9e0461a35278#diff-1bc4cea8a97434331b7833c4a4e720c5160e339759c73b9490d9c6277161871aR111
I've tested pure sniff ```sniff(iface="wlan0", timeout=100)```, that doesn't cause any cpu load
Thanks. Could you try the suggested fix?
No cpu load with your fix
The same thing happens when testing this locally on my laptop:

- Architecture: x86_64
- Python version: 3.9.4
- Operating System: Manjaro 21.0.3 / kernel 5.4.114-1-MANJARO

So it does not seem to be hardware or platform related.
Thanks for this report. I could confirm the bug on my system as well. I performed a git bisect test to identify the commit which caused the bug. 
@gpotter2 This bug is introduced in this commit: 880d0487c6885a5ddd0c5f747f6a9e0461a35278

I've tested this bug on 5.10.19-1-MANJARO. 
Thanks! Looking at the commit, it sets to_ms to 0 when calling open_pcap(). According to [man pcap_set_timeout](https://www.tcpdump.org/manpages/pcap_set_timeout.3pcap.html):

> _The behavior, if the timeout isn't specified, is undefined, as is
> the behavior if the timeout is set to zero or to a negative
> value.  We recommend always setting the timeout to a non-zero
> value unless immediate mode is set, in which case the timeout has
> no effect._

Immediate mode does not seem to be set anywhere.
That was reverted in a later commit: https://github.com/secdev/scapy/commit/2b96a8e3a48003c0b4a33a88a2ad570549058b6c not the issue here.
Thanks for the note polybassa I will have a look. TBH I'm rather intrigued, the commit you pointed out aimed at rewriting the behavior on Windows.. It shouldn't have changed anything on linux :/ I might have messed up something
Does this only happen when using `Pipetools` or also when using a plain `sniff` ?
If it's the former, try reverting this https://github.com/secdev/scapy/commit/880d0487c6885a5ddd0c5f747f6a9e0461a35278#diff-1bc4cea8a97434331b7833c4a4e720c5160e339759c73b9490d9c6277161871aR111
I've tested pure sniff ```sniff(iface="wlan0", timeout=100)```, that doesn't cause any cpu load
Thanks. Could you try the suggested fix?
No cpu load with your fix

Resolving patch:
diff --git a/scapy/pipetool.py b/scapy/pipetool.py
--- a/scapy/pipetool.py
+++ b/scapy/pipetool.py
@@ -108,7 +108,7 @@ def run(self):
             RUN = True
             STOP_IF_EXHAUSTED = False
             while RUN and (not STOP_IF_EXHAUSTED or len(sources) > 1):
-                fds = select_objects(sources, 0)
+                fds = select_objects(sources, 2)
                 for fd in fds:
                     if fd is self:
                         cmd = self._read_cmd()


Test patch: