zeromq/pyzmq — Issue #163: strange decrease in concurrency when pythoncom is imported
Author: kmanley
Created: 2012-01-03 19:04:17+00:00
State: open

I'm noticing a strange decrease in zmq concurrency when pythoncom is imported into the server process. This is with Python 2.7.2 on Windows.

To demonstrate the issue, use these 2 server scripts. The only difference is that one imports pythoncom

--- server_with_pythoncom.py ---
import zmq, xpersec
import pythoncom
counter = xpersec.XPerSec("reqs")
context = zmq.Context()
socket = context.socket(zmq.REP)
socket.bind("tcp://*:5555")
while True:
req = socket.recv()
counter.incr()
socket.send("OK")

--- server_without_pythoncom.py ---
(same as above without the pythoncom import)

--- xpersec.py ---
import time
class XPerSec:
def __init__(self, things):
self.things = things
self.start = time.clock()
self.count = 0

def incr(self):
now = time.clock()
elapsed = now - self.start
if elapsed > 1.0:
print "%.2f %s/sec" % (self.count/elapsed, self.things)
self.count = 1
self.start = now
else:
self.count += 1

and run at least 2 concurrent test clients

---client.py ---
import zmq, xpersec
counter = xpersec.XPerSec("req")
context = zmq.Context()
socket = context.socket(zmq.REQ)
socket.connect("tcp://127.0.0.1:5555")
while True:
socket.send("wtf?")
resp = socket.recv()
counter.incr()

--- end of code

When I run these, I get the following results. Without importing pythoncom, I consistently get 18K req/sec with 2 concurrent clients

C:\>c:\python27\python.exe server_without_pythoncom.py
18288.48 reqs/sec
18499.64 reqs/sec
18482.12 reqs/sec
18602.69 reqs/sec
18760.62 reqs/sec
18848.12 reqs/sec
18811.89 reqs/sec
18877.48 reqs/sec
18849.82 reqs/sec
18740.54 reqs/sec
18861.79 reqs/sec
18527.44 reqs/sec
18748.51 reqs/sec

However just by importing pythoncom into the server process, performance initially starts at 18K req/sec but quickly drops off to around 1K req/sec.

C:\data\personal\code\zmq_pythoncom_bug>c:\python27\python.exe server_with_pythoncom.py
18275.18 reqs/sec
18400.20 reqs/sec
18653.44 reqs/sec
18643.85 reqs/sec
18760.10 reqs/sec
15087.09 reqs/sec
1016.91 reqs/sec
1399.99 reqs/sec
1464.89 reqs/sec
1193.87 reqs/sec
1367.96 reqs/sec
1087.83 reqs/sec
1384.90 reqs/sec
1359.83 reqs/sec
1116.84 reqs/sec
1256.96 reqs/sec
1219.97 reqs/sec

I observed this behavior in a larger application and narrowed it down to this test case. Please let me know if you need any further details.