Repository: aio-libs/aiohttp

Issue:
Fast xoring in python
instead of 

```
def _websocket_mask_python(mask, data):
    return bytes(b ^ mask[i % 4] for i, b in enumerate(data))
```

please use this:

```
native_byteorder = sys.byteorder

def _websocket_mask_python(mask, data):
    assert len(mask) == 4
    datalen = len(data)
    if datalen == 0:
        return b''  #  everything work without this, but may be changed later in python.
    data = int.from_bytes(data, native_byteorder)
    mask = int.from_bytes(mask * (datalen // 4) + mask[: datalen % 4], native_byteorder)
    return (data ^ mask).to_bytes(datalen, native_byteorder)
```



Discussion:
Why?

this is MUCH faster

How much?

Could you elaborate on this? Could you post benchmark results or insights why this works faster?

depending on datasize, on my computer, from 1.1 to 31 times. The most "optimal" size for benchmarking is data size of 3 MB

```
#!/usr/bin/python3.5
import sys
import itertools
from time import monotonic

native_byteorder = sys.byteorder

def _websocket_mask_python(mask, data):
    assert len(mask) == 4
    datalen = len(data)
    if datalen == 0:
        return b''  #  everything work without this, but may be changed later in python.
    data = int.from_bytes(data, native_byteorder)
    mask = int.from_bytes(mask * (datalen // 4) + mask[: datalen % 4], native_byteorder)
    #mask = int.from_bytes(itertools.islice(itertools.cycle(mask), 0, datalen), native_byteorder)
    return (data ^ mask).to_bytes(datalen, native_byteorder)

def _websocket_mask_python1(mask, data):
    return bytes(b ^ mask[i % 4] for i, b in enumerate(data))


data = b'f' * (1024*1024*3)

a = monotonic()
res = _websocket_mask_python(b'1234', data)
b = monotonic()

a1 = monotonic()
res1 = _websocket_mask_python1(b'1234', data)
b1 = monotonic()

print (res==res1, (b1-a1)/(b-a))
```

it is faster even on one-byte message :)

Looks not oblivious why more operations(and code) is faster than shorter version:

```
Python 3.4.3 (default, Nov 12 2015, 20:43:56)
...
In [14]: %timeit websocket_mask(b'1234', data)
1 loops, best of 3: 967 ms per loop

In [15]: %timeit websocket_mask2(b'1234', data)
10 loops, best of 3: 22.6 ms per loop
```

Do you have an explanation for this?

sure, original implementation is executed in python, while my implementation is executed in C.

I don't care too much because cythonized version 7 times faster than your optimized python one.

You may make a pull request though.

@socketpair But enumeration and bytes operations are not made in Python, right? Math is the same. What I see in yours is more function/method calls which means more work to do. Still curious.

@asvetlov about cython, what you compare with what ?

@kxepal `b ^ mask[i % 4] for i, b in enumerate(data)` creates a new `int` for every byte.
@socketpair's version just creates one long `int` from the bytes and xors them.

This is a rule: block operations are always faster than byte-by-byte.

True, thanks!

would this be merged so ?

also, constructing big integer is cheap operation. xoring two bigintegers with same size is cheap too.

The most time (as I think) this function spent in constructing mask...I tried 4 approaches, maybe you know faster one ?

@socketpair I've compared your `_websocket_mask_python` with `aiohttp.websocket._websocket_mask_python` and `aiohttp.websocket._websocket_mask_cython`.
The last is 7 faster than your code.

Sorry, I cannot merge still nonexistent pull request :)


Resolving patch:
diff --git a/aiohttp/websocket.py b/aiohttp/websocket.py
--- a/aiohttp/websocket.py
+++ b/aiohttp/websocket.py
@@ -6,6 +6,7 @@
 import hashlib
 import os
 import random
+import sys
 
 from struct import Struct
 from aiohttp import errors, hdrs
@@ -172,6 +173,9 @@ def WebSocketParser(out, buf):
                     Message(OPCODE_BINARY, data, ''), len(data))
 
 
+native_byteorder = sys.byteorder
+
+
 def _websocket_mask_python(mask, data):
     """Websocket masking function.
 
@@ -184,7 +188,15 @@ def _websocket_mask_python(mask, data):
     version when available.
 
     """
-    return bytes(b ^ mask[i % 4] for i, b in enumerate(data))
+    assert len(mask) == 4
+    datalen = len(data)
+    if datalen == 0:
+        # everything work without this, but may be changed later in Python.
+        return b''
+    data = int.from_bytes(data, native_byteorder)
+    mask = int.from_bytes(mask * (datalen // 4) + mask[: datalen % 4],
+                          native_byteorder)
+    return (data ^ mask).to_bytes(datalen, native_byteorder)
 
 
 if bool(os.environ.get('AIOHTTP_NO_EXTENSIONS')):


Test patch: