Repository: OpenMined/PySyft

Issue:
`test_serde_simplify` is flaky
**Describe the bug**
`test_serde_simplify` frequently fails and appears to be applying the wrong tensor serialization format (sometimes but not always.)

**To Reproduce**
Run the test (or full suite) until it fails.

**Expected behavior**
The test should reliably pass.

**Screenshots**
```
2020-04-14T18:01:22.2134777Z =================================== FAILURES ===================================
2020-04-14T18:01:22.2135402Z __________________________ test_serde_simplify[State] __________________________
2020-04-14T18:01:22.2135862Z 
2020-04-14T18:01:22.2137173Z cls = <class 'syft.execution.state.State'>
2020-04-14T18:01:22.2138492Z workers = {'alice': <VirtualWorker id:alice #objects:0>, 'bob': <VirtualWorker id:bob #objects:0>, 'charlie': <VirtualWorker id:charlie #objects:0>, 'james': <VirtualWorker id:james #objects:0>, ...}
2020-04-14T18:01:22.2139188Z hook = <syft.frameworks.torch.hook.hook.TorchHook object at 0x7f343eeafe80>
2020-04-14T18:01:22.2140173Z start_remote_worker = <function start_remote_worker.<locals>._start_remote_worker at 0x7f343cc6aae8>
2020-04-14T18:01:22.2140630Z 
2020-04-14T18:01:22.2141200Z     @pytest.mark.parametrize("cls", samples)
2020-04-14T18:01:22.2142456Z     def test_serde_simplify(cls, workers, hook, start_remote_worker):
2020-04-14T18:01:22.2143904Z         """Checks that simplified structures match expected"""
2020-04-14T18:01:22.2144442Z         _samples = samples[cls](
2020-04-14T18:01:22.2144954Z             workers=workers,
2020-04-14T18:01:22.2145475Z             hook=hook,
2020-04-14T18:01:22.2146062Z             start_remote_worker=start_remote_worker,
2020-04-14T18:01:22.2146588Z             port=9001,
2020-04-14T18:01:22.2147100Z             id="simplify",
2020-04-14T18:01:22.2147606Z         )
2020-04-14T18:01:22.2148108Z         for sample in _samples:
2020-04-14T18:01:22.2148710Z             obj, expected_simplified_obj = sample.get("value"), sample.get("simplified")
2020-04-14T18:01:22.2149270Z             _simplify = (
2020-04-14T18:01:22.2149785Z                 msgpack.serde._simplify
2020-04-14T18:01:22.2150289Z                 if not sample.get("forced", False)
2020-04-14T18:01:22.2150818Z                 else msgpack.serde._force_full_simplify
2020-04-14T18:01:22.2151338Z             )
2020-04-14T18:01:22.2151890Z             serde_worker = syft.hook.local_worker
2020-04-14T18:01:22.2152420Z             serde_worker.framework = sample.get("framework", torch)
2020-04-14T18:01:22.2153946Z             simplified_obj = _simplify(syft.hook.local_worker, obj)
2020-04-14T18:01:22.2154461Z     
2020-04-14T18:01:22.2154872Z             if sample.get("cmp_simplified", None):
2020-04-14T18:01:22.2155283Z                 # Custom simplified objects comparison function.
2020-04-14T18:01:22.2155742Z                 assert sample.get("cmp_simplified")(simplified_obj, expected_simplified_obj) is True
2020-04-14T18:01:22.2156593Z             else:
2020-04-14T18:01:22.2157000Z >               assert simplified_obj == expected_simplified_obj
2020-04-14T18:01:22.2157445Z E               assert (22, ((1, ((4...one, ...)))))) == (22, ((1, ((4...one, ...))))))
2020-04-14T18:01:22.2170190Z E                 At index 1 diff: ((1, ((48, ((56, (22214076726,)), (3, ((5, (b'state1',)),)), None, None)), (48, ((56, (96071220832,)), (3, ((5, (b'state2',)),)), None, None)))), (1, ((14, (9826749836, b'\x80\x02\x8a\nl\xfc\x9cF\xf9 j\xa8P\x19.\x80\x02M\xe9\x03.\x80\x02}q\x00(X\x10\x00\x00\x00protocol_versionq\x01M\xe9\x03X\r\x00\x00\x00little_endianq\x02\x88X\n\x00\x00\x00type_sizesq\x03}q\x04(X\x05\x00\x00\x00shortq\x05K\x02X\x03\x00\x00\x00intq\x06K\x04X\x04\x00\x00\x00longq\x07K\x04uu.\x80\x02ctorch._utils\n_rebuild_tensor_v2\nq\x00((X\x07\x00\x00\x00storageq\x01ctorch\nFloatStorage\nq\x02X\x0e\x00\x00\x0094299942048352q\x03X\x03\x00\x00\x00cpuq\x04K\tNtq\x05QK\x00K\x03K\x03\x86q\x06K\x03K\x01\x86q\x07\x89ccollections\nOrderedDict\nq\x08)Rq\ttq\nRq\x0b.\x80\x02]q\x00X\x0e\x00\x00\x0094299942048352q\x01a.\t\x00\x00\x00\x00\x00\x00\x00#\x95{?\xf6\x19\x17\xbe\x9c\xa3\xec=\x01\x8b,<\x11-[\xbf\xa1d\xea\xbf\xb8\xcd5\xbe\xac\xbd\xb9\xbf\xb0JQ?', None, None, None, None, (5, (b'torch',)), None, None)), (14, (49440533498, b"\x80\x02\x8a\nl\xfc\x9cF\xf9 j\xa8P\x19.\x80\x02M\xe9\x03.\x80\x02}q\x00(X\x10\x00\x00\x00protocol_versionq\x01M\xe9\x03X\r\x00\x00\x00little_endianq\x02\x88X\n\x00\x00\x00type_sizesq\x03}q\x04(X\x05\x00\x00\x00shortq\x05K\x02X\x03\x00\x00\x00intq\x06K\x04X\x04\x00\x00\x00longq\x07K\x04uu.\x80\x02ctorch._utils\n_rebuild_tensor_v2\nq\x00((X\x07\x00\x00\x00storageq\x01ctorch\nFloatStorage\nq\x02X\x0e\x00\x00\x0094299946743552q\x03X\x03\x00\x00\x00cpuq\x04K\tNtq\x05QK\x00K\x03K\x03\x86q\x06K\x03K\x01\x86q\x07\x89ccollections\nOrderedDict\nq\x08)Rq\ttq\nRq\x0b.\x80\x02]q\x00X\x0e\x00\x00\x0094299946743552q\x01a.\t\x00\x00\x00\x00\x00\x00\x00p=D??*'\xbf\x11\x07\x97\xbdx\xa2\xd0=\x91\xb3v?\xad\xee->\xfd\xbc\xbc?\xef\x87\xe7\xbe\xa4\x1c\x00@", None, None, None, None, (5, (b'torch',)), None, None))))) != ((1, ((48, ((56, (22214076726,)), (3, ((5, (b'state1',)),)), None, None)), (48, ((56, (96071220832,)), (3, ((5, (b'state2',)),)), None, None)))), (1, ((14, (9826749836, (6, ((6, (3, 3)), (5, (b'float32',)), (1, (0.9827443957328796, -0.1475599706172943, 0.11554643511772156, 0.010531187988817692, -0.8561564087867737, -1.8311959505081177, -0.1775425672531128, -1.4511008262634277, 0.8175458908081055)))), None, None, None, None, (5, (b'all',)), None, None)), (14, (49440533498, (6, ((6, (3, 3)), (5, (b'float32',)), (1, (0.7665624618530273, -0.6529883742332458, -0.07374394685029984, 0.10187238454818726, 0.9636774659156799, 0.16985578835010529, 1.4745174646377563, -0.4522089660167694, 2.0017480850219727)))), None, None, None, None, (5, (b'all',)), None, None)))))
2020-04-14T18:01:22.2171445Z E                 Full diff:
2020-04-14T18:01:22.2171865Z E                   (
2020-04-14T18:01:22.2172266Z E                    22,
2020-04-14T18:01:22.2172660Z E                    ((1,
2020-04-14T18:01:22.2173053Z E                      ((48,
2020-04-14T18:01:22.2173436Z E                        ((56,
2020-04-14T18:01:22.2173834Z E                          (22214076726,)),
2020-04-14T18:01:22.2174238Z E                         (3,
2020-04-14T18:01:22.2174632Z E                          ((5,
2020-04-14T18:01:22.2175404Z E                            (b'state1',)),)),
2020-04-14T18:01:22.2175833Z E                         None,
2020-04-14T18:01:22.2176226Z E                         None)),
2020-04-14T18:01:22.2176620Z E                       (48,
2020-04-14T18:01:22.2177026Z E                        ((56,
2020-04-14T18:01:22.2177473Z E                          (96071220832,)),
2020-04-14T18:01:22.2177643Z E                         (3,
2020-04-14T18:01:22.2177828Z E                          ((5,
2020-04-14T18:01:22.2178170Z E                            (b'state2',)),)),
2020-04-14T18:01:22.2178369Z E                         None,
2020-04-14T18:01:22.2178554Z E                         None)))),
2020-04-14T18:01:22.2178739Z E                     (1,
2020-04-14T18:01:22.2178928Z E                      ((14,
2020-04-14T18:01:22.2179113Z E                        (9826749836,
2020-04-14T18:01:22.2179489Z E                 +       b'\x80\x02\x8a\nl\xfc\x9cF\xf9 j\xa8P\x19.\x80\x02M\xe9\x03.\x80\x02}'
2020-04-14T18:01:22.2179912Z E                 +       b'q\x00(X\x10\x00\x00\x00protocol_versionq\x01M\xe9\x03X\r\x00\x00\x00li'
2020-04-14T18:01:22.2180297Z E                 +       b'ttle_endianq\x02\x88X\n\x00\x00\x00type_sizesq\x03}q\x04(X'
2020-04-14T18:01:22.2180762Z E                 +       b'\x05\x00\x00\x00shortq\x05K\x02X\x03\x00\x00\x00intq\x06K\x04X\x04\x00'
2020-04-14T18:01:22.2181171Z E                 +       b'\x00\x00longq\x07K\x04uu.\x80\x02ctorch._utils\n_rebuild_tensor_v2\n'
2020-04-14T18:01:22.2181542Z E                 +       b'q\x00((X\x07\x00\x00\x00storageq\x01ctorch\nFloatStorage\nq\x02'
2020-04-14T18:01:22.2181933Z E                 +       b'X\x0e\x00\x00\x0094299942048352q\x03X\x03\x00\x00\x00cpuq\x04K\tNtq'
2020-04-14T18:01:22.2182528Z E                 +       b'\x05QK\x00K\x03K\x03\x86q\x06K\x03K\x01\x86q\x07\x89ccollections\nOrde'
2020-04-14T18:01:22.2182979Z E                 +       b'redDict\nq\x08)Rq\ttq\nRq\x0b.\x80\x02]q\x00X\x0e\x00\x00\x00942999420'
2020-04-14T18:01:22.2183422Z E                 +       b'48352q\x01a.\t\x00\x00\x00\x00\x00\x00\x00#\x95{?\xf6\x19\x17'
2020-04-14T18:01:22.2183887Z E                 +       b'\xbe\x9c\xa3\xec=\x01\x8b,<\x11-[\xbf\xa1d\xea\xbf\xb8\xcd5'
2020-04-14T18:01:22.2184281Z E                 +       b'\xbe\xac\xbd\xb9\xbf\xb0JQ?',
2020-04-14T18:01:22.2184668Z E                 -       (6,
2020-04-14T18:01:22.2185014Z E                 -        ((6,
2020-04-14T18:01:22.2185394Z E                 -          (3,
2020-04-14T18:01:22.2185742Z E                 -           3)),
2020-04-14T18:01:22.2186104Z E                 -         (5,
2020-04-14T18:01:22.2186487Z E                 -          (b'float32',)),
2020-04-14T18:01:22.2186851Z E                 -         (1,
2020-04-14T18:01:22.2187323Z E                 -          (0.9827443957328796,
2020-04-14T18:01:22.2187690Z E                 -           -0.1475599706172943,
2020-04-14T18:01:22.2188096Z E                 -           0.11554643511772156,
2020-04-14T18:01:22.2188465Z E                 -           0.010531187988817692,
2020-04-14T18:01:22.2188894Z E                 -           -0.8561564087867737,
2020-04-14T18:01:22.2189260Z E                 -           -1.8311959505081177,
2020-04-14T18:01:22.2189665Z E                 -           -0.1775425672531128,
2020-04-14T18:01:22.2190034Z E                 -           -1.4511008262634277,
2020-04-14T18:01:22.2190421Z E                 -           0.8175458908081055)))),
2020-04-14T18:01:22.2190649Z E                         None,
2020-04-14T18:01:22.2190860Z E                         None,
2020-04-14T18:01:22.2191066Z E                         None,
2020-04-14T18:01:22.2191256Z E                         None,
2020-04-14T18:01:22.2191485Z E                         (5,
2020-04-14T18:01:22.2191833Z E                 -        (b'all',)),
2020-04-14T18:01:22.2192094Z E                 ?           ^^^
2020-04-14T18:01:22.2192454Z E                 +        (b'torch',)),
2020-04-14T18:01:22.2192664Z E                 ?           ^^^^^
2020-04-14T18:01:22.2192875Z E                         None,
2020-04-14T18:01:22.2193084Z E                         None)),
2020-04-14T18:01:22.2193295Z E                       (14,
2020-04-14T18:01:22.2193506Z E                        (49440533498,
2020-04-14T18:01:22.2193939Z E                 +       b'\x80\x02\x8a\nl\xfc\x9cF\xf9 j\xa8P\x19.\x80\x02M\xe9\x03.\x80\x02}'
2020-04-14T18:01:22.2194376Z E                 +       b'q\x00(X\x10\x00\x00\x00protocol_versionq\x01M\xe9\x03X\r\x00\x00\x00li'
2020-04-14T18:01:22.2194829Z E                 +       b'ttle_endianq\x02\x88X\n\x00\x00\x00type_sizesq\x03}q\x04(X'
2020-04-14T18:01:22.2195269Z E                 +       b'\x05\x00\x00\x00shortq\x05K\x02X\x03\x00\x00\x00intq\x06K\x04X\x04\x00'
2020-04-14T18:01:22.2195729Z E                 +       b'\x00\x00longq\x07K\x04uu.\x80\x02ctorch._utils\n_rebuild_tensor_v2\n'
2020-04-14T18:01:22.2196719Z E                 +       b'q\x00((X\x07\x00\x00\x00storageq\x01ctorch\nFloatStorage\nq\x02'
2020-04-14T18:01:22.2197205Z E                 +       b'X\x0e\x00\x00\x0094299946743552q\x03X\x03\x00\x00\x00cpuq\x04K\tNtq'
2020-04-14T18:01:22.2197642Z E                 +       b'\x05QK\x00K\x03K\x03\x86q\x06K\x03K\x01\x86q\x07\x89ccollections\nOrde'
2020-04-14T18:01:22.2198217Z E                 +       b'redDict\nq\x08)Rq\ttq\nRq\x0b.\x80\x02]q\x00X\x0e\x00\x00\x00942999467'
2020-04-14T18:01:22.2198682Z E                 +       b"43552q\x01a.\t\x00\x00\x00\x00\x00\x00\x00p=D??*'\xbf\x11\x07\x97"
2020-04-14T18:01:22.2199134Z E                 +       b'\xbdx\xa2\xd0=\x91\xb3v?\xad\xee->\xfd\xbc\xbc?\xef\x87\xe7'
2020-04-14T18:01:22.2199514Z E                 +       b'\xbe\xa4\x1c\x00@',
2020-04-14T18:01:22.2199880Z E                 -       (6,
2020-04-14T18:01:22.2200242Z E                 -        ((6,
2020-04-14T18:01:22.2200612Z E                 -          (3,
2020-04-14T18:01:22.2200979Z E                 -           3)),
2020-04-14T18:01:22.2201498Z E                 -         (5,
2020-04-14T18:01:22.2201919Z E                 -          (b'float32',)),
2020-04-14T18:01:22.2202312Z E                 -         (1,
2020-04-14T18:01:22.2202708Z E                 -          (0.7665624618530273,
2020-04-14T18:01:22.2203142Z E                 -           -0.6529883742332458,
2020-04-14T18:01:22.2203545Z E                 -           -0.07374394685029984,
2020-04-14T18:01:22.2203979Z E                 -           0.10187238454818726,
2020-04-14T18:01:22.2204406Z E                 -           0.9636774659156799,
2020-04-14T18:01:22.2204821Z E                 -           0.16985578835010529,
2020-04-14T18:01:22.2205233Z E                 -           1.4745174646377563,
2020-04-14T18:01:22.2205629Z E                 -           -0.4522089660167694,
2020-04-14T18:01:22.2206066Z E                 -           2.0017480850219727)))),
2020-04-14T18:01:22.2206388Z E                         None,
2020-04-14T18:01:22.2206613Z E                         None,
2020-04-14T18:01:22.2206856Z E                         None,
2020-04-14T18:01:22.2207062Z E                         None,
2020-04-14T18:01:22.2207320Z E                         (5,
2020-04-14T18:01:22.2207701Z E                 -        (b'all',)),
2020-04-14T18:01:22.2207959Z E                 ?           ^^^
2020-04-14T18:01:22.2208337Z E                 +        (b'torch',)),
2020-04-14T18:01:22.2208586Z E                 ?           ^^^^^
2020-04-14T18:01:22.2208813Z E                         None,
2020-04-14T18:01:22.2209023Z E                         None))))),
2020-04-14T18:01:22.2209267Z E                   )
2020-04-14T18:01:22.2209433Z 
2020-04-14T18:01:22.2209663Z test/serde/msgpack/test_msgpack_serde_full.py:166: AssertionError
```

**Additional context**
This started being an issue when the order of the test suite was randomized (specifically to shake out flaky tests like this.)



Discussion:

Resolving patch:
diff --git a/syft/workers/base.py b/syft/workers/base.py
--- a/syft/workers/base.py
+++ b/syft/workers/base.py
@@ -1178,15 +1178,13 @@ def serializer(self, workers=None) -> codes.TENSOR_SERIALIZATION:
                 'torch': serialization will only work between workers that support PyTorch
                 (more to come: 'tensorflow', 'numpy', etc)
         """
-        if workers is not None:
-            if not isinstance(workers, list):
-                workers = [workers]
-        else:
+        if workers is None:
             workers = [w for w in self._known_workers.values() if isinstance(w, AbstractWorker)]
 
-        # self is not referenced in self._known_workers when auto_add=False
-        if self not in workers:
-            workers.append(self)
+        if not isinstance(workers, list):
+            workers = [workers]
+
+        workers.append(self)
 
         frameworks = set()
         for worker in workers:


Test patch: